diff options
| author | Reza Arbab <arbab@linux.ibm.com> | 2020-07-16 17:56:55 -0500 |
|---|---|---|
| committer | David Gibson <david@gibson.dropbear.id.au> | 2020-07-20 09:21:39 +1000 |
| commit | a6030d7e0b35a23c82e4a765b53dc3847bcdb4d1 (patch) | |
| tree | 0476fd6630d17bd727a20779aeae771348bd7011 /include | |
| parent | a4beb5f5d4672400a76cc0551c4f0878e42d921c (diff) | |
spapr: Add a new level of NUMA for GPUs
NUMA nodes corresponding to GPU memory currently have the same
affinity/distance as normal memory nodes. Add a third NUMA associativity
reference point enabling us to give GPU nodes more distance.
This is guest visible information, which shouldn't change under a
running guest across migration between different qemu versions, so make
the change effective only in new (pseries > 5.0) machine types.
Before, `numactl -H` output in a guest with 4 GPUs (nodes 2-5):
node distances:
node 0 1 2 3 4 5
0: 10 40 40 40 40 40
1: 40 10 40 40 40 40
2: 40 40 10 40 40 40
3: 40 40 40 10 40 40
4: 40 40 40 40 10 40
5: 40 40 40 40 40 10
After:
node distances:
node 0 1 2 3 4 5
0: 10 40 80 80 80 80
1: 40 10 80 80 80 80
2: 80 80 10 80 80 80
3: 80 80 80 10 80 80
4: 80 80 80 80 10 80
5: 80 80 80 80 80 10
These are the same distances as on the host, mirroring the change made
to host firmware in skiboot commit f845a648b8cb ("numa/associativity:
Add a new level of NUMA for GPU's").
Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
Message-Id: <20200716225655.24289-1-arbab@linux.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Diffstat (limited to 'include')
| -rw-r--r-- | include/hw/pci-host/spapr.h | 1 | ||||
| -rw-r--r-- | include/hw/ppc/spapr.h | 1 |
2 files changed, 2 insertions, 0 deletions
diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h index 8877ff51fb..600eb55c34 100644 --- a/include/hw/pci-host/spapr.h +++ b/include/hw/pci-host/spapr.h @@ -94,6 +94,7 @@ struct SpaprPhbState { hwaddr nv2_gpa_win_addr; hwaddr nv2_atsd_win_addr; SpaprPhbPciNvGpuConfig *nvgpus; + bool pre_5_1_assoc; }; #define SPAPR_PCI_MEM_WIN_BUS_OFFSET 0x80000000ULL diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index c421410e3f..3134d339e8 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -129,6 +129,7 @@ struct SpaprMachineClass { bool linux_pci_probe; bool smp_threads_vsmt; /* set VSMT to smp_threads by default */ hwaddr rma_limit; /* clamp the RMA to this size */ + bool pre_5_1_assoc_refpoints; void (*phb_placement)(SpaprMachineState *spapr, uint32_t index, uint64_t *buid, hwaddr *pio, |