diff options
38 files changed, 1516 insertions, 1194 deletions
diff --git a/docs/specs/ppc-spapr-xive.rst b/docs/specs/ppc-spapr-xive.rst index 539ce7ca4e..6159bc6eed 100644 --- a/docs/specs/ppc-spapr-xive.rst +++ b/docs/specs/ppc-spapr-xive.rst @@ -34,19 +34,118 @@ CAS Negotiation --------------- QEMU advertises the supported interrupt modes in the device tree -property "ibm,arch-vec-5-platform-support" in byte 23 and the OS -Selection for XIVE is indicated in the "ibm,architecture-vec-5" +property ``ibm,arch-vec-5-platform-support`` in byte 23 and the OS +Selection for XIVE is indicated in the ``ibm,architecture-vec-5`` property byte 23. The interrupt modes supported by the machine depend on the CPU type (POWER9 is required for XIVE) but also on the machine property ``ic-mode`` which can be set on the command line. It can take the -following values: ``xics``, ``xive``, ``dual`` and currently ``xics`` -is the default but it may change in the future. +following values: ``xics``, ``xive``, and ``dual`` which is the +default mode. ``dual`` means that both modes XICS **and** XIVE are +supported and if the guest OS supports XIVE, this mode will be +selected. The choosen interrupt mode is activated after a reconfiguration done in a machine reset. +KVM negotiation +--------------- + +When the guest starts under KVM, the capabilities of the host kernel +and QEMU are also negotiated. Depending on the version of the host +kernel, KVM will advertise the XIVE capability to QEMU or not. + +Nevertheless, the available interrupt modes in the machine should not +depend on the XIVE KVM capability of the host. On older kernels +without XIVE KVM support, QEMU will use the emulated XIVE device as a +fallback and on newer kernels (>=5.2), the KVM XIVE device. + +As a final refinement, the user can also switch the use of the KVM +device with the machine option ``kernel_irqchip``. + + +XIVE support in KVM +~~~~~~~~~~~~~~~~~~~ + +For guest OSes supporting XIVE, the resulting interrupt modes on host +kernels with XIVE KVM support are the following: + +============== ============= ============= ================ +ic-mode kernel_irqchip +-------------- ---------------------------------------------- +/ allowed off on + (default) +============== ============= ============= ================ +dual (default) XIVE KVM XIVE emul. XIVE KVM +xive XIVE KVM XIVE emul. XIVE KVM +xics XICS KVM XICS emul. XICS KVM +============== ============= ============= ================ + +For legacy guest OSes without XIVE support, the resulting interrupt +modes are the following: + +============== ============= ============= ================ +ic-mode kernel_irqchip +-------------- ---------------------------------------------- +/ allowed off on + (default) +============== ============= ============= ================ +dual (default) XICS KVM XICS emul. XICS KVM +xive QEMU error(3) QEMU error(3) QEMU error(3) +xics XICS KVM XICS emul. XICS KVM +============== ============= ============= ================ + +(3) QEMU fails at CAS with ``Guest requested unavailable interrupt + mode (XICS), either don't set the ic-mode machine property or try + ic-mode=xics or ic-mode=dual`` + + +No XIVE support in KVM +~~~~~~~~~~~~~~~~~~~~~~ + +For guest OSes supporting XIVE, the resulting interrupt modes on host +kernels without XIVE KVM support are the following: + +============== ============= ============= ================ +ic-mode kernel_irqchip +-------------- ---------------------------------------------- +/ allowed off on + (default) +============== ============= ============= ================ +dual (default) XIVE emul.(1) XIVE emul. QEMU error (2) +xive XIVE emul.(1) XIVE emul. QEMU error (2) +xics XICS KVM XICS emul. XICS KVM +============== ============= ============= ================ + + +(1) QEMU warns with ``warning: kernel_irqchip requested but unavailable: + IRQ_XIVE capability must be present for KVM`` +(2) QEMU fails with ``kernel_irqchip requested but unavailable: + IRQ_XIVE capability must be present for KVM`` + + +For legacy guest OSes without XIVE support, the resulting interrupt +modes are the following: + +============== ============= ============= ================ +ic-mode kernel_irqchip +-------------- ---------------------------------------------- +/ allowed off on + (default) +============== ============= ============= ================ +dual (default) QEMU error(4) XICS emul. QEMU error(4) +xive QEMU error(3) QEMU error(3) QEMU error(3) +xics XICS KVM XICS emul. XICS KVM +============== ============= ============= ================ + +(3) QEMU fails at CAS with ``Guest requested unavailable interrupt + mode (XICS), either don't set the ic-mode machine property or try + ic-mode=xics or ic-mode=dual`` +(4) QEMU/KVM incompatibility due to device destruction in reset. QEMU fails + with ``KVM is too old to support ic-mode=dual,kernel-irqchip=on`` + + XIVE Device tree properties --------------------------- @@ -92,10 +191,11 @@ for both interrupt mode. The different ranges are defined as follow : - ``0x0000 .. 0x0FFF`` 4K CPU IPIs (only used under XIVE) - ``0x1000 .. 0x1000`` 1 EPOW - ``0x1001 .. 0x1001`` 1 HOTPLUG +- ``0x1002 .. 0x10FF`` unused - ``0x1100 .. 0x11FF`` 256 VIO devices -- ``0x1200 .. 0x127F`` 32 PHBs devices +- ``0x1200 .. 0x127F`` 32x4 LSIs for PHB devices - ``0x1280 .. 0x12FF`` unused -- ``0x1300 .. 0x1FFF`` PHB MSIs +- ``0x1300 .. 0x1FFF`` PHB MSIs (dynamically allocated) Monitoring XIVE --------------- diff --git a/docs/specs/ppc-xive.rst b/docs/specs/ppc-xive.rst index b997dc0629..148d57eb6a 100644 --- a/docs/specs/ppc-xive.rst +++ b/docs/specs/ppc-xive.rst @@ -20,10 +20,11 @@ The XIVE IC is composed of three sub-engines, each taking care of a processing layer of external interrupts: - Interrupt Virtualization Source Engine (IVSE), or Source Controller - (SC). These are found in PCI PHBs, in the PSI host bridge - controller, but also inside the main controller for the core IPIs - and other sub-chips (NX, CAP, NPU) of the chip/processor. They are - configured to feed the IVRE with events. + (SC). These are found in PCI PHBs, in the Processor Service + Interface (PSI) host bridge Controller, but also inside the main + controller for the core IPIs and other sub-chips (NX, CAP, NPU) of + the chip/processor. They are configured to feed the IVRE with + events. - Interrupt Virtualization Routing Engine (IVRE) or Virtualization Controller (VC). It handles event coalescing and perform interrupt routing by matching an event source number with an Event diff --git a/hw/intc/pnv_xive.c b/hw/intc/pnv_xive.c index a55c2bbc88..4dc92ef1e3 100644 --- a/hw/intc/pnv_xive.c +++ b/hw/intc/pnv_xive.c @@ -169,7 +169,7 @@ static uint64_t pnv_xive_vst_addr_indirect(PnvXive *xive, uint32_t type, vsd = ldq_be_dma(&address_space_memory, vsd_addr); if (!(vsd & VSD_ADDRESS_MASK)) { - xive_error(xive, "VST: invalid %s entry %x !?", info->name, 0); + xive_error(xive, "VST: invalid %s entry %x !?", info->name, idx); return 0; } @@ -190,7 +190,7 @@ static uint64_t pnv_xive_vst_addr_indirect(PnvXive *xive, uint32_t type, vsd = ldq_be_dma(&address_space_memory, vsd_addr); if (!(vsd & VSD_ADDRESS_MASK)) { - xive_error(xive, "VST: invalid %s entry %x !?", info->name, 0); + xive_error(xive, "VST: invalid %s entry %x !?", info->name, idx); return 0; } @@ -294,8 +294,12 @@ static int pnv_xive_write_end(XiveRouter *xrtr, uint8_t blk, uint32_t idx, word_number); } -static int pnv_xive_end_update(PnvXive *xive, uint8_t blk, uint32_t idx) +static int pnv_xive_end_update(PnvXive *xive) { + uint8_t blk = GETFIELD(VC_EQC_CWATCH_BLOCKID, + xive->regs[(VC_EQC_CWATCH_SPEC >> 3)]); + uint32_t idx = GETFIELD(VC_EQC_CWATCH_OFFSET, + xive->regs[(VC_EQC_CWATCH_SPEC >> 3)]); int i; uint64_t eqc_watch[4]; @@ -307,6 +311,24 @@ static int pnv_xive_end_update(PnvXive *xive, uint8_t blk, uint32_t idx) XIVE_VST_WORD_ALL); } +static void pnv_xive_end_cache_load(PnvXive *xive) +{ + uint8_t blk = GETFIELD(VC_EQC_CWATCH_BLOCKID, + xive->regs[(VC_EQC_CWATCH_SPEC >> 3)]); + uint32_t idx = GETFIELD(VC_EQC_CWATCH_OFFSET, + xive->regs[(VC_EQC_CWATCH_SPEC >> 3)]); + uint64_t eqc_watch[4] = { 0 }; + int i; + + if (pnv_xive_vst_read(xive, VST_TSEL_EQDT, blk, idx, eqc_watch)) { + xive_error(xive, "VST: no END entry %x/%x !?", blk, idx); + } + + for (i = 0; i < ARRAY_SIZE(eqc_watch); i++) { + xive->regs[(VC_EQC_CWATCH_DAT0 >> 3) + i] = be64_to_cpu(eqc_watch[i]); + } +} + static int pnv_xive_get_nvt(XiveRouter *xrtr, uint8_t blk, uint32_t idx, XiveNVT *nvt) { @@ -320,8 +342,12 @@ static int pnv_xive_write_nvt(XiveRouter *xrtr, uint8_t blk, uint32_t idx, word_number); } -static int pnv_xive_nvt_update(PnvXive *xive, uint8_t blk, uint32_t idx) +static int pnv_xive_nvt_update(PnvXive *xive) { + uint8_t blk = GETFIELD(PC_VPC_CWATCH_BLOCKID, + xive->regs[(PC_VPC_CWATCH_SPEC >> 3)]); + uint32_t idx = GETFIELD(PC_VPC_CWATCH_OFFSET, + xive->regs[(PC_VPC_CWATCH_SPEC >> 3)]); int i; uint64_t vpc_watch[8]; @@ -333,6 +359,24 @@ static int pnv_xive_nvt_update(PnvXive *xive, uint8_t blk, uint32_t idx) XIVE_VST_WORD_ALL); } +static void pnv_xive_nvt_cache_load(PnvXive *xive) +{ + uint8_t blk = GETFIELD(PC_VPC_CWATCH_BLOCKID, + xive->regs[(PC_VPC_CWATCH_SPEC >> 3)]); + uint32_t idx = GETFIELD(PC_VPC_CWATCH_OFFSET, + xive->regs[(PC_VPC_CWATCH_SPEC >> 3)]); + uint64_t vpc_watch[8] = { 0 }; + int i; + + if (pnv_xive_vst_read(xive, VST_TSEL_VPDT, blk, idx, vpc_watch)) { + xive_error(xive, "VST: no NVT entry %x/%x !?", blk, idx); + } + + for (i = 0; i < ARRAY_SIZE(vpc_watch); i++) { + xive->regs[(PC_VPC_CWATCH_DAT0 >> 3) + i] = be64_to_cpu(vpc_watch[i]); + } +} + static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx, XiveEAS *eas) { @@ -346,12 +390,6 @@ static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx, return pnv_xive_vst_read(xive, VST_TSEL_IVT, blk, idx, eas); } -static int pnv_xive_eas_update(PnvXive *xive, uint8_t blk, uint32_t idx) -{ - /* All done. */ - return 0; -} - static XiveTCTX *pnv_xive_get_tctx(XiveRouter *xrtr, CPUState *cs) { PowerPCCPU *cpu = POWERPC_CPU(cs); @@ -781,8 +819,7 @@ static void pnv_xive_ic_reg_write(void *opaque, hwaddr offset, * support recently though) */ if (val & (VC_SBC_CONF_CPLX_CIST | VC_SBC_CONF_CIST_BOTH)) { - object_property_set_int(OBJECT(&xive->ipi_source), - XIVE_SRC_STORE_EOI, "flags", &error_fatal); + xive->ipi_source.esb_flags |= XIVE_SRC_STORE_EOI; } break; @@ -951,28 +988,43 @@ static void pnv_xive_ic_reg_write(void *opaque, hwaddr offset, * XIVE PC & VC cache updates for EAS, NVT and END */ case VC_IVC_SCRUB_MASK: - break; case VC_IVC_SCRUB_TRIG: - pnv_xive_eas_update(xive, GETFIELD(PC_SCRUB_BLOCK_ID, val), - GETFIELD(VC_SCRUB_OFFSET, val)); break; - case VC_EQC_SCRUB_MASK: case VC_EQC_CWATCH_SPEC: - case VC_EQC_CWATCH_DAT0 ... VC_EQC_CWATCH_DAT3: + val &= ~VC_EQC_CWATCH_CONFLICT; /* HW resets this bit */ + break; + case VC_EQC_CWATCH_DAT1 ... VC_EQC_CWATCH_DAT3: break; + case VC_EQC_CWATCH_DAT0: + /* writing to DATA0 triggers the cache write */ + xive->regs[reg] = val; + pnv_xive_end_update(xive); + break; + case VC_EQC_SCRUB_MASK: case VC_EQC_SCRUB_TRIG: - pnv_xive_end_update(xive, GETFIELD(VC_SCRUB_BLOCK_ID, val), - GETFIELD(VC_SCRUB_OFFSET, val)); + /* + * The scrubbing registers flush the cache in RAM and can also + * invalidate. + */ break; - case PC_VPC_SCRUB_MASK: case PC_VPC_CWATCH_SPEC: - case PC_VPC_CWATCH_DAT0 ... PC_VPC_CWATCH_DAT7: + val &= ~PC_VPC_CWATCH_CONFLICT; /* HW resets this bit */ + break; + case PC_VPC_CWATCH_DAT1 ... PC_VPC_CWATCH_DAT7: break; + case PC_VPC_CWATCH_DAT0: + /* writing to DATA0 triggers the cache write */ + xive->regs[reg] = val; + pnv_xive_nvt_update(xive); + break; + case PC_VPC_SCRUB_MASK: case PC_VPC_SCRUB_TRIG: - pnv_xive_nvt_update(xive, GETFIELD(PC_SCRUB_BLOCK_ID, val), - GETFIELD(PC_SCRUB_OFFSET, val)); + /* + * The scrubbing registers flush the cache in RAM and can also + * invalidate. + */ break; @@ -1023,15 +1075,6 @@ static uint64_t pnv_xive_ic_reg_read(void *opaque, hwaddr offset, unsigned size) case PC_GLOBAL_CONFIG: case PC_VPC_SCRUB_MASK: - case PC_VPC_CWATCH_SPEC: - case PC_VPC_CWATCH_DAT0: - case PC_VPC_CWATCH_DAT1: - case PC_VPC_CWATCH_DAT2: - case PC_VPC_CWATCH_DAT3: - case PC_VPC_CWATCH_DAT4: - case PC_VPC_CWATCH_DAT5: - case PC_VPC_CWATCH_DAT6: - case PC_VPC_CWATCH_DAT7: case VC_GLOBAL_CONFIG: case VC_AIB_TX_ORDER_TAG2: @@ -1044,12 +1087,6 @@ static uint64_t pnv_xive_ic_reg_read(void *opaque, hwaddr offset, unsigned size) case VC_IRQ_CONFIG_IPI_CASC: case VC_EQC_SCRUB_MASK: - case VC_EQC_CWATCH_DAT0: - case VC_EQC_CWATCH_DAT1: - case VC_EQC_CWATCH_DAT2: - case VC_EQC_CWATCH_DAT3: - - case VC_EQC_CWATCH_SPEC: case VC_IVC_SCRUB_MASK: case VC_SBC_CONFIG: case VC_AT_MACRO_KILL_MASK: @@ -1081,6 +1118,38 @@ static uint64_t pnv_xive_ic_reg_read(void *opaque, hwaddr offset, unsigned size) /* * XIVE PC & VC cache updates for EAS, NVT and END */ + case VC_EQC_CWATCH_SPEC: + xive->regs[reg] = ~(VC_EQC_CWATCH_FULL | VC_EQC_CWATCH_CONFLICT); + val = xive->regs[reg]; + break; + case VC_EQC_CWATCH_DAT0: + /* + * Load DATA registers from cache with data requested by the + * SPEC register + */ + pnv_xive_end_cache_load(xive); + val = xive->regs[reg]; + break; + case VC_EQC_CWATCH_DAT1 ... VC_EQC_CWATCH_DAT3: + val = xive->regs[reg]; + break; + + case PC_VPC_CWATCH_SPEC: + xive->regs[reg] = ~(PC_VPC_CWATCH_FULL | PC_VPC_CWATCH_CONFLICT); + val = xive->regs[reg]; + break; + case PC_VPC_CWATCH_DAT0: + /* + * Load DATA registers from cache with data requested by the + * SPEC register + */ + pnv_xive_nvt_cache_load(xive); + val = xive->regs[reg]; + break; + case PC_VPC_CWATCH_DAT1 ... PC_VPC_CWATCH_DAT7: + val = xive->regs[reg]; + break; + case PC_VPC_SCRUB_TRIG: case VC_IVC_SCRUB_TRIG: case VC_EQC_SCRUB_TRIG: diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c index 58c2e5d890..3ae311d9ff 100644 --- a/hw/intc/spapr_xive.c +++ b/hw/intc/spapr_xive.c @@ -194,13 +194,6 @@ void spapr_xive_pic_print_info(SpaprXive *xive, Monitor *mon) } } -void spapr_xive_map_mmio(SpaprXive *xive) -{ - sysbus_mmio_map(SYS_BUS_DEVICE(xive), 0, xive->vc_base); - sysbus_mmio_map(SYS_BUS_DEVICE(xive), 1, xive->end_base); - sysbus_mmio_map(SYS_BUS_DEVICE(xive), 2, xive->tm_base); -} - void spapr_xive_mmio_set_enabled(SpaprXive *xive, bool enable) { memory_region_set_enabled(&xive->source.esb_mmio, enable); @@ -305,6 +298,7 @@ static void spapr_xive_realize(DeviceState *dev, Error **errp) error_propagate(errp, local_err); return; } + sysbus_init_mmio(SYS_BUS_DEVICE(xive), &xsrc->esb_mmio); /* * Initialize the END ESB source @@ -318,6 +312,7 @@ static void spapr_xive_realize(DeviceState *dev, Error **errp) error_propagate(errp, local_err); return; } + sysbus_init_mmio(SYS_BUS_DEVICE(xive), &end_xsrc->esb_mmio); /* Set the mapping address of the END ESB pages after the source ESBs */ xive->end_base = xive->vc_base + (1ull << xsrc->esb_shift) * xsrc->nr_irqs; @@ -333,31 +328,18 @@ static void spapr_xive_realize(DeviceState *dev, Error **errp) qemu_register_reset(spapr_xive_reset, dev); - /* Define all XIVE MMIO regions on SysBus */ - sysbus_init_mmio(SYS_BUS_DEVICE(xive), &xsrc->esb_mmio); - sysbus_init_mmio(SYS_BUS_DEVICE(xive), &end_xsrc->esb_mmio); - sysbus_init_mmio(SYS_BUS_DEVICE(xive), &xive->tm_mmio); -} - -void spapr_xive_init(SpaprXive *xive, Error **errp) -{ - XiveSource *xsrc = &xive->source; - - /* - * The emulated XIVE device can only be initialized once. If the - * ESB memory region has been already mapped, it means we have been - * through there. - */ - if (memory_region_is_mapped(&xsrc->esb_mmio)) { - return; - } - /* TIMA initialization */ memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &xive_tm_ops, xive, "xive.tima", 4ull << TM_SHIFT); + sysbus_init_mmio(SYS_BUS_DEVICE(xive), &xive->tm_mmio); - /* Map all regions */ - spapr_xive_map_mmio(xive); + /* + * Map all regions. These will be enabled or disabled at reset and + * can also be overridden by KVM memory regions if active + */ + sysbus_mmio_map(SYS_BUS_DEVICE(xive), 0, xive->vc_base); + sysbus_mmio_map(SYS_BUS_DEVICE(xive), 1, xive->end_base); + sysbus_mmio_map(SYS_BUS_DEVICE(xive), 2, xive->tm_base); } static int spapr_xive_get_eas(XiveRouter *xrtr, uint8_t eas_blk, diff --git a/hw/intc/spapr_xive_kvm.c b/hw/intc/spapr_xive_kvm.c index b48f135838..3bf8e7a20e 100644 --- a/hw/intc/spapr_xive_kvm.c +++ b/hw/intc/spapr_xive_kvm.c @@ -724,12 +724,13 @@ void kvmppc_xive_connect(SpaprXive *xive, Error **errp) xsrc->esb_mmap = kvmppc_xive_mmap(xive, KVM_XIVE_ESB_PAGE_OFFSET, esb_len, &local_err); if (local_err) { - error_propagate(errp, local_err); - return; + goto fail; } - memory_region_init_ram_device_ptr(&xsrc->esb_mmio, OBJECT(xsrc), + memory_region_init_ram_device_ptr(&xsrc->esb_mmio_kvm, OBJECT(xsrc), "xive.esb", esb_len, xsrc->esb_mmap); + memory_region_add_subregion_overlap(&xsrc->esb_mmio, 0, + &xsrc->esb_mmio_kvm, 1); /* * 2. END ESB pages (No KVM support yet) @@ -741,11 +742,12 @@ void kvmppc_xive_connect(SpaprXive *xive, Error **errp) xive->tm_mmap = kvmppc_xive_mmap(xive, KVM_XIVE_TIMA_PAGE_OFFSET, tima_len, &local_err); if (local_err) { - error_propagate(errp, local_err); - return; + goto fail; } - memory_region_init_ram_device_ptr(&xive->tm_mmio, OBJECT(xive), + memory_region_init_ram_device_ptr(&xive->tm_mmio_kvm, OBJECT(xive), "xive.tima", tima_len, xive->tm_mmap); + memory_region_add_subregion_overlap(&xive->tm_mmio, 0, + &xive->tm_mmio_kvm, 1); xive->change = qemu_add_vm_change_state_handler( kvmppc_xive_change_state_handler, xive); @@ -756,24 +758,24 @@ void kvmppc_xive_connect(SpaprXive *xive, Error **errp) kvmppc_xive_cpu_connect(spapr_cpu_state(cpu)->tctx, &local_err); if (local_err) { - error_propagate(errp, local_err); - return; + goto fail; } } /* Update the KVM sources */ kvmppc_xive_source_reset(xsrc, &local_err); if (local_err) { - error_propagate(errp, local_err); - return; + goto fail; } kvm_kernel_irqchip = true; kvm_msi_via_irqfd_allowed = true; kvm_gsi_direct_mapping = true; + return; - /* Map all regions */ - spapr_xive_map_mmio(xive); +fail: + error_propagate(errp, local_err); + kvmppc_xive_disconnect(xive, NULL); } void kvmppc_xive_disconnect(SpaprXive *xive, Error **errp) @@ -795,21 +797,29 @@ void kvmppc_xive_disconnect(SpaprXive *xive, Error **errp) xsrc = &xive->source; esb_len = (1ull << xsrc->esb_shift) * xsrc->nr_irqs; - sysbus_mmio_unmap(SYS_BUS_DEVICE(xive), 0); - munmap(xsrc->esb_mmap, esb_len); - - sysbus_mmio_unmap(SYS_BUS_DEVICE(xive), 1); + if (xsrc->esb_mmap) { + memory_region_del_subregion(&xsrc->esb_mmio, &xsrc->esb_mmio_kvm); + object_unparent(OBJECT(&xsrc->esb_mmio_kvm)); + munmap(xsrc->esb_mmap, esb_len); + xsrc->esb_mmap = NULL; + } - sysbus_mmio_unmap(SYS_BUS_DEVICE(xive), 2); - munmap(xive->tm_mmap, 4ull << TM_SHIFT); + if (xive->tm_mmap) { + memory_region_del_subregion(&xive->tm_mmio, &xive->tm_mmio_kvm); + object_unparent(OBJECT(&xive->tm_mmio_kvm)); + munmap(xive->tm_mmap, 4ull << TM_SHIFT); + xive->tm_mmap = NULL; + } /* * When the KVM device fd is closed, the KVM device is destroyed * and removed from the list of devices of the VM. The VCPU * presenters are also detached from the device. */ - close(xive->fd); - xive->fd = -1; + if (xive->fd != -1) { + close(xive->fd); + xive->fd = -1; + } kvm_kernel_irqchip = false; kvm_msi_via_irqfd_allowed = false; @@ -819,5 +829,8 @@ void kvmppc_xive_disconnect(SpaprXive *xive, Error **errp) kvm_cpu_disable_all(); /* VM Change state handler is not needed anymore */ - qemu_del_vm_change_state_handler(xive->change); + if (xive->change) { + qemu_del_vm_change_state_handler(xive->change); + xive->change = NULL; + } } diff --git a/hw/intc/xics.c b/hw/intc/xics.c index 29f7d39781..faa976e2f8 100644 --- a/hw/intc/xics.c +++ b/hw/intc/xics.c @@ -267,7 +267,14 @@ static int icp_post_load(void *opaque, int version_id) ICPState *icp = opaque; if (kvm_irqchip_in_kernel()) { - return icp_set_kvm_state(icp); + Error *local_err = NULL; + int ret; + + ret = icp_set_kvm_state(icp, &local_err); + if (ret < 0) { + error_report_err(local_err); + return ret; + } } return 0; @@ -300,7 +307,12 @@ static void icp_reset_handler(void *dev) qemu_set_irq(icp->output, 0); if (kvm_irqchip_in_kernel()) { - icp_set_kvm_state(ICP(dev)); + Error *local_err = NULL; + + icp_set_kvm_state(ICP(dev), &local_err); + if (local_err) { + error_report_err(local_err); + } } } @@ -351,6 +363,7 @@ static void icp_realize(DeviceState *dev, Error **errp) return; } + /* Connect the presenter to the VCPU (required for CPU hotplug) */ if (kvm_irqchip_in_kernel()) { icp_kvm_realize(dev, &err); if (err) { @@ -563,7 +576,12 @@ static void ics_simple_reset(DeviceState *dev) icsc->parent_reset(dev); if (kvm_irqchip_in_kernel()) { - ics_set_kvm_state(ICS_BASE(dev)); + Error *local_err = NULL; + + ics_set_kvm_state(ICS_BASE(dev), &local_err); + if (local_err) { + error_report_err(local_err); + } } } @@ -679,7 +697,14 @@ static int ics_base_post_load(void *opaque, int version_id) ICSState *ics = opaque; if (kvm_irqchip_in_kernel()) { - return ics_set_kvm_state(ics); + Error *local_err = NULL; + int ret; + + ret = ics_set_kvm_state(ics, &local_err); + if (ret < 0) { + error_report_err(local_err); + return ret; + } } return 0; @@ -765,8 +790,13 @@ void ics_set_irq_type(ICSState *ics, int srcno, bool lsi) lsi ? XICS_FLAGS_IRQ_LSI : XICS_FLAGS_IRQ_MSI; if (kvm_irqchip_in_kernel()) { + Error *local_err = NULL; + ics_reset_irq(ics->irqs + srcno); - ics_set_kvm_state_one(ics, srcno); + ics_set_kvm_state_one(ics, srcno, &local_err); + if (local_err) { + error_report_err(local_err); + } } } diff --git a/hw/intc/xics_kvm.c b/hw/intc/xics_kvm.c index 5ba5b77561..51433b19b0 100644 --- a/hw/intc/xics_kvm.c +++ b/hw/intc/xics_kvm.c @@ -106,7 +106,7 @@ void icp_synchronize_state(ICPState *icp) } } -int icp_set_kvm_state(ICPState *icp) +int icp_set_kvm_state(ICPState *icp, Error **errp) { uint64_t state; int ret; @@ -126,10 +126,11 @@ int icp_set_kvm_state(ICPState *icp) | ((uint64_t)icp->pending_priority << KVM_REG_PPC_ICP_PPRI_SHIFT); ret = kvm_set_one_reg(icp->cs, KVM_REG_PPC_ICP_STATE, &state); - if (ret != 0) { - error_report("Unable to restore KVM interrupt controller state (0x%" - PRIx64 ") for CPU %ld: %s", state, kvm_arch_vcpu_id(icp->cs), - strerror(errno)); + if (ret < 0) { + error_setg_errno(errp, -ret, + "Unable to restore KVM interrupt controller state (0x%" + PRIx64 ") for CPU %ld", state, + kvm_arch_vcpu_id(icp->cs)); return ret; } @@ -240,10 +241,9 @@ void ics_synchronize_state(ICSState *ics) ics_get_kvm_state(ics); } -int ics_set_kvm_state_one(ICSState *ics, int srcno) +int ics_set_kvm_state_one(ICSState *ics, int srcno, Error **errp) { uint64_t state; - Error *local_err = NULL; ICSIRQState *irq = &ics->irqs[srcno]; int ret; @@ -278,16 +278,15 @@ int ics_set_kvm_state_one(ICSState *ics, int srcno) } ret = kvm_device_access(kernel_xics_fd, KVM_DEV_XICS_GRP_SOURCES, - srcno + ics->offset, &state, true, &local_err); - if (local_err) { - error_report_err(local_err); + srcno + ics->offset, &state, true, errp); + if (ret < 0) { return ret; } return 0; } -int ics_set_kvm_state(ICSState *ics) +int ics_set_kvm_state(ICSState *ics, Error **errp) { int i; @@ -297,10 +296,12 @@ int ics_set_kvm_state(ICSState *ics) } for (i = 0; i < ics->nr_irqs; i++) { + Error *local_err = NULL; int ret; - ret = ics_set_kvm_state_one(ics, i); - if (ret) { + ret = ics_set_kvm_state_one(ics, i, &local_err); + if (ret < 0) { + error_propagate(errp, local_err); return ret; } } @@ -331,16 +332,7 @@ void ics_kvm_set_irq(ICSState *ics, int srcno, int val) } } -static void rtas_dummy(PowerPCCPU *cpu, SpaprMachineState *spapr, - uint32_t token, - uint32_t nargs, target_ulong args, - uint32_t nret, target_ulong rets) -{ - error_report("pseries: %s must never be called for in-kernel XICS", - __func__); -} - -int xics_kvm_init(SpaprMachineState *spapr, Error **errp) +int xics_kvm_connect(SpaprMachineState *spapr, Error **errp) { int rc; CPUState *cs; @@ -357,42 +349,41 @@ int xics_kvm_init(SpaprMachineState *spapr, Error **errp) if (!kvm_enabled() || !kvm_check_extension(kvm_state, KVM_CAP_IRQ_XICS)) { error_setg(errp, "KVM and IRQ_XICS capability must be present for in-kernel XICS"); - goto fail; + return -1; } - spapr_rtas_register(RTAS_IBM_SET_XIVE, "ibm,set-xive", rtas_dummy); - spapr_rtas_register(RTAS_IBM_GET_XIVE, "ibm,get-xive", rtas_dummy); - spapr_rtas_register(RTAS_IBM_INT_OFF, "ibm,int-off", rtas_dummy); - spapr_rtas_register(RTAS_IBM_INT_ON, "ibm,int-on", rtas_dummy); - rc = kvmppc_define_rtas_kernel_token(RTAS_IBM_SET_XIVE, "ibm,set-xive"); if (rc < 0) { - error_setg(errp, "kvmppc_define_rtas_kernel_token: ibm,set-xive"); + error_setg_errno(&local_err, -rc, + "kvmppc_define_rtas_kernel_token: ibm,set-xive"); goto fail; } rc = kvmppc_define_rtas_kernel_token(RTAS_IBM_GET_XIVE, "ibm,get-xive"); if (rc < 0) { - error_setg(errp, "kvmppc_define_rtas_kernel_token: ibm,get-xive"); + error_setg_errno(&local_err, -rc, + "kvmppc_define_rtas_kernel_token: ibm,get-xive"); goto fail; } rc = kvmppc_define_rtas_kernel_token(RTAS_IBM_INT_ON, "ibm,int-on"); if (rc < 0) { - error_setg(errp, "kvmppc_define_rtas_kernel_token: ibm,int-on"); + error_setg_errno(&local_err, -rc, + "kvmppc_define_rtas_kernel_token: ibm,int-on"); goto fail; } rc = kvmppc_define_rtas_kernel_token(RTAS_IBM_INT_OFF, "ibm,int-off"); if (rc < 0) { - error_setg(errp, "kvmppc_define_rtas_kernel_token: ibm,int-off"); + error_setg_errno(&local_err, -rc, + "kvmppc_define_rtas_kernel_token: ibm,int-off"); goto fail; } /* Create the KVM XICS device */ rc = kvm_create_device(kvm_state, KVM_DEV_TYPE_XICS, false); if (rc < 0) { - error_setg_errno(errp, -rc, "Error on KVM_CREATE_DEVICE for XICS"); + error_setg_errno(&local_err, -rc, "Error on KVM_CREATE_DEVICE for XICS"); goto fail; } @@ -407,27 +398,30 @@ int xics_kvm_init(SpaprMachineState *spapr, Error **errp) icp_kvm_realize(DEVICE(spapr_cpu_state(cpu)->icp), &local_err); if (local_err) { - error_propagate(errp, local_err); goto fail; } } /* Update the KVM sources */ - ics_set_kvm_state(spapr->ics); + ics_set_kvm_state(spapr->ics, &local_err); + if (local_err) { + goto fail; + } /* Connect the presenters to the initial VCPUs of the machine */ CPU_FOREACH(cs) { PowerPCCPU *cpu = POWERPC_CPU(cs); - icp_set_kvm_state(spapr_cpu_state(cpu)->icp); + icp_set_kvm_state(spapr_cpu_state(cpu)->icp, &local_err); + if (local_err) { + goto fail; + } } return 0; fail: - kvmppc_define_rtas_kernel_token(0, "ibm,set-xive"); - kvmppc_define_rtas_kernel_token(0, "ibm,get-xive"); - kvmppc_define_rtas_kernel_token(0, "ibm,int-on"); - kvmppc_define_rtas_kernel_token(0, "ibm,int-off"); + error_propagate(errp, local_err); + xics_kvm_disconnect(spapr, NULL); return -1; } @@ -451,13 +445,10 @@ void xics_kvm_disconnect(SpaprMachineState *spapr, Error **errp) * removed from the list of devices of the VM. The VCPU presenters * are also detached from the device. */ - close(kernel_xics_fd); - kernel_xics_fd = -1; - - spapr_rtas_unregister(RTAS_IBM_SET_XIVE); - spapr_rtas_unregister(RTAS_IBM_GET_XIVE); - spapr_rtas_unregister(RTAS_IBM_INT_OFF); - spapr_rtas_unregister(RTAS_IBM_INT_ON); + if (kernel_xics_fd != -1) { + close(kernel_xics_fd); + kernel_xics_fd = -1; + } kvmppc_define_rtas_kernel_token(0, "ibm,set-xive"); kvmppc_define_rtas_kernel_token(0, "ibm,get-xive"); @@ -471,3 +462,33 @@ void xics_kvm_disconnect(SpaprMachineState *spapr, Error **errp) /* Clear the presenter from the VCPUs */ kvm_disable_icps(); } + +/* + * This is a heuristic to detect older KVMs on POWER9 hosts that don't + * support destruction of a KVM XICS device while the VM is running. + * Required to start a spapr machine with ic-mode=dual,kernel-irqchip=on. + */ +bool xics_kvm_has_broken_disconnect(SpaprMachineState *spapr) +{ + int rc; + + rc = kvm_create_device(kvm_state, KVM_DEV_TYPE_XICS, false); + if (rc < 0) { + /* + * The error is ignored on purpose. The KVM XICS setup code + * will catch it again anyway. The goal here is to see if + * close() actually destroys the device or not. + */ + return false; + } + + close(rc); + + rc = kvm_create_device(kvm_state, KVM_DEV_TYPE_XICS, false); + if (rc >= 0) { + close(rc); + return false; + } + + return errno == EEXIST; +} diff --git a/hw/intc/xics_spapr.c b/hw/intc/xics_spapr.c index 5a1835e8b1..7cd3c93d71 100644 --- a/hw/intc/xics_spapr.c +++ b/hw/intc/xics_spapr.c @@ -41,11 +41,32 @@ * Guest interfaces */ +static bool check_emulated_xics(SpaprMachineState *spapr, const char *func) +{ + if (spapr_ovec_test(spapr->ov5_cas, OV5_XIVE_EXPLOIT) || + kvm_irqchip_in_kernel()) { + error_report("pseries: %s must only be called for emulated XICS", + func); + return false; + } + + return true; +} + +#define CHECK_EMULATED_XICS_HCALL(spapr) \ + do { \ + if (!check_emulated_xics((spapr), __func__)) { \ + return H_HARDWARE; \ + } \ + } while (0) + static target_ulong h_cppr(PowerPCCPU *cpu, SpaprMachineState *spapr, target_ulong opcode, target_ulong *args) { target_ulong cppr = args[0]; + CHECK_EMULATED_XICS_HCALL(spapr); + icp_set_cppr(spapr_cpu_state(cpu)->icp, cppr); return H_SUCCESS; } @@ -56,6 +77,8 @@ static target_ulong h_ipi(PowerPCCPU *cpu, SpaprMachineState *spapr, target_ulong mfrr = args[1]; ICPState *icp = xics_icp_get(XICS_FABRIC(spapr), args[0]); + CHECK_EMULATED_XICS_HCALL(spapr); + if (!icp) { return H_PARAMETER; } @@ -69,6 +92,8 @@ static target_ulong h_xirr(PowerPCCPU *cpu, SpaprMachineState *spapr, { uint32_t xirr = icp_accept(spapr_cpu_state(cpu)->icp); + CHECK_EMULATED_XICS_HCALL(spapr); + args[0] = xirr; return H_SUCCESS; } @@ -78,6 +103,8 @@ static target_ulong h_xirr_x(PowerPCCPU *cpu, SpaprMachineState *spapr, { uint32_t xirr = icp_accept(spapr_cpu_state(cpu)->icp); + CHECK_EMULATED_XICS_HCALL(spapr); + args[0] = xirr; args[1] = cpu_get_host_ticks(); return H_SUCCESS; @@ -88,6 +115,8 @@ static target_ulong h_eoi(PowerPCCPU *cpu, SpaprMachineState *spapr, { target_ulong xirr = args[0]; + CHECK_EMULATED_XICS_HCALL(spapr); + icp_eoi(spapr_cpu_state(cpu)->icp, xirr); return H_SUCCESS; } @@ -99,6 +128,8 @@ static target_ulong h_ipoll(PowerPCCPU *cpu, SpaprMachineState *spapr, uint32_t mfrr; uint32_t xirr; + CHECK_EMULATED_XICS_HCALL(spapr); + if (!icp) { return H_PARAMETER; } @@ -111,6 +142,14 @@ static target_ulong h_ipoll(PowerPCCPU *cpu, SpaprMachineState *spapr, return H_SUCCESS; } +#define CHECK_EMULATED_XICS_RTAS(spapr, rets) \ + do { \ + if (!check_emulated_xics((spapr), __func__)) { \ + rtas_st((rets), 0, RTAS_OUT_HW_ERROR); \ + return; \ + } \ + } while (0) + static void rtas_set_xive(PowerPCCPU *cpu, SpaprMachineState *spapr, uint32_t token, uint32_t nargs, target_ulong args, @@ -119,6 +158,8 @@ static void rtas_set_xive(PowerPCCPU *cpu, SpaprMachineState *spapr, ICSState *ics = spapr->ics; uint32_t nr, srcno, server, priority; + CHECK_EMULATED_XICS_RTAS(spapr, rets); + if ((nargs != 3) || (nret != 1)) { rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR); return; @@ -152,6 +193,8 @@ static void rtas_get_xive(PowerPCCPU *cpu, SpaprMachineState *spapr, ICSState *ics = spapr->ics; uint32_t nr, srcno; + CHECK_EMULATED_XICS_RTAS(spapr, rets); + if ((nargs != 1) || (nret != 3)) { rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR); return; @@ -182,6 +225,8 @@ static void rtas_int_off(PowerPCCPU *cpu, SpaprMachineState *spapr, ICSState *ics = spapr->ics; uint32_t nr, srcno; + CHECK_EMULATED_XICS_RTAS(spapr, rets); + if ((nargs != 1) || (nret != 1)) { rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR); return; @@ -213,6 +258,8 @@ static void rtas_int_on(PowerPCCPU *cpu, SpaprMachineState *spapr, ICSState *ics = spapr->ics; uint32_t nr, srcno; + CHECK_EMULATED_XICS_RTAS(spapr, rets); + if ((nargs != 1) || (nret != 1)) { rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR); return; @@ -239,14 +286,6 @@ static void rtas_int_on(PowerPCCPU *cpu, SpaprMachineState *spapr, void xics_spapr_init(SpaprMachineState *spapr) { - /* Emulated mode can only be initialized once. */ - if (spapr->ics->init) { - return; - } - - spapr->ics->init = true; - - /* Registration of global state belongs into realize */ spapr_rtas_register(RTAS_IBM_SET_XIVE, "ibm,set-xive", rtas_set_xive); spapr_rtas_register(RTAS_IBM_GET_XIVE, "ibm,get-xive", rtas_get_xive); spapr_rtas_register(RTAS_IBM_INT_OFF, "ibm,int-off", rtas_int_off); diff --git a/hw/intc/xive.c b/hw/intc/xive.c index 6250c0414d..cf77bdb7d3 100644 --- a/hw/intc/xive.c +++ b/hw/intc/xive.c @@ -132,6 +132,11 @@ static void xive_tctx_set_cppr(XiveTCTX *tctx, uint8_t ring, uint8_t cppr) xive_tctx_notify(tctx, ring); } +static inline uint32_t xive_tctx_word2(uint8_t *ring) +{ + return *((uint32_t *) &ring[TM_WORD2]); +} + /* * XIVE Thread Interrupt Management Area (TIMA) */ @@ -150,11 +155,12 @@ static uint64_t xive_tm_ack_hv_reg(XiveTCTX *tctx, hwaddr offset, unsigned size) static uint64_t xive_tm_pull_pool_ctx(XiveTCTX *tctx, hwaddr offset, unsigned size) { - uint64_t ret; + uint32_t qw2w2_prev = xive_tctx_word2(&tctx->regs[TM_QW2_HV_POOL]); + uint32_t qw2w2; - ret = tctx->regs[TM_QW2_HV_POOL + TM_WORD2] & TM_QW2W2_POOL_CAM; - tctx->regs[TM_QW2_HV_POOL + TM_WORD2] &= ~TM_QW2W2_POOL_CAM; - return ret; + qw2w2 = xive_set_field32(TM_QW2W2_VP, qw2w2_prev, 0); + memcpy(&tctx->regs[TM_QW2_HV_POOL + TM_WORD2], &qw2w2, 4); + return qw2w2; } static void xive_tm_vt_push(XiveTCTX *tctx, hwaddr offset, @@ -182,31 +188,31 @@ static uint64_t xive_tm_vt_poll(XiveTCTX *tctx, hwaddr offset, unsigned size) */ static const uint8_t xive_tm_hw_view[] = { - /* QW-0 User */ 3, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0, 0, 0, 0, - /* QW-1 OS */ 3, 3, 3, 3, 3, 3, 0, 3, 3, 3, 3, 3, 0, 0, 0, 0, - /* QW-2 POOL */ 0, 0, 3, 3, 0, 0, 0, 0, 3, 3, 3, 3, 0, 0, 0, 0, - /* QW-3 PHYS */ 3, 3, 3, 3, 0, 3, 0, 3, 3, 0, 0, 3, 3, 3, 3, 0, + 3, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0, 0, 0, 0, /* QW-0 User */ + 3, 3, 3, 3, 3, 3, 0, 2, 3, 3, 3, 3, 0, 0, 0, 0, /* QW-1 OS */ + 0, 0, 3, 3, 0, 0, 0, 0, 3, 3, 3, 3, 0, 0, 0, 0, /* QW-2 POOL */ + 3, 3, 3, 3, 0, 3, 0, 2, 3, 0, 0, 3, 3, 3, 3, 0, /* QW-3 PHYS */ }; static const uint8_t xive_tm_hv_view[] = { - /* QW-0 User */ 3, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0, 0, 0, 0, - /* QW-1 OS */ 3, 3, 3, 3, 3, 3, 0, 3, 3, 3, 3, 3, 0, 0, 0, 0, - /* QW-2 POOL */ 0, 0, 3, 3, 0, 0, 0, 0, 0, 3, 3, 3, 0, 0, 0, 0, - /* QW-3 PHYS */ 3, 3, 3, 3, 0, 3, 0, 3, 3, 0, 0, 3, 0, 0, 0, 0, + 3, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0, 0, 0, 0, /* QW-0 User */ + 3, 3, 3, 3, 3, 3, 0, 2, 3, 3, 3, 3, 0, 0, 0, 0, /* QW-1 OS */ + 0, 0, 3, 3, 0, 0, 0, 0, 0, 3, 3, 3, 0, 0, 0, 0, /* QW-2 POOL */ + 3, 3, 3, 3, 0, 3, 0, 2, 3, 0, 0, 3, 0, 0, 0, 0, /* QW-3 PHYS */ }; static const uint8_t xive_tm_os_view[] = { - /* QW-0 User */ 3, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0, 0, 0, 0, - /* QW-1 OS */ 2, 3, 2, 2, 2, 2, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, - /* QW-2 POOL */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* QW-3 PHYS */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 3, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0, 0, 0, 0, /* QW-0 User */ + 2, 3, 2, 2, 2, 2, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, /* QW-1 OS */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* QW-2 POOL */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* QW-3 PHYS */ }; static const uint8_t xive_tm_user_view[] = { - /* QW-0 User */ 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* QW-1 OS */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* QW-2 POOL */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* QW-3 PHYS */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* QW-0 User */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* QW-1 OS */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* QW-2 POOL */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* QW-3 PHYS */ }; /* @@ -484,11 +490,6 @@ const MemoryRegionOps xive_tm_ops = { }, }; -static inline uint32_t xive_tctx_word2(uint8_t *ring) -{ - return *((uint32_t *) &ring[TM_WORD2]); -} - static char *xive_tctx_ring_print(uint8_t *ring) { uint32_t w2 = xive_tctx_word2(ring); @@ -1229,27 +1230,16 @@ XiveTCTX *xive_router_get_tctx(XiveRouter *xrtr, CPUState *cs) } /* - * By default on P9, the HW CAM line (23bits) is hardwired to : - * - * 0x000||0b1||4Bit chip number||7Bit Thread number. + * Encode the HW CAM line in the block group mode format : * - * When the block grouping is enabled, the CAM line is changed to : - * - * 4Bit chip number||0x001||7Bit Thread number. + * chip << 19 | 0000000 0 0001 thread (7Bit) */ -static uint32_t hw_cam_line(uint8_t chip_id, uint8_t tid) -{ - return 1 << 11 | (chip_id & 0xf) << 7 | (tid & 0x7f); -} - -static bool xive_presenter_tctx_match_hw(XiveTCTX *tctx, - uint8_t nvt_blk, uint32_t nvt_idx) +static uint32_t xive_tctx_hw_cam_line(XiveTCTX *tctx) { CPUPPCState *env = &POWERPC_CPU(tctx->cs)->env; uint32_t pir = env->spr_cb[SPR_PIR].default_value; - return hw_cam_line((pir >> 8) & 0xf, pir & 0x7f) == - hw_cam_line(nvt_blk, nvt_idx); + return xive_nvt_cam_line((pir >> 8) & 0xf, 1 << 7 | (pir & 0x7f)); } /* @@ -1285,7 +1275,7 @@ static int xive_presenter_tctx_match(XiveTCTX *tctx, uint8_t format, /* PHYS ring */ if ((be32_to_cpu(qw3w2) & TM_QW3W2_VT) && - xive_presenter_tctx_match_hw(tctx, nvt_blk, nvt_idx)) { + cam == xive_tctx_hw_cam_line(tctx)) { return TM_QW3_HV_PHYS; } diff --git a/hw/ppc/mac_newworld.c b/hw/ppc/mac_newworld.c index 4d835f32b5..c8d3245524 100644 --- a/hw/ppc/mac_newworld.c +++ b/hw/ppc/mac_newworld.c @@ -437,13 +437,11 @@ static void ppc_core99_init(MachineState *machine) } /* The NewWorld NVRAM is not located in the MacIO device */ -#ifdef CONFIG_KVM if (kvm_enabled() && getpagesize() > 4096) { /* We can't combine read-write and read-only in a single page, so move the NVRAM out of ROM again for KVM */ nvram_addr = 0xFFE00000; } -#endif dev = qdev_create(NULL, TYPE_MACIO_NVRAM); qdev_prop_set_uint32(dev, "size", 0x2000); qdev_prop_set_uint32(dev, "it_shift", 1); @@ -488,14 +486,12 @@ static void ppc_core99_init(MachineState *machine) fw_cfg_add_i32(fw_cfg, FW_CFG_PPC_IS_KVM, kvm_enabled()); if (kvm_enabled()) { -#ifdef CONFIG_KVM uint8_t *hypercall; hypercall = g_malloc(16); kvmppc_get_hypercall(env, hypercall, 16); fw_cfg_add_bytes(fw_cfg, FW_CFG_PPC_KVM_HC, hypercall, 16); fw_cfg_add_i32(fw_cfg, FW_CFG_PPC_KVM_PID, getpid()); -#endif } fw_cfg_add_i32(fw_cfg, FW_CFG_PPC_TBFREQ, tbfreq); /* Mac OS X requires a "known good" clock-frequency value; pass it one. */ diff --git a/hw/ppc/mac_oldworld.c b/hw/ppc/mac_oldworld.c index eddd005a7c..da751addc4 100644 --- a/hw/ppc/mac_oldworld.c +++ b/hw/ppc/mac_oldworld.c @@ -345,14 +345,12 @@ static void ppc_heathrow_init(MachineState *machine) fw_cfg_add_i32(fw_cfg, FW_CFG_PPC_IS_KVM, kvm_enabled()); if (kvm_enabled()) { -#ifdef CONFIG_KVM uint8_t *hypercall; hypercall = g_malloc(16); kvmppc_get_hypercall(env, hypercall, 16); fw_cfg_add_bytes(fw_cfg, FW_CFG_PPC_KVM_HC, hypercall, 16); fw_cfg_add_i32(fw_cfg, FW_CFG_PPC_KVM_PID, getpid()); -#endif } fw_cfg_add_i32(fw_cfg, FW_CFG_PPC_TBFREQ, tbfreq); /* Mac OS X requires a "known good" clock-frequency value; pass it one. */ diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c index 9db43916ac..b87e01e5b9 100644 --- a/hw/ppc/pnv.c +++ b/hw/ppc/pnv.c @@ -860,6 +860,14 @@ static void pnv_chip_power8_realize(DeviceState *dev, Error **errp) Pnv8Psi *psi8 = &chip8->psi; Error *local_err = NULL; + /* XSCOM bridge is first */ + pnv_xscom_realize(chip, PNV_XSCOM_SIZE, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + sysbus_mmio_map(SYS_BUS_DEVICE(chip), 0, PNV_XSCOM_BASE(chip)); + pcc->parent_realize(dev, &local_err); if (local_err) { error_propagate(errp, local_err); @@ -916,7 +924,6 @@ static void pnv_chip_power8e_class_init(ObjectClass *klass, void *data) k->isa_create = pnv_chip_power8_isa_create; k->dt_populate = pnv_chip_power8_dt_populate; k->pic_print_info = pnv_chip_power8_pic_print_info; - k->xscom_base = 0x003fc0000000000ull; dc->desc = "PowerNV Chip POWER8E"; device_class_set_parent_realize(dc, pnv_chip_power8_realize, @@ -936,7 +943,6 @@ static void pnv_chip_power8_class_init(ObjectClass *klass, void *data) k->isa_create = pnv_chip_power8_isa_create; k->dt_populate = pnv_chip_power8_dt_populate; k->pic_print_info = pnv_chip_power8_pic_print_info; - k->xscom_base = 0x003fc0000000000ull; dc->desc = "PowerNV Chip POWER8"; device_class_set_parent_realize(dc, pnv_chip_power8_realize, @@ -956,7 +962,6 @@ static void pnv_chip_power8nvl_class_init(ObjectClass *klass, void *data) k->isa_create = pnv_chip_power8nvl_isa_create; k->dt_populate = pnv_chip_power8_dt_populate; k->pic_print_info = pnv_chip_power8_pic_print_info; - k->xscom_base = 0x003fc0000000000ull; dc->desc = "PowerNV Chip POWER8NVL"; device_class_set_parent_realize(dc, pnv_chip_power8_realize, @@ -1024,6 +1029,14 @@ static void pnv_chip_power9_realize(DeviceState *dev, Error **errp) Pnv9Psi *psi9 = &chip9->psi; Error *local_err = NULL; + /* XSCOM bridge is first */ + pnv_xscom_realize(chip, PNV9_XSCOM_SIZE, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + sysbus_mmio_map(SYS_BUS_DEVICE(chip), 0, PNV9_XSCOM_BASE(chip)); + pcc->parent_realize(dev, &local_err); if (local_err) { error_propagate(errp, local_err); @@ -1099,7 +1112,6 @@ static void pnv_chip_power9_class_init(ObjectClass *klass, void *data) k->isa_create = pnv_chip_power9_isa_create; k->dt_populate = pnv_chip_power9_dt_populate; k->pic_print_info = pnv_chip_power9_pic_print_info; - k->xscom_base = 0x00603fc00000000ull; dc->desc = "PowerNV Chip POWER9"; device_class_set_parent_realize(dc, pnv_chip_power9_realize, @@ -1136,11 +1148,6 @@ static void pnv_chip_core_sanitize(PnvChip *chip, Error **errp) } } -static void pnv_chip_instance_init(Object *obj) -{ - PNV_CHIP(obj)->xscom_base = PNV_CHIP_GET_CLASS(obj)->xscom_base; -} - static void pnv_chip_core_realize(PnvChip *chip, Error **errp) { Error *error = NULL; @@ -1206,14 +1213,6 @@ static void pnv_chip_realize(DeviceState *dev, Error **errp) PnvChip *chip = PNV_CHIP(dev); Error *error = NULL; - /* XSCOM bridge */ - pnv_xscom_realize(chip, &error); - if (error) { - error_propagate(errp, error); - return; - } - sysbus_mmio_map(SYS_BUS_DEVICE(chip), 0, PNV_XSCOM_BASE(chip)); - /* Cores */ pnv_chip_core_realize(chip, &error); if (error) { @@ -1398,7 +1397,6 @@ static const TypeInfo types[] = { .name = TYPE_PNV_CHIP, .parent = TYPE_SYS_BUS_DEVICE, .class_init = pnv_chip_class_init, - .instance_init = pnv_chip_instance_init, .instance_size = sizeof(PnvChip), .class_size = sizeof(PnvChipClass), .abstract = true, diff --git a/hw/ppc/pnv_xscom.c b/hw/ppc/pnv_xscom.c index 4e52885c9e..2b81c75f56 100644 --- a/hw/ppc/pnv_xscom.c +++ b/hw/ppc/pnv_xscom.c @@ -213,17 +213,17 @@ const MemoryRegionOps pnv_xscom_ops = { .endianness = DEVICE_BIG_ENDIAN, }; -void pnv_xscom_realize(PnvChip *chip, Error **errp) +void pnv_xscom_realize(PnvChip *chip, uint64_t size, Error **errp) { SysBusDevice *sbd = SYS_BUS_DEVICE(chip); char *name; name = g_strdup_printf("xscom-%x", chip->chip_id); memory_region_init_io(&chip->xscom_mmio, OBJECT(chip), &pnv_xscom_ops, - chip, name, PNV_XSCOM_SIZE); + chip, name, size); sysbus_init_mmio(sbd, &chip->xscom_mmio); - memory_region_init(&chip->xscom, OBJECT(chip), name, PNV_XSCOM_SIZE); + memory_region_init(&chip->xscom, OBJECT(chip), name, size); address_space_init(&chip->xscom_as, &chip->xscom, name); g_free(name); } @@ -265,12 +265,19 @@ static const char compat_p9[] = "ibm,power9-xscom\0ibm,xscom"; int pnv_dt_xscom(PnvChip *chip, void *fdt, int root_offset) { - uint64_t reg[] = { cpu_to_be64(PNV_XSCOM_BASE(chip)), - cpu_to_be64(PNV_XSCOM_SIZE) }; + uint64_t reg[2]; int xscom_offset; ForeachPopulateArgs args; char *name; + if (pnv_chip_is_power9(chip)) { + reg[0] = cpu_to_be64(PNV9_XSCOM_BASE(chip)); + reg[1] = cpu_to_be64(PNV9_XSCOM_SIZE); + } else { + reg[0] = cpu_to_be64(PNV_XSCOM_BASE(chip)); + reg[1] = cpu_to_be64(PNV_XSCOM_SIZE); + } + name = g_strdup_printf("xscom@%" PRIx64, be64_to_cpu(reg[0])); xscom_offset = fdt_add_subnode(fdt, root_offset, name); _FDT(xscom_offset); diff --git a/hw/ppc/ppc.c b/hw/ppc/ppc.c index 9d91e8481b..a9e508c496 100644 --- a/hw/ppc/ppc.c +++ b/hw/ppc/ppc.c @@ -80,9 +80,7 @@ void ppc_set_irq(PowerPCCPU *cpu, int n_IRQ, int level) } if (old_pending != env->pending_interrupts) { -#ifdef CONFIG_KVM kvmppc_set_interrupt(cpu, n_IRQ, level); -#endif } @@ -1036,10 +1034,7 @@ static void timebase_load(PPCTimebase *tb) CPU_FOREACH(cpu) { PowerPCCPU *pcpu = POWERPC_CPU(cpu); pcpu->env.tb_env->tb_offset = tb_off_adj; -#if defined(CONFIG_KVM) - kvm_set_one_reg(cpu, KVM_REG_PPC_TB_OFFSET, - &pcpu->env.tb_env->tb_offset); -#endif + kvmppc_set_reg_tb_offset(pcpu, pcpu->env.tb_env->tb_offset); } } diff --git a/hw/ppc/prep.c b/hw/ppc/prep.c index 2a8009e20b..a248ce480d 100644 --- a/hw/ppc/prep.c +++ b/hw/ppc/prep.c @@ -780,7 +780,6 @@ static void ibm_40p_init(MachineState *machine) fw_cfg_add_i32(fw_cfg, FW_CFG_PPC_IS_KVM, kvm_enabled()); if (kvm_enabled()) { -#ifdef CONFIG_KVM uint8_t *hypercall; fw_cfg_add_i32(fw_cfg, FW_CFG_PPC_TBFREQ, kvmppc_get_tbfreq()); @@ -788,7 +787,6 @@ static void ibm_40p_init(MachineState *machine) kvmppc_get_hypercall(env, hypercall, 16); fw_cfg_add_bytes(fw_cfg, FW_CFG_PPC_KVM_HC, hypercall, 16); fw_cfg_add_i32(fw_cfg, FW_CFG_PPC_KVM_PID, getpid()); -#endif } else { fw_cfg_add_i32(fw_cfg, FW_CFG_PPC_TBFREQ, NANOSECONDS_PER_SECOND); } diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index 3156daf093..ff3df0bbd8 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -62,7 +62,7 @@ void spapr_irq_msi_reset(SpaprMachineState *spapr) bitmap_clear(spapr->irq_map, 0, spapr->irq_map_nr); } -static void spapr_irq_init_device(SpaprMachineState *spapr, +static void spapr_irq_init_kvm(SpaprMachineState *spapr, SpaprIrq *irq, Error **errp) { MachineState *machine = MACHINE(spapr); @@ -88,8 +88,6 @@ static void spapr_irq_init_device(SpaprMachineState *spapr, error_prepend(&local_err, "kernel_irqchip allowed but unavailable: "); warn_report_err(local_err); } - - irq->init_emu(spapr, errp); } /* @@ -114,6 +112,8 @@ static void spapr_irq_init_xics(SpaprMachineState *spapr, int nr_irqs, } spapr->ics = ICS_BASE(obj); + + xics_spapr_init(spapr); } #define ICS_IRQ_FREE(ics, srcno) \ @@ -222,7 +222,7 @@ static void spapr_irq_reset_xics(SpaprMachineState *spapr, Error **errp) { Error *local_err = NULL; - spapr_irq_init_device(spapr, &spapr_irq_xics, &local_err); + spapr_irq_init_kvm(spapr, &spapr_irq_xics, &local_err); if (local_err) { error_propagate(errp, local_err); return; @@ -234,15 +234,10 @@ static const char *spapr_irq_get_nodename_xics(SpaprMachineState *spapr) return XICS_NODENAME; } -static void spapr_irq_init_emu_xics(SpaprMachineState *spapr, Error **errp) -{ - xics_spapr_init(spapr); -} - static void spapr_irq_init_kvm_xics(SpaprMachineState *spapr, Error **errp) { if (kvm_enabled()) { - xics_kvm_init(spapr, errp); + xics_kvm_connect(spapr, errp); } } @@ -266,7 +261,6 @@ SpaprIrq spapr_irq_xics = { .reset = spapr_irq_reset_xics, .set_irq = spapr_irq_set_irq_xics, .get_nodename = spapr_irq_get_nodename_xics, - .init_emu = spapr_irq_init_emu_xics, .init_kvm = spapr_irq_init_kvm_xics, }; @@ -384,7 +378,7 @@ static void spapr_irq_reset_xive(SpaprMachineState *spapr, Error **errp) spapr_xive_set_tctx_os_cam(spapr_cpu_state(cpu)->tctx); } - spapr_irq_init_device(spapr, &spapr_irq_xive, &local_err); + spapr_irq_init_kvm(spapr, &spapr_irq_xive, &local_err); if (local_err) { error_propagate(errp, local_err); return; @@ -410,11 +404,6 @@ static const char *spapr_irq_get_nodename_xive(SpaprMachineState *spapr) return spapr->xive->nodename; } -static void spapr_irq_init_emu_xive(SpaprMachineState *spapr, Error **errp) -{ - spapr_xive_init(spapr->xive, errp); -} - static void spapr_irq_init_kvm_xive(SpaprMachineState *spapr, Error **errp) { if (kvm_enabled()) { @@ -446,7 +435,6 @@ SpaprIrq spapr_irq_xive = { .reset = spapr_irq_reset_xive, .set_irq = spapr_irq_set_irq_xive, .get_nodename = spapr_irq_get_nodename_xive, - .init_emu = spapr_irq_init_emu_xive, .init_kvm = spapr_irq_init_kvm_xive, }; @@ -624,7 +612,6 @@ SpaprIrq spapr_irq_dual = { .reset = spapr_irq_reset_dual, .set_irq = spapr_irq_set_irq_dual, .get_nodename = spapr_irq_get_nodename_dual, - .init_emu = NULL, /* should not be used */ .init_kvm = NULL, /* should not be used */ }; @@ -668,6 +655,19 @@ static void spapr_irq_check(SpaprMachineState *spapr, Error **errp) return; } } + + /* + * On a POWER9 host, some older KVM XICS devices cannot be destroyed and + * re-created. Detect that early to avoid QEMU to exit later when the + * guest reboots. + */ + if (kvm_enabled() && + spapr->irq == &spapr_irq_dual && + machine_kernel_irqchip_required(machine) && + xics_kvm_has_broken_disconnect(spapr)) { + error_setg(errp, "KVM is too old to support ic-mode=dual,kernel-irqchip=on"); + return; + } } /* @@ -827,6 +827,5 @@ SpaprIrq spapr_irq_xics_legacy = { .reset = spapr_irq_reset_xics, .set_irq = spapr_irq_set_irq_xics, .get_nodename = spapr_irq_get_nodename_xics, - .init_emu = spapr_irq_init_emu_xics, .init_kvm = spapr_irq_init_kvm_xics, }; diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c index 957ae88bbd..9003fe9010 100644 --- a/hw/ppc/spapr_pci.c +++ b/hw/ppc/spapr_pci.c @@ -1343,6 +1343,7 @@ static void spapr_dt_pci_device_cb(PCIBus *bus, PCIDevice *pdev, static int spapr_dt_pci_bus(SpaprPhbState *sphb, PCIBus *bus, void *fdt, int offset) { + Object *owner; PciWalkFdt cbinfo = { .fdt = fdt, .offset = offset, @@ -1356,15 +1357,20 @@ static int spapr_dt_pci_bus(SpaprPhbState *sphb, PCIBus *bus, _FDT(fdt_setprop_cell(fdt, offset, "#size-cells", RESOURCE_CELLS_SIZE)); - if (bus) { - pci_for_each_device_reverse(bus, pci_bus_num(bus), - spapr_dt_pci_device_cb, &cbinfo); - if (cbinfo.err) { - return cbinfo.err; - } + assert(bus); + pci_for_each_device_reverse(bus, pci_bus_num(bus), + spapr_dt_pci_device_cb, &cbinfo); + if (cbinfo.err) { + return cbinfo.err; } - ret = spapr_dt_drc(fdt, offset, OBJECT(bus->parent_dev), + if (pci_bus_is_root(bus)) { + owner = OBJECT(sphb); + } else { + owner = OBJECT(pci_bridge_get_device(bus)); + } + + ret = spapr_dt_drc(fdt, offset, owner, SPAPR_DR_CONNECTOR_TYPE_PCI); if (ret) { return ret; @@ -1782,6 +1788,12 @@ static void spapr_phb_unrealize(DeviceState *dev, Error **errp) memory_region_del_subregion(&sphb->iommu_root, &sphb->msiwindow); + /* + * An attached PCI device may have memory listeners, eg. VFIO PCI. We have + * unmapped all sections. Remove the listeners now, before destroying the + * address space. + */ + address_space_remove_listeners(&sphb->iommu_as); address_space_destroy(&sphb->iommu_as); qbus_set_hotplug_handler(BUS(phb->bus), NULL, &error_abort); @@ -1945,11 +1957,9 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp) * For KVM we want to ensure that this memory is a full page so that * our memory slot is of page size granularity. */ -#ifdef CONFIG_KVM if (kvm_enabled()) { msi_window_size = getpagesize(); } -#endif memory_region_init_io(&sphb->msiwindow, OBJECT(sphb), &spapr_msi_ops, spapr, "msi", msi_window_size); diff --git a/include/exec/memory.h b/include/exec/memory.h index e6140e8a04..2c5cdffa31 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -1758,6 +1758,16 @@ void address_space_init(AddressSpace *as, MemoryRegion *root, const char *name); void address_space_destroy(AddressSpace *as); /** + * address_space_remove_listeners: unregister all listeners of an address space + * + * Removes all callbacks previously registered with memory_listener_register() + * for @as. + * + * @as: an initialized #AddressSpace + */ +void address_space_remove_listeners(AddressSpace *as); + +/** * address_space_rw: read from or write to an address space. * * Return a MemTxResult indicating whether the operation succeeded diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h index fc4678f757..fb123edc4e 100644 --- a/include/hw/ppc/pnv.h +++ b/include/hw/ppc/pnv.h @@ -56,7 +56,6 @@ typedef struct PnvChip { uint64_t cores_mask; void *cores; - hwaddr xscom_base; MemoryRegion xscom_mmio; MemoryRegion xscom; AddressSpace xscom_as; @@ -105,8 +104,6 @@ typedef struct PnvChipClass { uint64_t chip_cfam_id; uint64_t cores_mask; - hwaddr xscom_base; - DeviceRealize parent_realize; uint32_t (*core_pir)(PnvChip *chip, uint32_t core_id); @@ -199,7 +196,7 @@ void pnv_bmc_powerdown(IPMIBmc *bmc); */ #define PNV_XSCOM_SIZE 0x800000000ull #define PNV_XSCOM_BASE(chip) \ - (chip->xscom_base + ((uint64_t)(chip)->chip_id) * PNV_XSCOM_SIZE) + (0x0003fc0000000000ull + ((uint64_t)(chip)->chip_id) * PNV_XSCOM_SIZE) /* * XSCOM 0x20109CA defines the ICP BAR: @@ -256,4 +253,7 @@ void pnv_bmc_powerdown(IPMIBmc *bmc); #define PNV9_PSIHB_ESB_SIZE 0x0000000000010000ull #define PNV9_PSIHB_ESB_BASE(chip) PNV9_CHIP_BASE(chip, 0x00060302031c0000ull) +#define PNV9_XSCOM_SIZE 0x0000000400000000ull +#define PNV9_XSCOM_BASE(chip) PNV9_CHIP_BASE(chip, 0x00603fc00000000ull) + #endif /* PPC_PNV_H */ diff --git a/include/hw/ppc/pnv_xscom.h b/include/hw/ppc/pnv_xscom.h index c842d950d2..67641ed278 100644 --- a/include/hw/ppc/pnv_xscom.h +++ b/include/hw/ppc/pnv_xscom.h @@ -87,7 +87,7 @@ typedef struct PnvXScomInterfaceClass { #define PNV9_XSCOM_XIVE_BASE 0x5013000 #define PNV9_XSCOM_XIVE_SIZE 0x300 -extern void pnv_xscom_realize(PnvChip *chip, Error **errp); +extern void pnv_xscom_realize(PnvChip *chip, uint64_t size, Error **errp); extern int pnv_dt_xscom(PnvChip *chip, void *fdt, int offset); extern void pnv_xscom_add_subregion(PnvChip *chip, hwaddr offset, diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index 4f5becf1f3..60553d32c4 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -676,10 +676,6 @@ typedef void (*spapr_rtas_fn)(PowerPCCPU *cpu, SpaprMachineState *sm, uint32_t nargs, target_ulong args, uint32_t nret, target_ulong rets); void spapr_rtas_register(int token, const char *name, spapr_rtas_fn fn); -static inline void spapr_rtas_unregister(int token) -{ - spapr_rtas_register(token, NULL, NULL); -} target_ulong spapr_rtas_call(PowerPCCPU *cpu, SpaprMachineState *sm, uint32_t token, uint32_t nargs, target_ulong args, uint32_t nret, target_ulong rets); diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h index 14cab73c9c..f965a58f89 100644 --- a/include/hw/ppc/spapr_irq.h +++ b/include/hw/ppc/spapr_irq.h @@ -48,7 +48,6 @@ typedef struct SpaprIrq { void (*reset)(SpaprMachineState *spapr, Error **errp); void (*set_irq)(void *opaque, int srcno, int val); const char *(*get_nodename)(SpaprMachineState *spapr); - void (*init_emu)(SpaprMachineState *spapr, Error **errp); void (*init_kvm)(SpaprMachineState *spapr, Error **errp); } SpaprIrq; diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h index b26befcf6b..7197144265 100644 --- a/include/hw/ppc/spapr_xive.h +++ b/include/hw/ppc/spapr_xive.h @@ -42,6 +42,7 @@ typedef struct SpaprXive { /* KVM support */ int fd; void *tm_mmap; + MemoryRegion tm_mmio_kvm; VMChangeStateEntry *change; } SpaprXive; @@ -66,7 +67,6 @@ void spapr_xive_map_mmio(SpaprXive *xive); int spapr_xive_end_to_target(uint8_t end_blk, uint32_t end_idx, uint32_t *out_server, uint8_t *out_prio); -void spapr_xive_init(SpaprXive *xive, Error **errp); /* * KVM XIVE device helpers diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h index d6f8e4c4c2..1eb7b5cd68 100644 --- a/include/hw/ppc/xics.h +++ b/include/hw/ppc/xics.h @@ -119,7 +119,6 @@ struct ICSState { uint32_t offset; ICSIRQState *irqs; XICSFabric *xics; - bool init; /* sPAPR ICS device initialized */ }; #define ICS_PROP_XICS "xics" @@ -191,13 +190,13 @@ Object *icp_create(Object *cpu, const char *type, XICSFabric *xi, /* KVM */ void icp_get_kvm_state(ICPState *icp); -int icp_set_kvm_state(ICPState *icp); +int icp_set_kvm_state(ICPState *icp, Error **errp); void icp_synchronize_state(ICPState *icp); void icp_kvm_realize(DeviceState *dev, Error **errp); void ics_get_kvm_state(ICSState *ics); -int ics_set_kvm_state_one(ICSState *ics, int srcno); -int ics_set_kvm_state(ICSState *ics); +int ics_set_kvm_state_one(ICSState *ics, int srcno, Error **errp); +int ics_set_kvm_state(ICSState *ics, Error **errp); void ics_synchronize_state(ICSState *ics); void ics_kvm_set_irq(ICSState *ics, int srcno, int val); diff --git a/include/hw/ppc/xics_spapr.h b/include/hw/ppc/xics_spapr.h index 2476b540ed..5dabc9a138 100644 --- a/include/hw/ppc/xics_spapr.h +++ b/include/hw/ppc/xics_spapr.h @@ -33,8 +33,9 @@ void spapr_dt_xics(SpaprMachineState *spapr, uint32_t nr_servers, void *fdt, uint32_t phandle); -int xics_kvm_init(SpaprMachineState *spapr, Error **errp); +int xics_kvm_connect(SpaprMachineState *spapr, Error **errp); void xics_kvm_disconnect(SpaprMachineState *spapr, Error **errp); +bool xics_kvm_has_broken_disconnect(SpaprMachineState *spapr); void xics_spapr_init(SpaprMachineState *spapr); #endif /* XICS_SPAPR_H */ diff --git a/include/hw/ppc/xive.h b/include/hw/ppc/xive.h index a6ee7e831d..55c53c7417 100644 --- a/include/hw/ppc/xive.h +++ b/include/hw/ppc/xive.h @@ -197,6 +197,7 @@ typedef struct XiveSource { /* KVM support */ void *esb_mmap; + MemoryRegion esb_mmio_kvm; XiveNotifier *xive; } XiveSource; @@ -2723,6 +2723,13 @@ void memory_listener_unregister(MemoryListener *listener) listener->address_space = NULL; } +void address_space_remove_listeners(AddressSpace *as) +{ + while (!QTAILQ_EMPTY(&as->listeners)) { + memory_listener_unregister(QTAILQ_FIRST(&as->listeners)); + } +} + void address_space_init(AddressSpace *as, MemoryRegion *root, const char *name) { memory_region_ref(root); diff --git a/pc-bios/spapr-rtas/Makefile b/pc-bios/spapr-rtas/Makefile index f26dd428b7..4b9bb12306 100644 --- a/pc-bios/spapr-rtas/Makefile +++ b/pc-bios/spapr-rtas/Makefile @@ -14,8 +14,11 @@ $(call set-vpath, $(SRC_PATH)/pc-bios/spapr-rtas) build-all: spapr-rtas.bin +%.o: %.S + $(call quiet-command,$(CCAS) -mbig -c -o $@ $<,"CCAS","$(TARGET_DIR)$@") + %.img: %.o - $(call quiet-command,$(CC) -nostdlib -o $@ $<,"Building","$(TARGET_DIR)$@") + $(call quiet-command,$(CC) -nostdlib -mbig -o $@ $<,"Building","$(TARGET_DIR)$@") %.bin: %.img $(call quiet-command,$(OBJCOPY) -O binary -j .text $< $@,"Building","$(TARGET_DIR)$@") diff --git a/target/ppc/fpu_helper.c b/target/ppc/fpu_helper.c index ffbd19afa1..f437c88aad 100644 --- a/target/ppc/fpu_helper.c +++ b/target/ppc/fpu_helper.c @@ -1801,37 +1801,35 @@ uint32_t helper_efdcmpeq(CPUPPCState *env, uint64_t op1, uint64_t op2) * sfprf - set FPRF */ #define VSX_ADD_SUB(name, op, nels, tp, fld, sfprf, r2sp) \ -void helper_##name(CPUPPCState *env, uint32_t opcode) \ +void helper_##name(CPUPPCState *env, ppc_vsr_t *xt, \ + ppc_vsr_t *xa, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xa, xb; \ + ppc_vsr_t t = *xt; \ int i; \ \ - getVSR(xA(opcode), &xa, env); \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ helper_reset_fpstatus(env); \ \ for (i = 0; i < nels; i++) { \ float_status tstat = env->fp_status; \ set_float_exception_flags(0, &tstat); \ - xt.fld = tp##_##op(xa.fld, xb.fld, &tstat); \ + t.fld = tp##_##op(xa->fld, xb->fld, &tstat); \ env->fp_status.float_exception_flags |= tstat.float_exception_flags; \ \ if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { \ float_invalid_op_addsub(env, sfprf, GETPC(), \ - tp##_classify(xa.fld) | \ - tp##_classify(xb.fld)); \ + tp##_classify(xa->fld) | \ + tp##_classify(xb->fld)); \ } \ \ if (r2sp) { \ - xt.fld = helper_frsp(env, xt.fld); \ + t.fld = helper_frsp(env, t.fld); \ } \ \ if (sfprf) { \ - helper_compute_fprf_float64(env, xt.fld); \ + helper_compute_fprf_float64(env, t.fld); \ } \ } \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -1844,14 +1842,12 @@ VSX_ADD_SUB(xssubsp, sub, 1, float64, VsrD(0), 1, 1) VSX_ADD_SUB(xvsubdp, sub, 2, float64, VsrD(i), 0, 0) VSX_ADD_SUB(xvsubsp, sub, 4, float32, VsrW(i), 0, 0) -void helper_xsaddqp(CPUPPCState *env, uint32_t opcode) +void helper_xsaddqp(CPUPPCState *env, uint32_t opcode, + ppc_vsr_t *xt, ppc_vsr_t *xa, ppc_vsr_t *xb) { - ppc_vsr_t xt, xa, xb; + ppc_vsr_t t = *xt; float_status tstat; - getVSR(rA(opcode) + 32, &xa, env); - getVSR(rB(opcode) + 32, &xb, env); - getVSR(rD(opcode) + 32, &xt, env); helper_reset_fpstatus(env); tstat = env->fp_status; @@ -1860,18 +1856,18 @@ void helper_xsaddqp(CPUPPCState *env, uint32_t opcode) } set_float_exception_flags(0, &tstat); - xt.f128 = float128_add(xa.f128, xb.f128, &tstat); + t.f128 = float128_add(xa->f128, xb->f128, &tstat); env->fp_status.float_exception_flags |= tstat.float_exception_flags; if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { float_invalid_op_addsub(env, 1, GETPC(), - float128_classify(xa.f128) | - float128_classify(xb.f128)); + float128_classify(xa->f128) | + float128_classify(xb->f128)); } - helper_compute_fprf_float128(env, xt.f128); + helper_compute_fprf_float128(env, t.f128); - putVSR(rD(opcode) + 32, &xt, env); + *xt = t; do_float_check_status(env, GETPC()); } @@ -1884,38 +1880,36 @@ void helper_xsaddqp(CPUPPCState *env, uint32_t opcode) * sfprf - set FPRF */ #define VSX_MUL(op, nels, tp, fld, sfprf, r2sp) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, \ + ppc_vsr_t *xa, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xa, xb; \ + ppc_vsr_t t = *xt; \ int i; \ \ - getVSR(xA(opcode), &xa, env); \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ helper_reset_fpstatus(env); \ \ for (i = 0; i < nels; i++) { \ float_status tstat = env->fp_status; \ set_float_exception_flags(0, &tstat); \ - xt.fld = tp##_mul(xa.fld, xb.fld, &tstat); \ + t.fld = tp##_mul(xa->fld, xb->fld, &tstat); \ env->fp_status.float_exception_flags |= tstat.float_exception_flags; \ \ if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { \ float_invalid_op_mul(env, sfprf, GETPC(), \ - tp##_classify(xa.fld) | \ - tp##_classify(xb.fld)); \ + tp##_classify(xa->fld) | \ + tp##_classify(xb->fld)); \ } \ \ if (r2sp) { \ - xt.fld = helper_frsp(env, xt.fld); \ + t.fld = helper_frsp(env, t.fld); \ } \ \ if (sfprf) { \ - helper_compute_fprf_float64(env, xt.fld); \ + helper_compute_fprf_float64(env, t.fld); \ } \ } \ \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -1924,15 +1918,12 @@ VSX_MUL(xsmulsp, 1, float64, VsrD(0), 1, 1) VSX_MUL(xvmuldp, 2, float64, VsrD(i), 0, 0) VSX_MUL(xvmulsp, 4, float32, VsrW(i), 0, 0) -void helper_xsmulqp(CPUPPCState *env, uint32_t opcode) +void helper_xsmulqp(CPUPPCState *env, uint32_t opcode, + ppc_vsr_t *xt, ppc_vsr_t *xa, ppc_vsr_t *xb) { - ppc_vsr_t xt, xa, xb; + ppc_vsr_t t = *xt; float_status tstat; - getVSR(rA(opcode) + 32, &xa, env); - getVSR(rB(opcode) + 32, &xb, env); - getVSR(rD(opcode) + 32, &xt, env); - helper_reset_fpstatus(env); tstat = env->fp_status; if (unlikely(Rc(opcode) != 0)) { @@ -1940,17 +1931,17 @@ void helper_xsmulqp(CPUPPCState *env, uint32_t opcode) } set_float_exception_flags(0, &tstat); - xt.f128 = float128_mul(xa.f128, xb.f128, &tstat); + t.f128 = float128_mul(xa->f128, xb->f128, &tstat); env->fp_status.float_exception_flags |= tstat.float_exception_flags; if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { float_invalid_op_mul(env, 1, GETPC(), - float128_classify(xa.f128) | - float128_classify(xb.f128)); + float128_classify(xa->f128) | + float128_classify(xb->f128)); } - helper_compute_fprf_float128(env, xt.f128); + helper_compute_fprf_float128(env, t.f128); - putVSR(rD(opcode) + 32, &xt, env); + *xt = t; do_float_check_status(env, GETPC()); } @@ -1963,41 +1954,39 @@ void helper_xsmulqp(CPUPPCState *env, uint32_t opcode) * sfprf - set FPRF */ #define VSX_DIV(op, nels, tp, fld, sfprf, r2sp) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, \ + ppc_vsr_t *xa, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xa, xb; \ + ppc_vsr_t t = *xt; \ int i; \ \ - getVSR(xA(opcode), &xa, env); \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ helper_reset_fpstatus(env); \ \ for (i = 0; i < nels; i++) { \ float_status tstat = env->fp_status; \ set_float_exception_flags(0, &tstat); \ - xt.fld = tp##_div(xa.fld, xb.fld, &tstat); \ + t.fld = tp##_div(xa->fld, xb->fld, &tstat); \ env->fp_status.float_exception_flags |= tstat.float_exception_flags; \ \ if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { \ float_invalid_op_div(env, sfprf, GETPC(), \ - tp##_classify(xa.fld) | \ - tp##_classify(xb.fld)); \ + tp##_classify(xa->fld) | \ + tp##_classify(xb->fld)); \ } \ if (unlikely(tstat.float_exception_flags & float_flag_divbyzero)) { \ float_zero_divide_excp(env, GETPC()); \ } \ \ if (r2sp) { \ - xt.fld = helper_frsp(env, xt.fld); \ + t.fld = helper_frsp(env, t.fld); \ } \ \ if (sfprf) { \ - helper_compute_fprf_float64(env, xt.fld); \ + helper_compute_fprf_float64(env, t.fld); \ } \ } \ \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -2006,15 +1995,12 @@ VSX_DIV(xsdivsp, 1, float64, VsrD(0), 1, 1) VSX_DIV(xvdivdp, 2, float64, VsrD(i), 0, 0) VSX_DIV(xvdivsp, 4, float32, VsrW(i), 0, 0) -void helper_xsdivqp(CPUPPCState *env, uint32_t opcode) +void helper_xsdivqp(CPUPPCState *env, uint32_t opcode, + ppc_vsr_t *xt, ppc_vsr_t *xa, ppc_vsr_t *xb) { - ppc_vsr_t xt, xa, xb; + ppc_vsr_t t = *xt; float_status tstat; - getVSR(rA(opcode) + 32, &xa, env); - getVSR(rB(opcode) + 32, &xb, env); - getVSR(rD(opcode) + 32, &xt, env); - helper_reset_fpstatus(env); tstat = env->fp_status; if (unlikely(Rc(opcode) != 0)) { @@ -2022,20 +2008,20 @@ void helper_xsdivqp(CPUPPCState *env, uint32_t opcode) } set_float_exception_flags(0, &tstat); - xt.f128 = float128_div(xa.f128, xb.f128, &tstat); + t.f128 = float128_div(xa->f128, xb->f128, &tstat); env->fp_status.float_exception_flags |= tstat.float_exception_flags; if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { float_invalid_op_div(env, 1, GETPC(), - float128_classify(xa.f128) | - float128_classify(xb.f128)); + float128_classify(xa->f128) | + float128_classify(xb->f128)); } if (unlikely(tstat.float_exception_flags & float_flag_divbyzero)) { float_zero_divide_excp(env, GETPC()); } - helper_compute_fprf_float128(env, xt.f128); - putVSR(rD(opcode) + 32, &xt, env); + helper_compute_fprf_float128(env, t.f128); + *xt = t; do_float_check_status(env, GETPC()); } @@ -2048,31 +2034,29 @@ void helper_xsdivqp(CPUPPCState *env, uint32_t opcode) * sfprf - set FPRF */ #define VSX_RE(op, nels, tp, fld, sfprf, r2sp) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t t = *xt; \ int i; \ \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ helper_reset_fpstatus(env); \ \ for (i = 0; i < nels; i++) { \ - if (unlikely(tp##_is_signaling_nan(xb.fld, &env->fp_status))) { \ + if (unlikely(tp##_is_signaling_nan(xb->fld, &env->fp_status))) { \ float_invalid_op_vxsnan(env, GETPC()); \ } \ - xt.fld = tp##_div(tp##_one, xb.fld, &env->fp_status); \ + t.fld = tp##_div(tp##_one, xb->fld, &env->fp_status); \ \ if (r2sp) { \ - xt.fld = helper_frsp(env, xt.fld); \ + t.fld = helper_frsp(env, t.fld); \ } \ \ if (sfprf) { \ - helper_compute_fprf_float64(env, xt.fld); \ + helper_compute_fprf_float64(env, t.fld); \ } \ } \ \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -2090,39 +2074,37 @@ VSX_RE(xvresp, 4, float32, VsrW(i), 0, 0) * sfprf - set FPRF */ #define VSX_SQRT(op, nels, tp, fld, sfprf, r2sp) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t t = *xt; \ int i; \ \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ helper_reset_fpstatus(env); \ \ for (i = 0; i < nels; i++) { \ float_status tstat = env->fp_status; \ set_float_exception_flags(0, &tstat); \ - xt.fld = tp##_sqrt(xb.fld, &tstat); \ + t.fld = tp##_sqrt(xb->fld, &tstat); \ env->fp_status.float_exception_flags |= tstat.float_exception_flags; \ \ if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { \ - if (tp##_is_neg(xb.fld) && !tp##_is_zero(xb.fld)) { \ + if (tp##_is_neg(xb->fld) && !tp##_is_zero(xb->fld)) { \ float_invalid_op_vxsqrt(env, sfprf, GETPC()); \ - } else if (tp##_is_signaling_nan(xb.fld, &tstat)) { \ + } else if (tp##_is_signaling_nan(xb->fld, &tstat)) { \ float_invalid_op_vxsnan(env, GETPC()); \ } \ } \ \ if (r2sp) { \ - xt.fld = helper_frsp(env, xt.fld); \ + t.fld = helper_frsp(env, t.fld); \ } \ \ if (sfprf) { \ - helper_compute_fprf_float64(env, xt.fld); \ + helper_compute_fprf_float64(env, t.fld); \ } \ } \ \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -2140,40 +2122,38 @@ VSX_SQRT(xvsqrtsp, 4, float32, VsrW(i), 0, 0) * sfprf - set FPRF */ #define VSX_RSQRTE(op, nels, tp, fld, sfprf, r2sp) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t t = *xt; \ int i; \ \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ helper_reset_fpstatus(env); \ \ for (i = 0; i < nels; i++) { \ float_status tstat = env->fp_status; \ set_float_exception_flags(0, &tstat); \ - xt.fld = tp##_sqrt(xb.fld, &tstat); \ - xt.fld = tp##_div(tp##_one, xt.fld, &tstat); \ + t.fld = tp##_sqrt(xb->fld, &tstat); \ + t.fld = tp##_div(tp##_one, t.fld, &tstat); \ env->fp_status.float_exception_flags |= tstat.float_exception_flags; \ \ if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { \ - if (tp##_is_neg(xb.fld) && !tp##_is_zero(xb.fld)) { \ + if (tp##_is_neg(xb->fld) && !tp##_is_zero(xb->fld)) { \ float_invalid_op_vxsqrt(env, sfprf, GETPC()); \ - } else if (tp##_is_signaling_nan(xb.fld, &tstat)) { \ + } else if (tp##_is_signaling_nan(xb->fld, &tstat)) { \ float_invalid_op_vxsnan(env, GETPC()); \ } \ } \ \ if (r2sp) { \ - xt.fld = helper_frsp(env, xt.fld); \ + t.fld = helper_frsp(env, t.fld); \ } \ \ if (sfprf) { \ - helper_compute_fprf_float64(env, xt.fld); \ + helper_compute_fprf_float64(env, t.fld); \ } \ } \ \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -2193,39 +2173,36 @@ VSX_RSQRTE(xvrsqrtesp, 4, float32, VsrW(i), 0, 0) * nbits - number of fraction bits */ #define VSX_TDIV(op, nels, tp, fld, emin, emax, nbits) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, uint32_t opcode, \ + ppc_vsr_t *xa, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xa, xb; \ int i; \ int fe_flag = 0; \ int fg_flag = 0; \ \ - getVSR(xA(opcode), &xa, env); \ - getVSR(xB(opcode), &xb, env); \ - \ for (i = 0; i < nels; i++) { \ - if (unlikely(tp##_is_infinity(xa.fld) || \ - tp##_is_infinity(xb.fld) || \ - tp##_is_zero(xb.fld))) { \ + if (unlikely(tp##_is_infinity(xa->fld) || \ + tp##_is_infinity(xb->fld) || \ + tp##_is_zero(xb->fld))) { \ fe_flag = 1; \ fg_flag = 1; \ } else { \ - int e_a = ppc_##tp##_get_unbiased_exp(xa.fld); \ - int e_b = ppc_##tp##_get_unbiased_exp(xb.fld); \ + int e_a = ppc_##tp##_get_unbiased_exp(xa->fld); \ + int e_b = ppc_##tp##_get_unbiased_exp(xb->fld); \ \ - if (unlikely(tp##_is_any_nan(xa.fld) || \ - tp##_is_any_nan(xb.fld))) { \ + if (unlikely(tp##_is_any_nan(xa->fld) || \ + tp##_is_any_nan(xb->fld))) { \ fe_flag = 1; \ } else if ((e_b <= emin) || (e_b >= (emax - 2))) { \ fe_flag = 1; \ - } else if (!tp##_is_zero(xa.fld) && \ + } else if (!tp##_is_zero(xa->fld) && \ (((e_a - e_b) >= emax) || \ ((e_a - e_b) <= (emin + 1)) || \ (e_a <= (emin + nbits)))) { \ fe_flag = 1; \ } \ \ - if (unlikely(tp##_is_zero_or_denormal(xb.fld))) { \ + if (unlikely(tp##_is_zero_or_denormal(xb->fld))) { \ /* \ * XB is not zero because of the above check and so \ * must be denormalized. \ @@ -2253,36 +2230,32 @@ VSX_TDIV(xvtdivsp, 4, float32, VsrW(i), -126, 127, 23) * nbits - number of fraction bits */ #define VSX_TSQRT(op, nels, tp, fld, emin, nbits) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, uint32_t opcode, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xa, xb; \ int i; \ int fe_flag = 0; \ int fg_flag = 0; \ \ - getVSR(xA(opcode), &xa, env); \ - getVSR(xB(opcode), &xb, env); \ - \ for (i = 0; i < nels; i++) { \ - if (unlikely(tp##_is_infinity(xb.fld) || \ - tp##_is_zero(xb.fld))) { \ + if (unlikely(tp##_is_infinity(xb->fld) || \ + tp##_is_zero(xb->fld))) { \ fe_flag = 1; \ fg_flag = 1; \ } else { \ - int e_b = ppc_##tp##_get_unbiased_exp(xb.fld); \ + int e_b = ppc_##tp##_get_unbiased_exp(xb->fld); \ \ - if (unlikely(tp##_is_any_nan(xb.fld))) { \ + if (unlikely(tp##_is_any_nan(xb->fld))) { \ fe_flag = 1; \ - } else if (unlikely(tp##_is_zero(xb.fld))) { \ + } else if (unlikely(tp##_is_zero(xb->fld))) { \ fe_flag = 1; \ - } else if (unlikely(tp##_is_neg(xb.fld))) { \ + } else if (unlikely(tp##_is_neg(xb->fld))) { \ fe_flag = 1; \ - } else if (!tp##_is_zero(xb.fld) && \ + } else if (!tp##_is_zero(xb->fld) && \ (e_b <= (emin + nbits))) { \ fe_flag = 1; \ } \ \ - if (unlikely(tp##_is_zero_or_denormal(xb.fld))) { \ + if (unlikely(tp##_is_zero_or_denormal(xb->fld))) { \ /* \ * XB is not zero because of the above check and \ * therefore must be denormalized. \ @@ -2307,30 +2280,15 @@ VSX_TSQRT(xvtsqrtsp, 4, float32, VsrW(i), -126, 23) * fld - vsr_t field (VsrD(*) or VsrW(*)) * maddflgs - flags for the float*muladd routine that control the * various forms (madd, msub, nmadd, nmsub) - * afrm - A form (1=A, 0=M) * sfprf - set FPRF */ -#define VSX_MADD(op, nels, tp, fld, maddflgs, afrm, sfprf, r2sp) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +#define VSX_MADD(op, nels, tp, fld, maddflgs, sfprf, r2sp) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, \ + ppc_vsr_t *xa, ppc_vsr_t *b, ppc_vsr_t *c) \ { \ - ppc_vsr_t xt_in, xa, xb, xt_out; \ - ppc_vsr_t *b, *c; \ + ppc_vsr_t t = *xt; \ int i; \ \ - if (afrm) { /* AxB + T */ \ - b = &xb; \ - c = &xt_in; \ - } else { /* AxT + B */ \ - b = &xt_in; \ - c = &xb; \ - } \ - \ - getVSR(xA(opcode), &xa, env); \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt_in, env); \ - \ - xt_out = xt_in; \ - \ helper_reset_fpstatus(env); \ \ for (i = 0; i < nels; i++) { \ @@ -2342,68 +2300,51 @@ void helper_##op(CPUPPCState *env, uint32_t opcode) \ * result to odd. \ */ \ set_float_rounding_mode(float_round_to_zero, &tstat); \ - xt_out.fld = tp##_muladd(xa.fld, b->fld, c->fld, \ - maddflgs, &tstat); \ - xt_out.fld |= (get_float_exception_flags(&tstat) & \ - float_flag_inexact) != 0; \ + t.fld = tp##_muladd(xa->fld, b->fld, c->fld, \ + maddflgs, &tstat); \ + t.fld |= (get_float_exception_flags(&tstat) & \ + float_flag_inexact) != 0; \ } else { \ - xt_out.fld = tp##_muladd(xa.fld, b->fld, c->fld, \ - maddflgs, &tstat); \ + t.fld = tp##_muladd(xa->fld, b->fld, c->fld, \ + maddflgs, &tstat); \ } \ env->fp_status.float_exception_flags |= tstat.float_exception_flags; \ \ if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { \ - tp##_maddsub_update_excp(env, xa.fld, b->fld, \ + tp##_maddsub_update_excp(env, xa->fld, b->fld, \ c->fld, maddflgs, GETPC()); \ } \ \ if (r2sp) { \ - xt_out.fld = helper_frsp(env, xt_out.fld); \ + t.fld = helper_frsp(env, t.fld); \ } \ \ if (sfprf) { \ - helper_compute_fprf_float64(env, xt_out.fld); \ + helper_compute_fprf_float64(env, t.fld); \ } \ } \ - putVSR(xT(opcode), &xt_out, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } -VSX_MADD(xsmaddadp, 1, float64, VsrD(0), MADD_FLGS, 1, 1, 0) -VSX_MADD(xsmaddmdp, 1, float64, VsrD(0), MADD_FLGS, 0, 1, 0) -VSX_MADD(xsmsubadp, 1, float64, VsrD(0), MSUB_FLGS, 1, 1, 0) -VSX_MADD(xsmsubmdp, 1, float64, VsrD(0), MSUB_FLGS, 0, 1, 0) -VSX_MADD(xsnmaddadp, 1, float64, VsrD(0), NMADD_FLGS, 1, 1, 0) -VSX_MADD(xsnmaddmdp, 1, float64, VsrD(0), NMADD_FLGS, 0, 1, 0) -VSX_MADD(xsnmsubadp, 1, float64, VsrD(0), NMSUB_FLGS, 1, 1, 0) -VSX_MADD(xsnmsubmdp, 1, float64, VsrD(0), NMSUB_FLGS, 0, 1, 0) - -VSX_MADD(xsmaddasp, 1, float64, VsrD(0), MADD_FLGS, 1, 1, 1) -VSX_MADD(xsmaddmsp, 1, float64, VsrD(0), MADD_FLGS, 0, 1, 1) -VSX_MADD(xsmsubasp, 1, float64, VsrD(0), MSUB_FLGS, 1, 1, 1) -VSX_MADD(xsmsubmsp, 1, float64, VsrD(0), MSUB_FLGS, 0, 1, 1) -VSX_MADD(xsnmaddasp, 1, float64, VsrD(0), NMADD_FLGS, 1, 1, 1) -VSX_MADD(xsnmaddmsp, 1, float64, VsrD(0), NMADD_FLGS, 0, 1, 1) -VSX_MADD(xsnmsubasp, 1, float64, VsrD(0), NMSUB_FLGS, 1, 1, 1) -VSX_MADD(xsnmsubmsp, 1, float64, VsrD(0), NMSUB_FLGS, 0, 1, 1) - -VSX_MADD(xvmaddadp, 2, float64, VsrD(i), MADD_FLGS, 1, 0, 0) -VSX_MADD(xvmaddmdp, 2, float64, VsrD(i), MADD_FLGS, 0, 0, 0) -VSX_MADD(xvmsubadp, 2, float64, VsrD(i), MSUB_FLGS, 1, 0, 0) -VSX_MADD(xvmsubmdp, 2, float64, VsrD(i), MSUB_FLGS, 0, 0, 0) -VSX_MADD(xvnmaddadp, 2, float64, VsrD(i), NMADD_FLGS, 1, 0, 0) -VSX_MADD(xvnmaddmdp, 2, float64, VsrD(i), NMADD_FLGS, 0, 0, 0) -VSX_MADD(xvnmsubadp, 2, float64, VsrD(i), NMSUB_FLGS, 1, 0, 0) -VSX_MADD(xvnmsubmdp, 2, float64, VsrD(i), NMSUB_FLGS, 0, 0, 0) - -VSX_MADD(xvmaddasp, 4, float32, VsrW(i), MADD_FLGS, 1, 0, 0) -VSX_MADD(xvmaddmsp, 4, float32, VsrW(i), MADD_FLGS, 0, 0, 0) -VSX_MADD(xvmsubasp, 4, float32, VsrW(i), MSUB_FLGS, 1, 0, 0) -VSX_MADD(xvmsubmsp, 4, float32, VsrW(i), MSUB_FLGS, 0, 0, 0) -VSX_MADD(xvnmaddasp, 4, float32, VsrW(i), NMADD_FLGS, 1, 0, 0) -VSX_MADD(xvnmaddmsp, 4, float32, VsrW(i), NMADD_FLGS, 0, 0, 0) -VSX_MADD(xvnmsubasp, 4, float32, VsrW(i), NMSUB_FLGS, 1, 0, 0) -VSX_MADD(xvnmsubmsp, 4, float32, VsrW(i), NMSUB_FLGS, 0, 0, 0) +VSX_MADD(xsmadddp, 1, float64, VsrD(0), MADD_FLGS, 1, 0) +VSX_MADD(xsmsubdp, 1, float64, VsrD(0), MSUB_FLGS, 1, 0) +VSX_MADD(xsnmadddp, 1, float64, VsrD(0), NMADD_FLGS, 1, 0) +VSX_MADD(xsnmsubdp, 1, float64, VsrD(0), NMSUB_FLGS, 1, 0) +VSX_MADD(xsmaddsp, 1, float64, VsrD(0), MADD_FLGS, 1, 1) +VSX_MADD(xsmsubsp, 1, float64, VsrD(0), MSUB_FLGS, 1, 1) +VSX_MADD(xsnmaddsp, 1, float64, VsrD(0), NMADD_FLGS, 1, 1) +VSX_MADD(xsnmsubsp, 1, float64, VsrD(0), NMSUB_FLGS, 1, 1) + +VSX_MADD(xvmadddp, 2, float64, VsrD(i), MADD_FLGS, 0, 0) +VSX_MADD(xvmsubdp, 2, float64, VsrD(i), MSUB_FLGS, 0, 0) +VSX_MADD(xvnmadddp, 2, float64, VsrD(i), NMADD_FLGS, 0, 0) +VSX_MADD(xvnmsubdp, 2, float64, VsrD(i), NMSUB_FLGS, 0, 0) + +VSX_MADD(xvmaddsp, 4, float32, VsrW(i), MADD_FLGS, 0, 0) +VSX_MADD(xvmsubsp, 4, float32, VsrW(i), MSUB_FLGS, 0, 0) +VSX_MADD(xvnmaddsp, 4, float32, VsrW(i), NMADD_FLGS, 0, 0) +VSX_MADD(xvnmsubsp, 4, float32, VsrW(i), NMSUB_FLGS, 0, 0) /* * VSX_SCALAR_CMP_DP - VSX scalar floating point compare double precision @@ -2413,24 +2354,21 @@ VSX_MADD(xvnmsubmsp, 4, float32, VsrW(i), NMSUB_FLGS, 0, 0, 0) * svxvc - set VXVC bit */ #define VSX_SCALAR_CMP_DP(op, cmp, exp, svxvc) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, \ + ppc_vsr_t *xa, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xa, xb; \ + ppc_vsr_t t = *xt; \ bool vxsnan_flag = false, vxvc_flag = false, vex_flag = false; \ \ - getVSR(xA(opcode), &xa, env); \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ - \ - if (float64_is_signaling_nan(xa.VsrD(0), &env->fp_status) || \ - float64_is_signaling_nan(xb.VsrD(0), &env->fp_status)) { \ + if (float64_is_signaling_nan(xa->VsrD(0), &env->fp_status) || \ + float64_is_signaling_nan(xb->VsrD(0), &env->fp_status)) { \ vxsnan_flag = true; \ if (fpscr_ve == 0 && svxvc) { \ vxvc_flag = true; \ } \ } else if (svxvc) { \ - vxvc_flag = float64_is_quiet_nan(xa.VsrD(0), &env->fp_status) || \ - float64_is_quiet_nan(xb.VsrD(0), &env->fp_status); \ + vxvc_flag = float64_is_quiet_nan(xa->VsrD(0), &env->fp_status) || \ + float64_is_quiet_nan(xb->VsrD(0), &env->fp_status); \ } \ if (vxsnan_flag) { \ float_invalid_op_vxsnan(env, GETPC()); \ @@ -2441,15 +2379,16 @@ void helper_##op(CPUPPCState *env, uint32_t opcode) \ vex_flag = fpscr_ve && (vxvc_flag || vxsnan_flag); \ \ if (!vex_flag) { \ - if (float64_##cmp(xb.VsrD(0), xa.VsrD(0), &env->fp_status) == exp) { \ - xt.VsrD(0) = -1; \ - xt.VsrD(1) = 0; \ + if (float64_##cmp(xb->VsrD(0), xa->VsrD(0), \ + &env->fp_status) == exp) { \ + t.VsrD(0) = -1; \ + t.VsrD(1) = 0; \ } else { \ - xt.VsrD(0) = 0; \ - xt.VsrD(1) = 0; \ + t.VsrD(0) = 0; \ + t.VsrD(1) = 0; \ } \ } \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -2458,20 +2397,17 @@ VSX_SCALAR_CMP_DP(xscmpgedp, le, 1, 1) VSX_SCALAR_CMP_DP(xscmpgtdp, lt, 1, 1) VSX_SCALAR_CMP_DP(xscmpnedp, eq, 0, 0) -void helper_xscmpexpdp(CPUPPCState *env, uint32_t opcode) +void helper_xscmpexpdp(CPUPPCState *env, uint32_t opcode, + ppc_vsr_t *xa, ppc_vsr_t *xb) { - ppc_vsr_t xa, xb; int64_t exp_a, exp_b; uint32_t cc; - getVSR(xA(opcode), &xa, env); - getVSR(xB(opcode), &xb, env); - - exp_a = extract64(xa.VsrD(0), 52, 11); - exp_b = extract64(xb.VsrD(0), 52, 11); + exp_a = extract64(xa->VsrD(0), 52, 11); + exp_b = extract64(xb->VsrD(0), 52, 11); - if (unlikely(float64_is_any_nan(xa.VsrD(0)) || - float64_is_any_nan(xb.VsrD(0)))) { + if (unlikely(float64_is_any_nan(xa->VsrD(0)) || + float64_is_any_nan(xb->VsrD(0)))) { cc = CRF_SO; } else { if (exp_a < exp_b) { @@ -2490,20 +2426,17 @@ void helper_xscmpexpdp(CPUPPCState *env, uint32_t opcode) do_float_check_status(env, GETPC()); } -void helper_xscmpexpqp(CPUPPCState *env, uint32_t opcode) +void helper_xscmpexpqp(CPUPPCState *env, uint32_t opcode, + ppc_vsr_t *xa, ppc_vsr_t *xb) { - ppc_vsr_t xa, xb; int64_t exp_a, exp_b; uint32_t cc; - getVSR(rA(opcode) + 32, &xa, env); - getVSR(rB(opcode) + 32, &xb, env); + exp_a = extract64(xa->VsrD(0), 48, 15); + exp_b = extract64(xb->VsrD(0), 48, 15); - exp_a = extract64(xa.VsrD(0), 48, 15); - exp_b = extract64(xb.VsrD(0), 48, 15); - - if (unlikely(float128_is_any_nan(xa.f128) || - float128_is_any_nan(xb.f128))) { + if (unlikely(float128_is_any_nan(xa->f128) || + float128_is_any_nan(xb->f128))) { cc = CRF_SO; } else { if (exp_a < exp_b) { @@ -2523,25 +2456,23 @@ void helper_xscmpexpqp(CPUPPCState *env, uint32_t opcode) } #define VSX_SCALAR_CMP(op, ordered) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, uint32_t opcode, \ + ppc_vsr_t *xa, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xa, xb; \ uint32_t cc = 0; \ bool vxsnan_flag = false, vxvc_flag = false; \ \ helper_reset_fpstatus(env); \ - getVSR(xA(opcode), &xa, env); \ - getVSR(xB(opcode), &xb, env); \ \ - if (float64_is_signaling_nan(xa.VsrD(0), &env->fp_status) || \ - float64_is_signaling_nan(xb.VsrD(0), &env->fp_status)) { \ + if (float64_is_signaling_nan(xa->VsrD(0), &env->fp_status) || \ + float64_is_signaling_nan(xb->VsrD(0), &env->fp_status)) { \ vxsnan_flag = true; \ cc = CRF_SO; \ if (fpscr_ve == 0 && ordered) { \ vxvc_flag = true; \ } \ - } else if (float64_is_quiet_nan(xa.VsrD(0), &env->fp_status) || \ - float64_is_quiet_nan(xb.VsrD(0), &env->fp_status)) { \ + } else if (float64_is_quiet_nan(xa->VsrD(0), &env->fp_status) || \ + float64_is_quiet_nan(xb->VsrD(0), &env->fp_status)) { \ cc = CRF_SO; \ if (ordered) { \ vxvc_flag = true; \ @@ -2554,9 +2485,9 @@ void helper_##op(CPUPPCState *env, uint32_t opcode) \ float_invalid_op_vxvc(env, 0, GETPC()); \ } \ \ - if (float64_lt(xa.VsrD(0), xb.VsrD(0), &env->fp_status)) { \ + if (float64_lt(xa->VsrD(0), xb->VsrD(0), &env->fp_status)) { \ cc |= CRF_LT; \ - } else if (!float64_le(xa.VsrD(0), xb.VsrD(0), &env->fp_status)) { \ + } else if (!float64_le(xa->VsrD(0), xb->VsrD(0), &env->fp_status)) { \ cc |= CRF_GT; \ } else { \ cc |= CRF_EQ; \ @@ -2573,25 +2504,23 @@ VSX_SCALAR_CMP(xscmpodp, 1) VSX_SCALAR_CMP(xscmpudp, 0) #define VSX_SCALAR_CMPQ(op, ordered) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, uint32_t opcode, \ + ppc_vsr_t *xa, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xa, xb; \ uint32_t cc = 0; \ bool vxsnan_flag = false, vxvc_flag = false; \ \ helper_reset_fpstatus(env); \ - getVSR(rA(opcode) + 32, &xa, env); \ - getVSR(rB(opcode) + 32, &xb, env); \ \ - if (float128_is_signaling_nan(xa.f128, &env->fp_status) || \ - float128_is_signaling_nan(xb.f128, &env->fp_status)) { \ + if (float128_is_signaling_nan(xa->f128, &env->fp_status) || \ + float128_is_signaling_nan(xb->f128, &env->fp_status)) { \ vxsnan_flag = true; \ cc = CRF_SO; \ if (fpscr_ve == 0 && ordered) { \ vxvc_flag = true; \ } \ - } else if (float128_is_quiet_nan(xa.f128, &env->fp_status) || \ - float128_is_quiet_nan(xb.f128, &env->fp_status)) { \ + } else if (float128_is_quiet_nan(xa->f128, &env->fp_status) || \ + float128_is_quiet_nan(xb->f128, &env->fp_status)) { \ cc = CRF_SO; \ if (ordered) { \ vxvc_flag = true; \ @@ -2604,9 +2533,9 @@ void helper_##op(CPUPPCState *env, uint32_t opcode) \ float_invalid_op_vxvc(env, 0, GETPC()); \ } \ \ - if (float128_lt(xa.f128, xb.f128, &env->fp_status)) { \ + if (float128_lt(xa->f128, xb->f128, &env->fp_status)) { \ cc |= CRF_LT; \ - } else if (!float128_le(xa.f128, xb.f128, &env->fp_status)) { \ + } else if (!float128_le(xa->f128, xb->f128, &env->fp_status)) { \ cc |= CRF_GT; \ } else { \ cc |= CRF_EQ; \ @@ -2631,24 +2560,21 @@ VSX_SCALAR_CMPQ(xscmpuqp, 0) * fld - vsr_t field (VsrD(*) or VsrW(*)) */ #define VSX_MAX_MIN(name, op, nels, tp, fld) \ -void helper_##name(CPUPPCState *env, uint32_t opcode) \ +void helper_##name(CPUPPCState *env, ppc_vsr_t *xt, \ + ppc_vsr_t *xa, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xa, xb; \ + ppc_vsr_t t = *xt; \ int i; \ \ - getVSR(xA(opcode), &xa, env); \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ - \ for (i = 0; i < nels; i++) { \ - xt.fld = tp##_##op(xa.fld, xb.fld, &env->fp_status); \ - if (unlikely(tp##_is_signaling_nan(xa.fld, &env->fp_status) || \ - tp##_is_signaling_nan(xb.fld, &env->fp_status))) { \ + t.fld = tp##_##op(xa->fld, xb->fld, &env->fp_status); \ + if (unlikely(tp##_is_signaling_nan(xa->fld, &env->fp_status) || \ + tp##_is_signaling_nan(xb->fld, &env->fp_status))) { \ float_invalid_op_vxsnan(env, GETPC()); \ } \ } \ \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -2660,29 +2586,26 @@ VSX_MAX_MIN(xvmindp, minnum, 2, float64, VsrD(i)) VSX_MAX_MIN(xvminsp, minnum, 4, float32, VsrW(i)) #define VSX_MAX_MINC(name, max) \ -void helper_##name(CPUPPCState *env, uint32_t opcode) \ +void helper_##name(CPUPPCState *env, uint32_t opcode, \ + ppc_vsr_t *xt, ppc_vsr_t *xa, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xa, xb; \ + ppc_vsr_t t = *xt; \ bool vxsnan_flag = false, vex_flag = false; \ \ - getVSR(rA(opcode) + 32, &xa, env); \ - getVSR(rB(opcode) + 32, &xb, env); \ - getVSR(rD(opcode) + 32, &xt, env); \ - \ - if (unlikely(float64_is_any_nan(xa.VsrD(0)) || \ - float64_is_any_nan(xb.VsrD(0)))) { \ - if (float64_is_signaling_nan(xa.VsrD(0), &env->fp_status) || \ - float64_is_signaling_nan(xb.VsrD(0), &env->fp_status)) { \ + if (unlikely(float64_is_any_nan(xa->VsrD(0)) || \ + float64_is_any_nan(xb->VsrD(0)))) { \ + if (float64_is_signaling_nan(xa->VsrD(0), &env->fp_status) || \ + float64_is_signaling_nan(xb->VsrD(0), &env->fp_status)) { \ vxsnan_flag = true; \ } \ - xt.VsrD(0) = xb.VsrD(0); \ + t.VsrD(0) = xb->VsrD(0); \ } else if ((max && \ - !float64_lt(xa.VsrD(0), xb.VsrD(0), &env->fp_status)) || \ + !float64_lt(xa->VsrD(0), xb->VsrD(0), &env->fp_status)) || \ (!max && \ - float64_lt(xa.VsrD(0), xb.VsrD(0), &env->fp_status))) { \ - xt.VsrD(0) = xa.VsrD(0); \ + float64_lt(xa->VsrD(0), xb->VsrD(0), &env->fp_status))) { \ + t.VsrD(0) = xa->VsrD(0); \ } else { \ - xt.VsrD(0) = xb.VsrD(0); \ + t.VsrD(0) = xb->VsrD(0); \ } \ \ vex_flag = fpscr_ve & vxsnan_flag; \ @@ -2690,7 +2613,7 @@ void helper_##name(CPUPPCState *env, uint32_t opcode) \ float_invalid_op_vxsnan(env, GETPC()); \ } \ if (!vex_flag) { \ - putVSR(rD(opcode) + 32, &xt, env); \ + *xt = t; \ } \ } \ @@ -2698,46 +2621,46 @@ VSX_MAX_MINC(xsmaxcdp, 1); VSX_MAX_MINC(xsmincdp, 0); #define VSX_MAX_MINJ(name, max) \ -void helper_##name(CPUPPCState *env, uint32_t opcode) \ +void helper_##name(CPUPPCState *env, uint32_t opcode, \ + ppc_vsr_t *xt, ppc_vsr_t *xa, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xa, xb; \ + ppc_vsr_t t = *xt; \ bool vxsnan_flag = false, vex_flag = false; \ \ - getVSR(rA(opcode) + 32, &xa, env); \ - getVSR(rB(opcode) + 32, &xb, env); \ - getVSR(rD(opcode) + 32, &xt, env); \ - \ - if (unlikely(float64_is_any_nan(xa.VsrD(0)))) { \ - if (float64_is_signaling_nan(xa.VsrD(0), &env->fp_status)) { \ + if (unlikely(float64_is_any_nan(xa->VsrD(0)))) { \ + if (float64_is_signaling_nan(xa->VsrD(0), &env->fp_status)) { \ vxsnan_flag = true; \ } \ - xt.VsrD(0) = xa.VsrD(0); \ - } else if (unlikely(float64_is_any_nan(xb.VsrD(0)))) { \ - if (float64_is_signaling_nan(xb.VsrD(0), &env->fp_status)) { \ + t.VsrD(0) = xa->VsrD(0); \ + } else if (unlikely(float64_is_any_nan(xb->VsrD(0)))) { \ + if (float64_is_signaling_nan(xb->VsrD(0), &env->fp_status)) { \ vxsnan_flag = true; \ } \ - xt.VsrD(0) = xb.VsrD(0); \ - } else if (float64_is_zero(xa.VsrD(0)) && float64_is_zero(xb.VsrD(0))) { \ + t.VsrD(0) = xb->VsrD(0); \ + } else if (float64_is_zero(xa->VsrD(0)) && \ + float64_is_zero(xb->VsrD(0))) { \ if (max) { \ - if (!float64_is_neg(xa.VsrD(0)) || !float64_is_neg(xb.VsrD(0))) { \ - xt.VsrD(0) = 0ULL; \ + if (!float64_is_neg(xa->VsrD(0)) || \ + !float64_is_neg(xb->VsrD(0))) { \ + t.VsrD(0) = 0ULL; \ } else { \ - xt.VsrD(0) = 0x8000000000000000ULL; \ + t.VsrD(0) = 0x8000000000000000ULL; \ } \ } else { \ - if (float64_is_neg(xa.VsrD(0)) || float64_is_neg(xb.VsrD(0))) { \ - xt.VsrD(0) = 0x8000000000000000ULL; \ + if (float64_is_neg(xa->VsrD(0)) || \ + float64_is_neg(xb->VsrD(0))) { \ + t.VsrD(0) = 0x8000000000000000ULL; \ } else { \ - xt.VsrD(0) = 0ULL; \ + t.VsrD(0) = 0ULL; \ } \ } \ } else if ((max && \ - !float64_lt(xa.VsrD(0), xb.VsrD(0), &env->fp_status)) || \ + !float64_lt(xa->VsrD(0), xb->VsrD(0), &env->fp_status)) || \ (!max && \ - float64_lt(xa.VsrD(0), xb.VsrD(0), &env->fp_status))) { \ - xt.VsrD(0) = xa.VsrD(0); \ + float64_lt(xa->VsrD(0), xb->VsrD(0), &env->fp_status))) { \ + t.VsrD(0) = xa->VsrD(0); \ } else { \ - xt.VsrD(0) = xb.VsrD(0); \ + t.VsrD(0) = xb->VsrD(0); \ } \ \ vex_flag = fpscr_ve & vxsnan_flag; \ @@ -2745,7 +2668,7 @@ void helper_##name(CPUPPCState *env, uint32_t opcode) \ float_invalid_op_vxsnan(env, GETPC()); \ } \ if (!vex_flag) { \ - putVSR(rD(opcode) + 32, &xt, env); \ + *xt = t; \ } \ } \ @@ -2763,46 +2686,42 @@ VSX_MAX_MINJ(xsminjdp, 0); * exp - expected result of comparison */ #define VSX_CMP(op, nels, tp, fld, cmp, svxvc, exp) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +uint32_t helper_##op(CPUPPCState *env, ppc_vsr_t *xt, \ + ppc_vsr_t *xa, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xa, xb; \ + ppc_vsr_t t = *xt; \ + uint32_t crf6 = 0; \ int i; \ int all_true = 1; \ int all_false = 1; \ \ - getVSR(xA(opcode), &xa, env); \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ - \ for (i = 0; i < nels; i++) { \ - if (unlikely(tp##_is_any_nan(xa.fld) || \ - tp##_is_any_nan(xb.fld))) { \ - if (tp##_is_signaling_nan(xa.fld, &env->fp_status) || \ - tp##_is_signaling_nan(xb.fld, &env->fp_status)) { \ + if (unlikely(tp##_is_any_nan(xa->fld) || \ + tp##_is_any_nan(xb->fld))) { \ + if (tp##_is_signaling_nan(xa->fld, &env->fp_status) || \ + tp##_is_signaling_nan(xb->fld, &env->fp_status)) { \ float_invalid_op_vxsnan(env, GETPC()); \ } \ if (svxvc) { \ float_invalid_op_vxvc(env, 0, GETPC()); \ } \ - xt.fld = 0; \ + t.fld = 0; \ all_true = 0; \ } else { \ - if (tp##_##cmp(xb.fld, xa.fld, &env->fp_status) == exp) { \ - xt.fld = -1; \ + if (tp##_##cmp(xb->fld, xa->fld, &env->fp_status) == exp) { \ + t.fld = -1; \ all_false = 0; \ } else { \ - xt.fld = 0; \ + t.fld = 0; \ all_true = 0; \ } \ } \ } \ \ - putVSR(xT(opcode), &xt, env); \ - if ((opcode >> (31 - 21)) & 1) { \ - env->crf[6] = (all_true ? 0x8 : 0) | (all_false ? 0x2 : 0); \ - } \ - do_float_check_status(env, GETPC()); \ - } + *xt = t; \ + crf6 = (all_true ? 0x8 : 0) | (all_false ? 0x2 : 0); \ + return crf6; \ +} VSX_CMP(xvcmpeqdp, 2, float64, VsrD(i), eq, 0, 1) VSX_CMP(xvcmpgedp, 2, float64, VsrD(i), le, 1, 1) @@ -2824,27 +2743,24 @@ VSX_CMP(xvcmpnesp, 4, float32, VsrW(i), eq, 0, 0) * sfprf - set FPRF */ #define VSX_CVT_FP_TO_FP(op, nels, stp, ttp, sfld, tfld, sfprf) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t t = *xt; \ int i; \ \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ - \ for (i = 0; i < nels; i++) { \ - xt.tfld = stp##_to_##ttp(xb.sfld, &env->fp_status); \ - if (unlikely(stp##_is_signaling_nan(xb.sfld, \ + t.tfld = stp##_to_##ttp(xb->sfld, &env->fp_status); \ + if (unlikely(stp##_is_signaling_nan(xb->sfld, \ &env->fp_status))) { \ float_invalid_op_vxsnan(env, GETPC()); \ - xt.tfld = ttp##_snan_to_qnan(xt.tfld); \ + t.tfld = ttp##_snan_to_qnan(t.tfld); \ } \ if (sfprf) { \ - helper_compute_fprf_##ttp(env, xt.tfld); \ + helper_compute_fprf_##ttp(env, t.tfld); \ } \ } \ \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -2864,27 +2780,25 @@ VSX_CVT_FP_TO_FP(xvcvspdp, 2, float32, float64, VsrW(2 * i), VsrD(i), 0) * sfprf - set FPRF */ #define VSX_CVT_FP_TO_FP_VECTOR(op, nels, stp, ttp, sfld, tfld, sfprf) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, uint32_t opcode, \ + ppc_vsr_t *xt, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t t = *xt; \ int i; \ \ - getVSR(rB(opcode) + 32, &xb, env); \ - getVSR(rD(opcode) + 32, &xt, env); \ - \ for (i = 0; i < nels; i++) { \ - xt.tfld = stp##_to_##ttp(xb.sfld, &env->fp_status); \ - if (unlikely(stp##_is_signaling_nan(xb.sfld, \ + t.tfld = stp##_to_##ttp(xb->sfld, &env->fp_status); \ + if (unlikely(stp##_is_signaling_nan(xb->sfld, \ &env->fp_status))) { \ float_invalid_op_vxsnan(env, GETPC()); \ - xt.tfld = ttp##_snan_to_qnan(xt.tfld); \ + t.tfld = ttp##_snan_to_qnan(t.tfld); \ } \ if (sfprf) { \ - helper_compute_fprf_##ttp(env, xt.tfld); \ + helper_compute_fprf_##ttp(env, t.tfld); \ } \ } \ \ - putVSR(rD(opcode) + 32, &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -2902,27 +2816,24 @@ VSX_CVT_FP_TO_FP_VECTOR(xscvdpqp, 1, float64, float128, VsrD(0), f128, 1) * sfprf - set FPRF */ #define VSX_CVT_FP_TO_FP_HP(op, nels, stp, ttp, sfld, tfld, sfprf) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t t = { }; \ int i; \ \ - getVSR(xB(opcode), &xb, env); \ - memset(&xt, 0, sizeof(xt)); \ - \ for (i = 0; i < nels; i++) { \ - xt.tfld = stp##_to_##ttp(xb.sfld, 1, &env->fp_status); \ - if (unlikely(stp##_is_signaling_nan(xb.sfld, \ + t.tfld = stp##_to_##ttp(xb->sfld, 1, &env->fp_status); \ + if (unlikely(stp##_is_signaling_nan(xb->sfld, \ &env->fp_status))) { \ float_invalid_op_vxsnan(env, GETPC()); \ - xt.tfld = ttp##_snan_to_qnan(xt.tfld); \ + t.tfld = ttp##_snan_to_qnan(t.tfld); \ } \ if (sfprf) { \ - helper_compute_fprf_##ttp(env, xt.tfld); \ + helper_compute_fprf_##ttp(env, t.tfld); \ } \ } \ \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -2935,28 +2846,26 @@ VSX_CVT_FP_TO_FP_HP(xvcvhpsp, 4, float16, float32, VsrH(2 * i + 1), VsrW(i), 0) * xscvqpdp isn't using VSX_CVT_FP_TO_FP() because xscvqpdpo will be * added to this later. */ -void helper_xscvqpdp(CPUPPCState *env, uint32_t opcode) +void helper_xscvqpdp(CPUPPCState *env, uint32_t opcode, + ppc_vsr_t *xt, ppc_vsr_t *xb) { - ppc_vsr_t xt, xb; + ppc_vsr_t t = { }; float_status tstat; - getVSR(rB(opcode) + 32, &xb, env); - memset(&xt, 0, sizeof(xt)); - tstat = env->fp_status; if (unlikely(Rc(opcode) != 0)) { tstat.float_rounding_mode = float_round_to_odd; } - xt.VsrD(0) = float128_to_float64(xb.f128, &tstat); + t.VsrD(0) = float128_to_float64(xb->f128, &tstat); env->fp_status.float_exception_flags |= tstat.float_exception_flags; - if (unlikely(float128_is_signaling_nan(xb.f128, &tstat))) { + if (unlikely(float128_is_signaling_nan(xb->f128, &tstat))) { float_invalid_op_vxsnan(env, GETPC()); - xt.VsrD(0) = float64_snan_to_qnan(xt.VsrD(0)); + t.VsrD(0) = float64_snan_to_qnan(t.VsrD(0)); } - helper_compute_fprf_float64(env, xt.VsrD(0)); + helper_compute_fprf_float64(env, t.VsrD(0)); - putVSR(rD(opcode) + 32, &xt, env); + *xt = t; do_float_check_status(env, GETPC()); } @@ -2987,27 +2896,24 @@ uint64_t helper_xscvspdpn(CPUPPCState *env, uint64_t xb) * rnan - resulting NaN */ #define VSX_CVT_FP_TO_INT(op, nels, stp, ttp, sfld, tfld, rnan) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb) \ { \ int all_flags = env->fp_status.float_exception_flags, flags; \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t t = *xt; \ int i; \ \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ - \ for (i = 0; i < nels; i++) { \ env->fp_status.float_exception_flags = 0; \ - xt.tfld = stp##_to_##ttp##_round_to_zero(xb.sfld, &env->fp_status); \ + t.tfld = stp##_to_##ttp##_round_to_zero(xb->sfld, &env->fp_status); \ flags = env->fp_status.float_exception_flags; \ if (unlikely(flags & float_flag_invalid)) { \ - float_invalid_cvt(env, 0, GETPC(), stp##_classify(xb.sfld)); \ - xt.tfld = rnan; \ + float_invalid_cvt(env, 0, GETPC(), stp##_classify(xb->sfld)); \ + t.tfld = rnan; \ } \ all_flags |= flags; \ } \ \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ env->fp_status.float_exception_flags = all_flags; \ do_float_check_status(env, GETPC()); \ } @@ -3040,20 +2946,18 @@ VSX_CVT_FP_TO_INT(xvcvspuxws, 4, float32, uint32, VsrW(i), VsrW(i), 0U) * rnan - resulting NaN */ #define VSX_CVT_FP_TO_INT_VECTOR(op, stp, ttp, sfld, tfld, rnan) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, uint32_t opcode, \ + ppc_vsr_t *xt, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t t = { }; \ \ - getVSR(rB(opcode) + 32, &xb, env); \ - memset(&xt, 0, sizeof(xt)); \ - \ - xt.tfld = stp##_to_##ttp##_round_to_zero(xb.sfld, &env->fp_status); \ + t.tfld = stp##_to_##ttp##_round_to_zero(xb->sfld, &env->fp_status); \ if (env->fp_status.float_exception_flags & float_flag_invalid) { \ - float_invalid_cvt(env, 0, GETPC(), stp##_classify(xb.sfld)); \ - xt.tfld = rnan; \ + float_invalid_cvt(env, 0, GETPC(), stp##_classify(xb->sfld)); \ + t.tfld = rnan; \ } \ \ - putVSR(rD(opcode) + 32, &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -3077,25 +2981,22 @@ VSX_CVT_FP_TO_INT_VECTOR(xscvqpuwz, float128, uint32, f128, VsrD(0), 0x0ULL) * sfprf - set FPRF */ #define VSX_CVT_INT_TO_FP(op, nels, stp, ttp, sfld, tfld, sfprf, r2sp) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t t = *xt; \ int i; \ \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ - \ for (i = 0; i < nels; i++) { \ - xt.tfld = stp##_to_##ttp(xb.sfld, &env->fp_status); \ + t.tfld = stp##_to_##ttp(xb->sfld, &env->fp_status); \ if (r2sp) { \ - xt.tfld = helper_frsp(env, xt.tfld); \ + t.tfld = helper_frsp(env, t.tfld); \ } \ if (sfprf) { \ - helper_compute_fprf_float64(env, xt.tfld); \ + helper_compute_fprf_float64(env, t.tfld); \ } \ } \ \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -3121,17 +3022,15 @@ VSX_CVT_INT_TO_FP(xvcvuxwsp, 4, uint32, float32, VsrW(i), VsrW(i), 0, 0) * tfld - target vsr_t field */ #define VSX_CVT_INT_TO_FP_VECTOR(op, stp, ttp, sfld, tfld) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, uint32_t opcode, \ + ppc_vsr_t *xt, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t t = *xt; \ \ - getVSR(rB(opcode) + 32, &xb, env); \ - getVSR(rD(opcode) + 32, &xt, env); \ + t.tfld = stp##_to_##ttp(xb->sfld, &env->fp_status); \ + helper_compute_fprf_##ttp(env, t.tfld); \ \ - xt.tfld = stp##_to_##ttp(xb.sfld, &env->fp_status); \ - helper_compute_fprf_##ttp(env, xt.tfld); \ - \ - putVSR(xT(opcode) + 32, &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -3155,27 +3054,25 @@ VSX_CVT_INT_TO_FP_VECTOR(xscvudqp, uint64, float128, VsrD(0), f128) * sfprf - set FPRF */ #define VSX_ROUND(op, nels, tp, fld, rmode, sfprf) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb) \ { \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t t = *xt; \ int i; \ - getVSR(xB(opcode), &xb, env); \ - getVSR(xT(opcode), &xt, env); \ \ if (rmode != FLOAT_ROUND_CURRENT) { \ set_float_rounding_mode(rmode, &env->fp_status); \ } \ \ for (i = 0; i < nels; i++) { \ - if (unlikely(tp##_is_signaling_nan(xb.fld, \ + if (unlikely(tp##_is_signaling_nan(xb->fld, \ &env->fp_status))) { \ float_invalid_op_vxsnan(env, GETPC()); \ - xt.fld = tp##_snan_to_qnan(xb.fld); \ + t.fld = tp##_snan_to_qnan(xb->fld); \ } else { \ - xt.fld = tp##_round_to_int(xb.fld, &env->fp_status); \ + t.fld = tp##_round_to_int(xb->fld, &env->fp_status); \ } \ if (sfprf) { \ - helper_compute_fprf_float64(env, xt.fld); \ + helper_compute_fprf_float64(env, t.fld); \ } \ } \ \ @@ -3189,7 +3086,7 @@ void helper_##op(CPUPPCState *env, uint32_t opcode) \ env->fp_status.float_exception_flags &= ~float_flag_inexact; \ } \ \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ do_float_check_status(env, GETPC()); \ } @@ -3223,46 +3120,41 @@ uint64_t helper_xsrsp(CPUPPCState *env, uint64_t xb) } #define VSX_XXPERM(op, indexed) \ -void helper_##op(CPUPPCState *env, uint32_t opcode) \ +void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, \ + ppc_vsr_t *xa, ppc_vsr_t *pcv) \ { \ - ppc_vsr_t xt, xa, pcv, xto; \ + ppc_vsr_t t = *xt; \ int i, idx; \ \ - getVSR(xA(opcode), &xa, env); \ - getVSR(xT(opcode), &xt, env); \ - getVSR(xB(opcode), &pcv, env); \ - \ for (i = 0; i < 16; i++) { \ - idx = pcv.VsrB(i) & 0x1F; \ + idx = pcv->VsrB(i) & 0x1F; \ if (indexed) { \ idx = 31 - idx; \ } \ - xto.VsrB(i) = (idx <= 15) ? xa.VsrB(idx) : xt.VsrB(idx - 16); \ + t.VsrB(i) = (idx <= 15) ? xa->VsrB(idx) \ + : xt->VsrB(idx - 16); \ } \ - putVSR(xT(opcode), &xto, env); \ + *xt = t; \ } VSX_XXPERM(xxperm, 0) VSX_XXPERM(xxpermr, 1) -void helper_xvxsigsp(CPUPPCState *env, uint32_t opcode) +void helper_xvxsigsp(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb) { - ppc_vsr_t xt, xb; + ppc_vsr_t t = { }; uint32_t exp, i, fraction; - getVSR(xB(opcode), &xb, env); - memset(&xt, 0, sizeof(xt)); - for (i = 0; i < 4; i++) { - exp = (xb.VsrW(i) >> 23) & 0xFF; - fraction = xb.VsrW(i) & 0x7FFFFF; + exp = (xb->VsrW(i) >> 23) & 0xFF; + fraction = xb->VsrW(i) & 0x7FFFFF; if (exp != 0 && exp != 255) { - xt.VsrW(i) = fraction | 0x00800000; + t.VsrW(i) = fraction | 0x00800000; } else { - xt.VsrW(i) = fraction; + t.VsrW(i) = fraction; } } - putVSR(xT(opcode), &xt, env); + *xt = t; } /* @@ -3279,27 +3171,28 @@ void helper_xvxsigsp(CPUPPCState *env, uint32_t opcode) #define VSX_TEST_DC(op, nels, xbn, tp, fld, tfld, fld_max, scrf) \ void helper_##op(CPUPPCState *env, uint32_t opcode) \ { \ - ppc_vsr_t xt, xb; \ + ppc_vsr_t *xt = &env->vsr[xT(opcode)]; \ + ppc_vsr_t *xb = &env->vsr[xbn]; \ + ppc_vsr_t t = { }; \ uint32_t i, sign, dcmx; \ uint32_t cc, match = 0; \ \ - getVSR(xbn, &xb, env); \ if (!scrf) { \ - memset(&xt, 0, sizeof(xt)); \ dcmx = DCMX_XV(opcode); \ } else { \ + t = *xt; \ dcmx = DCMX(opcode); \ } \ \ for (i = 0; i < nels; i++) { \ - sign = tp##_is_neg(xb.fld); \ - if (tp##_is_any_nan(xb.fld)) { \ + sign = tp##_is_neg(xb->fld); \ + if (tp##_is_any_nan(xb->fld)) { \ match = extract32(dcmx, 6, 1); \ - } else if (tp##_is_infinity(xb.fld)) { \ + } else if (tp##_is_infinity(xb->fld)) { \ match = extract32(dcmx, 4 + !sign, 1); \ - } else if (tp##_is_zero(xb.fld)) { \ + } else if (tp##_is_zero(xb->fld)) { \ match = extract32(dcmx, 2 + !sign, 1); \ - } else if (tp##_is_zero_or_denormal(xb.fld)) { \ + } else if (tp##_is_zero_or_denormal(xb->fld)) { \ match = extract32(dcmx, 0 + !sign, 1); \ } \ \ @@ -3309,12 +3202,12 @@ void helper_##op(CPUPPCState *env, uint32_t opcode) \ env->fpscr |= cc << FPSCR_FPRF; \ env->crf[BF(opcode)] = cc; \ } else { \ - xt.tfld = match ? fld_max : 0; \ + t.tfld = match ? fld_max : 0; \ } \ match = 0; \ } \ if (!scrf) { \ - putVSR(xT(opcode), &xt, env); \ + *xt = t; \ } \ } @@ -3323,31 +3216,29 @@ VSX_TEST_DC(xvtstdcsp, 4, xB(opcode), float32, VsrW(i), VsrW(i), UINT32_MAX, 0) VSX_TEST_DC(xststdcdp, 1, xB(opcode), float64, VsrD(0), VsrD(0), 0, 1) VSX_TEST_DC(xststdcqp, 1, (rB(opcode) + 32), float128, f128, VsrD(0), 0, 1) -void helper_xststdcsp(CPUPPCState *env, uint32_t opcode) +void helper_xststdcsp(CPUPPCState *env, uint32_t opcode, ppc_vsr_t *xb) { - ppc_vsr_t xb; uint32_t dcmx, sign, exp; uint32_t cc, match = 0, not_sp = 0; - getVSR(xB(opcode), &xb, env); dcmx = DCMX(opcode); - exp = (xb.VsrD(0) >> 52) & 0x7FF; + exp = (xb->VsrD(0) >> 52) & 0x7FF; - sign = float64_is_neg(xb.VsrD(0)); - if (float64_is_any_nan(xb.VsrD(0))) { + sign = float64_is_neg(xb->VsrD(0)); + if (float64_is_any_nan(xb->VsrD(0))) { match = extract32(dcmx, 6, 1); - } else if (float64_is_infinity(xb.VsrD(0))) { + } else if (float64_is_infinity(xb->VsrD(0))) { match = extract32(dcmx, 4 + !sign, 1); - } else if (float64_is_zero(xb.VsrD(0))) { + } else if (float64_is_zero(xb->VsrD(0))) { match = extract32(dcmx, 2 + !sign, 1); - } else if (float64_is_zero_or_denormal(xb.VsrD(0)) || + } else if (float64_is_zero_or_denormal(xb->VsrD(0)) || (exp > 0 && exp < 0x381)) { match = extract32(dcmx, 0 + !sign, 1); } - not_sp = !float64_eq(xb.VsrD(0), + not_sp = !float64_eq(xb->VsrD(0), float32_to_float64( - float64_to_float32(xb.VsrD(0), &env->fp_status), + float64_to_float32(xb->VsrD(0), &env->fp_status), &env->fp_status), &env->fp_status); cc = sign << CRF_LT_BIT | match << CRF_EQ_BIT | not_sp << CRF_SO_BIT; @@ -3356,18 +3247,16 @@ void helper_xststdcsp(CPUPPCState *env, uint32_t opcode) env->crf[BF(opcode)] = cc; } -void helper_xsrqpi(CPUPPCState *env, uint32_t opcode) +void helper_xsrqpi(CPUPPCState *env, uint32_t opcode, + ppc_vsr_t *xt, ppc_vsr_t *xb) { - ppc_vsr_t xb; - ppc_vsr_t xt; + ppc_vsr_t t = { }; uint8_t r = Rrm(opcode); uint8_t ex = Rc(opcode); uint8_t rmc = RMC(opcode); uint8_t rmode = 0; float_status tstat; - getVSR(rB(opcode) + 32, &xb, env); - memset(&xt, 0, sizeof(xt)); helper_reset_fpstatus(env); if (r == 0 && rmc == 0) { @@ -3396,13 +3285,13 @@ void helper_xsrqpi(CPUPPCState *env, uint32_t opcode) tstat = env->fp_status; set_float_exception_flags(0, &tstat); set_float_rounding_mode(rmode, &tstat); - xt.f128 = float128_round_to_int(xb.f128, &tstat); + t.f128 = float128_round_to_int(xb->f128, &tstat); env->fp_status.float_exception_flags |= tstat.float_exception_flags; if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { - if (float128_is_signaling_nan(xb.f128, &tstat)) { + if (float128_is_signaling_nan(xb->f128, &tstat)) { float_invalid_op_vxsnan(env, GETPC()); - xt.f128 = float128_snan_to_qnan(xt.f128); + t.f128 = float128_snan_to_qnan(t.f128); } } @@ -3410,23 +3299,21 @@ void helper_xsrqpi(CPUPPCState *env, uint32_t opcode) env->fp_status.float_exception_flags &= ~float_flag_inexact; } - helper_compute_fprf_float128(env, xt.f128); + helper_compute_fprf_float128(env, t.f128); do_float_check_status(env, GETPC()); - putVSR(rD(opcode) + 32, &xt, env); + *xt = t; } -void helper_xsrqpxp(CPUPPCState *env, uint32_t opcode) +void helper_xsrqpxp(CPUPPCState *env, uint32_t opcode, + ppc_vsr_t *xt, ppc_vsr_t *xb) { - ppc_vsr_t xb; - ppc_vsr_t xt; + ppc_vsr_t t = { }; uint8_t r = Rrm(opcode); uint8_t rmc = RMC(opcode); uint8_t rmode = 0; floatx80 round_res; float_status tstat; - getVSR(rB(opcode) + 32, &xb, env); - memset(&xt, 0, sizeof(xt)); helper_reset_fpstatus(env); if (r == 0 && rmc == 0) { @@ -3455,30 +3342,28 @@ void helper_xsrqpxp(CPUPPCState *env, uint32_t opcode) tstat = env->fp_status; set_float_exception_flags(0, &tstat); set_float_rounding_mode(rmode, &tstat); - round_res = float128_to_floatx80(xb.f128, &tstat); - xt.f128 = floatx80_to_float128(round_res, &tstat); + round_res = float128_to_floatx80(xb->f128, &tstat); + t.f128 = floatx80_to_float128(round_res, &tstat); env->fp_status.float_exception_flags |= tstat.float_exception_flags; if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { - if (float128_is_signaling_nan(xb.f128, &tstat)) { + if (float128_is_signaling_nan(xb->f128, &tstat)) { float_invalid_op_vxsnan(env, GETPC()); - xt.f128 = float128_snan_to_qnan(xt.f128); + t.f128 = float128_snan_to_qnan(t.f128); } } - helper_compute_fprf_float128(env, xt.f128); - putVSR(rD(opcode) + 32, &xt, env); + helper_compute_fprf_float128(env, t.f128); + *xt = t; do_float_check_status(env, GETPC()); } -void helper_xssqrtqp(CPUPPCState *env, uint32_t opcode) +void helper_xssqrtqp(CPUPPCState *env, uint32_t opcode, + ppc_vsr_t *xt, ppc_vsr_t *xb) { - ppc_vsr_t xb; - ppc_vsr_t xt; + ppc_vsr_t t = { }; float_status tstat; - getVSR(rB(opcode) + 32, &xb, env); - memset(&xt, 0, sizeof(xt)); helper_reset_fpstatus(env); tstat = env->fp_status; @@ -3487,34 +3372,32 @@ void helper_xssqrtqp(CPUPPCState *env, uint32_t opcode) } set_float_exception_flags(0, &tstat); - xt.f128 = float128_sqrt(xb.f128, &tstat); + t.f128 = float128_sqrt(xb->f128, &tstat); env->fp_status.float_exception_flags |= tstat.float_exception_flags; if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { - if (float128_is_signaling_nan(xb.f128, &tstat)) { + if (float128_is_signaling_nan(xb->f128, &tstat)) { float_invalid_op_vxsnan(env, GETPC()); - xt.f128 = float128_snan_to_qnan(xb.f128); - } else if (float128_is_quiet_nan(xb.f128, &tstat)) { - xt.f128 = xb.f128; - } else if (float128_is_neg(xb.f128) && !float128_is_zero(xb.f128)) { + t.f128 = float128_snan_to_qnan(xb->f128); + } else if (float128_is_quiet_nan(xb->f128, &tstat)) { + t.f128 = xb->f128; + } else if (float128_is_neg(xb->f128) && !float128_is_zero(xb->f128)) { float_invalid_op_vxsqrt(env, 1, GETPC()); - xt.f128 = float128_default_nan(&env->fp_status); + t.f128 = float128_default_nan(&env->fp_status); } } - helper_compute_fprf_float128(env, xt.f128); - putVSR(rD(opcode) + 32, &xt, env); + helper_compute_fprf_float128(env, t.f128); + *xt = t; do_float_check_status(env, GETPC()); } -void helper_xssubqp(CPUPPCState *env, uint32_t opcode) +void helper_xssubqp(CPUPPCState *env, uint32_t opcode, + ppc_vsr_t *xt, ppc_vsr_t *xa, ppc_vsr_t *xb) { - ppc_vsr_t xt, xa, xb; + ppc_vsr_t t = *xt; float_status tstat; - getVSR(rA(opcode) + 32, &xa, env); - getVSR(rB(opcode) + 32, &xb, env); - getVSR(rD(opcode) + 32, &xt, env); helper_reset_fpstatus(env); tstat = env->fp_status; @@ -3523,16 +3406,16 @@ void helper_xssubqp(CPUPPCState *env, uint32_t opcode) } set_float_exception_flags(0, &tstat); - xt.f128 = float128_sub(xa.f128, xb.f128, &tstat); + t.f128 = float128_sub(xa->f128, xb->f128, &tstat); env->fp_status.float_exception_flags |= tstat.float_exception_flags; if (unlikely(tstat.float_exception_flags & float_flag_invalid)) { float_invalid_op_addsub(env, 1, GETPC(), - float128_classify(xa.f128) | - float128_classify(xb.f128)); + float128_classify(xa->f128) | + float128_classify(xb->f128)); } - helper_compute_fprf_float128(env, xt.f128); - putVSR(rD(opcode) + 32, &xt, env); + helper_compute_fprf_float128(env, t.f128); + *xt = t; do_float_check_status(env, GETPC()); } diff --git a/target/ppc/helper.h b/target/ppc/helper.h index 02b67a333e..380c9b1e2a 100644 --- a/target/ppc/helper.h +++ b/target/ppc/helper.h @@ -108,6 +108,10 @@ DEF_HELPER_FLAGS_1(ftsqrt, TCG_CALL_NO_RWG_SE, i32, i64) #define dh_ctype_avr ppc_avr_t * #define dh_is_signed_avr dh_is_signed_ptr +#define dh_alias_vsr ptr +#define dh_ctype_vsr ppc_vsr_t * +#define dh_is_signed_vsr dh_is_signed_ptr + DEF_HELPER_3(vavgub, void, avr, avr, avr) DEF_HELPER_3(vavguh, void, avr, avr, avr) DEF_HELPER_3(vavguw, void, avr, avr, avr) @@ -275,10 +279,10 @@ DEF_HELPER_3(stvebx, void, env, avr, tl) DEF_HELPER_3(stvehx, void, env, avr, tl) DEF_HELPER_3(stvewx, void, env, avr, tl) #if defined(TARGET_PPC64) -DEF_HELPER_4(lxvl, void, env, tl, tl, tl) -DEF_HELPER_4(lxvll, void, env, tl, tl, tl) -DEF_HELPER_4(stxvl, void, env, tl, tl, tl) -DEF_HELPER_4(stxvll, void, env, tl, tl, tl) +DEF_HELPER_4(lxvl, void, env, tl, vsr, tl) +DEF_HELPER_4(lxvll, void, env, tl, vsr, tl) +DEF_HELPER_4(stxvl, void, env, tl, vsr, tl) +DEF_HELPER_4(stxvll, void, env, tl, vsr, tl) #endif DEF_HELPER_4(vsumsws, void, env, avr, avr, avr) DEF_HELPER_4(vsum2sws, void, env, avr, avr, avr) @@ -361,178 +365,162 @@ DEF_HELPER_4(bcdsr, i32, avr, avr, avr, i32) DEF_HELPER_4(bcdtrunc, i32, avr, avr, avr, i32) DEF_HELPER_4(bcdutrunc, i32, avr, avr, avr, i32) -DEF_HELPER_2(xsadddp, void, env, i32) -DEF_HELPER_2(xsaddqp, void, env, i32) -DEF_HELPER_2(xssubdp, void, env, i32) -DEF_HELPER_2(xsmuldp, void, env, i32) -DEF_HELPER_2(xsmulqp, void, env, i32) -DEF_HELPER_2(xsdivdp, void, env, i32) -DEF_HELPER_2(xsdivqp, void, env, i32) -DEF_HELPER_2(xsredp, void, env, i32) -DEF_HELPER_2(xssqrtdp, void, env, i32) -DEF_HELPER_2(xsrsqrtedp, void, env, i32) -DEF_HELPER_2(xstdivdp, void, env, i32) -DEF_HELPER_2(xstsqrtdp, void, env, i32) -DEF_HELPER_2(xsmaddadp, void, env, i32) -DEF_HELPER_2(xsmaddmdp, void, env, i32) -DEF_HELPER_2(xsmsubadp, void, env, i32) -DEF_HELPER_2(xsmsubmdp, void, env, i32) -DEF_HELPER_2(xsnmaddadp, void, env, i32) -DEF_HELPER_2(xsnmaddmdp, void, env, i32) -DEF_HELPER_2(xsnmsubadp, void, env, i32) -DEF_HELPER_2(xsnmsubmdp, void, env, i32) -DEF_HELPER_2(xscmpeqdp, void, env, i32) -DEF_HELPER_2(xscmpgtdp, void, env, i32) -DEF_HELPER_2(xscmpgedp, void, env, i32) -DEF_HELPER_2(xscmpnedp, void, env, i32) -DEF_HELPER_2(xscmpexpdp, void, env, i32) -DEF_HELPER_2(xscmpexpqp, void, env, i32) -DEF_HELPER_2(xscmpodp, void, env, i32) -DEF_HELPER_2(xscmpudp, void, env, i32) -DEF_HELPER_2(xscmpoqp, void, env, i32) -DEF_HELPER_2(xscmpuqp, void, env, i32) -DEF_HELPER_2(xsmaxdp, void, env, i32) -DEF_HELPER_2(xsmindp, void, env, i32) -DEF_HELPER_2(xsmaxcdp, void, env, i32) -DEF_HELPER_2(xsmincdp, void, env, i32) -DEF_HELPER_2(xsmaxjdp, void, env, i32) -DEF_HELPER_2(xsminjdp, void, env, i32) -DEF_HELPER_2(xscvdphp, void, env, i32) -DEF_HELPER_2(xscvdpqp, void, env, i32) -DEF_HELPER_2(xscvdpsp, void, env, i32) +DEF_HELPER_4(xsadddp, void, env, vsr, vsr, vsr) +DEF_HELPER_5(xsaddqp, void, env, i32, vsr, vsr, vsr) +DEF_HELPER_4(xssubdp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xsmuldp, void, env, vsr, vsr, vsr) +DEF_HELPER_5(xsmulqp, void, env, i32, vsr, vsr, vsr) +DEF_HELPER_4(xsdivdp, void, env, vsr, vsr, vsr) +DEF_HELPER_5(xsdivqp, void, env, i32, vsr, vsr, vsr) +DEF_HELPER_3(xsredp, void, env, vsr, vsr) +DEF_HELPER_3(xssqrtdp, void, env, vsr, vsr) +DEF_HELPER_3(xsrsqrtedp, void, env, vsr, vsr) +DEF_HELPER_4(xstdivdp, void, env, i32, vsr, vsr) +DEF_HELPER_3(xstsqrtdp, void, env, i32, vsr) +DEF_HELPER_5(xsmadddp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xsmsubdp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xsnmadddp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xsnmsubdp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_4(xscmpeqdp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xscmpgtdp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xscmpgedp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xscmpnedp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xscmpexpdp, void, env, i32, vsr, vsr) +DEF_HELPER_4(xscmpexpqp, void, env, i32, vsr, vsr) +DEF_HELPER_4(xscmpodp, void, env, i32, vsr, vsr) +DEF_HELPER_4(xscmpudp, void, env, i32, vsr, vsr) +DEF_HELPER_4(xscmpoqp, void, env, i32, vsr, vsr) +DEF_HELPER_4(xscmpuqp, void, env, i32, vsr, vsr) +DEF_HELPER_4(xsmaxdp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xsmindp, void, env, vsr, vsr, vsr) +DEF_HELPER_5(xsmaxcdp, void, env, i32, vsr, vsr, vsr) +DEF_HELPER_5(xsmincdp, void, env, i32, vsr, vsr, vsr) +DEF_HELPER_5(xsmaxjdp, void, env, i32, vsr, vsr, vsr) +DEF_HELPER_5(xsminjdp, void, env, i32, vsr, vsr, vsr) +DEF_HELPER_3(xscvdphp, void, env, vsr, vsr) +DEF_HELPER_4(xscvdpqp, void, env, i32, vsr, vsr) +DEF_HELPER_3(xscvdpsp, void, env, vsr, vsr) DEF_HELPER_2(xscvdpspn, i64, env, i64) -DEF_HELPER_2(xscvqpdp, void, env, i32) -DEF_HELPER_2(xscvqpsdz, void, env, i32) -DEF_HELPER_2(xscvqpswz, void, env, i32) -DEF_HELPER_2(xscvqpudz, void, env, i32) -DEF_HELPER_2(xscvqpuwz, void, env, i32) -DEF_HELPER_2(xscvhpdp, void, env, i32) -DEF_HELPER_2(xscvsdqp, void, env, i32) -DEF_HELPER_2(xscvspdp, void, env, i32) +DEF_HELPER_4(xscvqpdp, void, env, i32, vsr, vsr) +DEF_HELPER_4(xscvqpsdz, void, env, i32, vsr, vsr) +DEF_HELPER_4(xscvqpswz, void, env, i32, vsr, vsr) +DEF_HELPER_4(xscvqpudz, void, env, i32, vsr, vsr) +DEF_HELPER_4(xscvqpuwz, void, env, i32, vsr, vsr) +DEF_HELPER_3(xscvhpdp, void, env, vsr, vsr) +DEF_HELPER_4(xscvsdqp, void, env, i32, vsr, vsr) +DEF_HELPER_3(xscvspdp, void, env, vsr, vsr) DEF_HELPER_2(xscvspdpn, i64, env, i64) -DEF_HELPER_2(xscvdpsxds, void, env, i32) -DEF_HELPER_2(xscvdpsxws, void, env, i32) -DEF_HELPER_2(xscvdpuxds, void, env, i32) -DEF_HELPER_2(xscvdpuxws, void, env, i32) -DEF_HELPER_2(xscvsxddp, void, env, i32) -DEF_HELPER_2(xscvuxdsp, void, env, i32) -DEF_HELPER_2(xscvsxdsp, void, env, i32) -DEF_HELPER_2(xscvudqp, void, env, i32) -DEF_HELPER_2(xscvuxddp, void, env, i32) -DEF_HELPER_2(xststdcsp, void, env, i32) +DEF_HELPER_3(xscvdpsxds, void, env, vsr, vsr) +DEF_HELPER_3(xscvdpsxws, void, env, vsr, vsr) +DEF_HELPER_3(xscvdpuxds, void, env, vsr, vsr) +DEF_HELPER_3(xscvdpuxws, void, env, vsr, vsr) +DEF_HELPER_3(xscvsxddp, void, env, vsr, vsr) +DEF_HELPER_3(xscvuxdsp, void, env, vsr, vsr) +DEF_HELPER_3(xscvsxdsp, void, env, vsr, vsr) +DEF_HELPER_4(xscvudqp, void, env, i32, vsr, vsr) +DEF_HELPER_3(xscvuxddp, void, env, vsr, vsr) +DEF_HELPER_3(xststdcsp, void, env, i32, vsr) DEF_HELPER_2(xststdcdp, void, env, i32) DEF_HELPER_2(xststdcqp, void, env, i32) -DEF_HELPER_2(xsrdpi, void, env, i32) -DEF_HELPER_2(xsrdpic, void, env, i32) -DEF_HELPER_2(xsrdpim, void, env, i32) -DEF_HELPER_2(xsrdpip, void, env, i32) -DEF_HELPER_2(xsrdpiz, void, env, i32) -DEF_HELPER_2(xsrqpi, void, env, i32) -DEF_HELPER_2(xsrqpxp, void, env, i32) -DEF_HELPER_2(xssqrtqp, void, env, i32) -DEF_HELPER_2(xssubqp, void, env, i32) +DEF_HELPER_3(xsrdpi, void, env, vsr, vsr) +DEF_HELPER_3(xsrdpic, void, env, vsr, vsr) +DEF_HELPER_3(xsrdpim, void, env, vsr, vsr) +DEF_HELPER_3(xsrdpip, void, env, vsr, vsr) +DEF_HELPER_3(xsrdpiz, void, env, vsr, vsr) +DEF_HELPER_4(xsrqpi, void, env, i32, vsr, vsr) +DEF_HELPER_4(xsrqpxp, void, env, i32, vsr, vsr) +DEF_HELPER_4(xssqrtqp, void, env, i32, vsr, vsr) +DEF_HELPER_5(xssubqp, void, env, i32, vsr, vsr, vsr) -DEF_HELPER_2(xsaddsp, void, env, i32) -DEF_HELPER_2(xssubsp, void, env, i32) -DEF_HELPER_2(xsmulsp, void, env, i32) -DEF_HELPER_2(xsdivsp, void, env, i32) -DEF_HELPER_2(xsresp, void, env, i32) +DEF_HELPER_4(xsaddsp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xssubsp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xsmulsp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xsdivsp, void, env, vsr, vsr, vsr) +DEF_HELPER_3(xsresp, void, env, vsr, vsr) DEF_HELPER_2(xsrsp, i64, env, i64) -DEF_HELPER_2(xssqrtsp, void, env, i32) -DEF_HELPER_2(xsrsqrtesp, void, env, i32) -DEF_HELPER_2(xsmaddasp, void, env, i32) -DEF_HELPER_2(xsmaddmsp, void, env, i32) -DEF_HELPER_2(xsmsubasp, void, env, i32) -DEF_HELPER_2(xsmsubmsp, void, env, i32) -DEF_HELPER_2(xsnmaddasp, void, env, i32) -DEF_HELPER_2(xsnmaddmsp, void, env, i32) -DEF_HELPER_2(xsnmsubasp, void, env, i32) -DEF_HELPER_2(xsnmsubmsp, void, env, i32) +DEF_HELPER_3(xssqrtsp, void, env, vsr, vsr) +DEF_HELPER_3(xsrsqrtesp, void, env, vsr, vsr) +DEF_HELPER_5(xsmaddsp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xsmsubsp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xsnmaddsp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xsnmsubsp, void, env, vsr, vsr, vsr, vsr) -DEF_HELPER_2(xvadddp, void, env, i32) -DEF_HELPER_2(xvsubdp, void, env, i32) -DEF_HELPER_2(xvmuldp, void, env, i32) -DEF_HELPER_2(xvdivdp, void, env, i32) -DEF_HELPER_2(xvredp, void, env, i32) -DEF_HELPER_2(xvsqrtdp, void, env, i32) -DEF_HELPER_2(xvrsqrtedp, void, env, i32) -DEF_HELPER_2(xvtdivdp, void, env, i32) -DEF_HELPER_2(xvtsqrtdp, void, env, i32) -DEF_HELPER_2(xvmaddadp, void, env, i32) -DEF_HELPER_2(xvmaddmdp, void, env, i32) -DEF_HELPER_2(xvmsubadp, void, env, i32) -DEF_HELPER_2(xvmsubmdp, void, env, i32) -DEF_HELPER_2(xvnmaddadp, void, env, i32) -DEF_HELPER_2(xvnmaddmdp, void, env, i32) -DEF_HELPER_2(xvnmsubadp, void, env, i32) -DEF_HELPER_2(xvnmsubmdp, void, env, i32) -DEF_HELPER_2(xvmaxdp, void, env, i32) -DEF_HELPER_2(xvmindp, void, env, i32) -DEF_HELPER_2(xvcmpeqdp, void, env, i32) -DEF_HELPER_2(xvcmpgedp, void, env, i32) -DEF_HELPER_2(xvcmpgtdp, void, env, i32) -DEF_HELPER_2(xvcmpnedp, void, env, i32) -DEF_HELPER_2(xvcvdpsp, void, env, i32) -DEF_HELPER_2(xvcvdpsxds, void, env, i32) -DEF_HELPER_2(xvcvdpsxws, void, env, i32) -DEF_HELPER_2(xvcvdpuxds, void, env, i32) -DEF_HELPER_2(xvcvdpuxws, void, env, i32) -DEF_HELPER_2(xvcvsxddp, void, env, i32) -DEF_HELPER_2(xvcvuxddp, void, env, i32) -DEF_HELPER_2(xvcvsxwdp, void, env, i32) -DEF_HELPER_2(xvcvuxwdp, void, env, i32) -DEF_HELPER_2(xvrdpi, void, env, i32) -DEF_HELPER_2(xvrdpic, void, env, i32) -DEF_HELPER_2(xvrdpim, void, env, i32) -DEF_HELPER_2(xvrdpip, void, env, i32) -DEF_HELPER_2(xvrdpiz, void, env, i32) +DEF_HELPER_4(xvadddp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xvsubdp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xvmuldp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xvdivdp, void, env, vsr, vsr, vsr) +DEF_HELPER_3(xvredp, void, env, vsr, vsr) +DEF_HELPER_3(xvsqrtdp, void, env, vsr, vsr) +DEF_HELPER_3(xvrsqrtedp, void, env, vsr, vsr) +DEF_HELPER_4(xvtdivdp, void, env, i32, vsr, vsr) +DEF_HELPER_3(xvtsqrtdp, void, env, i32, vsr) +DEF_HELPER_5(xvmadddp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xvmsubdp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xvnmadddp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xvnmsubdp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_4(xvmaxdp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xvmindp, void, env, vsr, vsr, vsr) +DEF_HELPER_FLAGS_4(xvcmpeqdp, TCG_CALL_NO_RWG, i32, env, vsr, vsr, vsr) +DEF_HELPER_FLAGS_4(xvcmpgedp, TCG_CALL_NO_RWG, i32, env, vsr, vsr, vsr) +DEF_HELPER_FLAGS_4(xvcmpgtdp, TCG_CALL_NO_RWG, i32, env, vsr, vsr, vsr) +DEF_HELPER_FLAGS_4(xvcmpnedp, TCG_CALL_NO_RWG, i32, env, vsr, vsr, vsr) +DEF_HELPER_3(xvcvdpsp, void, env, vsr, vsr) +DEF_HELPER_3(xvcvdpsxds, void, env, vsr, vsr) +DEF_HELPER_3(xvcvdpsxws, void, env, vsr, vsr) +DEF_HELPER_3(xvcvdpuxds, void, env, vsr, vsr) +DEF_HELPER_3(xvcvdpuxws, void, env, vsr, vsr) +DEF_HELPER_3(xvcvsxddp, void, env, vsr, vsr) +DEF_HELPER_3(xvcvuxddp, void, env, vsr, vsr) +DEF_HELPER_3(xvcvsxwdp, void, env, vsr, vsr) +DEF_HELPER_3(xvcvuxwdp, void, env, vsr, vsr) +DEF_HELPER_3(xvrdpi, void, env, vsr, vsr) +DEF_HELPER_3(xvrdpic, void, env, vsr, vsr) +DEF_HELPER_3(xvrdpim, void, env, vsr, vsr) +DEF_HELPER_3(xvrdpip, void, env, vsr, vsr) +DEF_HELPER_3(xvrdpiz, void, env, vsr, vsr) -DEF_HELPER_2(xvaddsp, void, env, i32) -DEF_HELPER_2(xvsubsp, void, env, i32) -DEF_HELPER_2(xvmulsp, void, env, i32) -DEF_HELPER_2(xvdivsp, void, env, i32) -DEF_HELPER_2(xvresp, void, env, i32) -DEF_HELPER_2(xvsqrtsp, void, env, i32) -DEF_HELPER_2(xvrsqrtesp, void, env, i32) -DEF_HELPER_2(xvtdivsp, void, env, i32) -DEF_HELPER_2(xvtsqrtsp, void, env, i32) -DEF_HELPER_2(xvmaddasp, void, env, i32) -DEF_HELPER_2(xvmaddmsp, void, env, i32) -DEF_HELPER_2(xvmsubasp, void, env, i32) -DEF_HELPER_2(xvmsubmsp, void, env, i32) -DEF_HELPER_2(xvnmaddasp, void, env, i32) -DEF_HELPER_2(xvnmaddmsp, void, env, i32) -DEF_HELPER_2(xvnmsubasp, void, env, i32) -DEF_HELPER_2(xvnmsubmsp, void, env, i32) -DEF_HELPER_2(xvmaxsp, void, env, i32) -DEF_HELPER_2(xvminsp, void, env, i32) -DEF_HELPER_2(xvcmpeqsp, void, env, i32) -DEF_HELPER_2(xvcmpgesp, void, env, i32) -DEF_HELPER_2(xvcmpgtsp, void, env, i32) -DEF_HELPER_2(xvcmpnesp, void, env, i32) -DEF_HELPER_2(xvcvspdp, void, env, i32) -DEF_HELPER_2(xvcvsphp, void, env, i32) -DEF_HELPER_2(xvcvhpsp, void, env, i32) -DEF_HELPER_2(xvcvspsxds, void, env, i32) -DEF_HELPER_2(xvcvspsxws, void, env, i32) -DEF_HELPER_2(xvcvspuxds, void, env, i32) -DEF_HELPER_2(xvcvspuxws, void, env, i32) -DEF_HELPER_2(xvcvsxdsp, void, env, i32) -DEF_HELPER_2(xvcvuxdsp, void, env, i32) -DEF_HELPER_2(xvcvsxwsp, void, env, i32) -DEF_HELPER_2(xvcvuxwsp, void, env, i32) +DEF_HELPER_4(xvaddsp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xvsubsp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xvmulsp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xvdivsp, void, env, vsr, vsr, vsr) +DEF_HELPER_3(xvresp, void, env, vsr, vsr) +DEF_HELPER_3(xvsqrtsp, void, env, vsr, vsr) +DEF_HELPER_3(xvrsqrtesp, void, env, vsr, vsr) +DEF_HELPER_4(xvtdivsp, void, env, i32, vsr, vsr) +DEF_HELPER_3(xvtsqrtsp, void, env, i32, vsr) +DEF_HELPER_5(xvmaddsp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xvmsubsp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xvnmaddsp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_5(xvnmsubsp, void, env, vsr, vsr, vsr, vsr) +DEF_HELPER_4(xvmaxsp, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xvminsp, void, env, vsr, vsr, vsr) +DEF_HELPER_FLAGS_4(xvcmpeqsp, TCG_CALL_NO_RWG, i32, env, vsr, vsr, vsr) +DEF_HELPER_FLAGS_4(xvcmpgesp, TCG_CALL_NO_RWG, i32, env, vsr, vsr, vsr) +DEF_HELPER_FLAGS_4(xvcmpgtsp, TCG_CALL_NO_RWG, i32, env, vsr, vsr, vsr) +DEF_HELPER_FLAGS_4(xvcmpnesp, TCG_CALL_NO_RWG, i32, env, vsr, vsr, vsr) +DEF_HELPER_3(xvcvspdp, void, env, vsr, vsr) +DEF_HELPER_3(xvcvsphp, void, env, vsr, vsr) +DEF_HELPER_3(xvcvhpsp, void, env, vsr, vsr) +DEF_HELPER_3(xvcvspsxds, void, env, vsr, vsr) +DEF_HELPER_3(xvcvspsxws, void, env, vsr, vsr) +DEF_HELPER_3(xvcvspuxds, void, env, vsr, vsr) +DEF_HELPER_3(xvcvspuxws, void, env, vsr, vsr) +DEF_HELPER_3(xvcvsxdsp, void, env, vsr, vsr) +DEF_HELPER_3(xvcvuxdsp, void, env, vsr, vsr) +DEF_HELPER_3(xvcvsxwsp, void, env, vsr, vsr) +DEF_HELPER_3(xvcvuxwsp, void, env, vsr, vsr) DEF_HELPER_2(xvtstdcsp, void, env, i32) DEF_HELPER_2(xvtstdcdp, void, env, i32) -DEF_HELPER_2(xvrspi, void, env, i32) -DEF_HELPER_2(xvrspic, void, env, i32) -DEF_HELPER_2(xvrspim, void, env, i32) -DEF_HELPER_2(xvrspip, void, env, i32) -DEF_HELPER_2(xvrspiz, void, env, i32) -DEF_HELPER_2(xxperm, void, env, i32) -DEF_HELPER_2(xxpermr, void, env, i32) -DEF_HELPER_4(xxextractuw, void, env, tl, tl, i32) -DEF_HELPER_4(xxinsertw, void, env, tl, tl, i32) -DEF_HELPER_2(xvxsigsp, void, env, i32) +DEF_HELPER_3(xvrspi, void, env, vsr, vsr) +DEF_HELPER_3(xvrspic, void, env, vsr, vsr) +DEF_HELPER_3(xvrspim, void, env, vsr, vsr) +DEF_HELPER_3(xvrspip, void, env, vsr, vsr) +DEF_HELPER_3(xvrspiz, void, env, vsr, vsr) +DEF_HELPER_4(xxperm, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xxpermr, void, env, vsr, vsr, vsr) +DEF_HELPER_4(xxextractuw, void, env, vsr, vsr, i32) +DEF_HELPER_4(xxinsertw, void, env, vsr, vsr, i32) +DEF_HELPER_3(xvxsigsp, void, env, vsr, vsr) DEF_HELPER_2(efscfsi, i32, env, i32) DEF_HELPER_2(efscfui, i32, env, i32) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index 8ce89f2ad9..5c07ef3e4d 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -1899,41 +1899,35 @@ VEXTRACT(uw, u32) VEXTRACT(d, u64) #undef VEXTRACT -void helper_xxextractuw(CPUPPCState *env, target_ulong xtn, - target_ulong xbn, uint32_t index) +void helper_xxextractuw(CPUPPCState *env, ppc_vsr_t *xt, + ppc_vsr_t *xb, uint32_t index) { - ppc_vsr_t xt, xb; + ppc_vsr_t t = { }; size_t es = sizeof(uint32_t); uint32_t ext_index; int i; - getVSR(xbn, &xb, env); - memset(&xt, 0, sizeof(xt)); - ext_index = index; for (i = 0; i < es; i++, ext_index++) { - xt.VsrB(8 - es + i) = xb.VsrB(ext_index % 16); + t.VsrB(8 - es + i) = xb->VsrB(ext_index % 16); } - putVSR(xtn, &xt, env); + *xt = t; } -void helper_xxinsertw(CPUPPCState *env, target_ulong xtn, - target_ulong xbn, uint32_t index) +void helper_xxinsertw(CPUPPCState *env, ppc_vsr_t *xt, + ppc_vsr_t *xb, uint32_t index) { - ppc_vsr_t xt, xb; + ppc_vsr_t t = *xt; size_t es = sizeof(uint32_t); int ins_index, i = 0; - getVSR(xbn, &xb, env); - getVSR(xtn, &xt, env); - ins_index = index; for (i = 0; i < es && ins_index < 16; i++, ins_index++) { - xt.VsrB(ins_index) = xb.VsrB(8 - es + i); + t.VsrB(ins_index) = xb->VsrB(8 - es + i); } - putVSR(xtn, &xt, env); + *xt = t; } #define VEXT_SIGNED(name, element, cast) \ diff --git a/target/ppc/internal.h b/target/ppc/internal.h index fb6f64ed1e..d3d327e548 100644 --- a/target/ppc/internal.h +++ b/target/ppc/internal.h @@ -204,18 +204,6 @@ EXTRACT_HELPER(IMM8, 11, 8); EXTRACT_HELPER(DCMX, 16, 7); EXTRACT_HELPER_SPLIT_3(DCMX_XV, 5, 16, 0, 1, 2, 5, 1, 6, 6); -static inline void getVSR(int n, ppc_vsr_t *vsr, CPUPPCState *env) -{ - vsr->VsrD(0) = env->vsr[n].VsrD(0); - vsr->VsrD(1) = env->vsr[n].VsrD(1); -} - -static inline void putVSR(int n, ppc_vsr_t *vsr, CPUPPCState *env) -{ - env->vsr[n].VsrD(0) = vsr->VsrD(0); - env->vsr[n].VsrD(1) = vsr->VsrD(1); -} - void helper_compute_fprf_float16(CPUPPCState *env, float16 arg); void helper_compute_fprf_float32(CPUPPCState *env, float32 arg); void helper_compute_fprf_float128(CPUPPCState *env, float128 arg); diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c index 4b4989c0af..8a06d3171e 100644 --- a/target/ppc/kvm.c +++ b/target/ppc/kvm.c @@ -2650,7 +2650,7 @@ int kvmppc_define_rtas_kernel_token(uint32_t token, const char *function) return -ENOENT; } - strncpy(args.name, function, sizeof(args.name)); + strncpy(args.name, function, sizeof(args.name) - 1); return kvm_vm_ioctl(kvm_state, KVM_PPC_RTAS_DEFINE_TOKEN, &args); } @@ -2944,3 +2944,12 @@ void kvmppc_set_reg_ppc_online(PowerPCCPU *cpu, unsigned int online) kvm_set_one_reg(cs, KVM_REG_PPC_ONLINE, &online); } } + +void kvmppc_set_reg_tb_offset(PowerPCCPU *cpu, int64_t tb_offset) +{ + CPUState *cs = CPU(cpu); + + if (kvm_enabled()) { + kvm_set_one_reg(cs, KVM_REG_PPC_TB_OFFSET, &tb_offset); + } +} diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h index 45776cad79..98bd7d5da6 100644 --- a/target/ppc/kvm_ppc.h +++ b/target/ppc/kvm_ppc.h @@ -80,6 +80,7 @@ bool kvmppc_pvr_workaround_required(PowerPCCPU *cpu); bool kvmppc_hpt_needs_host_contiguous_pages(void); void kvm_check_mmu(PowerPCCPU *cpu, Error **errp); void kvmppc_set_reg_ppc_online(PowerPCCPU *cpu, unsigned int online); +void kvmppc_set_reg_tb_offset(PowerPCCPU *cpu, int64_t tb_offset); #else @@ -206,6 +207,10 @@ static inline void kvmppc_set_reg_ppc_online(PowerPCCPU *cpu, return; } +static inline void kvmppc_set_reg_tb_offset(PowerPCCPU *cpu, int64_t tb_offset) +{ +} + #ifndef CONFIG_USER_ONLY static inline bool kvmppc_spapr_use_multitce(void) { @@ -394,6 +399,11 @@ static inline int kvmppc_resize_hpt_commit(PowerPCCPU *cpu, return -ENOSYS; } +static inline bool kvmppc_pvr_workaround_required(PowerPCCPU *cpu) +{ + return false; +} + #endif #ifndef CONFIG_KVM diff --git a/target/ppc/machine.c b/target/ppc/machine.c index 5ad7b40f45..e82f5de9db 100644 --- a/target/ppc/machine.c +++ b/target/ppc/machine.c @@ -378,11 +378,9 @@ static int cpu_post_load(void *opaque, int version_id) * receive the PVR it expects as a workaround. * */ -#if defined(CONFIG_KVM) if (kvmppc_pvr_workaround_required(cpu)) { env->spr[SPR_PVR] = env->spr_cb[SPR_PVR].default_value; } -#endif env->lr = env->spr[SPR_LR]; env->ctr = env->spr[SPR_CTR]; diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c index 5b0f9ee50d..6f4ffa3661 100644 --- a/target/ppc/mem_helper.c +++ b/target/ppc/mem_helper.c @@ -415,28 +415,28 @@ STVE(stvewx, cpu_stl_data_ra, bswap32, u32) #define VSX_LXVL(name, lj) \ void helper_##name(CPUPPCState *env, target_ulong addr, \ - target_ulong xt_num, target_ulong rb) \ + ppc_vsr_t *xt, target_ulong rb) \ { \ - int i; \ - ppc_vsr_t xt; \ + ppc_vsr_t t; \ uint64_t nb = GET_NB(rb); \ + int i; \ \ - xt.s128 = int128_zero(); \ + t.s128 = int128_zero(); \ if (nb) { \ nb = (nb >= 16) ? 16 : nb; \ if (msr_le && !lj) { \ for (i = 16; i > 16 - nb; i--) { \ - xt.VsrB(i - 1) = cpu_ldub_data_ra(env, addr, GETPC()); \ + t.VsrB(i - 1) = cpu_ldub_data_ra(env, addr, GETPC()); \ addr = addr_add(env, addr, 1); \ } \ } else { \ for (i = 0; i < nb; i++) { \ - xt.VsrB(i) = cpu_ldub_data_ra(env, addr, GETPC()); \ + t.VsrB(i) = cpu_ldub_data_ra(env, addr, GETPC()); \ addr = addr_add(env, addr, 1); \ } \ } \ } \ - putVSR(xt_num, &xt, env); \ + *xt = t; \ } VSX_LXVL(lxvl, 0) @@ -445,25 +445,24 @@ VSX_LXVL(lxvll, 1) #define VSX_STXVL(name, lj) \ void helper_##name(CPUPPCState *env, target_ulong addr, \ - target_ulong xt_num, target_ulong rb) \ + ppc_vsr_t *xt, target_ulong rb) \ { \ - int i; \ - ppc_vsr_t xt; \ target_ulong nb = GET_NB(rb); \ + int i; \ \ if (!nb) { \ return; \ } \ - getVSR(xt_num, &xt, env); \ + \ nb = (nb >= 16) ? 16 : nb; \ if (msr_le && !lj) { \ for (i = 16; i > 16 - nb; i--) { \ - cpu_stb_data_ra(env, addr, xt.VsrB(i - 1), GETPC()); \ + cpu_stb_data_ra(env, addr, xt->VsrB(i - 1), GETPC()); \ addr = addr_add(env, addr, 1); \ } \ } else { \ for (i = 0; i < nb; i++) { \ - cpu_stb_data_ra(env, addr, xt.VsrB(i), GETPC()); \ + cpu_stb_data_ra(env, addr, xt->VsrB(i), GETPC()); \ addr = addr_add(env, addr, 1); \ } \ } \ diff --git a/target/ppc/translate/vsx-impl.inc.c b/target/ppc/translate/vsx-impl.inc.c index e9b7562f84..3922686ad6 100644 --- a/target/ppc/translate/vsx-impl.inc.c +++ b/target/ppc/translate/vsx-impl.inc.c @@ -20,6 +20,13 @@ static inline void set_cpu_vsrl(int n, TCGv_i64 src) tcg_gen_st_i64(src, cpu_env, vsr64_offset(n, false)); } +static inline TCGv_ptr gen_vsr_ptr(int reg) +{ + TCGv_ptr r = tcg_temp_new_ptr(); + tcg_gen_addi_ptr(r, cpu_env, vsr_full_offset(reg)); + return r; +} + #define VSX_LOAD_SCALAR(name, operation) \ static void gen_##name(DisasContext *ctx) \ { \ @@ -337,29 +344,30 @@ VSX_VECTOR_STORE(stxv, st_i64, 0) VSX_VECTOR_STORE(stxvx, st_i64, 1) #ifdef TARGET_PPC64 -#define VSX_VECTOR_LOAD_STORE_LENGTH(name) \ -static void gen_##name(DisasContext *ctx) \ -{ \ - TCGv EA, xt; \ - \ - if (xT(ctx->opcode) < 32) { \ - if (unlikely(!ctx->vsx_enabled)) { \ - gen_exception(ctx, POWERPC_EXCP_VSXU); \ - return; \ - } \ - } else { \ - if (unlikely(!ctx->altivec_enabled)) { \ - gen_exception(ctx, POWERPC_EXCP_VPU); \ - return; \ - } \ - } \ - EA = tcg_temp_new(); \ - xt = tcg_const_tl(xT(ctx->opcode)); \ - gen_set_access_type(ctx, ACCESS_INT); \ - gen_addr_register(ctx, EA); \ - gen_helper_##name(cpu_env, EA, xt, cpu_gpr[rB(ctx->opcode)]); \ - tcg_temp_free(EA); \ - tcg_temp_free(xt); \ +#define VSX_VECTOR_LOAD_STORE_LENGTH(name) \ +static void gen_##name(DisasContext *ctx) \ +{ \ + TCGv EA; \ + TCGv_ptr xt; \ + \ + if (xT(ctx->opcode) < 32) { \ + if (unlikely(!ctx->vsx_enabled)) { \ + gen_exception(ctx, POWERPC_EXCP_VSXU); \ + return; \ + } \ + } else { \ + if (unlikely(!ctx->altivec_enabled)) { \ + gen_exception(ctx, POWERPC_EXCP_VPU); \ + return; \ + } \ + } \ + EA = tcg_temp_new(); \ + xt = gen_vsr_ptr(xT(ctx->opcode)); \ + gen_set_access_type(ctx, ACCESS_INT); \ + gen_addr_register(ctx, EA); \ + gen_helper_##name(cpu_env, EA, xt, cpu_gpr[rB(ctx->opcode)]); \ + tcg_temp_free(EA); \ + tcg_temp_free_ptr(xt); \ } VSX_VECTOR_LOAD_STORE_LENGTH(lxvl) @@ -958,6 +966,57 @@ VSX_VECTOR_MOVE(xvnabssp, OP_NABS, SGN_MASK_SP) VSX_VECTOR_MOVE(xvnegsp, OP_NEG, SGN_MASK_SP) VSX_VECTOR_MOVE(xvcpsgnsp, OP_CPSGN, SGN_MASK_SP) +#define VSX_CMP(name, op1, op2, inval, type) \ +static void gen_##name(DisasContext *ctx) \ +{ \ + TCGv_i32 ignored; \ + TCGv_ptr xt, xa, xb; \ + if (unlikely(!ctx->vsx_enabled)) { \ + gen_exception(ctx, POWERPC_EXCP_VSXU); \ + return; \ + } \ + xt = gen_vsr_ptr(xT(ctx->opcode)); \ + xa = gen_vsr_ptr(xA(ctx->opcode)); \ + xb = gen_vsr_ptr(xB(ctx->opcode)); \ + if ((ctx->opcode >> (31 - 21)) & 1) { \ + gen_helper_##name(cpu_crf[6], cpu_env, xt, xa, xb); \ + } else { \ + ignored = tcg_temp_new_i32(); \ + gen_helper_##name(ignored, cpu_env, xt, xa, xb); \ + tcg_temp_free_i32(ignored); \ + } \ + gen_helper_float_check_status(cpu_env); \ + tcg_temp_free_ptr(xt); \ + tcg_temp_free_ptr(xa); \ + tcg_temp_free_ptr(xb); \ +} + +VSX_CMP(xvcmpeqdp, 0x0C, 0x0C, 0, PPC2_VSX) +VSX_CMP(xvcmpgedp, 0x0C, 0x0E, 0, PPC2_VSX) +VSX_CMP(xvcmpgtdp, 0x0C, 0x0D, 0, PPC2_VSX) +VSX_CMP(xvcmpnedp, 0x0C, 0x0F, 0, PPC2_ISA300) +VSX_CMP(xvcmpeqsp, 0x0C, 0x08, 0, PPC2_VSX) +VSX_CMP(xvcmpgesp, 0x0C, 0x0A, 0, PPC2_VSX) +VSX_CMP(xvcmpgtsp, 0x0C, 0x09, 0, PPC2_VSX) +VSX_CMP(xvcmpnesp, 0x0C, 0x0B, 0, PPC2_VSX) + +static void gen_xscvqpdp(DisasContext *ctx) +{ + TCGv_i32 opc; + TCGv_ptr xt, xb; + if (unlikely(!ctx->vsx_enabled)) { + gen_exception(ctx, POWERPC_EXCP_VSXU); + return; + } + opc = tcg_const_i32(ctx->opcode); + xt = gen_vsr_ptr(xT(ctx->opcode)); + xb = gen_vsr_ptr(xB(ctx->opcode)); + gen_helper_xscvqpdp(cpu_env, opc, xt, xb); + tcg_temp_free_i32(opc); + tcg_temp_free_ptr(xt); + tcg_temp_free_ptr(xb); +} + #define GEN_VSX_HELPER_2(name, op1, op2, inval, type) \ static void gen_##name(DisasContext *ctx) \ { \ @@ -971,6 +1030,128 @@ static void gen_##name(DisasContext *ctx) \ tcg_temp_free_i32(opc); \ } +#define GEN_VSX_HELPER_X3(name, op1, op2, inval, type) \ +static void gen_##name(DisasContext *ctx) \ +{ \ + TCGv_ptr xt, xa, xb; \ + if (unlikely(!ctx->vsx_enabled)) { \ + gen_exception(ctx, POWERPC_EXCP_VSXU); \ + return; \ + } \ + xt = gen_vsr_ptr(xT(ctx->opcode)); \ + xa = gen_vsr_ptr(xA(ctx->opcode)); \ + xb = gen_vsr_ptr(xB(ctx->opcode)); \ + gen_helper_##name(cpu_env, xt, xa, xb); \ + tcg_temp_free_ptr(xt); \ + tcg_temp_free_ptr(xa); \ + tcg_temp_free_ptr(xb); \ +} + +#define GEN_VSX_HELPER_X2(name, op1, op2, inval, type) \ +static void gen_##name(DisasContext *ctx) \ +{ \ + TCGv_ptr xt, xb; \ + if (unlikely(!ctx->vsx_enabled)) { \ + gen_exception(ctx, POWERPC_EXCP_VSXU); \ + return; \ + } \ + xt = gen_vsr_ptr(xT(ctx->opcode)); \ + xb = gen_vsr_ptr(xB(ctx->opcode)); \ + gen_helper_##name(cpu_env, xt, xb); \ + tcg_temp_free_ptr(xt); \ + tcg_temp_free_ptr(xb); \ +} + +#define GEN_VSX_HELPER_X2_AB(name, op1, op2, inval, type) \ +static void gen_##name(DisasContext *ctx) \ +{ \ + TCGv_i32 opc; \ + TCGv_ptr xa, xb; \ + if (unlikely(!ctx->vsx_enabled)) { \ + gen_exception(ctx, POWERPC_EXCP_VSXU); \ + return; \ + } \ + opc = tcg_const_i32(ctx->opcode); \ + xa = gen_vsr_ptr(xA(ctx->opcode)); \ + xb = gen_vsr_ptr(xB(ctx->opcode)); \ + gen_helper_##name(cpu_env, opc, xa, xb); \ + tcg_temp_free_i32(opc); \ + tcg_temp_free_ptr(xa); \ + tcg_temp_free_ptr(xb); \ +} + +#define GEN_VSX_HELPER_X1(name, op1, op2, inval, type) \ +static void gen_##name(DisasContext *ctx) \ +{ \ + TCGv_i32 opc; \ + TCGv_ptr xb; \ + if (unlikely(!ctx->vsx_enabled)) { \ + gen_exception(ctx, POWERPC_EXCP_VSXU); \ + return; \ + } \ + opc = tcg_const_i32(ctx->opcode); \ + xb = gen_vsr_ptr(xB(ctx->opcode)); \ + gen_helper_##name(cpu_env, opc, xb); \ + tcg_temp_free_i32(opc); \ + tcg_temp_free_ptr(xb); \ +} + +#define GEN_VSX_HELPER_R3(name, op1, op2, inval, type) \ +static void gen_##name(DisasContext *ctx) \ +{ \ + TCGv_i32 opc; \ + TCGv_ptr xt, xa, xb; \ + if (unlikely(!ctx->vsx_enabled)) { \ + gen_exception(ctx, POWERPC_EXCP_VSXU); \ + return; \ + } \ + opc = tcg_const_i32(ctx->opcode); \ + xt = gen_vsr_ptr(rD(ctx->opcode) + 32); \ + xa = gen_vsr_ptr(rA(ctx->opcode) + 32); \ + xb = gen_vsr_ptr(rB(ctx->opcode) + 32); \ + gen_helper_##name(cpu_env, opc, xt, xa, xb); \ + tcg_temp_free_i32(opc); \ + tcg_temp_free_ptr(xt); \ + tcg_temp_free_ptr(xa); \ + tcg_temp_free_ptr(xb); \ +} + +#define GEN_VSX_HELPER_R2(name, op1, op2, inval, type) \ +static void gen_##name(DisasContext *ctx) \ +{ \ + TCGv_i32 opc; \ + TCGv_ptr xt, xb; \ + if (unlikely(!ctx->vsx_enabled)) { \ + gen_exception(ctx, POWERPC_EXCP_VSXU); \ + return; \ + } \ + opc = tcg_const_i32(ctx->opcode); \ + xt = gen_vsr_ptr(rD(ctx->opcode) + 32); \ + xb = gen_vsr_ptr(rB(ctx->opcode) + 32); \ + gen_helper_##name(cpu_env, opc, xt, xb); \ + tcg_temp_free_i32(opc); \ + tcg_temp_free_ptr(xt); \ + tcg_temp_free_ptr(xb); \ +} + +#define GEN_VSX_HELPER_R2_AB(name, op1, op2, inval, type) \ +static void gen_##name(DisasContext *ctx) \ +{ \ + TCGv_i32 opc; \ + TCGv_ptr xa, xb; \ + if (unlikely(!ctx->vsx_enabled)) { \ + gen_exception(ctx, POWERPC_EXCP_VSXU); \ + return; \ + } \ + opc = tcg_const_i32(ctx->opcode); \ + xa = gen_vsr_ptr(rA(ctx->opcode) + 32); \ + xb = gen_vsr_ptr(rB(ctx->opcode) + 32); \ + gen_helper_##name(cpu_env, opc, xa, xb); \ + tcg_temp_free_i32(opc); \ + tcg_temp_free_ptr(xa); \ + tcg_temp_free_ptr(xb); \ +} + #define GEN_VSX_HELPER_XT_XB_ENV(name, op1, op2, inval, type) \ static void gen_##name(DisasContext *ctx) \ { \ @@ -989,176 +1170,180 @@ static void gen_##name(DisasContext *ctx) \ tcg_temp_free_i64(t1); \ } -GEN_VSX_HELPER_2(xsadddp, 0x00, 0x04, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsaddqp, 0x04, 0x00, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xssubdp, 0x00, 0x05, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsmuldp, 0x00, 0x06, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsmulqp, 0x04, 0x01, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xsdivdp, 0x00, 0x07, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsdivqp, 0x04, 0x11, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xsredp, 0x14, 0x05, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xssqrtdp, 0x16, 0x04, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsrsqrtedp, 0x14, 0x04, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xstdivdp, 0x14, 0x07, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xstsqrtdp, 0x14, 0x06, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsmaddadp, 0x04, 0x04, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsmaddmdp, 0x04, 0x05, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsmsubadp, 0x04, 0x06, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsmsubmdp, 0x04, 0x07, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsnmaddadp, 0x04, 0x14, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsnmaddmdp, 0x04, 0x15, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsnmsubadp, 0x04, 0x16, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsnmsubmdp, 0x04, 0x17, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xscmpeqdp, 0x0C, 0x00, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xscmpgtdp, 0x0C, 0x01, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xscmpgedp, 0x0C, 0x02, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xscmpnedp, 0x0C, 0x03, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xscmpexpdp, 0x0C, 0x07, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xscmpexpqp, 0x04, 0x05, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xscmpodp, 0x0C, 0x05, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xscmpudp, 0x0C, 0x04, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xscmpoqp, 0x04, 0x04, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xscmpuqp, 0x04, 0x14, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsmaxdp, 0x00, 0x14, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsmindp, 0x00, 0x15, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsmaxcdp, 0x00, 0x10, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xsmincdp, 0x00, 0x11, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xsmaxjdp, 0x00, 0x12, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xsminjdp, 0x00, 0x12, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xscvdphp, 0x16, 0x15, 0x11, PPC2_ISA300) -GEN_VSX_HELPER_2(xscvdpsp, 0x12, 0x10, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xscvdpqp, 0x04, 0x1A, 0x16, PPC2_ISA300) +GEN_VSX_HELPER_X3(xsadddp, 0x00, 0x04, 0, PPC2_VSX) +GEN_VSX_HELPER_R3(xsaddqp, 0x04, 0x00, 0, PPC2_ISA300) +GEN_VSX_HELPER_X3(xssubdp, 0x00, 0x05, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xsmuldp, 0x00, 0x06, 0, PPC2_VSX) +GEN_VSX_HELPER_R3(xsmulqp, 0x04, 0x01, 0, PPC2_ISA300) +GEN_VSX_HELPER_X3(xsdivdp, 0x00, 0x07, 0, PPC2_VSX) +GEN_VSX_HELPER_R3(xsdivqp, 0x04, 0x11, 0, PPC2_ISA300) +GEN_VSX_HELPER_X2(xsredp, 0x14, 0x05, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xssqrtdp, 0x16, 0x04, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xsrsqrtedp, 0x14, 0x04, 0, PPC2_VSX) +GEN_VSX_HELPER_X2_AB(xstdivdp, 0x14, 0x07, 0, PPC2_VSX) +GEN_VSX_HELPER_X1(xstsqrtdp, 0x14, 0x06, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xscmpeqdp, 0x0C, 0x00, 0, PPC2_ISA300) +GEN_VSX_HELPER_X3(xscmpgtdp, 0x0C, 0x01, 0, PPC2_ISA300) +GEN_VSX_HELPER_X3(xscmpgedp, 0x0C, 0x02, 0, PPC2_ISA300) +GEN_VSX_HELPER_X3(xscmpnedp, 0x0C, 0x03, 0, PPC2_ISA300) +GEN_VSX_HELPER_X2_AB(xscmpexpdp, 0x0C, 0x07, 0, PPC2_ISA300) +GEN_VSX_HELPER_R2_AB(xscmpexpqp, 0x04, 0x05, 0, PPC2_ISA300) +GEN_VSX_HELPER_X2_AB(xscmpodp, 0x0C, 0x05, 0, PPC2_VSX) +GEN_VSX_HELPER_X2_AB(xscmpudp, 0x0C, 0x04, 0, PPC2_VSX) +GEN_VSX_HELPER_R2_AB(xscmpoqp, 0x04, 0x04, 0, PPC2_VSX) +GEN_VSX_HELPER_R2_AB(xscmpuqp, 0x04, 0x14, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xsmaxdp, 0x00, 0x14, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xsmindp, 0x00, 0x15, 0, PPC2_VSX) +GEN_VSX_HELPER_R3(xsmaxcdp, 0x00, 0x10, 0, PPC2_ISA300) +GEN_VSX_HELPER_R3(xsmincdp, 0x00, 0x11, 0, PPC2_ISA300) +GEN_VSX_HELPER_R3(xsmaxjdp, 0x00, 0x12, 0, PPC2_ISA300) +GEN_VSX_HELPER_R3(xsminjdp, 0x00, 0x12, 0, PPC2_ISA300) +GEN_VSX_HELPER_X2(xscvdphp, 0x16, 0x15, 0x11, PPC2_ISA300) +GEN_VSX_HELPER_X2(xscvdpsp, 0x12, 0x10, 0, PPC2_VSX) +GEN_VSX_HELPER_R2(xscvdpqp, 0x04, 0x1A, 0x16, PPC2_ISA300) GEN_VSX_HELPER_XT_XB_ENV(xscvdpspn, 0x16, 0x10, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xscvqpdp, 0x04, 0x1A, 0x14, PPC2_ISA300) -GEN_VSX_HELPER_2(xscvqpsdz, 0x04, 0x1A, 0x19, PPC2_ISA300) -GEN_VSX_HELPER_2(xscvqpswz, 0x04, 0x1A, 0x09, PPC2_ISA300) -GEN_VSX_HELPER_2(xscvqpudz, 0x04, 0x1A, 0x11, PPC2_ISA300) -GEN_VSX_HELPER_2(xscvqpuwz, 0x04, 0x1A, 0x01, PPC2_ISA300) -GEN_VSX_HELPER_2(xscvhpdp, 0x16, 0x15, 0x10, PPC2_ISA300) -GEN_VSX_HELPER_2(xscvsdqp, 0x04, 0x1A, 0x0A, PPC2_ISA300) -GEN_VSX_HELPER_2(xscvspdp, 0x12, 0x14, 0, PPC2_VSX) +GEN_VSX_HELPER_R2(xscvqpsdz, 0x04, 0x1A, 0x19, PPC2_ISA300) +GEN_VSX_HELPER_R2(xscvqpswz, 0x04, 0x1A, 0x09, PPC2_ISA300) +GEN_VSX_HELPER_R2(xscvqpudz, 0x04, 0x1A, 0x11, PPC2_ISA300) +GEN_VSX_HELPER_R2(xscvqpuwz, 0x04, 0x1A, 0x01, PPC2_ISA300) +GEN_VSX_HELPER_X2(xscvhpdp, 0x16, 0x15, 0x10, PPC2_ISA300) +GEN_VSX_HELPER_R2(xscvsdqp, 0x04, 0x1A, 0x0A, PPC2_ISA300) +GEN_VSX_HELPER_X2(xscvspdp, 0x12, 0x14, 0, PPC2_VSX) GEN_VSX_HELPER_XT_XB_ENV(xscvspdpn, 0x16, 0x14, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xscvdpsxds, 0x10, 0x15, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xscvdpsxws, 0x10, 0x05, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xscvdpuxds, 0x10, 0x14, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xscvdpuxws, 0x10, 0x04, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xscvsxddp, 0x10, 0x17, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xscvudqp, 0x04, 0x1A, 0x02, PPC2_ISA300) -GEN_VSX_HELPER_2(xscvuxddp, 0x10, 0x16, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsrdpi, 0x12, 0x04, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsrdpic, 0x16, 0x06, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsrdpim, 0x12, 0x07, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsrdpip, 0x12, 0x06, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xsrdpiz, 0x12, 0x05, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xscvdpsxds, 0x10, 0x15, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xscvdpsxws, 0x10, 0x05, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xscvdpuxds, 0x10, 0x14, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xscvdpuxws, 0x10, 0x04, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xscvsxddp, 0x10, 0x17, 0, PPC2_VSX) +GEN_VSX_HELPER_R2(xscvudqp, 0x04, 0x1A, 0x02, PPC2_ISA300) +GEN_VSX_HELPER_X2(xscvuxddp, 0x10, 0x16, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xsrdpi, 0x12, 0x04, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xsrdpic, 0x16, 0x06, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xsrdpim, 0x12, 0x07, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xsrdpip, 0x12, 0x06, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xsrdpiz, 0x12, 0x05, 0, PPC2_VSX) GEN_VSX_HELPER_XT_XB_ENV(xsrsp, 0x12, 0x11, 0, PPC2_VSX207) - -GEN_VSX_HELPER_2(xsrqpi, 0x05, 0x00, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xsrqpxp, 0x05, 0x01, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xssqrtqp, 0x04, 0x19, 0x1B, PPC2_ISA300) -GEN_VSX_HELPER_2(xssubqp, 0x04, 0x10, 0, PPC2_ISA300) - -GEN_VSX_HELPER_2(xsaddsp, 0x00, 0x00, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xssubsp, 0x00, 0x01, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsmulsp, 0x00, 0x02, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsdivsp, 0x00, 0x03, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsresp, 0x14, 0x01, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xssqrtsp, 0x16, 0x00, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsrsqrtesp, 0x14, 0x00, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsmaddasp, 0x04, 0x00, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsmaddmsp, 0x04, 0x01, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsmsubasp, 0x04, 0x02, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsmsubmsp, 0x04, 0x03, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsnmaddasp, 0x04, 0x10, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsnmaddmsp, 0x04, 0x11, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsnmsubasp, 0x04, 0x12, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xsnmsubmsp, 0x04, 0x13, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xscvsxdsp, 0x10, 0x13, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xscvuxdsp, 0x10, 0x12, 0, PPC2_VSX207) -GEN_VSX_HELPER_2(xststdcsp, 0x14, 0x12, 0, PPC2_ISA300) +GEN_VSX_HELPER_R2(xsrqpi, 0x05, 0x00, 0, PPC2_ISA300) +GEN_VSX_HELPER_R2(xsrqpxp, 0x05, 0x01, 0, PPC2_ISA300) +GEN_VSX_HELPER_R2(xssqrtqp, 0x04, 0x19, 0x1B, PPC2_ISA300) +GEN_VSX_HELPER_R3(xssubqp, 0x04, 0x10, 0, PPC2_ISA300) +GEN_VSX_HELPER_X3(xsaddsp, 0x00, 0x00, 0, PPC2_VSX207) +GEN_VSX_HELPER_X3(xssubsp, 0x00, 0x01, 0, PPC2_VSX207) +GEN_VSX_HELPER_X3(xsmulsp, 0x00, 0x02, 0, PPC2_VSX207) +GEN_VSX_HELPER_X3(xsdivsp, 0x00, 0x03, 0, PPC2_VSX207) +GEN_VSX_HELPER_X2(xsresp, 0x14, 0x01, 0, PPC2_VSX207) +GEN_VSX_HELPER_X2(xssqrtsp, 0x16, 0x00, 0, PPC2_VSX207) +GEN_VSX_HELPER_X2(xsrsqrtesp, 0x14, 0x00, 0, PPC2_VSX207) +GEN_VSX_HELPER_X2(xscvsxdsp, 0x10, 0x13, 0, PPC2_VSX207) +GEN_VSX_HELPER_X2(xscvuxdsp, 0x10, 0x12, 0, PPC2_VSX207) +GEN_VSX_HELPER_X1(xststdcsp, 0x14, 0x12, 0, PPC2_ISA300) GEN_VSX_HELPER_2(xststdcdp, 0x14, 0x16, 0, PPC2_ISA300) GEN_VSX_HELPER_2(xststdcqp, 0x04, 0x16, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xvadddp, 0x00, 0x0C, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvsubdp, 0x00, 0x0D, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmuldp, 0x00, 0x0E, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvdivdp, 0x00, 0x0F, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvredp, 0x14, 0x0D, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvsqrtdp, 0x16, 0x0C, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrsqrtedp, 0x14, 0x0C, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvtdivdp, 0x14, 0x0F, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvtsqrtdp, 0x14, 0x0E, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmaddadp, 0x04, 0x0C, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmaddmdp, 0x04, 0x0D, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmsubadp, 0x04, 0x0E, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmsubmdp, 0x04, 0x0F, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvnmaddadp, 0x04, 0x1C, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvnmaddmdp, 0x04, 0x1D, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvnmsubadp, 0x04, 0x1E, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvnmsubmdp, 0x04, 0x1F, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmaxdp, 0x00, 0x1C, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmindp, 0x00, 0x1D, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcmpeqdp, 0x0C, 0x0C, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcmpgtdp, 0x0C, 0x0D, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcmpgedp, 0x0C, 0x0E, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcmpnedp, 0x0C, 0x0F, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xvcvdpsp, 0x12, 0x18, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvdpsxds, 0x10, 0x1D, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvdpsxws, 0x10, 0x0D, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvdpuxds, 0x10, 0x1C, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvdpuxws, 0x10, 0x0C, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvsxddp, 0x10, 0x1F, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvuxddp, 0x10, 0x1E, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvsxwdp, 0x10, 0x0F, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvuxwdp, 0x10, 0x0E, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrdpi, 0x12, 0x0C, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrdpic, 0x16, 0x0E, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrdpim, 0x12, 0x0F, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrdpip, 0x12, 0x0E, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrdpiz, 0x12, 0x0D, 0, PPC2_VSX) - -GEN_VSX_HELPER_2(xvaddsp, 0x00, 0x08, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvsubsp, 0x00, 0x09, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmulsp, 0x00, 0x0A, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvdivsp, 0x00, 0x0B, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvresp, 0x14, 0x09, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvsqrtsp, 0x16, 0x08, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrsqrtesp, 0x14, 0x08, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvtdivsp, 0x14, 0x0B, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvtsqrtsp, 0x14, 0x0A, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmaddasp, 0x04, 0x08, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmaddmsp, 0x04, 0x09, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmsubasp, 0x04, 0x0A, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmsubmsp, 0x04, 0x0B, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvnmaddasp, 0x04, 0x18, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvnmaddmsp, 0x04, 0x19, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvnmsubasp, 0x04, 0x1A, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvnmsubmsp, 0x04, 0x1B, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvmaxsp, 0x00, 0x18, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvminsp, 0x00, 0x19, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcmpeqsp, 0x0C, 0x08, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcmpgtsp, 0x0C, 0x09, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcmpgesp, 0x0C, 0x0A, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcmpnesp, 0x0C, 0x0B, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvspdp, 0x12, 0x1C, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvhpsp, 0x16, 0x1D, 0x18, PPC2_ISA300) -GEN_VSX_HELPER_2(xvcvsphp, 0x16, 0x1D, 0x19, PPC2_ISA300) -GEN_VSX_HELPER_2(xvcvspsxds, 0x10, 0x19, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvspsxws, 0x10, 0x09, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvspuxds, 0x10, 0x18, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvspuxws, 0x10, 0x08, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvsxdsp, 0x10, 0x1B, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvuxdsp, 0x10, 0x1A, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvsxwsp, 0x10, 0x0B, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvcvuxwsp, 0x10, 0x0A, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrspi, 0x12, 0x08, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrspic, 0x16, 0x0A, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrspim, 0x12, 0x0B, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrspip, 0x12, 0x0A, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xvrspiz, 0x12, 0x09, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xvadddp, 0x00, 0x0C, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xvsubdp, 0x00, 0x0D, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xvmuldp, 0x00, 0x0E, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xvdivdp, 0x00, 0x0F, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvredp, 0x14, 0x0D, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvsqrtdp, 0x16, 0x0C, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrsqrtedp, 0x14, 0x0C, 0, PPC2_VSX) +GEN_VSX_HELPER_X2_AB(xvtdivdp, 0x14, 0x0F, 0, PPC2_VSX) +GEN_VSX_HELPER_X1(xvtsqrtdp, 0x14, 0x0E, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xvmaxdp, 0x00, 0x1C, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xvmindp, 0x00, 0x1D, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvdpsp, 0x12, 0x18, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvdpsxds, 0x10, 0x1D, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvdpsxws, 0x10, 0x0D, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvdpuxds, 0x10, 0x1C, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvdpuxws, 0x10, 0x0C, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvsxddp, 0x10, 0x1F, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvuxddp, 0x10, 0x1E, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvsxwdp, 0x10, 0x0F, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvuxwdp, 0x10, 0x0E, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrdpi, 0x12, 0x0C, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrdpic, 0x16, 0x0E, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrdpim, 0x12, 0x0F, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrdpip, 0x12, 0x0E, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrdpiz, 0x12, 0x0D, 0, PPC2_VSX) + +GEN_VSX_HELPER_X3(xvaddsp, 0x00, 0x08, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xvsubsp, 0x00, 0x09, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xvmulsp, 0x00, 0x0A, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xvdivsp, 0x00, 0x0B, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvresp, 0x14, 0x09, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvsqrtsp, 0x16, 0x08, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrsqrtesp, 0x14, 0x08, 0, PPC2_VSX) +GEN_VSX_HELPER_X2_AB(xvtdivsp, 0x14, 0x0B, 0, PPC2_VSX) +GEN_VSX_HELPER_X1(xvtsqrtsp, 0x14, 0x0A, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xvmaxsp, 0x00, 0x18, 0, PPC2_VSX) +GEN_VSX_HELPER_X3(xvminsp, 0x00, 0x19, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvspdp, 0x12, 0x1C, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvhpsp, 0x16, 0x1D, 0x18, PPC2_ISA300) +GEN_VSX_HELPER_X2(xvcvsphp, 0x16, 0x1D, 0x19, PPC2_ISA300) +GEN_VSX_HELPER_X2(xvcvspsxds, 0x10, 0x19, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvspsxws, 0x10, 0x09, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvspuxds, 0x10, 0x18, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvspuxws, 0x10, 0x08, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvsxdsp, 0x10, 0x1B, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvuxdsp, 0x10, 0x1A, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvsxwsp, 0x10, 0x0B, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvcvuxwsp, 0x10, 0x0A, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrspi, 0x12, 0x08, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrspic, 0x16, 0x0A, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrspim, 0x12, 0x0B, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrspip, 0x12, 0x0A, 0, PPC2_VSX) +GEN_VSX_HELPER_X2(xvrspiz, 0x12, 0x09, 0, PPC2_VSX) GEN_VSX_HELPER_2(xvtstdcsp, 0x14, 0x1A, 0, PPC2_VSX) GEN_VSX_HELPER_2(xvtstdcdp, 0x14, 0x1E, 0, PPC2_VSX) -GEN_VSX_HELPER_2(xxperm, 0x08, 0x03, 0, PPC2_ISA300) -GEN_VSX_HELPER_2(xxpermr, 0x08, 0x07, 0, PPC2_ISA300) +GEN_VSX_HELPER_X3(xxperm, 0x08, 0x03, 0, PPC2_ISA300) +GEN_VSX_HELPER_X3(xxpermr, 0x08, 0x07, 0, PPC2_ISA300) + +#define GEN_VSX_HELPER_VSX_MADD(name, op1, aop, mop, inval, type) \ +static void gen_##name(DisasContext *ctx) \ +{ \ + TCGv_ptr xt, xa, b, c; \ + if (unlikely(!ctx->vsx_enabled)) { \ + gen_exception(ctx, POWERPC_EXCP_VSXU); \ + return; \ + } \ + xt = gen_vsr_ptr(xT(ctx->opcode)); \ + xa = gen_vsr_ptr(xA(ctx->opcode)); \ + if (ctx->opcode & PPC_BIT(25)) { \ + /* \ + * AxT + B \ + */ \ + b = gen_vsr_ptr(xT(ctx->opcode)); \ + c = gen_vsr_ptr(xB(ctx->opcode)); \ + } else { \ + /* \ + * AxB + T \ + */ \ + b = gen_vsr_ptr(xB(ctx->opcode)); \ + c = gen_vsr_ptr(xT(ctx->opcode)); \ + } \ + gen_helper_##name(cpu_env, xt, xa, b, c); \ + tcg_temp_free_ptr(xt); \ + tcg_temp_free_ptr(xa); \ + tcg_temp_free_ptr(b); \ + tcg_temp_free_ptr(c); \ +} + +GEN_VSX_HELPER_VSX_MADD(xsmadddp, 0x04, 0x04, 0x05, 0, PPC2_VSX) +GEN_VSX_HELPER_VSX_MADD(xsmsubdp, 0x04, 0x06, 0x07, 0, PPC2_VSX) +GEN_VSX_HELPER_VSX_MADD(xsnmadddp, 0x04, 0x14, 0x15, 0, PPC2_VSX) +GEN_VSX_HELPER_VSX_MADD(xsnmsubdp, 0x04, 0x16, 0x17, 0, PPC2_VSX) +GEN_VSX_HELPER_VSX_MADD(xsmaddsp, 0x04, 0x00, 0x01, 0, PPC2_VSX207) +GEN_VSX_HELPER_VSX_MADD(xsmsubsp, 0x04, 0x02, 0x03, 0, PPC2_VSX207) +GEN_VSX_HELPER_VSX_MADD(xsnmaddsp, 0x04, 0x10, 0x11, 0, PPC2_VSX207) +GEN_VSX_HELPER_VSX_MADD(xsnmsubsp, 0x04, 0x12, 0x13, 0, PPC2_VSX207) +GEN_VSX_HELPER_VSX_MADD(xvmadddp, 0x04, 0x0C, 0x0D, 0, PPC2_VSX) +GEN_VSX_HELPER_VSX_MADD(xvmsubdp, 0x04, 0x0E, 0x0F, 0, PPC2_VSX) +GEN_VSX_HELPER_VSX_MADD(xvnmadddp, 0x04, 0x1C, 0x1D, 0, PPC2_VSX) +GEN_VSX_HELPER_VSX_MADD(xvnmsubdp, 0x04, 0x1E, 0x1F, 0, PPC2_VSX) +GEN_VSX_HELPER_VSX_MADD(xvmaddsp, 0x04, 0x08, 0x09, 0, PPC2_VSX) +GEN_VSX_HELPER_VSX_MADD(xvmsubsp, 0x04, 0x0A, 0x0B, 0, PPC2_VSX) +GEN_VSX_HELPER_VSX_MADD(xvnmaddsp, 0x04, 0x18, 0x19, 0, PPC2_VSX) +GEN_VSX_HELPER_VSX_MADD(xvnmsubsp, 0x04, 0x1A, 0x1B, 0, PPC2_VSX) static void gen_xxbrd(DisasContext *ctx) { @@ -1460,7 +1645,7 @@ static void gen_xxsldwi(DisasContext *ctx) #define VSX_EXTRACT_INSERT(name) \ static void gen_##name(DisasContext *ctx) \ { \ - TCGv xt, xb; \ + TCGv_ptr xt, xb; \ TCGv_i32 t0; \ TCGv_i64 t1; \ uint8_t uimm = UIMM4(ctx->opcode); \ @@ -1469,8 +1654,8 @@ static void gen_##name(DisasContext *ctx) \ gen_exception(ctx, POWERPC_EXCP_VSXU); \ return; \ } \ - xt = tcg_const_tl(xT(ctx->opcode)); \ - xb = tcg_const_tl(xB(ctx->opcode)); \ + xt = gen_vsr_ptr(xT(ctx->opcode)); \ + xb = gen_vsr_ptr(xB(ctx->opcode)); \ t0 = tcg_temp_new_i32(); \ t1 = tcg_temp_new_i64(); \ /* \ @@ -1485,8 +1670,8 @@ static void gen_##name(DisasContext *ctx) \ } \ tcg_gen_movi_i32(t0, uimm); \ gen_helper_##name(cpu_env, xt, xb, t0); \ - tcg_temp_free(xb); \ - tcg_temp_free(xt); \ + tcg_temp_free_ptr(xb); \ + tcg_temp_free_ptr(xt); \ tcg_temp_free_i32(t0); \ tcg_temp_free_i64(t1); \ } @@ -1813,7 +1998,7 @@ static void gen_xvxexpdp(DisasContext *ctx) tcg_temp_free_i64(xbl); } -GEN_VSX_HELPER_2(xvxsigsp, 0x00, 0x04, 0, PPC2_ISA300) +GEN_VSX_HELPER_X2(xvxsigsp, 0x00, 0x04, 0, PPC2_ISA300) static void gen_xvxsigdp(DisasContext *ctx) { diff --git a/target/ppc/translate/vsx-ops.inc.c b/target/ppc/translate/vsx-ops.inc.c index 5030c4aceb..7fd3942b84 100644 --- a/target/ppc/translate/vsx-ops.inc.c +++ b/target/ppc/translate/vsx-ops.inc.c @@ -63,6 +63,12 @@ GEN_HANDLER2_E(name, #name, 0x3C, opc2 | 1, opc3, 0, PPC_NONE, fl2), \ GEN_HANDLER2_E(name, #name, 0x3C, opc2 | 2, opc3, 0, PPC_NONE, fl2), \ GEN_HANDLER2_E(name, #name, 0x3C, opc2 | 3, opc3, 0, PPC_NONE, fl2) +#define GEN_XX3FORM_NAME(name, opcname, opc2, opc3, fl2) \ +GEN_HANDLER2_E(name, opcname, 0x3C, opc2 | 0, opc3, 0, PPC_NONE, fl2), \ +GEN_HANDLER2_E(name, opcname, 0x3C, opc2 | 1, opc3, 0, PPC_NONE, fl2), \ +GEN_HANDLER2_E(name, opcname, 0x3C, opc2 | 2, opc3, 0, PPC_NONE, fl2), \ +GEN_HANDLER2_E(name, opcname, 0x3C, opc2 | 3, opc3, 0, PPC_NONE, fl2) + #define GEN_XX2IFORM(name, opc2, opc3, fl2) \ GEN_HANDLER2_E(name, #name, 0x3C, opc2 | 0, opc3, 1, PPC_NONE, fl2), \ GEN_HANDLER2_E(name, #name, 0x3C, opc2 | 1, opc3, 1, PPC_NONE, fl2), \ @@ -182,14 +188,14 @@ GEN_XX2FORM(xssqrtdp, 0x16, 0x04, PPC2_VSX), GEN_XX2FORM(xsrsqrtedp, 0x14, 0x04, PPC2_VSX), GEN_XX3FORM(xstdivdp, 0x14, 0x07, PPC2_VSX), GEN_XX2FORM(xstsqrtdp, 0x14, 0x06, PPC2_VSX), -GEN_XX3FORM(xsmaddadp, 0x04, 0x04, PPC2_VSX), -GEN_XX3FORM(xsmaddmdp, 0x04, 0x05, PPC2_VSX), -GEN_XX3FORM(xsmsubadp, 0x04, 0x06, PPC2_VSX), -GEN_XX3FORM(xsmsubmdp, 0x04, 0x07, PPC2_VSX), -GEN_XX3FORM(xsnmaddadp, 0x04, 0x14, PPC2_VSX), -GEN_XX3FORM(xsnmaddmdp, 0x04, 0x15, PPC2_VSX), -GEN_XX3FORM(xsnmsubadp, 0x04, 0x16, PPC2_VSX), -GEN_XX3FORM(xsnmsubmdp, 0x04, 0x17, PPC2_VSX), +GEN_XX3FORM_NAME(xsmadddp, "xsmaddadp", 0x04, 0x04, PPC2_VSX), +GEN_XX3FORM_NAME(xsmadddp, "xsmaddmdp", 0x04, 0x05, PPC2_VSX), +GEN_XX3FORM_NAME(xsmsubdp, "xsmsubadp", 0x04, 0x06, PPC2_VSX), +GEN_XX3FORM_NAME(xsmsubdp, "xsmsubmdp", 0x04, 0x07, PPC2_VSX), +GEN_XX3FORM_NAME(xsnmadddp, "xsnmaddadp", 0x04, 0x14, PPC2_VSX), +GEN_XX3FORM_NAME(xsnmadddp, "xsnmaddmdp", 0x04, 0x15, PPC2_VSX), +GEN_XX3FORM_NAME(xsnmsubdp, "xsnmsubadp", 0x04, 0x16, PPC2_VSX), +GEN_XX3FORM_NAME(xsnmsubdp, "xsnmsubmdp", 0x04, 0x17, PPC2_VSX), GEN_XX3FORM(xscmpeqdp, 0x0C, 0x00, PPC2_ISA300), GEN_XX3FORM(xscmpgtdp, 0x0C, 0x01, PPC2_ISA300), GEN_XX3FORM(xscmpgedp, 0x0C, 0x02, PPC2_ISA300), @@ -235,14 +241,14 @@ GEN_XX2FORM(xsresp, 0x14, 0x01, PPC2_VSX207), GEN_XX2FORM(xsrsp, 0x12, 0x11, PPC2_VSX207), GEN_XX2FORM(xssqrtsp, 0x16, 0x00, PPC2_VSX207), GEN_XX2FORM(xsrsqrtesp, 0x14, 0x00, PPC2_VSX207), -GEN_XX3FORM(xsmaddasp, 0x04, 0x00, PPC2_VSX207), -GEN_XX3FORM(xsmaddmsp, 0x04, 0x01, PPC2_VSX207), -GEN_XX3FORM(xsmsubasp, 0x04, 0x02, PPC2_VSX207), -GEN_XX3FORM(xsmsubmsp, 0x04, 0x03, PPC2_VSX207), -GEN_XX3FORM(xsnmaddasp, 0x04, 0x10, PPC2_VSX207), -GEN_XX3FORM(xsnmaddmsp, 0x04, 0x11, PPC2_VSX207), -GEN_XX3FORM(xsnmsubasp, 0x04, 0x12, PPC2_VSX207), -GEN_XX3FORM(xsnmsubmsp, 0x04, 0x13, PPC2_VSX207), +GEN_XX3FORM_NAME(xsmaddsp, "xsmaddasp", 0x04, 0x00, PPC2_VSX207), +GEN_XX3FORM_NAME(xsmaddsp, "xsmaddmsp", 0x04, 0x01, PPC2_VSX207), +GEN_XX3FORM_NAME(xsmsubsp, "xsmsubasp", 0x04, 0x02, PPC2_VSX207), +GEN_XX3FORM_NAME(xsmsubsp, "xsmsubmsp", 0x04, 0x03, PPC2_VSX207), +GEN_XX3FORM_NAME(xsnmaddsp, "xsnmaddasp", 0x04, 0x10, PPC2_VSX207), +GEN_XX3FORM_NAME(xsnmaddsp, "xsnmaddmsp", 0x04, 0x11, PPC2_VSX207), +GEN_XX3FORM_NAME(xsnmsubsp, "xsnmsubasp", 0x04, 0x12, PPC2_VSX207), +GEN_XX3FORM_NAME(xsnmsubsp, "xsnmsubmsp", 0x04, 0x13, PPC2_VSX207), GEN_XX2FORM(xscvsxdsp, 0x10, 0x13, PPC2_VSX207), GEN_XX2FORM(xscvuxdsp, 0x10, 0x12, PPC2_VSX207), @@ -255,14 +261,14 @@ GEN_XX2FORM(xvsqrtdp, 0x16, 0x0C, PPC2_VSX), GEN_XX2FORM(xvrsqrtedp, 0x14, 0x0C, PPC2_VSX), GEN_XX3FORM(xvtdivdp, 0x14, 0x0F, PPC2_VSX), GEN_XX2FORM(xvtsqrtdp, 0x14, 0x0E, PPC2_VSX), -GEN_XX3FORM(xvmaddadp, 0x04, 0x0C, PPC2_VSX), -GEN_XX3FORM(xvmaddmdp, 0x04, 0x0D, PPC2_VSX), -GEN_XX3FORM(xvmsubadp, 0x04, 0x0E, PPC2_VSX), -GEN_XX3FORM(xvmsubmdp, 0x04, 0x0F, PPC2_VSX), -GEN_XX3FORM(xvnmaddadp, 0x04, 0x1C, PPC2_VSX), -GEN_XX3FORM(xvnmaddmdp, 0x04, 0x1D, PPC2_VSX), -GEN_XX3FORM(xvnmsubadp, 0x04, 0x1E, PPC2_VSX), -GEN_XX3FORM(xvnmsubmdp, 0x04, 0x1F, PPC2_VSX), +GEN_XX3FORM_NAME(xvmadddp, "xvmaddadp", 0x04, 0x0C, PPC2_VSX), +GEN_XX3FORM_NAME(xvmadddp, "xvmaddmdp", 0x04, 0x0D, PPC2_VSX), +GEN_XX3FORM_NAME(xvmsubdp, "xvmsubadp", 0x04, 0x0E, PPC2_VSX), +GEN_XX3FORM_NAME(xvmsubdp, "xvmsubmdp", 0x04, 0x0F, PPC2_VSX), +GEN_XX3FORM_NAME(xvnmadddp, "xvnmaddadp", 0x04, 0x1C, PPC2_VSX), +GEN_XX3FORM_NAME(xvnmadddp, "xvnmaddmdp", 0x04, 0x1D, PPC2_VSX), +GEN_XX3FORM_NAME(xvnmsubdp, "xvnmsubadp", 0x04, 0x1E, PPC2_VSX), +GEN_XX3FORM_NAME(xvnmsubdp, "xvnmsubmdp", 0x04, 0x1F, PPC2_VSX), GEN_XX3FORM(xvmaxdp, 0x00, 0x1C, PPC2_VSX), GEN_XX3FORM(xvmindp, 0x00, 0x1D, PPC2_VSX), GEN_XX3_RC_FORM(xvcmpeqdp, 0x0C, 0x0C, PPC2_VSX), @@ -293,14 +299,14 @@ GEN_XX2FORM(xvsqrtsp, 0x16, 0x08, PPC2_VSX), GEN_XX2FORM(xvrsqrtesp, 0x14, 0x08, PPC2_VSX), GEN_XX3FORM(xvtdivsp, 0x14, 0x0B, PPC2_VSX), GEN_XX2FORM(xvtsqrtsp, 0x14, 0x0A, PPC2_VSX), -GEN_XX3FORM(xvmaddasp, 0x04, 0x08, PPC2_VSX), -GEN_XX3FORM(xvmaddmsp, 0x04, 0x09, PPC2_VSX), -GEN_XX3FORM(xvmsubasp, 0x04, 0x0A, PPC2_VSX), -GEN_XX3FORM(xvmsubmsp, 0x04, 0x0B, PPC2_VSX), -GEN_XX3FORM(xvnmaddasp, 0x04, 0x18, PPC2_VSX), -GEN_XX3FORM(xvnmaddmsp, 0x04, 0x19, PPC2_VSX), -GEN_XX3FORM(xvnmsubasp, 0x04, 0x1A, PPC2_VSX), -GEN_XX3FORM(xvnmsubmsp, 0x04, 0x1B, PPC2_VSX), +GEN_XX3FORM_NAME(xvmaddsp, "xvmaddasp", 0x04, 0x08, PPC2_VSX), +GEN_XX3FORM_NAME(xvmaddsp, "xvmaddmsp", 0x04, 0x09, PPC2_VSX), +GEN_XX3FORM_NAME(xvmsubsp, "xvmsubasp", 0x04, 0x0A, PPC2_VSX), +GEN_XX3FORM_NAME(xvmsubsp, "xvmsubmsp", 0x04, 0x0B, PPC2_VSX), +GEN_XX3FORM_NAME(xvnmaddsp, "xvnmaddasp", 0x04, 0x18, PPC2_VSX), +GEN_XX3FORM_NAME(xvnmaddsp, "xvnmaddmsp", 0x04, 0x19, PPC2_VSX), +GEN_XX3FORM_NAME(xvnmsubsp, "xvnmsubasp", 0x04, 0x1A, PPC2_VSX), +GEN_XX3FORM_NAME(xvnmsubsp, "xvnmsubmsp", 0x04, 0x1B, PPC2_VSX), GEN_XX3FORM(xvmaxsp, 0x00, 0x18, PPC2_VSX), GEN_XX3FORM(xvminsp, 0x00, 0x19, PPC2_VSX), GEN_XX3_RC_FORM(xvcmpeqsp, 0x0C, 0x08, PPC2_VSX), |