aboutsummaryrefslogtreecommitdiff
path: root/include/tcg
AgeCommit message (Collapse)Author
2023-07-01tcg: Reduce tcg_assert_listed_vecop() scopePhilippe Mathieu-Daudé
tcg_assert_listed_vecop() is only used in tcg-op-vec.c. Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20230629091107.74384-1-philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-26accel/tcg: remove CONFIG_PROFILERFei Wu
TBStats will be introduced to replace CONFIG_PROFILER totally, here remove all CONFIG_PROFILER related stuffs first. Signed-off-by: Vanderson M. do Rosario <vandersonmr2@gmail.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Fei Wu <fei2.wu@intel.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20230607122411.3394702-2-fei2.wu@intel.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Remove target-specific headers from tcg.[ch]Richard Henderson
This finally paves the way for tcg/ to be built once per mode. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Split tcg/tcg-op-gvec.hRichard Henderson
Create tcg/tcg-op-gvec-common.h, moving everything that does not concern TARGET_LONG_BITS. Adjust tcg-op-gvec.c to use the new header. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Move TLB_FLAGS_MASK check out of get_alignment_bitsRichard Henderson
The replacement isn't ideal, as the raw count of bits is not easily synced with exec/cpu-all.h, but it does remove from tcg.h the target dependency on TARGET_PAGE_BITS_MIN which is built into TLB_FLAGS_MASK. Reviewed-by: Anton Johansson <anjo@rev.ng> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Add guest_mo to TCGContextRichard Henderson
This replaces of TCG_GUEST_DEFAULT_MO in tcg-op-ldst.c. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Add insn_start_words to TCGContextRichard Henderson
This will enable replacement of TARGET_INSN_START_WORDS in tcg.c. Split out "tcg/insn-start-words.h" and use it in target/. Reviewed-by: Anton Johansson <anjo@rev.ng> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Split helper-proto.hRichard Henderson
Create helper-proto-common.h without the target specific portion. Use that in tcg-op-common.h. Include helper-proto.h in target/arm and target/hexagon before helper-info.c.inc; all other targets are already correct in this regard. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Split helper-gen.hRichard Henderson
Create helper-gen-common.h without the target specific portion. Use that in tcg-op-common.h. Reorg headers in target/arm to ensure that helper-gen.h is included before helper-info.c.inc. All other targets are already correct in this regard. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Split tcg_gen_callNRichard Henderson
Make tcg_gen_callN a static function. Create tcg_gen_call[0-7] functions for use by helper-gen.h.inc. Removes a multiplicty of calls to __stack_chk_fail, saving up to 143kiB of .text space as measured on an x86_64 host. Old New Less %Change 8888680 8741816 146864 1.65% qemu-system-aarch64 5911832 5856152 55680 0.94% qemu-system-riscv64 5816728 5767512 49216 0.85% qemu-system-mips64 6707832 6659144 48688 0.73% qemu-system-ppc64 Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Move temp_idx and tcgv_i32_temp debug out of lineRichard Henderson
Removes a multiplicity of calls to __assert_fail, saving up to 360kiB of .text space as measured on an x86_64 host. Old New Less %Change 9257272 8888680 368592 3.98% qemu-system-aarch64 6100968 5911832 189136 3.10% qemu-system-riscv64 5839112 5707032 132080 2.26% qemu-system-mips 4447608 4341752 105856 2.38% qemu-system-s390x Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Pass TCGHelperInfo to tcg_gen_callNRichard Henderson
In preparation for compiling tcg/ only once, eliminate the all_helpers array. Instantiate the info structs for the generic helpers in accel/tcg/, and the structs for the target-specific helpers in each translate.c. Since we don't see all of the info structs at startup, initialize at first use, using g_once_init_* to make sure we don't race while doing so. Reviewed-by: Anton Johansson <anjo@rev.ng> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Move TCGHelperInfo and dependencies to tcg/helper-info.hRichard Henderson
This will be required outside of tcg-internal.h soon. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Split tcg/tcg-op-common.h from tcg/tcg-op.hRichard Henderson
Create tcg/tcg-op-common.h, moving everything that does not concern TARGET_LONG_BITS or TCGv. Adjust tcg/*.c to use the new header instead of tcg-op.h, in preparation for compiling tcg/ only once. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Move TCGv, dup_const_tl definitions to tcg-op.hRichard Henderson
These two items are the last uses of TARGET_LONG_BITS within tcg.h, and are more in common with the other "_tl" definitions within that file. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Split out tcg/oversized-guest.hRichard Henderson
Move a use of TARGET_LONG_BITS out of tcg/tcg.h. Include the new file only where required. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Split out tcg-target-reg-bits.hRichard Henderson
Often, the only thing we need to know about the TCG host is the register size. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Add tlb_fast_offset to TCGContextRichard Henderson
Disconnect the layout of ArchCPU from TCG compilation. Pass the relative offset of 'env' and 'neg.tlb.f' as a parameter. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-05tcg: Move TCG_TYPE_TL from tcg.h to tcg-op.hRichard Henderson
Removes the only use of TARGET_LONG_BITS from tcg.h, which is to be target independent. Move the symbol to a define in tcg-op.h, which will continue to be target dependent. Rather than complicate matters for the use in tb_gen_code(), expand the definition there. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-23tcg: Split out tcg/debug-assert.hRichard Henderson
Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16tcg: Add tlb_dyn_max_bits to TCGContextRichard Henderson
Disconnect guest tlb parameters from TCG compilation. Reviewed-by: Anton Johansson <anjo@rev.ng> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16tcg: Add page_bits and page_mask to TCGContextRichard Henderson
Disconnect guest page size from TCG compilation. While this could be done via exec/target_page.h, we want to cache the value across multiple memory access operations, so we might as well initialize this early. The changes within tcg/ are entirely mechanical: sed -i s/TARGET_PAGE_BITS/s->page_bits/g sed -i s/TARGET_PAGE_MASK/s->page_mask/g Reviewed-by: Anton Johansson <anjo@rev.ng> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16tcg: Split INDEX_op_qemu_{ld,st}* for guest address sizeRichard Henderson
For 32-bit hosts, we cannot simply rely on TCGContext.addr_bits, as we need one or two host registers to represent the guest address. Create the new opcodes and update all users. Since we have not yet eliminated TARGET_LONG_BITS, only one of the two opcodes will ever be used, so we can get away with treating them the same in the backends. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16tcg: Remove TCGv from tcg_gen_atomic_*Richard Henderson
Expand from TCGv to TCGTemp inline in the translators, and validate that the size matches tcg_ctx->addr_type. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16tcg: Remove TCGv from tcg_gen_qemu_{ld,st}_*Richard Henderson
Expand from TCGv to TCGTemp inline in the translators, and validate that the size matches tcg_ctx->addr_type. These inlines will eventually be seen only by target-specific code. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16tcg: Add addr_type to TCGContextRichard Henderson
This will enable replacement of TARGET_LONG_BITS within tcg/. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16tcg: Widen tcg_gen_code pc_start argument to uint64_tRichard Henderson
Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16accel/tcg: Widen tcg-ldst.h addresses to uint64_tRichard Henderson
Always pass the target address as uint64_t. Adjust tcg_out_{ld,st}_helper_args to match. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16tcg: Widen gen_insn_data to uint64_tRichard Henderson
We already pass uint64_t to restore_state_to_opc; this changes all of the other uses from insn_start through the encoding to decoding. Reviewed-by: Anton Johansson <anjo@rev.ng> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16tcg: Add INDEX_op_qemu_{ld,st}_i128Richard Henderson
Add opcodes for backend support for 128-bit memory operations. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16accel/tcg: Remove helper_unaligned_{ld,st}Richard Henderson
These functions are now unused. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16tcg: Add 128-bit guest memory primitivesRichard Henderson
Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16accel/tcg: Implement helper_{ld,st}*_mmu for user-onlyRichard Henderson
TCG backends may need to defer to a helper to implement the atomicity required by a given operation. Mirror the interface used in system mode. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-16tcg: Unify helper_{be,le}_{ld,st}*Richard Henderson
With the current structure of cputlb.c, there is no difference between the little-endian and big-endian entry points, aside from the assert. Unify the pairs of functions. Hoist the qemu_{ld,st}_helpers arrays to tcg.c. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-05tcg: Widen helper_*_st[bw]_mmu val argumentsRichard Henderson
While the old type was correct in the ideal sense, some ABIs require the argument to be zero-extended. Using uint32_t for all such values is a decent compromise. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-05tcg: Remove compatability helpers for qemu ld/stRichard Henderson
Remove the old interfaces with the implicit MemOp argument. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Acked-by: David Hildenbrand <david@redhat.com> Message-Id: <20230502135741.1158035-10-richard.henderson@linaro.org>
2023-05-02tcg: Add tcg_gen_gvec_rotrsNazar Kazakov
Add tcg expander and helper functions for rotate right vector with scalar operand. Signed-off-by: Nazar Kazakov <nazar.kazakov@codethink.co.uk> Message-Id: <20230428144757.57530-10-lawrence.hunter@codethink.co.uk> [rth: Split out of larger patch; mask rotation count.] Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-02tcg: Add tcg_gen_gvec_andcsNazar Kazakov
Add tcg expander and helper functions for and-compliment vector with scalar operand. Signed-off-by: Nazar Kazakov <nazar.kazakov@codethink.co.uk> Message-Id: <20230428144757.57530-10-lawrence.hunter@codethink.co.uk> [rth: Split out of larger patch.] Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-04-23tcg: Replace tcg_abort with g_assert_not_reachedRichard Henderson
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-03-13tcg: Drop tcg_const_*Richard Henderson
These functions are no longer used. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-03-13tcg: Drop tcg_const_*_vecRichard Henderson
Replace with tcg_constant_vec*. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-03-13tcg: Create tcg/tcg-temp-internal.hRichard Henderson
Move the tcg_temp_free_* and tcg_temp_ebb_new_* declarations and inlines to the new header. These are private to the implementation, and will prevent tcg_temp_free_* from creeping back into the guest front ends. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-03-05tcg: Remove tcg_check_temp_count, tcg_clear_temp_countRichard Henderson
Since all temps allocated by guest front-ends are now TEMP_TB, and we don't recycle TEMP_TB, there's no point in requiring that the front-ends free the temps at all. Begin by dropping the inner-most checks that all temps have been freed. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-03-05tcg: Link branches to the labelsRichard Henderson
This allows us to easily find all branches that use a label. Since 'refs' is only tested vs zero, remove it and test for an empty list instead. Drop the use of bitfields, which had been used to pack refs into a single 32-bit word. Reviewed-by: Taylor Simpson <tsimpson@quicinc.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-03-01tcg: Remove tcg_temp_local_new_*, tcg_const_local_*Richard Henderson
These symbols are now unused. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-03-01tcg: Change default temp lifetime to TEMP_TBRichard Henderson
Guest front-ends now get temps that span the lifetime of the translation block by default, which avoids accidentally using the temp across branches and invalidating the data. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-03-01tcg: Don't re-use TEMP_TB temporariesRichard Henderson
Reusing TEMP_TB interferes with detecting whether the temp can be adjusted to TEMP_EBB. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-03-01tcg: Add tcg_temp_ebb_new_{i32,i64,ptr}Richard Henderson
TCG internals will want to be able to allocate and reuse explicitly life-limited temporaries. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-03-01tcg: Add tcg_gen_movi_ptrRichard Henderson
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-03-01tcg: Pass TCGTempKind to tcg_temp_new_internalRichard Henderson
While the argument can only be TEMP_EBB or TEMP_TB, it's more obvious this way. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>