275 lines · plain
1============================2Transactional Memory support3============================4 5POWER kernel support for this feature is currently limited to supporting6its use by user programs. It is not currently used by the kernel itself.7 8This file aims to sum up how it is supported by Linux and what behaviour you9can expect from your user programs.10 11 12Basic overview13==============14 15Hardware Transactional Memory is supported on POWER8 processors, and is a16feature that enables a different form of atomic memory access. Several new17instructions are presented to delimit transactions; transactions are18guaranteed to either complete atomically or roll back and undo any partial19changes.20 21A simple transaction looks like this::22 23 begin_move_money:24 tbegin25 beq abort_handler26 27 ld r4, SAVINGS_ACCT(r3)28 ld r5, CURRENT_ACCT(r3)29 subi r5, r5, 130 addi r4, r4, 131 std r4, SAVINGS_ACCT(r3)32 std r5, CURRENT_ACCT(r3)33 34 tend35 36 b continue37 38 abort_handler:39 ... test for odd failures ...40 41 /* Retry the transaction if it failed because it conflicted with42 * someone else: */43 b begin_move_money44 45 46The 'tbegin' instruction denotes the start point, and 'tend' the end point.47Between these points the processor is in 'Transactional' state; any memory48references will complete in one go if there are no conflicts with other49transactional or non-transactional accesses within the system. In this50example, the transaction completes as though it were normal straight-line code51IF no other processor has touched SAVINGS_ACCT(r3) or CURRENT_ACCT(r3); an52atomic move of money from the current account to the savings account has been53performed. Even though the normal ld/std instructions are used (note no54lwarx/stwcx), either *both* SAVINGS_ACCT(r3) and CURRENT_ACCT(r3) will be55updated, or neither will be updated.56 57If, in the meantime, there is a conflict with the locations accessed by the58transaction, the transaction will be aborted by the CPU. Register and memory59state will roll back to that at the 'tbegin', and control will continue from60'tbegin+4'. The branch to abort_handler will be taken this second time; the61abort handler can check the cause of the failure, and retry.62 63Checkpointed registers include all GPRs, FPRs, VRs/VSRs, LR, CCR/CR, CTR, FPCSR64and a few other status/flag regs; see the ISA for details.65 66Causes of transaction aborts67============================68 69- Conflicts with cache lines used by other processors70- Signals71- Context switches72- See the ISA for full documentation of everything that will abort transactions.73 74 75Syscalls76========77 78Syscalls made from within an active transaction will not be performed and the79transaction will be doomed by the kernel with the failure code TM_CAUSE_SYSCALL80| TM_CAUSE_PERSISTENT.81 82Syscalls made from within a suspended transaction are performed as normal and83the transaction is not explicitly doomed by the kernel. However, what the84kernel does to perform the syscall may result in the transaction being doomed85by the hardware. The syscall is performed in suspended mode so any side86effects will be persistent, independent of transaction success or failure. No87guarantees are provided by the kernel about which syscalls will affect88transaction success.89 90Care must be taken when relying on syscalls to abort during active transactions91if the calls are made via a library. Libraries may cache values (which may92give the appearance of success) or perform operations that cause transaction93failure before entering the kernel (which may produce different failure codes).94Examples are glibc's getpid() and lazy symbol resolution.95 96 97Signals98=======99 100Delivery of signals (both sync and async) during transactions provides a second101thread state (ucontext/mcontext) to represent the second transactional register102state. Signal delivery 'treclaim's to capture both register states, so signals103abort transactions. The usual ucontext_t passed to the signal handler104represents the checkpointed/original register state; the signal appears to have105arisen at 'tbegin+4'.106 107If the sighandler ucontext has uc_link set, a second ucontext has been108delivered. For future compatibility the MSR.TS field should be checked to109determine the transactional state -- if so, the second ucontext in uc->uc_link110represents the active transactional registers at the point of the signal.111 112For 64-bit processes, uc->uc_mcontext.regs->msr is a full 64-bit MSR and its TS113field shows the transactional mode.114 115For 32-bit processes, the mcontext's MSR register is only 32 bits; the top 32116bits are stored in the MSR of the second ucontext, i.e. in117uc->uc_link->uc_mcontext.regs->msr. The top word contains the transactional118state TS.119 120However, basic signal handlers don't need to be aware of transactions121and simply returning from the handler will deal with things correctly:122 123Transaction-aware signal handlers can read the transactional register state124from the second ucontext. This will be necessary for crash handlers to125determine, for example, the address of the instruction causing the SIGSEGV.126 127Example signal handler::128 129 void crash_handler(int sig, siginfo_t *si, void *uc)130 {131 ucontext_t *ucp = uc;132 ucontext_t *transactional_ucp = ucp->uc_link;133 134 if (ucp_link) {135 u64 msr = ucp->uc_mcontext.regs->msr;136 /* May have transactional ucontext! */137 #ifndef __powerpc64__138 msr |= ((u64)transactional_ucp->uc_mcontext.regs->msr) << 32;139 #endif140 if (MSR_TM_ACTIVE(msr)) {141 /* Yes, we crashed during a transaction. Oops. */142 fprintf(stderr, "Transaction to be restarted at 0x%llx, but "143 "crashy instruction was at 0x%llx\n",144 ucp->uc_mcontext.regs->nip,145 transactional_ucp->uc_mcontext.regs->nip);146 }147 }148 149 fix_the_problem(ucp->dar);150 }151 152When in an active transaction that takes a signal, we need to be careful with153the stack. It's possible that the stack has moved back up after the tbegin.154The obvious case here is when the tbegin is called inside a function that155returns before a tend. In this case, the stack is part of the checkpointed156transactional memory state. If we write over this non transactionally or in157suspend, we are in trouble because if we get a tm abort, the program counter and158stack pointer will be back at the tbegin but our in memory stack won't be valid159anymore.160 161To avoid this, when taking a signal in an active transaction, we need to use162the stack pointer from the checkpointed state, rather than the speculated163state. This ensures that the signal context (written tm suspended) will be164written below the stack required for the rollback. The transaction is aborted165because of the treclaim, so any memory written between the tbegin and the166signal will be rolled back anyway.167 168For signals taken in non-TM or suspended mode, we use the169normal/non-checkpointed stack pointer.170 171Any transaction initiated inside a sighandler and suspended on return172from the sighandler to the kernel will get reclaimed and discarded.173 174Failure cause codes used by kernel175==================================176 177These are defined in <asm/reg.h>, and distinguish different reasons why the178kernel aborted a transaction:179 180 ====================== ================================181 TM_CAUSE_RESCHED Thread was rescheduled.182 TM_CAUSE_TLBI Software TLB invalid.183 TM_CAUSE_FAC_UNAV FP/VEC/VSX unavailable trap.184 TM_CAUSE_SYSCALL Syscall from active transaction.185 TM_CAUSE_SIGNAL Signal delivered.186 TM_CAUSE_MISC Currently unused.187 TM_CAUSE_ALIGNMENT Alignment fault.188 TM_CAUSE_EMULATE Emulation that touched memory.189 ====================== ================================190 191These can be checked by the user program's abort handler as TEXASR[0:7]. If192bit 7 is set, it indicates that the error is considered persistent. For example193a TM_CAUSE_ALIGNMENT will be persistent while a TM_CAUSE_RESCHED will not.194 195GDB196===197 198GDB and ptrace are not currently TM-aware. If one stops during a transaction,199it looks like the transaction has just started (the checkpointed state is200presented). The transaction cannot then be continued and will take the failure201handler route. Furthermore, the transactional 2nd register state will be202inaccessible. GDB can currently be used on programs using TM, but not sensibly203in parts within transactions.204 205POWER9206======207 208TM on POWER9 has issues with storing the complete register state. This209is described in this commit::210 211 commit 4bb3c7a0208fc13ca70598efd109901a7cd45ae7212 Author: Paul Mackerras <paulus@ozlabs.org>213 Date: Wed Mar 21 21:32:01 2018 +1100214 KVM: PPC: Book3S HV: Work around transactional memory bugs in POWER9215 216To account for this different POWER9 chips have TM enabled in217different ways.218 219On POWER9N DD2.01 and below, TM is disabled. ie220HWCAP2[PPC_FEATURE2_HTM] is not set.221 222On POWER9N DD2.1 TM is configured by firmware to always abort a223transaction when tm suspend occurs. So tsuspend will cause a224transaction to be aborted and rolled back. Kernel exceptions will also225cause the transaction to be aborted and rolled back and the exception226will not occur. If userspace constructs a sigcontext that enables TM227suspend, the sigcontext will be rejected by the kernel. This mode is228advertised to users with HWCAP2[PPC_FEATURE2_HTM_NO_SUSPEND] set.229HWCAP2[PPC_FEATURE2_HTM] is not set in this mode.230 231On POWER9N DD2.2 and above, KVM and POWERVM emulate TM for guests (as232described in commit 4bb3c7a0208f), hence TM is enabled for guests233ie. HWCAP2[PPC_FEATURE2_HTM] is set for guest userspace. Guests that234makes heavy use of TM suspend (tsuspend or kernel suspend) will result235in traps into the hypervisor and hence will suffer a performance236degradation. Host userspace has TM disabled237ie. HWCAP2[PPC_FEATURE2_HTM] is not set. (although we make enable it238at some point in the future if we bring the emulation into host239userspace context switching).240 241POWER9C DD1.2 and above are only available with POWERVM and hence242Linux only runs as a guest. On these systems TM is emulated like on243POWER9N DD2.2.244 245Guest migration from POWER8 to POWER9 will work with POWER9N DD2.2 and246POWER9C DD1.2. Since earlier POWER9 processors don't support TM247emulation, migration from POWER8 to POWER9 is not supported there.248 249Kernel implementation250=====================251 252h/rfid mtmsrd quirk253-------------------254 255As defined in the ISA, rfid has a quirk which is useful in early256exception handling. When in a userspace transaction and we enter the257kernel via some exception, MSR will end up as TM=0 and TS=01 (ie. TM258off but TM suspended). Regularly the kernel will want change bits in259the MSR and will perform an rfid to do this. In this case rfid can260have SRR0 TM = 0 and TS = 00 (ie. TM off and non transaction) and the261resulting MSR will retain TM = 0 and TS=01 from before (ie. stay in262suspend). This is a quirk in the architecture as this would normally263be a transition from TS=01 to TS=00 (ie. suspend -> non transactional)264which is an illegal transition.265 266This quirk is described the architecture in the definition of rfid267with these lines:268 269 if (MSR 29:31 ¬ = 0b010 | SRR1 29:31 ¬ = 0b000) then270 MSR 29:31 <- SRR1 29:31271 272hrfid and mtmsrd have the same quirk.273 274The Linux kernel uses this quirk in its early exception handling.275