223 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3=================================4The PPC KVM paravirtual interface5=================================6 7The basic execution principle by which KVM on PowerPC works is to run all kernel8space code in PR=1 which is user space. This way we trap all privileged9instructions and can emulate them accordingly.10 11Unfortunately that is also the downfall. There are quite some privileged12instructions that needlessly return us to the hypervisor even though they13could be handled differently.14 15This is what the PPC PV interface helps with. It takes privileged instructions16and transforms them into unprivileged ones with some help from the hypervisor.17This cuts down virtualization costs by about 50% on some of my benchmarks.18 19The code for that interface can be found in arch/powerpc/kernel/kvm*20 21Querying for existence22======================23 24To find out if we're running on KVM or not, we leverage the device tree. When25Linux is running on KVM, a node /hypervisor exists. That node contains a26compatible property with the value "linux,kvm".27 28Once you determined you're running under a PV capable KVM, you can now use29hypercalls as described below.30 31KVM hypercalls32==============33 34Inside the device tree's /hypervisor node there's a property called35'hypercall-instructions'. This property contains at most 4 opcodes that make36up the hypercall. To call a hypercall, just call these instructions.37 38The parameters are as follows:39 40 ======== ================ ================41 Register IN OUT42 ======== ================ ================43 r0 - volatile44 r3 1st parameter Return code45 r4 2nd parameter 1st output value46 r5 3rd parameter 2nd output value47 r6 4th parameter 3rd output value48 r7 5th parameter 4th output value49 r8 6th parameter 5th output value50 r9 7th parameter 6th output value51 r10 8th parameter 7th output value52 r11 hypercall number 8th output value53 r12 - volatile54 ======== ================ ================55 56Hypercall definitions are shared in generic code, so the same hypercall numbers57apply for x86 and powerpc alike with the exception that each KVM hypercall58also needs to be ORed with the KVM vendor code which is (42 << 16).59 60Return codes can be as follows:61 62 ==== =========================63 Code Meaning64 ==== =========================65 0 Success66 12 Hypercall not implemented67 <0 Error68 ==== =========================69 70The magic page71==============72 73To enable communication between the hypervisor and guest there is a new shared74page that contains parts of supervisor visible register state. The guest can75map this shared page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE.76 77With this hypercall issued the guest always gets the magic page mapped at the78desired location. The first parameter indicates the effective address when the79MMU is enabled. The second parameter indicates the address in real mode, if80applicable to the target. For now, we always map the page to -4096. This way we81can access it using absolute load and store functions. The following82instruction reads the first field of the magic page::83 84 ld rX, -4096(0)85 86The interface is designed to be extensible should there be need later to add87additional registers to the magic page. If you add fields to the magic page,88also define a new hypercall feature to indicate that the host can give you more89registers. Only if the host supports the additional features, make use of them.90 91The magic page layout is described by struct kvm_vcpu_arch_shared92in arch/powerpc/include/uapi/asm/kvm_para.h.93 94Magic page features95===================96 97When mapping the magic page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE,98a second return value is passed to the guest. This second return value contains99a bitmap of available features inside the magic page.100 101The following enhancements to the magic page are currently available:102 103 ============================ =======================================104 KVM_MAGIC_FEAT_SR Maps SR registers r/w in the magic page105 KVM_MAGIC_FEAT_MAS0_TO_SPRG7 Maps MASn, ESR, PIR and high SPRGs106 ============================ =======================================107 108For enhanced features in the magic page, please check for the existence of the109feature before using them!110 111Magic page flags112================113 114In addition to features that indicate whether a host is capable of a particular115feature we also have a channel for a guest to tell the host whether it's capable116of something. This is what we call "flags".117 118Flags are passed to the host in the low 12 bits of the Effective Address.119 120The following flags are currently available for a guest to expose:121 122 MAGIC_PAGE_FLAG_NOT_MAPPED_NX Guest handles NX bits correctly wrt magic page123 124MSR bits125========126 127The MSR contains bits that require hypervisor intervention and bits that do128not require direct hypervisor intervention because they only get interpreted129when entering the guest or don't have any impact on the hypervisor's behavior.130 131The following bits are safe to be set inside the guest:132 133 - MSR_EE134 - MSR_RI135 136If any other bit changes in the MSR, please still use mtmsr(d).137 138Patched instructions139====================140 141The "ld" and "std" instructions are transformed to "lwz" and "stw" instructions142respectively on 32-bit systems with an added offset of 4 to accommodate for big143endianness.144 145The following is a list of mapping the Linux kernel performs when running as146guest. Implementing any of those mappings is optional, as the instruction traps147also act on the shared page. So calling privileged instructions still works as148before.149 150======================= ================================151From To152======================= ================================153mfmsr rX ld rX, magic_page->msr154mfsprg rX, 0 ld rX, magic_page->sprg0155mfsprg rX, 1 ld rX, magic_page->sprg1156mfsprg rX, 2 ld rX, magic_page->sprg2157mfsprg rX, 3 ld rX, magic_page->sprg3158mfsrr0 rX ld rX, magic_page->srr0159mfsrr1 rX ld rX, magic_page->srr1160mfdar rX ld rX, magic_page->dar161mfdsisr rX lwz rX, magic_page->dsisr162 163mtmsr rX std rX, magic_page->msr164mtsprg 0, rX std rX, magic_page->sprg0165mtsprg 1, rX std rX, magic_page->sprg1166mtsprg 2, rX std rX, magic_page->sprg2167mtsprg 3, rX std rX, magic_page->sprg3168mtsrr0 rX std rX, magic_page->srr0169mtsrr1 rX std rX, magic_page->srr1170mtdar rX std rX, magic_page->dar171mtdsisr rX stw rX, magic_page->dsisr172 173tlbsync nop174 175mtmsrd rX, 0 b <special mtmsr section>176mtmsr rX b <special mtmsr section>177 178mtmsrd rX, 1 b <special mtmsrd section>179 180[Book3S only]181mtsrin rX, rY b <special mtsrin section>182 183[BookE only]184wrteei [0|1] b <special wrteei section>185======================= ================================186 187Some instructions require more logic to determine what's going on than a load188or store instruction can deliver. To enable patching of those, we keep some189RAM around where we can live translate instructions to. What happens is the190following:191 192 1) copy emulation code to memory193 2) patch that code to fit the emulated instruction194 3) patch that code to return to the original pc + 4195 4) patch the original instruction to branch to the new code196 197That way we can inject an arbitrary amount of code as replacement for a single198instruction. This allows us to check for pending interrupts when setting EE=1199for example.200 201Hypercall ABIs in KVM on PowerPC202=================================203 2041) KVM hypercalls (ePAPR)205 206These are ePAPR compliant hypercall implementation (mentioned above). Even207generic hypercalls are implemented here, like the ePAPR idle hcall. These are208available on all targets.209 2102) PAPR hypercalls211 212PAPR hypercalls are needed to run server PowerPC PAPR guests (-M pseries in QEMU).213These are the same hypercalls that pHyp, the POWER hypervisor, implements. Some of214them are handled in the kernel, some are handled in user space. This is only215available on book3s_64.216 2173) OSI hypercalls218 219Mac-on-Linux is another user of KVM on PowerPC, which has its own hypercall (long220before KVM). This is supported to maintain compatibility. All these hypercalls get221forwarded to user space. This is only useful on book3s_32, but can be used with222book3s_64 as well.223