206 lines · plain
1===================2Testing BPF on s3903===================4 51. Introduction6***************7 8IBM Z are mainframe computers, which are descendants of IBM System/360 from9year 1964. They are supported by the Linux kernel under the name "s390". This10document describes how to test BPF in an s390 QEMU guest.11 122. One-time setup13*****************14 15The following is required to build and run the test suite:16 17 * s390 GCC18 * s390 development headers and libraries19 * Clang with BPF support20 * QEMU with s390 support21 * Disk image with s390 rootfs22 23Debian supports installing compiler and libraries for s390 out of the box.24Users of other distros may use debootstrap in order to set up a Debian chroot::25 26 sudo debootstrap \27 --variant=minbase \28 --include=sudo \29 testing \30 ./s390-toolchain31 sudo mount --rbind /dev ./s390-toolchain/dev32 sudo mount --rbind /proc ./s390-toolchain/proc33 sudo mount --rbind /sys ./s390-toolchain/sys34 sudo chroot ./s390-toolchain35 36Once on Debian, the build prerequisites can be installed as follows::37 38 sudo dpkg --add-architecture s390x39 sudo apt-get update40 sudo apt-get install \41 bc \42 bison \43 cmake \44 debootstrap \45 dwarves \46 flex \47 g++ \48 gcc \49 g++-s390x-linux-gnu \50 gcc-s390x-linux-gnu \51 gdb-multiarch \52 git \53 make \54 python3 \55 qemu-system-misc \56 qemu-utils \57 rsync \58 libcap-dev:s390x \59 libelf-dev:s390x \60 libncurses-dev61 62Latest Clang targeting BPF can be installed as follows::63 64 git clone https://github.com/llvm/llvm-project.git65 ln -s ../../clang llvm-project/llvm/tools/66 mkdir llvm-project-build67 cd llvm-project-build68 cmake \69 -DLLVM_TARGETS_TO_BUILD=BPF \70 -DCMAKE_BUILD_TYPE=Release \71 -DCMAKE_INSTALL_PREFIX=/opt/clang-bpf \72 ../llvm-project/llvm73 make74 sudo make install75 export PATH=/opt/clang-bpf/bin:$PATH76 77The disk image can be prepared using a loopback mount and debootstrap::78 79 qemu-img create -f raw ./s390.img 1G80 sudo losetup -f ./s390.img81 sudo mkfs.ext4 /dev/loopX82 mkdir ./s390.rootfs83 sudo mount /dev/loopX ./s390.rootfs84 sudo debootstrap \85 --foreign \86 --arch=s390x \87 --variant=minbase \88 --include=" \89 iproute2, \90 iputils-ping, \91 isc-dhcp-client, \92 kmod, \93 libcap2, \94 libelf1, \95 netcat, \96 procps" \97 testing \98 ./s390.rootfs99 sudo umount ./s390.rootfs100 sudo losetup -d /dev/loopX101 1023. Compilation103**************104 105In addition to the usual Kconfig options required to run the BPF test suite, it106is also helpful to select::107 108 CONFIG_NET_9P=y109 CONFIG_9P_FS=y110 CONFIG_NET_9P_VIRTIO=y111 CONFIG_VIRTIO_PCI=y112 113as that would enable a very easy way to share files with the s390 virtual114machine.115 116Compiling kernel, modules and testsuite, as well as preparing gdb scripts to117simplify debugging, can be done using the following commands::118 119 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- menuconfig120 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- bzImage modules scripts_gdb121 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- \122 -C tools/testing/selftests \123 TARGETS=bpf \124 INSTALL_PATH=$PWD/tools/testing/selftests/kselftest_install \125 install126 1274. Running the test suite128*************************129 130The virtual machine can be started as follows::131 132 qemu-system-s390x \133 -cpu max,zpci=on \134 -smp 2 \135 -m 4G \136 -kernel linux/arch/s390/boot/compressed/vmlinux \137 -drive file=./s390.img,if=virtio,format=raw \138 -nographic \139 -append 'root=/dev/vda rw console=ttyS1' \140 -virtfs local,path=./linux,security_model=none,mount_tag=linux \141 -object rng-random,filename=/dev/urandom,id=rng0 \142 -device virtio-rng-ccw,rng=rng0 \143 -netdev user,id=net0 \144 -device virtio-net-ccw,netdev=net0145 146When using this on a real IBM Z, ``-enable-kvm`` may be added for better147performance. When starting the virtual machine for the first time, disk image148setup must be finalized using the following command::149 150 /debootstrap/debootstrap --second-stage151 152Directory with the code built on the host as well as ``/proc`` and ``/sys``153need to be mounted as follows::154 155 mkdir -p /linux156 mount -t 9p linux /linux157 mount -t proc proc /proc158 mount -t sysfs sys /sys159 160After that, the test suite can be run using the following commands::161 162 cd /linux/tools/testing/selftests/kselftest_install163 ./run_kselftest.sh164 165As usual, tests can be also run individually::166 167 cd /linux/tools/testing/selftests/bpf168 ./test_verifier169 1705. Debugging171************172 173It is possible to debug the s390 kernel using QEMU GDB stub, which is activated174by passing ``-s`` to QEMU.175 176It is preferable to turn KASLR off, so that gdb would know where to find the177kernel image in memory, by building the kernel with::178 179 RANDOMIZE_BASE=n180 181GDB can then be attached using the following command::182 183 gdb-multiarch -ex 'target remote localhost:1234' ./vmlinux184 1856. Network186**********187 188In case one needs to use the network in the virtual machine in order to e.g.189install additional packages, it can be configured using::190 191 dhclient eth0192 1937. Links194********195 196This document is a compilation of techniques, whose more comprehensive197descriptions can be found by following these links:198 199- `Debootstrap <https://wiki.debian.org/EmDebian/CrossDebootstrap>`_200- `Multiarch <https://wiki.debian.org/Multiarch/HOWTO>`_201- `Building LLVM <https://llvm.org/docs/CMake.html>`_202- `Cross-compiling the kernel <https://wiki.gentoo.org/wiki/Embedded_Handbook/General/Cross-compiling_the_kernel>`_203- `QEMU s390x Guest Support <https://wiki.qemu.org/Documentation/Platforms/S390X>`_204- `Plan 9 folder sharing over Virtio <https://wiki.qemu.org/Documentation/9psetup>`_205- `Using GDB with QEMU <https://wiki.osdev.org/Kernel_Debugging#Use_GDB_with_QEMU>`_206