# BST-TIMEOUT: 300 # MEASURED: 33 s on an idle local guest (see the # # ==BST-SUITE== line on any run), so ~9x headroom. Twenty- # # five of the cases are one exec each of a 4.4 MB static # # wasm binary that exits inside arptc_init(), and four more # # exec perl, which is the expensive half. The runner's 60 s # # default is not enough for that on a loaded host. . "$BST_LIB" # arptables 0.0.5+git (SRCREV efae8949) -- meta-openembedded/meta-networking # recipes-support/arptables. THE SUITE'S SHAPE IS DECIDED BY ONE LINE OF THE # SOURCE, SO IT IS QUOTED HERE RATHER THAN SUMMARISED. # # WHICH arptables THIS IS. The standalone netfilter project from # git://git.netfilter.org/arptables: arptables.c + libarptc/ + four extensions # in extensions/arpt_*.c, its own kernel ABI (get/setsockopt on IPPROTO_IP, # ARPT_SO_GET_INFO = 96). It is NOT `arptables-nft`, which is a front-end over # iptables' xtables-nft-multi -- BRINTOS-1362 builds the legacy backend only, so # no nft arm and no arptables-nft exists anywhere in this distro. A18 asserts # that absence so a later change that quietly restores it comes past this suite. # # --------------------------------------------------------------------------- # THERE IS NO OFFLINE SURFACE IN THIS TOOL, AND GROUP C IS THE PROOF, NOT THE # WORKAROUND. # # do_command() (arptables.c:1785) opens like this: # # /* first figure out if this is a 2.6 or a 2.4 kernel */ # *handle = arptc_init(*table); # if (!*handle) { # arptables_insmod("arp_tables", modprobe); # ... # exit_error(VERSION_PROBLEM, "can't initialize arptables table ... # # That is the FIRST executable statement of the function, before getopt_long() # is entered and before argv is looked at. So on a kernel with no arp_tables # EVERY command dies in the same place with the same text and the same exit # status 3 -- `-V`, `-h`, a well-formed rule, a malformed rule, and `-t # some-bogus-table`, whose diagnostic still names `filter' because -t has not # been read yet. # # The ticket that ordered this port expected otherwise: it named the rule parser # and the save-format writer as "deterministic work over a file [that] does not # depend on the kernel". Neither is reachable, and the two escape hatches its # sibling ports used are both absent here -- ebtables' --atomic-file arm has no # analogue (`grep -r atomic` over the whole tree is empty) and iptables' # socket-free iptables-xml has none either. arptables-save and arptables-restore # are perl scripts that shell out to the same binary, so they inherit the wall. # # So group C measures the degeneracy itself: the transcript of twenty-five # distinct command lines, compared BYTE FOR BYTE against the same twenty-five # run on an x86-64 build of the same git revision, on a host made to present # this kernel (src/arptables_nokernel.c intercepts socket/getsockopt/open so the # oracle fails at the same call with the same errno). Twenty-five different # commands agreeing with each other AND with native is a real claim about this # port: a miscompile, a dropped extension object, a broken exit_error() or a # different errno all break it. It is the largest claim this kernel admits. # # WHEN brintos/linux#86 LANDS, GROUP C GOES RED AND GROUP S GOES GREEN. That is # by design and is the signal to re-run src/arptables_mknative.sh against a # kernel that answers -- at which point the matrix stops being degenerate and # the comparison becomes the parser-and-printer comparison the ticket wanted. # # --------------------------------------------------------------------------- # GROUP D IS WHY THIS PORT NEEDED A BBAPPEND, AND IT IS AN ATTRIBUTION FIX. # # arptables-save and arptables-restore both open with # my $tool = "__EXEC_PATH__/arptables"; # unless (-x $tool) { print "ERROR: ... isn't executable"; exit -1; }; # where __EXEC_PATH__ is sed'd to ${sbindir}. The same Makefile's install target # installs exactly one binary and its name is arptables-LEGACY. Upstream leaves # `arptables` to update-alternatives, which this recipe does not inherit -- so as # packaged BOTH FRONT-ENDS DIE ON THEIR OWN FIRST LINE. arptables_%.bbappend # adds the symlink. # # It matters here because the two failures are indistinguishable by exit status: # both are 255. Without the symlink D01/D02 would fail for a PACKAGING reason # and read exactly like the kernel-gated reds in group S -- reporting # brintos/linux#86's blast radius as two cases wider than it is. The assertions # are therefore on the OUTPUT: the front-end must reach the tool and report the # tool's own diagnostic, not its own "isn't executable". # # --------------------------------------------------------------------------- # THE EXPECTED REDS: THREE CASES, ONE GAP, LEFT RED ON PURPOSE. # # S01-S03 -- list, save and restore all reach the kernel. arch/wasm32/configs/ # wasm32_defconfig sets CONFIG_NET=y and CONFIG_INET=y and has no # CONFIG_NETFILTER line at all, so ip_getsockopt()'s nf_getsockopt() call is not # compiled in and do_ip_getsockopt()'s default arm answers -ENOPROTOOPT for # optname 96. brintos/linux#86 (https://brintos.io/brintos/linux/issues/86), the # same gap iptables, ebtables, libnfnetlink and the two libnetfilter_* ports are # held by. Reported failing and NOT skipped, so they flip green by themselves # the day the option lands. # # THE CONTROL THAT SEPARATES "NO NETFILTER" FROM "arptables IS BROKEN" IS GROUP # C ITSELF, and it needs no extra case. The transcript pins the errno: had # socket(AF_INET, SOCK_RAW, IPPROTO_RAW) failed instead -- no CONFIG_INET, or no # CAP_NET_RAW -- TC_INIT would return one call earlier and TC_STRERROR would # print "Permission denied (you must be root)" or a bare strerror, not # "arptables who?". Matching the oracle byte for byte therefore says the socket # opened and the failure is exactly the missing netfilter option. A05 adds the # other half: /proc/sys/kernel/modprobe does not exist here (no CONFIG_MODULES), # which is why the transcript carries no modprobe noise -- get_modprobe() # returns NULL and arptables_insmod() returns -1 without forking. PATH="$PATH:/usr/sbin:/sbin"; export PATH require arptables-legacy arptables/present tmp=${TMPDIR:-/tmp}/bst-arptables.$$ mkdir -p "$tmp" 2>/dev/null BIN=/usr/sbin/arptables-legacy # A suite's FIRST BST result line does not reach the host -- the kernel's benign # post-==BST-BEGIN== audit line displaces it (BRINTOS-1149 host-side, # brintos/linux#96 and linux-in-the-browser#138 for the console loss). The # harness counts total-minus-failures, so a dropped line is counted as a PASS, # which means the eaten position must never carry an assertion anyone depends # on. Remove when BRINTOS-1149 is fixed. bst_ok arptables/suite-canary # --- A: identity, install paths, and the shape of the port ----------------- check arptables/bin-path test -x "$BIN" check arptables/save-path test -x /usr/sbin/arptables-save check arptables/restore-path test -x /usr/sbin/arptables-restore # The symlink the bbappend adds -- both front-ends resolve this exact path. check arptables/alias-path test -x /usr/sbin/arptables check arptables/alias-is-link test -L /usr/sbin/arptables check_out arptables/alias-target "arptables-legacy" readlink /usr/sbin/arptables # A05 -- the control named above: no CONFIG_MODULES, so get_modprobe() finds # nothing and arptables_insmod() cannot fork. Its absence is what keeps the # transcript in group C free of modprobe output. check_rc arptables/no-modprobe-sysctl 1 test -e /proc/sys/kernel/modprobe # perl is a hard runtime dependency: BOTH front-ends are perl scripts. check arptables/perl-present test -x /usr/bin/perl check_out arptables/save-is-perl "/usr/bin/perl" head -1 /usr/sbin/arptables-save check_out arptables/restore-is-perl "/usr/bin/perl" head -1 /usr/sbin/arptables-restore # The front-ends must point at the path the bbappend creates, not at some other # ${sbindir}: the Makefile seds __EXEC_PATH__ at install time and a changed # sbindir would silently re-break group D. check_out arptables/save-tool-path 'my $tool = "/usr/sbin/arptables";' cat /usr/sbin/arptables-save check_out arptables/restore-tool-path 'my $tool = "/usr/sbin/arptables";' cat /usr/sbin/arptables-restore # A18 -- the nft arm is deliberately not part of this distro. Assert its absence # so a later change that restores it has to come past this case. check_rc arptables/no-nft 1 test -e /usr/sbin/arptables-nft check_rc arptables/no-nft-restore 1 test -e /usr/sbin/arptables-nft-restore # --- B: the four extensions are LINKED IN -------------------------------- # Makefile links extensions/arpt_*.o as PLAIN OBJECTS on the arptables-legacy # rule, so unlike ebtables there is no archive for the linker to prune and no # --whole-archive needed. That is a claim, and these cases are what check it: a # dropped object takes its help string with it. The strings are the FORMAT # strings (v%s, not v0.0.5) because that is what is in .rodata. # # Asserting on the binary rather than on `-h ` is forced, not preferred: # the registry cannot be reached at run time on this kernel, because the -h # handler is behind arptc_init(). B05 is the control -- a string that is in no # extension must be absent, or the four above would pass against any file. check arptables/ext-mangle grep -qF -- 'mangle target v%s options:' "$BIN" check arptables/ext-classify grep -qF -- 'CLASSIFY target v%s options:' "$BIN" check arptables/ext-mark grep -qF -- 'MARK target v%s options:' "$BIN" check arptables/ext-standard grep -qF -- 'Standard v%s options:' "$BIN" check_rc arptables/ext-control 1 grep -qF -- 'BRINTOS-no-such-extension' "$BIN" # --- C: the command matrix, byte for byte against the x86-64 oracle -------- . "$BST_DATA/arptables-runcases.sh" mkdir -p "$tmp/scratch" { printf '# arptables 0.0.5 (efae8949e31f8b2eb6290f377a28384cecaf105a) -- x86_64 control, kernel presented by arptables_nokernel.c\n' printf '# regenerate: src/arptables_mknative.sh (BRINTOS-1365)\n' arptables_runcases "$BIN" "$BST_DATA/arptables-cases" "$tmp/scratch" } > "$tmp/got.out" 2>"$tmp/got.err" check arptables/matrix-quiet test ! -s "$tmp/got.err" check arptables/matrix-nonempty test -s "$tmp/got.out" check arptables/matrix-identical cmp -s "$tmp/got.out" "$BST_DATA/arptables-native.out" check_eq arptables/matrix-cases "$(grep -c '^== ' "$BST_DATA/arptables-native.out")" \ "$(grep -c '^== ' "$tmp/got.out")" check_eq arptables/matrix-bytes "$(wc -c < "$BST_DATA/arptables-native.out")" \ "$(wc -c < "$tmp/got.out")" # The one assertion that would survive a regenerated oracle: the table named in # the diagnostic is `filter' even when -t asked for something else. That is the # single most direct statement that argv is not parsed. # # Captured to a file rather than asserted with check_out, and that is not a # style choice: check_out requires rc=0 AND the needle, so every case here whose # WHOLE POINT is a non-zero exit fails with `want:'X' got:'X'`. Measured in the # native dry run before this suite shipped -- three cases reddened that way, the # same trap 99-ebtables.test hit and answered with its own expect_fail. "$BIN" -t brintos-no-such-table -L > "$tmp/tbogus.out" 2>&1 || : check arptables/no-argv-parse \ grep -qF "can't initialize arptables table \`filter'" "$tmp/tbogus.out" # --- D: the perl front-ends REACH the tool --------------------------------- # Both exit 255 whether they reached the tool or not, so the exit status cannot # tell the two apart and the assertion has to be on the TEXT. "arptables who?" # is the tool's own voice, reached only through the bbappend's symlink; # "executable" appears only in the front-end's own -x refusal, which is what # this port would report without it. Each is asserted in both directions -- # present and absent -- because either alone would pass on a file that is empty. # # Captured once into a file and grepped, rather than run under check_out: these # runs exit 255 by design, and check_out requires rc=0 (see group C). # # RUN THROUGH `sh -c`, WHICH IS NOT STYLE. A command whose stdin THIS shell # opens on the virtiofs root, and which reads it, kills THIS shell -- silently: # no exit status, no stderr, no kernel line. The suite simply stops at that # statement and the runner reports a shorter run that reads as all-green. It # cost this port four boots to find, because the first symptom was group D and # group S missing from an otherwise passing suite. # # ALREADY TRACKED: brintos/linux-in-the-browser#172 # (https://brintos.io/brintos/linux-in-the-browser/issues/172), which reports # the same defect with a wider matrix -- it is not the program (cat, tr and # `wc -l` all die), not the size, not the write side, and not reading a virtiofs # file as such, since the same file passed as a filename ARGUMENT is fine. What # survives is one extra fork level. Measured again here on this image, in one # boot, five controls green against the one failing arm: the file as an argument # (`cat /etc/passwd > o`), stdin from a DEVICE (`cat < /dev/null`), the same # redirection inside `$(...)`, inside `( )`, and inside `sh -c` -- only the bare # form in this shell dies, and it dies on `cat`, with no perl and no arptables # involved. # # #172 records that 33 suites on this image already truncate this way and still # report green, so this is not a novel idiom -- it is a trap the corpus is # already in, and the reason group D is written the long way round. # BRINTOS-WORKAROUND: brintos/linux-in-the-browser#172 remove -- drop the sh -c # wrappers once a command can read a shell-opened virtiofs fd as stdin. sh -c 'exec arptables-save' > "$tmp/save.out" 2>&1 || : sh -c 'exec arptables-restore < "$0"' "$BST_DATA/arptables-ref.rules" \ > "$tmp/restore.out" 2>&1 || : check arptables/save-reaches-tool \ grep -qF 'arptables who? (do you need to insmod?)' "$tmp/save.out" check arptables/restore-reaches-tool \ grep -qF 'arptables who? (do you need to insmod?)' "$tmp/restore.out" check_rc arptables/save-not-missing-tool 1 grep -qF 'executable' "$tmp/save.out" check_rc arptables/restore-not-missing-tool 1 grep -qF 'executable' "$tmp/restore.out" # --- S: expected red -- brintos/linux#86, no CONFIG_NETFILTER in this kernel - check arptables/list "$BIN" -L -n check arptables/save arptables-save check arptables/restore sh -c 'arptables-restore < "$0"' "$BST_DATA/arptables-ref.rules" rm -rf "$tmp"