# BST-TIMEOUT: 300 # The suite execs the tool ~45 times. Every case that can # # reach a socket is run under the bounded runner below at # # NBOUND=10s, and there are 12 of them, so the worst case # # -- every socket command hanging -- is ~120s plus the # # userland group's handful of instant execs. 300 is the # # bound the ticket gates and the nightly pass. . "$BST_LIB" # nfacct 1.0.2 (meta-networking recipes-filter/nfacct) -- the command-line tool # for the kernel's extended accounting infrastructure: it creates, lists, # reads, resets, flushes and deletes the named packet/byte counters that # netfilter rules increment. # # --------------------------------------------------------------------------- # WHAT THIS SUITE IS FOR, GIVEN THAT ONE ALREADY DRIVES THIS BINARY. # # 79-libnetfilter-acct's T group drives nfacct too, and deliberately: that # ticket's pass criterion was written against the tool, so the tool is how it # is scored. This suite is not a copy of it and does not restate it. The split # is by SUBJECT, not by binary: # # * 79-libnetfilter-acct asks "does the LIBRARY frame, parse and print the # accounting messages correctly" -- 111 of its cases run on a kernel with # no netfilter at all, through a probe -- and drives the tool only for the # object-lifecycle criterion its own ticket names. # * THIS suite asks "does the TOOL behave", which is a different surface: # the argument grammar, the six usage-error paths, the prefix dispatch, # the two format words, the empty-table answers, the object-name length # rule, and -- the assertion the ticket turns on -- that every command # TERMINATES rather than blocking in recvmsg. # # The overlap is four cases (version, help, no-args, unknown-command) which # both suites pin because both need them as the "the binary really runs" # floor. That is cheap and it is the right kind of duplication: either suite # must be readable alone. # # --------------------------------------------------------------------------- # THREE THINGS THE TICKET ASSERTS THAT THE SOURCE DOES NOT DO. # # All three were MEASURED on an x86-64 build of the identical three sources # (nfacct 1.0.2 + libnetfilter_acct 1.0.3 + libmnl 1.0.5), driven under # `unshare -rn` against a kernel that has nfnetlink_acct. They are pinned here # as the upstream behaviour they are, not filed as defects: # # 1. "`nfacct list` ... report an empty table". IT PRINTS NOTHING AT ALL. # The header, the rows and the footer are all emitted from nfacct_cb(), # which the dump never calls when the table is empty, so the output is # zero bytes and the status is 0. An empty table and a successful dump of # nothing are indistinguishable from the outside. # # 2. "`nfacct list --xml` ... emit well-formed XML for that same empty set". # TWO ERRORS IN ONE PHRASE. # (a) `--xml` IS NOT THE SPELLING. The format is a positional word -- # `nfacct list xml` -- and `--xml` falls through the argument loop # to `nfacct_perror("unknown argument")`, exit 1. Pinned as # list-dashdash-xml below, because it is the spelling a reader of # this ticket will try first. # (b) EVEN SPELLED RIGHT IT EMITS NO XML for an empty set. `` # and `` are printed inside nfacct_cb() on the first row, # and `_print_footer()` emits `` only `if (fmt_header)`. # No rows means no header, so no footer: zero bytes, status 0. Zero # bytes is not a well-formed XML document -- a document needs a root # element -- so the ticket's assertion cannot be satisfied by this # release on any kernel, and asserting it would have produced a red # nobody could ever clear. # # 3. "the documented rejection of an object name longer than the protocol # allows". THERE IS NO REJECTION ANYWHERE. nfacct does not check the # length, and libnetfilter_acct's setter is # # strncpy(nfacct->name, data, NFACCT_NAME_MAX); /* 32 */ # nfacct->name[NFACCT_NAME_MAX-1] = '\0'; # # -- a SILENT TRUNCATION to 31 characters. Measured: `nfacct add` with a # 64-character name exits 0 and creates an object named 31 characters # long, and `nfacct get` with the same 64-character argument finds it, # because the query is truncated by the same setter. Two distinct names # differing only after character 31 therefore denote ONE object, silently. # That is a real hazard for a caller and it is pinned as behaviour in the # N group rather than asserted as a rejection that does not exist. # # --------------------------------------------------------------------------- # WHAT THIS IMAGE CAN AND CANNOT RUN, AND HOW THE GROUPS SPLIT ON IT. # # Every command that touches an accounting object opens # mnl_socket_open(NETLINK_NETFILTER) -- protocol 12, which this kernel does # not register at all: wasm32_defconfig sets CONFIG_NET=y, CONFIG_INET=y, # CONFIG_UNIX=y and no CONFIG_NETFILTER of any kind (brintos/linux#86). So # socket(AF_NETLINK, SOCK_RAW, 12) answers EPROTONOSUPPORT and the tool prints # `nfacct v1.0.2: mnl_socket_open: Protocol not supported` and exits 1. # # * U -- the userland grammar. No socket is reached: every one of these # validates and returns before mnl_socket_open(). These are GREEN on this # image, and they are what says the port is not merely installed but runs. # * T -- TERMINATION. Every socket-touching command, run under the bounded # runner, asserted to EXIT rather than block. GREEN on this image, and the # half of the ticket that is actually measurable here. # * S -- the documented RESULTS of those same commands. RED on this image, # every one attributed to brintos/linux#86, and they flip green by # themselves on the day the kernel option lands. That is the regression # signal this port is bought for. # * N -- the object-name length rule, split the same way: the half that # needs no kernel is green, the half that does is red and attributed. # # T AND S ARE DELIBERATELY SEPARATE CASES OVER THE SAME COMMAND. Folding them # would mean asserting the documented result and reading a hang as "wrong # output", which is exactly the confusion the ticket asks to be able to tell # apart: "to exit rather than hang" is a different claim from "to answer # correctly", and on this image the first is true and the second is not. # # --------------------------------------------------------------------------- # WHAT THIS SUITE DELIBERATELY DOES NOT DO. # # `nfacct monitor` is not run. It is an unbounded recv() loop waiting for # overquota notifications with no self-terminating path, so on a kernel that # HAS netfilter it blocks forever by design -- asserting that it terminates # would be asserting a bug. It is the one command whose T-group case would be # wrong on the platform this port is aiming at. # # The counters INCREMENTING is not exercised. That needs an iptables/nftables # rule carrying an `-m nfacct` match, and therefore the whole netfilter hook # machinery -- two gaps past what this port is about. R=${BST_NFACCT_ROOT:-} NFACCT=${BST_NFACCT:-$R/usr/sbin/nfacct} # Asserted rather than assumed: a recipe bumped without regenerating the pinned # strings below must go red on ONE named case instead of silently changing the # subject of the other forty. WANT_VERSION=1.0.2 # The bound for one socket-touching command. Ten seconds is two orders of # magnitude above what an EPROTONOSUPPORT socket(2) costs and still leaves the # whole T group inside the suite's own 300s. NBOUND=10 if [ ! -x "$NFACCT" ]; then bst_skip nfacct/present "$NFACCT absent from the image" exit 0 fi bst_ok nfacct/present NW=${TMPDIR:-/tmp}/bst-nfacct.$$ mkdir -p "$NW" 2>/dev/null # --------------------------------------------------------------------------- # THE BOUNDED RUNNER. # # bst_probe cannot be used here: it emits its own case and scores rc=0 as the # pass, while every socket case in this suite expects rc=1 today and rc=0 only # once the kernel gains netfilter. So the bounding is reused and the scoring is # this suite's own. # # THE LOOP'S SHAPE IS bst_probe'S AND THE REASONS ARE ITS REASONS, restated # because getting any of them wrong reintroduces a hang the suite exists to # detect: # * NOT timeout(1). It can kill nothing this loop cannot (brintos/linux#21), # and before the brintos/linux#77 fix its timer never fired at all, which # made it a second hang rather than a bound. # * A COMPLETION SENTINEL file, not `kill -0`: a fast command's zombie is not # reaped inside a tight poll, so `kill -0` reads it as alive forever. # * /proc/uptime, not a counter -- it is the clock that advances here. # * `sleep 1` and BUILTINS ONLY in the loop body. Without the sleep the poll # runs as fast as the guest schedules it and grows the host process without # bound (brintos/hardwarejs#20); and the sleep must not follow a forked # child, because a relative sleep placed directly after the shell reaps a # foreground child returns instantly (brintos/linux-in-the-browser#120). # # Sets _n_rc and leaves the merged output at $NW/out. Returns 124 on overrun. _n_bounded() { _n_bound=$1; shift _n_rcf=$NW/out.rc rm -f "$NW/out" "$_n_rcf" { "$@" >"$NW/out" 2>&1; echo $? >"$_n_rcf"; } & _n_pid=$! _n_t0= if [ -r /proc/uptime ] && read -r _n_u _ < /proc/uptime; then _n_t0=${_n_u%.*}; fi _n_ticks=0 while [ ! -e "$_n_rcf" ]; do _n_d=$_n_ticks if [ -n "$_n_t0" ] && read -r _n_u _ < /proc/uptime; then _n_d=$(( ${_n_u%.*} - _n_t0 )) fi if [ "$_n_d" -ge "$_n_bound" ]; then kill -9 "$_n_pid" 2>/dev/null return 124 fi sleep 1 _n_ticks=$((_n_ticks + 1)) done _n_rc=0 # BST-REDIRECT-OK: $NW is ${TMPDIR:-/tmp}, tmpfs, never the virtiofs rootfs [ -s "$_n_rcf" ] && read -r _n_rc < "$_n_rcf" 2>/dev/null case "$_n_rc" in ''|*[!0-9]*) _n_rc=0 ;; esac rm -f "$_n_rcf" return 0 } # The attribution, keyed on WHAT THE TOOL ITSELF PRINTED and only then on # anything else. Keying on "a socket command failed, therefore brintos/linux#86" # is the obvious rule and it is WRONG: run this same binary natively without a # namespace and the socket opens fine while every command past it answers # EPERM, so that rule prints "protocol 12 is not registered" over a red whose # actual cause is that we may not use it -- sending the reader to a kernel # .config for a permissions problem. That arm is not hypothetical; it is one of # the two arms this port was measured on. _nattrib() { case "$1" in *"mnl_socket_open"*"Protocol not supported"*) printf '%s' "brintos/linux#86: NETLINK_NETFILTER (protocol 12) is not registered -- wasm32_defconfig carries no CONFIG_NETFILTER" ;; *"mnl_socket_open"*) printf '%s' "mnl_socket_open failed, but NOT with EPROTONOSUPPORT -- read the errno in got:; brintos/linux#86 covers the unregistered-protocol case only" ;; *"Operation not permitted"*) printf '%s' "EPERM -- protocol 12 IS registered here and the process lacks CAP_NET_ADMIN; this is NOT brintos/linux#86" ;; *) printf '%s' "unattributed" ;; esac } # n_case -- # # AN EMPTY MEANS THE TOOL MUST PRINT NOTHING, not "any output # will do": `nfacct add`, `delete`, `flush` and an empty `list` all succeed # silently, and a substring test against "" matches every string including an # error message. That distinction is the whole of case S01 -- an empty `list` # is scored by the absence of output, because there is nothing else to score. n_case() { _nid=$1; _nwrc=$2; _nsub=$3; shift 3 if ! _n_bounded "$NBOUND" "$@"; then bst_notok "nfacct/$_nid" "DID NOT EXIT within ${NBOUND}s -- the command blocked instead of returning; last line: $(_bst_1line "$(tail -n 1 "$NW/out" 2>/dev/null)")" return fi _nout=$(cat "$NW/out" 2>/dev/null) if [ -z "$_nsub" ]; then [ -z "$_nout" ] && _nhit=1 || _nhit=0 else case "$_nout" in *"$_nsub"*) _nhit=1 ;; *) _nhit=0 ;; esac fi if [ "$_n_rc" -eq "$_nwrc" ] && [ "$_nhit" -eq 1 ]; then bst_ok "nfacct/$_nid" elif bst_execfail "$_nout"; then bst_unrun "nfacct/$_nid" "$_nout" else bst_notok "nfacct/$_nid" "rc=$_n_rc want-rc=$_nwrc want:'${_nsub:-}' got:'$(_bst_1line "$_nout")' -- $(_nattrib "$_nout")" fi } # n_exits -- : the T-group form. Asserts ONLY that the command # returned inside the bound and did so through exit(2) rather than a signal -- # nothing about what it printed or which status it chose, because on this image # the status is 1 and on a netfilter kernel it is 0 and BOTH are correct here. # # A status above 128 is a death by signal, which is the crash half of the # ticket's "rather than to block on recvmsg or abort". n_exits() { _nid=$1; shift if ! _n_bounded "$NBOUND" "$@"; then bst_notok "nfacct/$_nid" "DID NOT EXIT within ${NBOUND}s -- blocked rather than returning; last line: $(_bst_1line "$(tail -n 1 "$NW/out" 2>/dev/null)")" return fi _nout=$(cat "$NW/out" 2>/dev/null) if bst_execfail "$_nout"; then bst_unrun "nfacct/$_nid" "$_nout" elif [ "$_n_rc" -gt 128 ]; then bst_notok "nfacct/$_nid" "exited on a signal (rc=$_n_rc), not through exit(2) -- got:'$(_bst_1line "$_nout")'" else bst_ok "nfacct/$_nid" fi } # --------------------------------------------------------------------------- # P -- packaging. Where the binary is, which a consumer's scripts depend on. # Non-empty regular file: the presence guard above already proved it is # executable, so re-asserting that would be a case that cannot fail. check nfacct/regular-file test -s "$NFACCT" # ${sbindir}, not ${bindir}: the recipe installs it as a superuser tool and the # bbappend's do_install:append bbfatals if it is not there. Pinned so a move # shows up here rather than as "command not found" in somebody else's suite. case "$NFACCT" in */sbin/nfacct) bst_ok nfacct/sbin-path ;; *) bst_notok nfacct/sbin-path "expected .../sbin/nfacct, got '$NFACCT'" ;; esac # --------------------------------------------------------------------------- # U -- THE USERLAND GRAMMAR. No socket is opened on any path below; every one # of these validates and returns first. These are the port's GREENS, and they # are what separates "the binary is installed" from "the binary runs". n_case version 0 "nfacct v$WANT_VERSION: utility for the Netfilter extended accounting infrastructure" "$NFACCT" version n_case version-license 0 "ABSOLUTELY NO WARRANTY" "$NFACCT" version n_case help 0 "Usage:" "$NFACCT" help # The help text is the tool's own list of what it can do, so it is asserted # command by command: a release that drops one from the table has changed the # interface, and a bare "Usage:" match would not notice. n_case help-list 0 "list [reset]" "$NFACCT" help n_case help-add 0 "add object-name" "$NFACCT" help n_case help-delete 0 "delete object-name" "$NFACCT" help n_case help-get 0 "get object-name" "$NFACCT" help n_case help-flush 0 "flush" "$NFACCT" help n_case help-restore 0 "restore" "$NFACCT" help n_case help-monitor 0 "monitor" "$NFACCT" help # No arguments at all: main() prints the usage to STDERR and exits EXIT_FAILURE. n_case no-args 1 "Usage:" "$NFACCT" # The ticket's "usage error for an unknown subcommand". n_case unknown-command 1 "Unknown command: frobnicate" "$NFACCT" frobnicate # ... and the usage line that follows it, which is the half that tells the user # what to do instead. n_case unknown-command-usage 1 "Usage:" "$NFACCT" frobnicate # THE DISPATCH IS A PREFIX MATCH, and that is upstream's design rather than an # accident: main() compares with strncmp(argv[1], "version", strlen(argv[1])), # so any unambiguous leading substring selects the command. Pinned because a # consumer's script written against `nfacct v` keeps working only as long as # this holds, and because it is the reason an EMPTY argv[1] would select the # first arm of the chain -- strlen("") is 0 and strncmp with 0 always matches. n_case prefix-version 0 "nfacct v$WANT_VERSION" "$NFACCT" ver n_case prefix-version-1char 0 "nfacct v$WANT_VERSION" "$NFACCT" v n_case prefix-help 0 "Commands:" "$NFACCT" hel # A longer string is NOT a prefix and must not match: strlen is the user's, so # "listing" compares seven bytes against "list\0" and differs. n_case prefix-overlong-rejected 1 "Unknown command: listing" "$NFACCT" listing # The ticket's "usage error ... for a missing argument", on all three commands # that take an object name. Validated before any socket, which is what makes # these green here. n_case add-missing-name 1 "missing object name" "$NFACCT" add n_case get-missing-name 1 "missing object name" "$NFACCT" get n_case delete-missing-name 1 "missing object name" "$NFACCT" delete # The quota grammar of `add`, all three of its error arms. # argc==4, i.e. a quota KIND with no value after it. `add bst-n` alone is # argc==3 and is the plain create, which opens a socket -- it belongs in the S # group, not here, and putting it here was a bug this suite's own first run # caught. n_case add-missing-quota-value 1 "missing quota value" "$NFACCT" add bst-n byte n_case add-bad-quota-kind 1 'argument must "byte" or "packet"' "$NFACCT" add bst-n frobnicate 4096 n_case add-too-many-args 1 "too many arguments" "$NFACCT" add bst-n byte 4096 extra # `delete` and `flush` take a fixed arity and say so before opening anything. n_case delete-too-many-args 1 "too many arguments" "$NFACCT" delete bst-n extra n_case flush-too-many-args 1 "too many arguments" "$NFACCT" flush extra # The format/filter word loop of `list` and `get`, which also runs before the # socket. An unrecognised word is rejected rather than ignored. n_case list-unknown-argument 1 "unknown argument" "$NFACCT" list frobnicate n_case get-unknown-argument 1 "unknown argument" "$NFACCT" get bst-n frobnicate # THE TICKET'S OWN SPELLING, pinned as the error it is. `--xml` is not an # option; the format is a positional word. See note 2(a) in the header -- this # is the first thing a reader of the ticket will type, and it must have a case # saying what happens, or the next reader re-derives it from the source. n_case list-dashdash-xml 1 "unknown argument" "$NFACCT" list --xml n_case get-dashdash-xml 1 "unknown argument" "$NFACCT" get bst-n --xml # Two format words at once is a duplicate-option error, and it is upstream's # only message that does NOT carry the `nfacct v1.0.2:` prefix -- duparg() # writes its own. Pinned as emitted. n_case list-duplicate-format 1 'Error: duplicate "json" or collision with another option' "$NFACCT" list xml json # `restore` reads its table from stdin and rejects a line with no `;`. The pipe # is deliberate: a top-level stdin redirect from the virtiofs root makes dup2(2) # return 513 and ends the suite (brintos/linux#73), while a pipe forks first. _nout=$(printf 'no-semicolon-here\n' | "$NFACCT" restore 2>&1); _n_rc=$? case "$_nout" in *"invalid line"*) [ "$_n_rc" -eq 1 ] && bst_ok nfacct/restore-invalid-line \ || bst_notok nfacct/restore-invalid-line "right message, rc=$_n_rc want-rc=1" ;; *) bst_notok nfacct/restore-invalid-line "want:'invalid line' got:'$(_bst_1line "$_nout")' -- $(_nattrib "$_nout")" ;; esac # --------------------------------------------------------------------------- # T -- TERMINATION. The ticket's "to exit rather than hang", asserted directly # and separately from what the command answers. Every socket-touching command # is run under the bound; a case here goes red only if the tool BLOCKED or died # on a signal, which on this image it does not. # # These are green now and stay green on a netfilter kernel -- they are the one # group whose meaning does not change when brintos/linux#86 closes. n_exits list-exits "$NFACCT" list n_exits list-xml-exits "$NFACCT" list xml n_exits list-json-exits "$NFACCT" list json n_exits list-reset-exits "$NFACCT" list reset n_exits add-exits "$NFACCT" add bst-n n_exits get-exits "$NFACCT" get bst-n n_exits delete-exits "$NFACCT" delete bst-n n_exits flush-exits "$NFACCT" flush # --------------------------------------------------------------------------- # S -- THE DOCUMENTED RESULTS of those same commands, which is what the tool is # FOR. Every case here is RED on this image and every one is attributed to # brintos/linux#86 by _nattrib; they flip green together on the day # CONFIG_NETFILTER + CONFIG_NETFILTER_NETLINK_ACCT land, with no edit to this # file. That is the regression signal the port is bought for. # # THE EXPECTED VALUES ARE MEASURED, NOT GUESSED: an x86-64 build of the same # three sources under `unshare -rn` against a kernel that has nfnetlink_acct # produced exactly these, and the header records the run. # Leave no object behind from an earlier boot: on an image where these work, a # stale bst-n would make the add below answer EBUSY and read as a defect. "$NFACCT" delete bst-n >/dev/null 2>&1 # An empty table prints NOTHING and exits 0 -- see header note 1. The empty # want-substring is the assertion, not an omission. n_case list-empty 0 "" "$NFACCT" list # ... and the same for the XML format, which also prints nothing, because the # header and footer are both conditional on a row having been seen. See note # 2(b): this is why the ticket's "well-formed XML for the empty set" is not # assertable against this release. n_case list-xml-empty 0 "" "$NFACCT" list xml n_case flush-empty 0 "" "$NFACCT" flush # The object lifecycle. `add` and `delete` succeed silently; `get` prints the # object in the tool's own fixed-width form. n_case add 0 "" "$NFACCT" add bst-n n_case get-after-add 0 "{ pkts = 00000000000000000000, bytes = 00000000000000000000 } = bst-n;" "$NFACCT" get bst-n n_case list-after-add 0 "{ pkts = 00000000000000000000, bytes = 00000000000000000000 } = bst-n;" "$NFACCT" list n_case get-xml-after-add 0 "bst-n00000000000000000000" "$NFACCT" get bst-n xml n_case delete 0 "" "$NFACCT" delete bst-n # The case the lifecycle turns on: without it, the delete above could be an # accepted no-op and every case before it would still be green. n_case get-after-delete 1 "No such file or directory" "$NFACCT" get bst-n n_case delete-again 1 "No such file or directory" "$NFACCT" delete bst-n # A name that was never added, asked for on a table that exists. n_case get-absent 1 "No such file or directory" "$NFACCT" get bst-n-never-added # --------------------------------------------------------------------------- # N -- THE OBJECT-NAME LENGTH RULE. The ticket asks for "the documented # rejection of an object name longer than the protocol allows"; there is no # rejection, at any layer, and the header's note 3 records what was measured # instead. The rule is pinned in the two halves that can be scored separately. # 64 'a's. Built by a loop rather than a printf format so that busybox's # printf, which has no %*s padding, is not relied on. NLONG= _ni=0 while [ "$_ni" -lt 64 ]; do NLONG=${NLONG}a; _ni=$((_ni + 1)); done NTRUNC= _ni=0 while [ "$_ni" -lt 31 ]; do NTRUNC=${NTRUNC}a; _ni=$((_ni + 1)); done # THE HALF THAT NEEDS NO KERNEL, and so is green here: an over-long name is not # rejected in userland. The tool reaches the socket with it and fails there for # the socket's reason -- exactly as a one-character name does -- rather than # returning a distinct validation error first. Scored as "the over-long name # produces the same failure as a legal one", which is what "no length check" # means from the outside. _n_bounded "$NBOUND" "$NFACCT" add "$NLONG" _nlong_rc=$_n_rc; _nlong_out=$(cat "$NW/out" 2>/dev/null) _n_bounded "$NBOUND" "$NFACCT" add bst-n _nshort_rc=$_n_rc; _nshort_out=$(cat "$NW/out" 2>/dev/null) "$NFACCT" delete bst-n >/dev/null 2>&1 if [ "$_nlong_rc" -eq "$_nshort_rc" ] && [ "$_nlong_out" = "$_nshort_out" ]; then bst_ok nfacct/name-overlong-not-rejected else bst_notok nfacct/name-overlong-not-rejected \ "a 64-char name behaved differently from a legal one, so something now validates length: long(rc=$_nlong_rc)='$(_bst_1line "$_nlong_out")' short(rc=$_nshort_rc)='$(_bst_1line "$_nshort_out")'" fi # THE HALF THAT NEEDS THE KERNEL, and so is red here and attributed with the # rest of the S group: the name is TRUNCATED to 31 characters, and the object # that appears carries the short name. This is the case that would catch the # collision hazard being fixed -- an upstream that started rejecting instead # would red here and green the case above, and the pair says which happened. "$NFACCT" delete "$NLONG" >/dev/null 2>&1 n_case name-overlong-truncated-add 0 "" "$NFACCT" add "$NLONG" n_case name-overlong-truncated-get 0 "} = $NTRUNC;" "$NFACCT" get "$NTRUNC" # ... and the collision itself, stated as a case: the 64-character name finds # the 31-character object, because the query is truncated by the same setter. n_case name-overlong-collides 0 "} = $NTRUNC;" "$NFACCT" get "$NLONG" "$NFACCT" delete "$NLONG" >/dev/null 2>&1 rm -rf "$NW"