. "$BST_LIB" # BST-TIMEOUT: 180 # ~45 execs of one small wasm module, no loops, no corpus # and nothing that waits: every invocation below either # fails in argv parsing or fails at the first open(2) and # returns at once. NOT MEASURED -- this port's suite has # never run on wasm32; 180 is the 60s default tripled for a # module this arm has not yet sized. Trim it against the # first `==BST-SUITE sgpio elapsed=..==` line rather than # leaving a bound nobody has checked. # sgpio 1.2-0.10 (sgpio_1.2.0.10.bb, meta-openembedded meta-oe/recipes-extended/ # sgpio, carrying Makefile-error-fix.patch and # 0001-makefile-Add-LDFLAGS-to-linking-rule.patch) -- Intel's SGPIO captive # backplane enclosure-management utility. ONE binary, /usr/sbin/sgpio, out of one # 683-line sgpio.c. No library, no configure step, no target dependency beyond # libc. The -dev and -dbg packages are empty and are not on the image. # # THERE IS NO BACKPLANE BEHIND THIS MACHINE, SO REFUSING CLEANLY IS THE WHOLE # ASSERTION. The program's only real code path here is the refusal: every # LED-setting route ends at open("/sys/class/scsi_host/hostN/em_message", # O_RDWR), which cannot succeed on a guest with no SCSI host at all. Everything # above that line -- getopt_long, the status keyword table, the frequency range # check, the port-number strtol -- is arithmetic on argv and calls to printf, # and a wasm32 build that got any of it wrong has no environmental excuse. That # is what the bulk of this file asserts. # # EVERY EXPECTATION BELOW WAS READ OFF A NATIVE x86-64 BUILD of the identical # 1.2-0.10 tarball with the same two meta-oe patches applied, not transcribed # from the source. Where that oracle and the ticket disagree, the oracle wins # and the disagreement is written down rather than smoothed over. # # FOUR OF THE TICKET'S STATED EXPECTATIONS DO NOT MATCH THE PROGRAM, and the # cases record what is actually there: # # (1) THE DIAGNOSTICS ARE ON STDOUT, NOT STDERR. There is no fprintf(stderr, # ...) and no perror() on any reachable path -- every message in this file # is a plain printf. The only thing sgpio ever puts on stderr is getopt's # own line for an unrecognised option, which getopt writes, not sgpio. # Asserted directly by sgpio/refusal-stream-is-stdout. # # (2) IT DOES EXIT 0 ON FAILURES -- three of them, and they are the ticket's # "must not exit 0" written as measurements rather than as a wish. A # missing target (`-s locate` with no -p/-d), a stray non-option argument, # and -h/-V alone all print an Error: line and then reach `exit(0)`. The # return-code half of main() is simply inconsistent with the message half: # the argv-validation failures `return -1` (rc 255) while the # missing-required-argument failures `exit(0)`. Asserted, as the defect it # is, by the three sgpio/exits-zero-* cases. # # (3) ARGUMENT VALIDATION DOES NOT PRECEDE DEVICE ACCESS. led_set() -- which # opens the sysfs path -- is called from inside the getopt_long loop, on # the `-p` arm, so the device is touched before a later `-s` has been # looked at at all. `sgpio -p 0 -s bogus` therefore fails at the open and # never reports the bad status keyword. Asserted by # sgpio/device-access-precedes-status-check, which is the one case here # that would go red if upstream ever fixed the ordering -- deliberately, # because that is a change this image should be told about. # # (4) A VALID -f DOES NOT TERMINATE. `if(_led_rate != 0)` enters a `while(1)` # that re-arms an itimer and spins on a sig_atomic_t flag, and the loop # has no exit at all -- not after the LEDs are written, not when there are # no LEDs to write. Measured on the oracle: `sgpio -f 3` was still running # at the 4s bound with no output. That is an unbounded busy loop, and it # is the one thing this file will NOT run on the guest -- see the skip at # the end of the frequency block for why. # # NONE OF THE ABOVE IS HARDCODED AS "the machine has no backplane". The refusal # cases assert the diagnostic and the status the program gives when the open # fails, and they name the port they were asked for, so the day this image grows # a real SGPIO host they fail loudly rather than passing on a stale premise -- # which is the correct direction for a suite whose subject is an absence. bst_ok sgpio/suite-canary # first line: absorbs brintos/linux#96, which drops it SG=/usr/sbin/sgpio MAN=/usr/share/man/man1/sgpio.1 # The absolute path, not the PATH name: /usr/sbin is not on every login's PATH # here, and a `require sgpio` that missed it would SKIP this whole file rather # than red -- an absent suite that reads as a legitimate one. Whether the name # also resolves through PATH is asserted below as its own case. require "$SG" sgpio/present # sg -- # stdout and stderr TOGETHER, for the cases where which stream carried the # text is not what is being asserted. Run inside a command substitution with # stdin on /dev/null: the fork is what keeps any invocation away from the # suite's own shell, and /dev/null is devtmpfs, not the virtiofs root the # top-level-redirect rule is about (brintos/linux#73). sg() { _id=$1; _want=$2; _needle=$3; shift 3 _o=$("$SG" "$@" 2>&1 -- sg_absent() { _id=$1; _want=$2; _needle=$3; shift 3 _o=$("$SG" "$@" 2>&1 /dev/null &1 >/dev/null` # points fd 2 at the current stdout FIRST and only then silences fd 1, which is # what leaves stderr on the pipe and stdout discarded. sg_err() { "$SG" "$@" 2>&1 >/dev/null -- : the needle must be on STDERR. # Not check_out: that helper requires rc 0, and every invocation whose stderr # is interesting here exits non-zero, so check_out would red them all on the # status before ever looking at the text. sg_stderr() { _id=$1; _want=$2; _needle=$3; shift 3 _e=$("$SG" "$@" 2>&1 >/dev/null /dev/null | wc -l | tr -d ' ')" # THE MAN PAGE IS NOT ON THIS IMAGE, AND THAT IS THE IMAGE'S RULE RATHER THAN A # PACKAGING FAILURE -- which is a correction to the ticket, whose fourth # assertion is that "the binary and its manual page land where the packaging # promises". Half of that is satisfiable here and half is not. # # The recipe passes MANDIR=${D}${mandir} and the patched Makefile appends /man1, # so do_install DOES produce /usr/share/man/man1/sgpio.1 -- MEASURED at 1822 # bytes in sgpio-doc_1.2.0.10-r0_wasm32.deb. It lands in the -doc package, and # core-image-brintos-emcraft installs no port's -doc package: the rule is # written against giflib-doc in the image recipe and restated against # xorg-docs-doc, and it distinguishes documentation SPLIT OFF a functional port # -- which this is -- from a standalone documentation corpus, which man-pages # and xorg-docs are. EMCRAFT_TOOLS therefore names `sgpio` and not `sgpio-doc`. # # So the packaging promise the image can be held to is the BINARY, asserted # above. The page is asserted in BOTH ARMS rather than pinned to its absence: a # case that required the page to be missing would go red on a later image that # legitimately decided to ship -doc packages, which is a change nobody should # have to debug through a red. Present -> it must be the real sgpio page. # Absent -> that is the -doc rule, recorded as a skip naming it. if [ -s "$MAN" ]; then check_out sgpio/manpage-is-sgpio "captive backplane LED control utility" cat "$MAN" check_out sgpio/manpage-documents-freq-range "between 1 and 10" cat "$MAN" else bst_skip sgpio/manpage-is-sgpio \ "sgpio.1 is packaged in sgpio-doc (measured, 1822 B) and this image installs no port's -doc package -- see the giflib-doc rule in core-image-brintos-emcraft.bb" bst_skip sgpio/manpage-documents-freq-range \ "same: the page ships in sgpio-doc, which is not on this image" fi # --- the usage text ---------------------------------------------------------- # # With no arguments at all: usage, and a NON-ZERO status. This is the one # no-argument path in the program that gets the exit code right (`exit(1)` at # the argc<2 guard), and it is the ticket's headline "refuse cleanly" case for # the empty command line. sg sgpio/no-args-rc1 1 "SGPIO tool for intelligent hard disk drive backplanes" sg sgpio/no-args-usage-body 1 "[-s|--status] " # The usage line is printed with the program's OWN name, recovered by strrchr on # argv[0]. Invoked through the absolute path above, that must come back as the # bare `sgpio` and not as `/usr/sbin/sgpio` -- which is a real assertion, since # the code takes the null-strrchr branch when there is no '/' and would print # the whole path if the truncation were dropped. sg sgpio/usage-names-basename 1 " sgpio [[-d|--disk]" sg_absent sgpio/usage-not-full-path 1 "/usr/sbin/sgpio [[-d|--disk]" # --- --help and --version ---------------------------------------------------- # # BOTH OF THESE EXIT 0 WHILE PRINTING AN Error: LINE, and both do it for the # same reason: the 'h' and 'V' arms of the switch fall through the getopt loop # instead of exiting, so control reaches the missing-argument checks below and # takes their exit(0). The status is asserted as 0 because that is what the # program does; the accompanying Error: line is asserted alongside it so the # pair is on the record together rather than the 0 looking like success. sg sgpio/help-short-usage 0 "SGPIO tool for intelligent hard disk drive backplanes" -h sg sgpio/help-long-usage 0 "SGPIO tool for intelligent hard disk drive backplanes" --help check_eq sgpio/help-short-equals-long "$(sg_out -h)" "$(sg_out --help)" sg sgpio/version-utility 0 "Utility Version 0.10" -V sg sgpio/version-spec 0 "SGPIO Specification Version 1.2" -V check_eq sgpio/version-short-equals-long "$(sg_out -V)" "$(sg_out --version)" # The version arm prints the two compile-time constants from version.h and # nothing else -- no banner, no build id, and in particular no usage text, which # is what separates it from the help arm. sg_absent sgpio/version-is-not-help 0 "SGPIO tool for intelligent" -V # --- unrecognised options ---------------------------------------------------- # # getopt_long returns '?', which lands in the `default:` arm: sgpio prints # "Unknown command", repeats the usage, and exit(1)s. getopt has ALREADY printed # its own complaint on stderr by then, so the two streams carry different text # and both are asserted. sg sgpio/unknown-short-rc1 1 "Unknown command" -Z sg sgpio/unknown-long-rc1 1 "Unknown command" --bogus sg_stderr sgpio/unknown-short-getopt-stderr 1 "invalid option" -Z sg_stderr sgpio/unknown-long-getopt-stderr 1 "unrecognized option" --bogus # --- --status keyword validation --------------------------------------------- # # An unknown keyword is caught by the else arm of the strcmp chain and returns # -1 from main, which the shell reports as 255. That 255 is asserted literally # throughout: `return -1` is the argv-validation failure convention in this # program, and collapsing it to "non-zero" would hide the inconsistency with the # exit(0) paths below, which is exactly what these cases exist to expose. sg sgpio/status-bad-rc255 255 "Error: Unknown option: bogus!" -s bogus sg sgpio/status-bad-long 255 "Error: Unknown option: bogus!" --status bogus sg sgpio/status-empty 255 "Error: Unknown option: !" -s "" # The four keywords the man page documents are accepted -- reaching the # missing-target message rather than the unknown-keyword one is the proof, since # nothing here supplies a -p or -d. sg sgpio/status-locate-accepted 0 "Error: Port number or disk name must be specified!" -s locate sg sgpio/status-fault-accepted 0 "Error: Port number or disk name must be specified!" -s fault sg sgpio/status-rebuild-accepted 0 "Error: Port number or disk name must be specified!" -s rebuild sg sgpio/status-off-accepted 0 "Error: Port number or disk name must be specified!" -s off # A FIFTH keyword is accepted and is in NEITHER the man page nor the usage text: # `activity`, mapping to ACTIV_LED (2^16). It is asserted because it is real # surface -- a script may depend on it -- and because its absence from both # documents is the actual defect, not its presence in the code. sg sgpio/status-activity-accepted-undocumented 0 "Error: Port number or disk name must be specified!" -s activity # --- --freq range validation ------------------------------------------------- # # strtol into a range check: 1..10 inclusive, and any trailing garbage is # refused. This is the program's one genuine pre-device value check and it # behaves correctly, so these are ordinary passes rather than defect records. sg sgpio/freq-zero-refused 255 "Error: Invalid frequency, please use number between 1 and 10!" -f 0 sg sgpio/freq-eleven-refused 255 "Error: Invalid frequency, please use number between 1 and 10!" -f 11 sg sgpio/freq-negative-refused 255 "Error: Invalid frequency, please use number between 1 and 10!" -f -1 sg sgpio/freq-nonnumeric-refused 255 "Error: Invalid frequency, please use number between 1 and 10!" -f abc sg sgpio/freq-trailing-refused 255 "Error: Invalid frequency, please use number between 1 and 10!" -f 3x sg sgpio/freq-long-refused 255 "Error: Invalid frequency, please use number between 1 and 10!" --freq 99 # A VALID -f IS DELIBERATELY NOT RUN ON THIS GUEST, and the skip is the case. # # `-f 1` through `-f 10` set _led_rate, and main() then enters # # while(1){ signal(SIGALRM, ...); if(sigal_flag){ led_timer(); led_change_state(); sigal_flag = 0; } } # # which has no exit condition of any kind -- not "the LEDs were written", not # "there are no LEDs to write". With no disk initialised led_change_state() does # nothing and returns 0, so the loop simply spins on the flag forever. Measured # on the native oracle: `sgpio -f 3` was alive at a 4s bound having printed # nothing. # # bst_probe would bound it, and that is not the risk. The risk is what bounding # it COSTS on this platform: the bound is enforced by `kill -9` from the suite's # shell, and a guest that cannot deliver that signal to a process spinning # entirely in compute leaves the runner with an unkillable child -- which wedges # the suite, and a wedged suite is quarantined and takes every case after this # line with it. Trading ~40 real assertions for one already-known defect is the # wrong side of that trade, so the loop is recorded here and exercised nowhere. # Reinstate it as a bst_probe case once signal delivery to a compute-bound wasm # process is established on this arm. bst_skip sgpio/freq-valid-loops-forever \ "not run on purpose: a valid -f enters an unbounded busy while(1) with no exit condition (sgpio.c main); measured alive at 4s on a native oracle. Running it risks an unkillable child wedging the whole suite" # --- --port argument validation ---------------------------------------------- # # strtol with a trailing-garbage check and a >=0 bound, and the rejected token # is echoed back verbatim -- which is what these assert, since a diagnostic that # named the wrong token would be worse than none. sg sgpio/port-nonnumeric-refused 255 "Error: abc is not a valid port number!" -p abc sg sgpio/port-negative-refused 255 "Error: -1 is not a valid port number!" -p -1 sg sgpio/port-trailing-refused 255 "Error: 0x is not a valid port number!" -p 0x sg sgpio/port-partial-refused 255 "Error: 1abc is not a valid port number!" -p 1abc sg sgpio/port-long-refused 255 "Error: abc is not a valid port number!" --port abc # --- --disk argument validation ---------------------------------------------- # # whatportis() refuses anything that is not exactly 3 characters before it looks # at the filesystem at all, and returns -1 for a name it cannot map. Both arrive # at the same message from main. sg sgpio/disk-too-short-refused 255 "Error: can not extract port number!" -d sd sg sgpio/disk-too-long-refused 255 "Error: can not extract port number!" -d sdaa sg sgpio/disk-long-opt-refused 255 "Error: can not extract port number!" --disk sdaa # --- THE REFUSAL: no backplane behind this machine --------------------------- # # led_set() builds /sys/class/scsi_host/host/em_message and opens it O_RDWR. # On a guest with no SCSI host that open fails for every N, and the program # prints the port-specific line, then main prints the generic one, then returns # -1 (rc 255). BOTH lines are asserted, and each names the port it was actually # given -- so a build that silently substituted a port, or lost the number in # the format string, is caught. sg sgpio/refusal-port0-names-port 255 "Error: Unable to find path to requested disk: port 0!" -p 0 -s locate sg sgpio/refusal-port0-generic-line 255 "Error: can not initialize data for disks LED!" -p 0 -s locate sg sgpio/refusal-port99-names-port 255 "Error: Unable to find path to requested disk: port 99!" -p 99 -s locate sg sgpio/refusal-long-opts 255 "Error: Unable to find path to requested disk: port 0!" --port 0 --status locate sg sgpio/refusal-port-list-first 255 "Error: Unable to find path to requested disk: port 0!" -p 0,1 -s locate sg sgpio/refusal-by-disk-name 255 "Error: can not extract port number!" -d sda -s fault # THE TICKET'S TWO NEGATIVE ASSERTIONS, as their own cases: on a machine with no # backplane the tool must not claim success, and must not go quiet. A bare # `[ $rc -ne 0 ]` would be satisfied by a crash, so the status is pinned to the # exact 255 the refusal path produces, and the output is required to be # non-empty independently of what it says. # # "set for " is led_execute()'s success text -- the one string in the program a # caller could read as "the backplane is fine". It is unreachable on this # machine because led_execute() is only called once a disk has been INITIATED, # which the failed open prevents; asserting its absence is what makes that # reasoning checkable rather than argued. sg_absent sgpio/refusal-claims-no-success 255 "set for " -p 0 -s locate check sgpio/refusal-rc-is-not-zero sh -c '"$0" -p 0 -s locate >/dev/null 2>&1 &1 /dev/null