# BST-TIMEOUT: 300 # 48 cases. The fixture tree is ~355 files built by a # # sourced shell fragment (~40 execs, no per-file dirname), # # then 24 lsscsi runs over it. Measured on this arm at well # # under a minute, but the tree is written to tmpfs one # # `printf >` at a time and this guest is slowest exactly # # there, so 300 s is declared rather than the 60 s default # # -- the same value and the same reason as 99-nvme-cli. # # lsscsi 0.30 (svn r154, 2018-06-12) -- meta-openembedded/meta-oe, # recipes-bsp/lsscsi. /usr/bin/lsscsi, the SCSI device lister. # # THIS SUITE MEASURES THE WHOLE TOOL, NOT A CORNER OF IT, and it can because # lsscsi is unusually testable: it reads /sys and nothing else. It issues no # ioctl, opens no device node for I/O, and takes the sysfs mount point as an # option -- so pointing it at a captured tree makes its ENTIRE output a pure # function of that tree. There is no expected-red half here and no # device-present precondition: every case below is a real assertion about what # the ported binary computed, and a disagreement is a real defect. # # THE REFERENCE IS A NATIVE x86-64 BUILD OF THE SAME SOURCE. Each # data/lsscsi-expected-*.txt was captured by configuring and building # lsscsi-030r154 from the very tarball meta-oe's recipe fetches # (sha256 619a2187...b8b15, checked by bitbake) on x86-64, and running it # against the tree data/lsscsi-sysfs-fixture.sh produces. So the comparison is # wasm32-hwjs output against x86-64 output for one source at one version, which # is the only comparison that can find a codegen or libc difference. It is NOT a # re-hash of our own build, which would be tautological. # # --------------------------------------------------------------------------- # THE TWO THINGS THE TICKET SAID TO CHECK RATHER THAN ASSUME, AND WHAT THEY ARE # --------------------------------------------------------------------------- # # (1) THE SYSFS-ROOT OPTION IS `--sysfsroot=PATH` / `-y PATH` IN THIS VERSION. # It is a getopt long option with a required argument (`{"sysfsroot", # required_argument, 0, 'y'}`), assigned straight to the `sysfsroot` prefix # variable. It is NOT a positional argument here. That matters because 0.30 # DOES take positionals -- up to four of them -- and they mean something else # entirely: they are an [H:C:T:L] device filter (`lsscsi 1:0:0:0`, `lsscsi # host1`, `lsscsi 0 - - -`). So a caller who passes the fixture root as a bare # argument does not get a fixture-rooted listing; they get the REAL /sys # filtered by a garbage tuple. C01-C04 pin both halves. # # (2) TWO PATHS ARE HARDCODED PAST THE OPTION, AND THEY ARE `/dev` AND # `/dev/disk/by-id`. `--sysfsroot` is a prefix on the sysfs scan only. The # device-node column of the default listing is resolved by collect_dev_nodes(), # which opendir()s the literal "/dev" and matches by st_rdev; and `--wwn` calls # collect_disk_wwn_nodes(), which opendir()s the literal "/dev/disk/by-id". # Neither is rerouted. This is exactly the failure the ticket named -- a tool # that honours the option for the scan and still opens an absolute path past it # would read green on a machine that happens to have one -- and it is REAL here. # D01-D04 establish it positively rather than by inspection: the tool KNOWS the # device number from the fixture (`--device` prints [247:0]) and CAN name the # node from the fixture (`--kname` prints /dev/sda), yet the default listing # still prints `-` for that same LU. The only way both are true at once is that # the default path looked the number up somewhere the fixture is not. # # THAT IS NOT A brintOS DEFECT AND NOTHING WAS FILED FOR IT. `--sysfsroot=PATH` # is documented as "set sysfs mount point to PATH", not as a chroot, and the # behaviour is identical on the native x86-64 build -- which is why the expected # files carry the same `-` and the comparison stays byte-for-byte on both. It is # a fact about lsscsi worth writing down once, not a bug in the port. # # THE FIXTURE ROOT IS DELIBERATELY NOWHERE NEAR /sys, so a leak fails loudly # rather than passing on a machine that happens to mirror the real tree. E01-E03 # make that a measurement: pointed at a path that does not exist, lsscsi prints # NOTHING and exits 0 -- it does not fall back to /sys -- and under -v it names # the exact path it tried. # # WHY THE VERBOSE FORMS ARE COMPARED THROUGH sed AND EVERYTHING ELSE IS NOT. # `-v` prints the absolute path of every device directory it visits, and that # path is the fixture root the caller chose -- a tmpfs path containing this # suite's pid. There is no fixed value it could be captured at: /tmp is shared # on the build host where the reference was taken, and this platform's rootfs is # read-only at browser runtime, so the tree cannot live anywhere stable. The # root is therefore substituted for the token @ROOT@ on BOTH sides. Every other # form embeds no path and is compared raw. B20 then takes a sha256 over the # concatenation of all eighteen raw streams, which is the byte-for-byte # statement including the trailing newlines that $( ) would have eaten. . "$BST_LIB" S=lsscsi # THE CANARY, which asserts nothing about lsscsi. # # A suite's FIRST BST result line is lost between the guest's printf and the # host's read -- per-suite, deterministic, and NOT a defect in the host tool # (BRINTOS-1149 established that and filed it as linux-in-the-browser#177). The # harness reports the deficit but computes its pass count as total-minus- # failures, so THE DROPPED LINE IS COUNTED AS A PASS. A nothing-case therefore # stands in that position and the eaten line costs nothing. It sits ahead of the # `require` guard deliberately, because a `require` that skips would emit its # skip as the first line and that line would be eaten too. # # Remove this case when linux-in-the-browser#177 is fixed. bst_ok $S/suite-canary require lsscsi $S/present bst_ok $S/present D=${BST_DATA:-/usr/lib/brintos-tests/data} W=${TMPDIR:-/tmp}/bst-lsscsi.$$ F=$W/sys # --------------------------------------------------------------------------- # A -- identity # --------------------------------------------------------------------------- # The version is pinned exactly because every expected file below was captured # from THIS version's output. A recipe bump that changed the version without # refreshing the references would otherwise show up as eighteen mysterious # diffs instead of one clear one. check_eq $S/A01-version 'version: 0.30 2018/06/12 [svn: r154]' "$(lsscsi --version 2>&1)" # --help must document the option in the form this version actually implements. # This is the assertion that would have caught the ticket's open question if it # had gone the other way, and it is checked against the tool rather than against # the manual. check_out $S/A02-help-sysfsroot '--sysfsroot=PATH|-y PATH' lsscsi --help # An unknown option is a refusal, not a silent default -- so a typo in any case # below cannot quietly become "lsscsi with no options". check_rc $S/A03-bad-option-refused 1 lsscsi --no-such-option # --------------------------------------------------------------------------- # Build the fixture. # --------------------------------------------------------------------------- mkdir -p "$F" if [ -f "$D/lsscsi-sysfs-fixture.sh" ]; then . "$D/lsscsi-sysfs-fixture.sh" "$F" fi # The fixture has to exist before any comparison means anything: a tree that # failed to materialise would make every listing empty, and eighteen empty # listings compared against eighteen non-empty references produce eighteen reds # that all say the same thing and none of which says THIS. So it is asserted # once, by cause, first. if [ -d "$F/bus/scsi/devices" ] && [ -d "$F/class/scsi_device/0:0:0:0" ] && \ [ -f "$F/class/scsi_device/0:0:0:0/device/model" ]; then bst_ok $S/A04-fixture-materialised else bst_notok $S/A04-fixture-materialised "fixture tree absent or incomplete under $F" fi # The six LUs and the four hosts the fixture declares, counted from the tree # rather than from this file's opinion of it -- so a generator that silently # stopped half way is a red here and not a puzzle downstream. _n=0 for e in "$F"/bus/scsi/devices/*:*:*:*; do case "$e" in *"*"*) continue ;; esac _n=$((_n + 1)) done check_eq $S/A05-fixture-six-lus 6 "$_n" # --------------------------------------------------------------------------- # B -- the listing forms, against the native x86-64 reference # --------------------------------------------------------------------------- # One case per form. The comparison is on the whole output of the run, so a # single wrong character anywhere -- a mis-decoded VPD designator, a size # computed with the wrong integer width, a field padded by a printf that rounds # differently -- fails the case that produced it and names the form. # # cmp_form cmp_form() { _cid=$1; _cf=$D/lsscsi-expected-$2.txt; shift 2 if [ ! -f "$_cf" ]; then bst_notok "$_cid" "reference $_cf missing from the installed data payload" return fi check_eq "$_cid" "$(cat "$_cf")" "$(lsscsi -y "$F" "$@" 2>&1)" } cmp_form $S/B01-plain plain cmp_form $S/B02-classic classic --classic cmp_form $S/B03-generic generic --generic cmp_form $S/B04-device device --device cmp_form $S/B05-size size --size cmp_form $S/B06-transport transport --transport cmp_form $S/B07-long long --long cmp_form $S/B08-list list --list cmp_form $S/B09-hosts hosts --hosts cmp_form $S/B10-hosts-long hostslong --hosts --long cmp_form $S/B11-brief brief --brief cmp_form $S/B12-pdt pdt --pdt cmp_form $S/B13-lunhex lunhex --lunhex cmp_form $S/B14-kname kname --kname cmp_form $S/B15-protection protection --protection --protmode cmp_form $S/B16-unit unit --unit cmp_form $S/B17-no-nvme nonvme --no-nvme cmp_form $S/B18-transport-list translist --transport --list # The verbose forms, path-substituted on both sides (see the header). cmp_verbose() { _cid=$1; _cf=$D/lsscsi-expected-$2.txt; shift 2 if [ ! -f "$_cf" ]; then bst_notok "$_cid" "reference $_cf missing from the installed data payload" return fi check_eq "$_cid" "$(cat "$_cf")" "$(lsscsi -y "$F" "$@" 2>&1 | sed "s|$F|@ROOT@|g")" } cmp_verbose $S/B19-verbose verbose -v cmp_verbose $S/B19b-verbose2 verbose2 -vv # THE BYTE-FOR-BYTE STATEMENT. Every case above compares through $( ), which # strips trailing newlines from both sides -- symmetric, so it cannot hide a # difference INSIDE the output, but it does not see the final newline. This one # concatenates the eighteen raw streams and digests them, so trailing newlines, # trailing spaces and stream boundaries are all inside the assertion. The # expected digest is over the same eighteen native reference files concatenated # in the same order. _bfb=$W/allforms.raw : > "$_bfb" for a in "" "--classic" "--generic" "--device" "--size" "--transport" \ "--long" "--list" "--hosts" "--brief" "--pdt" "--lunhex" \ "--kname" "--unit" "--no-nvme"; do if [ -z "$a" ]; then lsscsi -y "$F" >> "$_bfb" 2>&1 else lsscsi -y "$F" $a >> "$_bfb" 2>&1; fi done lsscsi -y "$F" --hosts --long >> "$_bfb" 2>&1 lsscsi -y "$F" --protection --protmode >> "$_bfb" 2>&1 lsscsi -y "$F" --transport --list >> "$_bfb" 2>&1 _ref=$W/allforms.ref : > "$_ref" for n in plain classic generic device size transport long list hosts brief \ pdt lunhex kname unit nonvme hostslong protection translist; do cat "$D/lsscsi-expected-$n.txt" >> "$_ref" 2>/dev/null done # Digested by NAME, not from stdin: the two operands are different files, so a # stdin form would have to compare digests of differently-named streams anyway, # and this platform's virtiofs stdin redirect is the one thing a suite must not # do in its own shell (brintos/linux#73). check_eq $S/B20-byte-for-byte-digest \ "$(sha256sum "$_ref" 2>/dev/null | awk '{print $1}')" \ "$(sha256sum "$_bfb" 2>/dev/null | awk '{print $1}')" # --------------------------------------------------------------------------- # C -- what the positional argument actually is # --------------------------------------------------------------------------- # The ticket's premise was that the tool "takes the root of that tree as an # argument". In 0.30 it does not, and these four cases are the evidence. # # C01/C02: a positional is an [H:C:T:L] filter and a `hostN` filter, applied to # whatever --sysfsroot named. Both are compared against the native reference. cmp_form $S/C01-filter-lu filter-lu 1:0:0:0 cmp_form $S/C02-filter-host filter-host host1 # C03: the four-positional `H C T L` form, dashes meaning "any". cmp_form $S/C03-filter-tuple filter-h0 0 - - - # C04: THE CONSEQUENCE. Passing the fixture root as a bare positional -- the # form the ticket guessed at -- is not a rooted listing. The path parses as a # filter tuple, which fails, and the tool refuses with rc=1 rather than doing # something plausible-looking. That refusal is the reason this is safe to have # gotten wrong: it cannot silently read the real /sys instead. check_rc $S/C04-root-as-positional-refused 1 lsscsi "$F" # C05: A RELATIVE --sysfsroot IS BROKEN, and it is worth pinning because it # fails PARTIALLY, which is the shape that gets believed. list_sdevices() finds # the first LU, one_sdev_entry() chdir()s into its device directory to resolve # the node, and never chdir()s back -- so every subsequent relative lookup is # resolved from inside the previous device and comes back empty. The first entry # renders correctly and the rest print `type? vendor? model? rev?`. Same on the # native x86-64 build, so it is upstream behaviour and not this arm's; the # assertion is simply that an absolute root is the supported spelling. ( cd "$W" && lsscsi -y sys > "$W/rel.out" 2>&1 ) if grep -q 'vendor?' "$W/rel.out" 2>/dev/null; then bst_ok $S/C05-relative-root-degrades else bst_notok $S/C05-relative-root-degrades \ "a relative --sysfsroot no longer degrades after the first LU -- upstream behaviour changed, refresh the note" fi # --------------------------------------------------------------------------- # D -- the /dev leak, established positively # --------------------------------------------------------------------------- # D01: the tool reads the device NUMBER out of the fixture. [247:0] for # 0:0:0:0 is the content of $F/.../block/sda/dev and appears nowhere else. check_out $S/D01-devnum-from-fixture '[247:0]' lsscsi -y "$F" --device # D02: with --kname it NAMES the node from the fixture too -- /dev/sda is # built by concatenating "/dev" with the basename of the sysfs block directory, # with no lookup at all. check_out $S/D02-kname-from-fixture '/dev/sda' lsscsi -y "$F" --kname # D03: and the DEFAULT listing still prints `-` for that same LU. Knowing the # number (D01) and being able to spell the name (D02), the only thing that can # make the default path fail is that it went looking for major 247 minor 0 in a # directory the fixture does not own. That directory is the literal "/dev". _d03=$(lsscsi -y "$F" 2>&1) case "$_d03" in *'BRINTOS DISK A 1.00 -'*) bst_ok $S/D03-default-node-not-from-fixture ;; *) bst_notok $S/D03-default-node-not-from-fixture \ "the default listing resolved a node for 0:0:0:0 -- /dev now carries 247:0, so this assertion needs a different unused number" ;; esac # D04: --wwn is the second hardcoded path, /dev/disk/by-id, and it is wholly # outside the fixture -- so the WWN column is blank here and would be blank on # any machine without that directory. Compared against the native reference like # every other form, which is what makes "blank on both" a measurement. cmp_form $S/D04-wwn-outside-fixture wwn --wwn # --------------------------------------------------------------------------- # E -- the fixture root is honoured, and there is no fallback to /sys # --------------------------------------------------------------------------- # E01: pointed at a tree that does not exist, lsscsi lists NOTHING. If # --sysfsroot were advisory, or if any scan fell back to the real /sys, this # would print whatever the guest has. check_eq $S/E01-absent-root-lists-nothing '' "$(lsscsi -y /nonexistent/brintos/lsscsi 2>&1)" # E02: and it says so under -v, naming the exact path it tried -- so the # negative in E01 is a measurement rather than an absence of output. check_out $S/E02-absent-root-names-path \ '/nonexistent/brintos/lsscsi/bus/scsi/devices' lsscsi -y /nonexistent/brintos/lsscsi -v # E03: an empty-but-present root is the other half. The directory exists and # holds nothing, so the scan succeeds and finds no devices -- distinct from E01, # where the scan itself failed, and both must list nothing. mkdir -p "$W/emptyroot/bus/scsi/devices" "$W/emptyroot/class/nvme" check_eq $S/E03-empty-root-lists-nothing '' "$(lsscsi -y "$W/emptyroot" 2>&1)" # E04: THE GUEST'S OWN /sys, asserted rather than assumed. This kernel carries # no SCSI mid-layer and no NVMe driver, so an unrooted lsscsi must list nothing. # It is a real assertion about the platform: were it ever to list something, the # fixture-based cases above would still pass while the tool's actual subject had # changed underneath them, and nothing else here would notice. check_eq $S/E04-guest-sys-has-no-scsi '' "$(lsscsi 2>&1)" # E05: the same statement under --hosts, which takes a different scan path # (class/scsi_host rather than bus/scsi/devices) and would not be covered by # E04 alone. The two paths part company in one respect, and it is the reason # this case reads stdout alone: the device scan is SILENT about a sysfs tree # that is not there (E01), while the host scan NAMES it. So "lists no hosts" # and "says nothing" are different states here, and only the first is an # assertion about the guest. check_eq $S/E05-guest-sys-has-no-hosts '' "$(lsscsi --hosts 2>/dev/null)" # E05b: and the diagnostic is what makes E05's silence a measurement rather than # an absence of one -- exactly as E02 does for E01. It names the directory this # kernel does not carry, so an empty host list is evidence the scan ran and # found no tree, not evidence that it never ran. check_out $S/E05b-guest-sys-names-absent-host-dir \ '/sys/class/scsi_host/: No such file or directory' lsscsi --hosts # E05c: and the diagnostic tracks an ABSENT directory specifically, not the # --hosts form -- an empty-but-present class/scsi_host is silent. This is the # E01/E03 distinction carried onto the host path, and unlike E05/E05b it is # rooted at the fixture, so it is an ordinary assertion on a build host too. mkdir -p "$W/hostless/bus/scsi/devices" "$W/emptyhosts/bus/scsi/devices" \ "$W/emptyhosts/class/scsi_host" check_out $S/E05c-absent-host-dir-named-under-root \ "$W/hostless/class/scsi_host/: No such file or directory" \ lsscsi -y "$W/hostless" --hosts check_eq $S/E05d-empty-host-dir-is-silent '' "$(lsscsi -y "$W/emptyhosts" --hosts 2>&1)" # --------------------------------------------------------------------------- # F -- the port's shape on the image # --------------------------------------------------------------------------- # E04, E05, E05b and F01 are the four cases that assert something about THIS # GUEST rather than about the fixture, so they are the four that legitimately # differ when the suite is dry-run on a build host (which has real SCSI hosts # and an lsscsi somewhere other than /usr/bin). On the target they are ordinary # assertions and any red among them is real. # # That list is also a blind spot, and it cost this suite a wrong expectation: # E05 sat on it, so its red on the build host carried no information and the # first run that could have contradicted it was the on-target one. A case # excluded from the dry run is a case whose expectation nothing checks until # the gate -- which is why E05c/E05d were added rooted at the fixture, where a # build host can still contradict them. check_eq $S/F01-install-path /usr/bin/lsscsi "$(command -v lsscsi 2>/dev/null)" # THE REFERENCES ARE ONLY VALID IF THE OUTPUT IS LOCALE-INDEPENDENT, and one # form makes that a live question: --size renders capacities through a # floating-point format ("1.07GB"), and a locale whose LC_NUMERIC uses a comma # would render "1,07GB" -- turning every checked-in reference into a red that # says nothing about the port. So it is measured rather than assumed: the same # run under the C locale and under whatever the guest's ambient locale is must # agree. If they ever diverge, THIS case names the cause and B05 stops being a # mystery. Same reason 99-nvme-cli pins its locale. check_eq $S/F02-size-locale-independent \ "$(LC_ALL=C lsscsi -y "$F" --size 2>&1)" "$(lsscsi -y "$F" --size 2>&1)" # The manual page ships in lsscsi-doc, which this image does not install, so its # ABSENCE from the main package is the assertion -- a doc file landing in ${PN} # would mean the packaging split had broken. if [ -e /usr/share/man/man8/lsscsi.8 ] || [ -e /usr/share/man/man8/lsscsi.8.gz ]; then bst_notok $S/F03-doc-split "lsscsi.8 is present, so it landed in the main package rather than lsscsi-doc" else bst_ok $S/F03-doc-split fi rm -rf "$W"