# BST-TIMEOUT: 900 # measured 33 s on target 2026-08-12 (56 cases, of which 19 # # are fresh wasm module instantiations). A marker is needed # # even so: the runner's own default is 60 s, and 33 s is an # # idle-host number. The bound is set far above 12x that, # # deliberately: while hardwarejs#12 stands the eight # # regression entries abort in under a second each, and when # # it is fixed they will instead run full OTR conversations # # of up to 50 messages at up to 500 ms intervals -- ~40 s # # apiece natively, and more here. 900 survives that day # # without a second visit to this line. . "$BST_LIB" # libotr 4.1.1 -- the Off-the-Record messaging library: deniable authenticated # encryption for chat. Two parties negotiate a shared secret over Diffie-Hellman, # authenticate each other WITHOUT signing anything an outsider could later use as # proof, encrypt each message under a key that is then thrown away, and finally # PUBLISH the old MAC keys so that anybody could have forged the transcript after # the fact. That last step is the "off the record" part, and it is why this # library exists rather than one of the image's other crypto stacks. # # It is built entirely on libgcrypt (79-libgcrypt), which supplies the MPI layer # the Diffie-Hellman runs on, AES-CTR, SHA-1/SHA-256 and the CSPRNG. That port # landing green is what makes this one possible; see the note on hwjs-cc#26 below. # # The distro is static-only, so the image gets /usr/lib/libotr.a, the sixteen # headers under /usr/include/libotr/, libotr.pc, the libotr.m4 aclocal macro, and # the six toolkit programmes. See libotr_%.bbappend for what the port adds. # # --------------------------------------------------------------------------- # WHAT THIS SUITE IS, AND WHY IT IS NOT A PROBE # # It closes on the PACKAGE'S OWN UPSTREAM SUITE. libotr ships tests/test_list -- # its own declaration of its suite, 19 entries -- and the bbappend stages exactly # what that file names, so what runs here is upstream's suite and not our reading # of it. Eleven unit programmes over the library's internals and eight protocol # regressions that drive BOTH ends of a real OTR conversation in one process. # # The native reference, taken on x86-64 before any wasm32 build was spent: # # All tests successful. # Files=19, Tests=3832, 55 wallclock secs # # --------------------------------------------------------------------------- # THE ONE THING TO UNDERSTAND BEFORE READING THE COUNTS BELOW # # THE EXIT STATUS OF THIS SUITE IS MEANINGLESS; THE TAP STREAM IS THE VERDICT. # All eleven unit programmes end `return 0` unconditionally and never call # libtap's exit_status(); each regression entry is a bash wrapper whose own # tap.sh EXIT trap reports the WRAPPER's test count (zero) and discards the # client's status. A harness keying on `$? -eq 0` scores this suite 19/19 green # with every assertion inside it failing. run-ptest is therefore a TAP parser, # which is also what upstream's own runner is (`prove`). This was not theory: the # first native staging of this port shipped libtool WRAPPER SCRIPTS instead of # the programmes, and every one of them exited 0 while printing no TAP at all. # # AND ONLY HALF OF THE ASSERTIONS ARE COUNTABLE. libtap has two plan modes and # libotr uses both: # # plan_tests(N) -- the eleven unit files. Their plans are fixed properties of # the source and sum to exactly 189: # auth 5, proto 48, dh 38, b64 10, context 22, userstate 1, # tlv 11, mem 5, sm 23, instag 13, privkey 13. # That number is pinned below, and it is the strongest # assertion in this file -- it cannot drift without either # the package version changing or a test silently dying. # plan_no_plan() -- the eight regression files, whose client sends `rand() % 50` # messages of `rand() % 600` bytes each. Their totals are # RANDOM PER RUN: measured 3375 and 2793 on two consecutive # native runs of the identical tree, and one single file # measured 262 / 262 / 278 on three. Pinning THAT number # would red this suite on nothing at all. # While hardwarejs#12 stands the question does not arise -- # the eight die after three deterministic points each, long # before any randomness, so what is pinned below (24) is how # far they get and not how much they produce. The day that # defect closes, the number becomes random and the assertion # must become a floor; see want_regression_reach. # # run-ptest reports the two separately, which is what makes both assertions # possible. TAP itself is what distinguishes them -- plan_tests prints `1..N` # BEFORE the points, plan_no_plan prints it after. # # --------------------------------------------------------------------------- # THE SIX TOOLKIT PROGRAMMES ARE COVERED HERE AND NOWHERE ELSE # # Upstream registers no test for them -- its suite exercises the library through # its own C programmes -- and they are the part of this package a user actually # types. They are the OTR protocol forensics kit: given captured ciphertext and # whatever key material you hold, read it, re-MAC it, forge it, or modify it in # place. So this file drives one of them over known vectors and asserts what it # prints. ptestdir=/usr/lib/libotr/ptest runner=/usr/bin/ptest-runner # Upstream's own counts, as staged by do_install_ptest out of tests/test_list. # Asserted rather than assumed: a suite that is silently shorter than upstream # declared reads exactly like a suite that passed. want_total=19 # test_list entries: 11 unit + 8 regression want_unit=11 want_regression=8 # THE FIXED HALF, ON THIS ARM. Upstream's eleven plans sum to 189; test_instag # contributes 0 of its 13 because it dies at readlink(/proc/self/exe) before its # first assertion (see KNOWN_RED below), so ten files' worth is what runs here: # # 189 (upstream) - 13 (test_instag, held by the /proc/self/exe gap) = 176 # # Written as the subtraction rather than as the literal, because the two numbers # mean different things and only one of them is a platform fact. When the gap # closes this goes red at 189 and is updated to upstream's number -- which is the # point of pinning it. want_fixed_points=176 # HOW FAR THE EIGHT REGRESSIONS GET before the futex kills them. Each emits # exactly three points -- library init, userstate creation, key load -- all of # them before a thread is created, so this is deterministic rather than a sample # of the random conversation length. 8 x 3 = 24. # # It is pinned, not floored, precisely BECAUSE it is small: while hardwarejs#12 # stands, the meaningful assertion is "every one of the eight reached the same # point and died there", which a floor would not say. When that defect closes, # these files start emitting hundreds of RANDOM points each (measured natively: # 2793 and 3375 whole-suite on two consecutive runs) -- this assertion goes red, # and whoever sees it replaces it with a floor, since a random total must never # be pinned. want_regression_reach=24 tmp=${TMPDIR:-/tmp}/bst-libotr.$$ mkdir -p "$tmp" 2>/dev/null || tmp=/tmp # ----------------------------------------------------------- what shipped ---- # The six toolkit programmes. Unlike the recent library-only ports, the MAIN # package here carries a command line and it is part of the deliverable. check_rc libotr/otr_parse 0 test -x /usr/bin/otr_parse check_rc libotr/otr_sesskeys 0 test -x /usr/bin/otr_sesskeys check_rc libotr/otr_mackey 0 test -x /usr/bin/otr_mackey check_rc libotr/otr_readforge 0 test -x /usr/bin/otr_readforge check_rc libotr/otr_modify 0 test -x /usr/bin/otr_modify check_rc libotr/otr_remac 0 test -x /usr/bin/otr_remac # The library half. Static-only arm: libotr.a IS the library. check_rc libotr/ar 0 test -s /usr/lib/libotr.a # The standing static-only invariant. check_rc libotr/no-so 1 test -e /usr/lib/libotr.so # The headers live under /usr/include/libotr/, not at the top level -- upstream's # own layout (otrincdir = $(includedir)/libotr), which is why libotr.pc says # `Cflags: -I${includedir}` and consumers spell it . The absence # of a top-level copy is asserted so a future flattening cannot pass silently. check_rc libotr/hdr-dir 0 test -d /usr/include/libotr check_rc libotr/hdr-proto 0 test -s /usr/include/libotr/proto.h check_rc libotr/hdr-version 0 test -s /usr/include/libotr/version.h check_rc libotr/no-top-hdr 1 test -e /usr/include/proto.h check_rc libotr/pc 0 test -s /usr/lib/pkgconfig/libotr.pc # libotr.m4 is how a consumer's configure finds this library (AM_PATH_LIBOTR). # It is the one part of the -dev package that is not a header or a .pc file, and # nothing else on the image would notice if it went missing. check_rc libotr/m4 0 test -s /usr/share/aclocal/libotr.m4 # ------------------------------------------------------- the ptest payload ---- check_rc libotr/ptest-dir 0 test -d "$ptestdir" check_rc libotr/ptest-script 0 test -x "$ptestdir/run-ptest" check_rc libotr/ptest-list 0 test -s "$ptestdir/test_list" # tap.sh is sourced by all eight regression wrappers as $TESTDIR/utils/tap/tap.sh. # Missing, every one of them dies in the shell before reaching the client -- and # with no TAP emitted, which run-ptest reports as a plan-less FAIL. Asserted so # the cause is legible rather than eight identical reds. check_rc libotr/ptest-tapsh 0 test -s "$ptestdir/utils/tap/tap.sh" # instag.txt must sit beside the BINARY, not beside test_list: test_instag builds # its path from readlink("/proc/self/exe"). If that readlink fails on this # platform the test returns -ENOMEM from main having printed only its plan, which # run-ptest scores a short-count FAIL -- a different red from a missing fixture, # and this assertion is what tells them apart. check_rc libotr/ptest-instag 0 test -s "$ptestdir/unit/instag.txt" # otr.key is upstream's pre-generated DSA private key. The regression client # --load-keys it rather than generating one, because generating one costs minutes # of parameter search; without it the eight regressions would each pay that. check_rc libotr/ptest-otrkey 0 test -s "$ptestdir/regression/client/otr.key" check_rc libotr/ptest-client 0 test -x "$ptestdir/regression/client/client" # The staged entry counts, by glob expansion rather than `ls | wc -l`, which would # fork twice. A short payload would leave the suite passing over fewer files with # nothing saying so. set -- "$ptestdir"/unit/test_* check_eq libotr/unit-count "$want_unit" "$#" set -- "$ptestdir"/regression/*.sh check_eq libotr/regression-count "$want_regression" "$#" # ...and test_list itself, which is upstream's declaration and the file run-ptest # iterates. Counted from the file rather than from the payload, so a payload that # grew and a test_list that did not (or the reverse) is caught. declared=$(grep -c . "$ptestdir/test_list") check_eq libotr/test-list-count "$want_total" "$declared" if [ -x "$runner" ]; then bst_ok libotr/ptest-runner-present else bst_skip libotr/ptest-runner-present "this image ships no ptest-runner" fi # --------------------------------------------------------------- the tools ---- # otr_mackey derives an OTR MAC key from an AES key -- which OTR defines as the # SHA-1 of the AES key. Both vectors are deterministic and platform-independent, # so these two cases are a hard assertion that real libgcrypt SHA-1 runs # correctly through libotr's toolkit on this arm, not a presence check. The # second is SHA-1 of sixteen zero bytes, a value checkable against any SHA-1 # implementation anywhere. # # THEY ARE ALSO THIS SUITE'S WITNESS FOR hwjs-cc#26. Every one of these # programmes links libgcrypt.a, and for most of this Epic anything that did # TRAPPED IN __wasm32_main_with_ctors BEFORE main's first instruction ("orphan at # clear") -- 79-libgcrypt carried mpicalc as a standing red for exactly that. Our # hwjs-cc fork now carries the F139 CTOR-DRIVE patch and the trap is gone # (measured: mpicalc 2.0 answers, and 79-libgcrypt is 126/126). If that patch is # ever dropped before upstream lands it, these two cases and the whole suite # below go red together, which is the regression evidence this port buys. check_out libotr/mackey-vector1 \ '56178b86a57fac22899a9964185c2cc96e7da589' \ otr_mackey 000102030405060708090a0b0c0d0e0f check_out libotr/mackey-vector2 \ 'e129f27c5103bc5cc44bcdf0a15e160d445066ff' \ otr_mackey 00000000000000000000000000000000 # Each of the other five refuses a bare invocation with its usage text AND exit # status 1. That is a thin assertion on its own, but it is not nothing on this # arm: it says the programme instantiated, linked libotr.a and libgcrypt.a, # reached main and wrote to stderr -- which is precisely the sequence hwjs-cc#26 # used to break. # # BOTH HALVES ARE ASSERTED TOGETHER, and the status is 1 rather than 0 -- measured # on the native build, where all five behave identically. check_out alone would be # wrong here: it requires rc=0, so it reds a tool that printed exactly the right # usage text and exited exactly as upstream intends. (It did, on the first # on-target run of this suite -- five reds that were this file's bug and not the # port's.) otr_parse is the one that exits 0 on empty input, so it is not in this # group; it is covered by the ptest suite instead. usage_case() { _id=$1; _prog=$2 _out=$("$_prog" 2>&1); _rc=$? case "$_out" in "Usage: $_prog"*) _hit=1 ;; *) _hit=0 ;; esac if [ "$_rc" = 1 ] && [ "$_hit" = 1 ]; then bst_ok "$_id" else bst_notok "$_id" "rc=$_rc (want 1) out='$(printf '%s' "$_out" | head -c 60)'"; fi } usage_case libotr/sesskeys-usage otr_sesskeys usage_case libotr/mackey-usage otr_mackey usage_case libotr/readforge-usage otr_readforge usage_case libotr/modify-usage otr_modify usage_case libotr/remac-usage otr_remac # ------------------------------------------------- run upstream's own suite ---- # One run, one capture, then a `while read` that spawns nothing: printf and test # are ash builtins and the parse below is not a per-case command substitution. out=$tmp/suite.out sh "$ptestdir/run-ptest" > "$out" 2>&1 # THE EXPECTED REDS: NINE OF THE NINETEEN, and every one of them is ONE OF TWO # PLATFORM GAPS -- not nine problems, and nothing about libotr itself. Ten of the # eleven unit files pass, which is where the library's own correctness is proven: # Diffie-Hellman, the socialist millionaires' protocol, AES-CTR message framing, # TLVs, contexts, the secure allocator and private-key generation all run here # exactly as they do natively. # # unit/test_instag /proc/self/exe IS NOT READABLE on this kernel. The # symlink exists and stats, but readlink(2) returns # ENOENT (measured on target: `ls: cannot read symbolic # link '/proc/self/exe': No such file or directory`). # test_instag derives the directory holding its # instag.txt fixture from that readlink and returns # -ENOMEM from main when it fails -- so it prints its # plan `1..13` and runs ZERO of the thirteen. Nothing at # the port level can work around it: the path is # computed inside the test. # Filed as brintos/linux#69. # # regression/*.sh (8) THREADS. The regression client drives Alice and Bob as # two pthreads over an AF_UNIX socketpair with epoll, # and glibc's futex path is unusable here: # FUTEX_WAIT_BITSET returns ENOSYS in the Worker # fast-path and __futex_abstimed_wait treats that as # unrecoverable, calling futex_fatal_error(). Measured # on target, identically for all eight: # ok 1 - OTR library initialization done. # ok 2 - OTR userstate creation done. # ok 3 - Loading key from given file # The futex facility returned an unexpected error code. # Already tracked as hardwarejs#12 -- the same defect # fftw's threaded arm and libevent's thread-basic hit. # They abort CLEANLY (no guest kill, no lost sentinel), # so unlike fftw's threaded case they are RUN rather # than asserted-without-running: a real red measured is # better evidence than a predicted one. # # The set is asserted as a SET rather than a count, so that a fixed platform gap # and a newly broken case cannot cancel out into a green tally. When either gap # closes, the corresponding entries go green, THIS assertion goes red, and # whoever sees it updates the set -- which is the regression evidence this # lifecycle buys. KNOWN_RED='unit/test_instag regression/random-msg.sh regression/random-msg-auth.sh regression/random-msg-fast.sh regression/random-msg-frag.sh regression/random-msg-disconnect.sh regression/random-msg-disconnect-frag.sh regression/random-msg-disconnect-auth.sh regression/random-msg-disconnect-frag-auth.sh' red_why() { case "$1" in regression/*) echo "hardwarejs#12: pthreads unusable -- FUTEX_WAIT_BITSET ENOSYS, glibc calls futex_fatal_error() after 3 points" ;; unit/test_instag) echo "brintos/linux#69: readlink(/proc/self/exe) returns ENOENT, so the test cannot locate instag.txt and runs 0 of its 13" ;; *) echo "upstream unit test failed; no red is expected here on this arm" ;; esac } npass=0 nfail=0 nskip=0 failed= while read -r verdict name _rest; do case "$verdict" in PASS:) bst_ok "libotr/t-$name"; npass=$((npass + 1)) ;; FAIL:) bst_notok "libotr/t-$name" "$(red_why "$name")"; nfail=$((nfail + 1)) failed="$failed $name" ;; SKIP:) bst_skip "libotr/t-$name" "upstream self-skip"; nskip=$((nskip + 1)) ;; esac done < "$out" # The tallies are asserted separately from the individual cases because they fail # differently: a red above is one broken behaviour, whereas a short total means # entries never ran at all -- a truncated capture, a scratch directory the run # could not create, or a payload that lost files after do_install_ptest. Without # this, a run that produced three PASS lines and died would read as three green # cases and nothing else. check_eq libotr/suite-total "$want_total" "$((npass + nfail + nskip))" # THE RED SET, NOT THE RED COUNT -- so a fixed platform gap and a newly broken # case cannot cancel out into a green tally. got=$(printf '%s\n' $failed | sort | tr '\n' ' ') want=$(printf '%s\n' $KNOWN_RED | sort | tr '\n' ' ') check_eq libotr/known-red-set "$want" "$got" # ------------------------------------------------------- the assertion tally ---- # Nineteen green files still says nothing about how much ran INSIDE them: a # programme that printed its plan and exited would be caught by run-ptest's # short-count rule, but a suite whose files all shrank together would not be. The # fixed half is pinned exactly; the dynamic half is bounded from below only, for # the reason set out at the top of this file. fixed=$(sed -n 's/^libotr-tap-points: .* fixed=\([0-9]*\) .*$/\1/p' "$out") dynamic=$(sed -n 's/^libotr-tap-points: .* dynamic=\([0-9]*\)$/\1/p' "$out") notok=$(sed -n 's/^libotr-tap-points: ok=[0-9]* not_ok=\([0-9]*\) .*$/\1/p' "$out") check_eq libotr/fixed-points "$want_fixed_points" "${fixed:-}" check_eq libotr/regression-reach "$want_regression_reach" "${dynamic:-}" # NO `not ok` ANYWHERE. This is the assertion that separates the two ways this # suite can be red, and it is why the nine known reds above do not make it # meaningless: all nine die WITHOUT emitting a failing assertion -- test_instag # runs zero of its thirteen, and the regressions are killed mid-stream by # futex_fatal_error() -- so a `not ok` on this arm means the LIBRARY computed a # wrong answer, which nothing here is expected to do and nothing here does. check_eq libotr/tap-not-ok 0 "${notok:-}" rm -rf "$tmp"