# BST-TIMEOUT: 600 # 145 cases, measured 41s on a host under full build load # # (==BST-SUITE libidn2 elapsed=41s cases=145). Cost is one # # 2.6 MB wasm module start plus 145 shell-level compares, so # # the figure is start-up dominated and does not grow with a # # few more cases. The declared bound is well over the 60s # # default because 41s already sits too close to it. . "$BST_LIB" # libidn2 2.3.7 -- GNU's IDNA2008 + Unicode TR46 implementation (poky # recipes-extended/libidn). What ships is /usr/lib/libidn2.a, /usr/include/idn2.h, # /usr/lib/pkgconfig/libidn2.pc (libidn2-dev + -staticdev) and /usr/bin/idn2 # (libidn2-bin). # # --------------------------------------------------------------------------- # WHY THE CLI IS *NOT* THIS LIBRARY'S COVERAGE HERE, THOUGH IT IS FOR MOST PORTS # # 79-libsmi drives smilint/smidump because those tools ARE libsmi's callers and # everything they print is a value the library computed. idn2(1) is the same shape # of tool -- and on this image it cannot play that role, for a reason that is # about the image and not about libidn2. # # Every entry point idn2(1) uses is a LOCALE-flavoured one -- idn2_to_ascii_lz, # idn2_to_unicode_lzlz, idn2_register_ul -- and each begins by transcoding its # argument from locale_charset() to UTF-8. This image ships no locale data at all: # no /usr/lib/locale, no locale-archive, no gconv modules. So setlocale(LC_ALL,"") # lands on "C", nl_langinfo(CODESET) answers ANSI_X3.4-1968, and every non-ASCII # argument is refused with IDN2_ICONV_FAIL *before any IDNA code runs*. The CLI # covers the tool and the error surface; it cannot reach one codepoint of the # Unicode semantics libidn2 exists for. # # That boundary is ASSERTED below (idn2/locale-charset, idn2/nonascii-refused) # rather than left to be discovered, and it is the image's boundary, not the # library's: the day this image gains a UTF-8 locale those two cases change, and # they are the prompt to widen the CLI coverage rather than a regression. # # AND SEPARATELY, idn2(1) ABORTS on this arm -- `free(): invalid size` on any # input string, while the library underneath is clean. Nine CLI cases are red for # that reason and each one says so in its own detail field, naming # brintos/hwjs-cc#46; the full analysis and what rules libidn2 out is in the CLI # block below. Read that before treating any of those reds as a fault in this # port. # # --------------------------------------------------------------------------- # SO THE LIBRARY IS COVERED BY A CROSS-LINKED PROBE, WHICH IS ALSO THE POINT # # libidn2's own callers are C programs -- curl, wget and libpsl all call # idn2_lookup_u8 or the _8z forms, none of which touches a locale -- and this # ticket ships libidn2 as an ENABLER for exactly those. data/idn2-probe is # src/libidn2_probe.c cross-linked with wasmcc-glibc against THE VERY ARCHIVE # THAT SHIPS, so it answers the question a library port has to answer: can a # consumer link this .a and get right answers? It is the BRINTOS-217 instrument # (oniguruma compiled and packaged perfectly, then returned wrong offsets to # every caller), and a payload-shape-only suite would go green on that. # # The probe MEASURES and does not judge: 122 lines of `IDN2- # `, scored against data/libidn2-native.txt -- the byte-identical output of # the same source built natively against the same libidn2 2.3.7 and libunistring # 1.2 tarballs (src/libidn2_mknative.sh, which also runs upstream's own `make # check` against that build: 15/15 PASS). Upstream's suite is sound and is not the # oracle only because it cannot run here -- its drivers are in-tree test programs # no package installs, IdnaTestV2.txt is not installed either, and there is no # compiler on the image to build them with. # # Nothing the probe prints carries a pointer width, a sizeof, an address, a locale # or a clock, so no value is allowed to differ between the arms -- with the single # exception of its first line, the pointer-width banner, which is stripped before # the comparison and asserted on its own. ORACLE="$BST_DATA/libidn2-native.txt" PROBE="$BST_DATA/idn2-probe" TMP=${TMPDIR:-/tmp} # --- payload shape ---------------------------------------------------------- check idn2/lib-archive test -f /usr/lib/libidn2.a check idn2/header test -f /usr/include/idn2.h check idn2/pkgconfig test -f /usr/lib/pkgconfig/libidn2.pc check idn2/cli test -x /usr/bin/idn2 # Static-only arm's standing invariant. `set --` so the glob is evaluated once and # its non-expansion (no match) is the passing case. set -- /usr/lib/libidn2.so* if [ "$1" = '/usr/lib/libidn2.so*' ]; then bst_ok idn2/no-shared else bst_notok idn2/no-shared "shared object present: $*"; fi # --- the CLI ---------------------------------------------------------------- # # THESE CASES ARE RED ON PURPOSE AND EACH SAYS WHY. idn2(1) aborts with # `free(): invalid size` on ANY input string on wasm32-hwjs -- so the ten cases # below that give it one are reds attributed to that defect, not to libidn2. # They are NOT downgraded to skip: the tool is present and broken, which is # exactly what `not ok` is for, and turning it green would hide the one thing # this suite found that the port itself did not. # # What is known about the defect (BRINTOS-544 Investigation Log 5, measured on # this image): # - `idn2 --version` and `--help` are clean, rc=0. They return before # process_input(); every input path dies there, before printing, so the # computed value never reaches even a line-buffered tty. # - glibc's own iconv_open/iconv/iconv_close for UTF-8 <- ANSI_X3.4-1968 is # clean on this image. # - the library is clean: 129/129 of the probe cases below pass, including # idn2_to_ascii_lz() -- the very call process_input() makes -- returning the # right answer and freeing cleanly, in a probe linked against the same two # archives. # - so the fault is in the shipped binary's own module, not in libidn2, not in # libunistring's conversion, and not in glibc's iconv. It reproduces 100% in # /usr/bin/idn2 and not at all in a differently-linked probe of the same # calls, which is a heap corruption sensitive to module layout -- a # miscompile signature. Filed as brintos/hwjs-cc#46. `free(): invalid size` # from a gnulib malloca/freea path is the same shape as brintos/hwjs-cc#27 # (u16_asnprintf %s) and the coroutinize pass's own ALLOCSIZE-STRIP note on # libunistring_mmalloca (brintos/hwjs-cc#12), but SAME ROOT CAUSE IS NOT # ESTABLISHED here. # # One helper runs every CLI case so the abort is recognised and attributed # uniformly -- and so that when the defect is fixed these cases start asserting # the real behaviour again with no edit. IDN2_ABORT='free(): invalid size' IDN2_ATTR='idn2(1) aborts with free(): invalid size in process_input() on wasm32-hwjs; the LIBRARY is clean (129/129 probe cases, incl. idn2_to_ascii_lz). Not libidn2 and not this port -- brintos/hwjs-cc#46 https://brintos.io/brintos/hwjs-cc/issues/46 ; see 78-libidn2.test header' # cli cli() { _cid=$1; _crc=$2; _cw=$3; shift 3 _co=$(idn2 "$@" 2>&1); _cr=$? case "$_co" in *"$IDN2_ABORT"*) bst_notok "$_cid" "$IDN2_ATTR (rc=$_cr)" ; return ;; esac if bst_execfail "$_co"; then bst_unrun "$_cid" "$_co"; return; fi case "$_co" in *"$_cw"*) _ch=1 ;; *) _ch=0 ;; esac if [ "$_cr" -eq "$_crc" ] && [ "$_ch" -eq 1 ]; then bst_ok "$_cid" else bst_notok "$_cid" "rc=$_cr want=$_crc want:'$_cw' got:'$(_bst_1line "$_co")'"; fi } cli idn2/cli-version 0 "idn2 (Libidn2) 2.3.7" --version cli idn2/cli-ascii 0 "example.com" example.com cli idn2/cli-alabel 0 "xn--rksmrgs-5wao1o.se" xn--rksmrgs-5wao1o.se cli idn2/cli-decode-ascii 0 "example.com" -d example.com # stdin rather than argv -- a different path into the same process_input() sout=$(printf 'example.com\n' | idn2 2>&1); src=$? case "$sout" in *"$IDN2_ABORT"*) bst_notok idn2/cli-stdin "$IDN2_ATTR (rc=$src)" ;; *example.com*) [ "$src" -eq 0 ] && bst_ok idn2/cli-stdin \ || bst_notok idn2/cli-stdin "rc=$src" ;; *) bst_notok idn2/cli-stdin "rc=$src got:'$(_bst_1line "$sout")'" ;; esac # A 64-octet label is one over the IDNA limit; the CLI reports it and exits 1. long=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa cli idn2/cli-rc-toolong 1 "" "$long.com" cli idn2/cli-msg-toolong 1 "domain label longer than 63 characters" "$long.com" cli idn2/cli-rc-hyphen 1 "string start/ends with forbidden hyphen" -- -bad.com # THE IMAGE'S LOCALE BOUNDARY, asserted rather than discovered. See the header: # these three describe the image, not libidn2, and they change the day a UTF-8 # locale ships. The UTF-8 bytes are written as octal escapes so this file stays # pure ASCII in transport -- "raksmorgas.se" with a-diaeresis, o-diaeresis and # a-ring. # # locale-charset is the one that survives the abort today: idn2 --debug prints # Charset: to STDERR, unbuffered, before process_input() dies -- so it is scored # on the charset alone and the abort is attributed separately. dbg=$(idn2 --debug example.com 2>&1); dbgrc=$? case "$dbg" in *"Charset: ANSI_X3.4-1968"*) bst_ok idn2/locale-charset ;; *"$IDN2_ABORT"*) bst_notok idn2/locale-charset "$IDN2_ATTR (rc=$dbgrc)" ;; *) bst_notok idn2/locale-charset "rc=$dbgrc got:'$(_bst_1line "$dbg")'" ;; esac uarg=$(printf 'r\303\244ksm\303\266rg\303\245s.se') cli idn2/nonascii-refused 1 "" "$uarg" cli idn2/nonascii-message 1 "could not convert string to UTF-8" "$uarg" # --- the library RUNNING, via the cross-linked probe ------------------------- if [ ! -f "$ORACLE" ]; then bst_notok idn2/oracle "native control $ORACLE not installed" exit 0 fi bst_ok idn2/oracle body=$TMP/bst-idn2-out.$$ ctl=$TMP/bst-idn2-ctl.$$ probe=$TMP/bst-idn2-probe.$$ trap 'rm -f "$body" "$ctl" "$probe"' EXIT if ! cp "$PROBE" "$probe" 2>/dev/null || ! chmod +x "$probe" 2>/dev/null; then bst_notok idn2/probe-stage "cannot stage $PROBE" exit 0 fi bst_ok idn2/probe-stage # Run ONCE and keep both the output and the status: on this runtime a multi-MB # wasm module's start-up dominates the whole suite's cost, so a second run to # re-read the exit code would roughly double it. raw=$("$probe" 2>&1); prc=$? if bst_execfail "$raw"; then bst_unrun idn2/probe-exit "$raw" exit 0 fi check_eq idn2/probe-exit 0 "$prc" # The banner is the one value that legitimately differs between the arms, so it # is asserted here and stripped from everything below. banner=$(printf '%s\n' "$raw" | sed -n 1p) check_eq idn2/probe-pointer-width "IDN2-PROBE libidn2 probe, 32-bit pointers" "$banner" printf '%s\n' "$raw" | sed 1d > "$body" grep -v '^#' "$ORACLE" > "$ctl" # Completeness first: a per-line comparison that silently ran out of lines would # report every remaining case as a mismatch against an empty string, or worse # stop early and read as coverage. The count and the whole-file compare bound # that before any individual case is scored. check_eq idn2/native-line-count "$(wc -l < "$ctl" | tr -d ' ')" "$(wc -l < "$body" | tr -d ' ')" if cmp -s "$ctl" "$body"; then bst_ok idn2/native-byte-identical else bst_notok idn2/native-byte-identical "probe output differs from the native control -- see the per-case results below"; fi # One case per measurement, read in lockstep off two descriptors so the loop # forks nothing: check_eq is a shell function and the two files are already here. # The key is compared as well as the value, so a misalignment reports itself # instead of scoring case N's value against case N+1's expectation. exec 3< "$ctl" exec 4< "$body" while read -r want <&3; do read -r got <&4 || got="" id=${want%% *} gid=${got%% *} case "$id" in IDN2-*) ;; *) continue ;; esac if [ "$id" != "$gid" ]; then bst_notok "idn2/${id#IDN2-}" "line misaligned: expected key $id, got '${gid:-}'" continue fi check_eq "idn2/${id#IDN2-}" "$want" "$got" done exec 3<&- exec 4<&-