. "$BST_LIB" require libc-bench libc-bench/present # libc-bench (musl/etalabs, SRCREV b6b2ce5f) — 29 micro-benchmarks over the libc # primitives a C program actually leans on: malloc, string, stdio, regex, utf8 and # pthread. /usr/bin/libc-bench, no arguments, everything runs. # # WHY THIS SUITE LOOKS LIKE A COVERAGE MATRIX RATHER THAN A BENCHMARK. libc-bench # takes no selector -- main() calls all 29 unconditionally -- and each one runs in # its OWN fork()ed child, timed by the child with clock_gettime(CLOCK_REALTIME) # around the workload. Two consequences shape everything below: # - A benchmark that dies takes only its own child with it. The parent wait()s # and moves on, so a partial run is the NORMAL failure mode, not a crash, and # "which of the 29 produced a time line" is the real signal on this arm. # - The clock is WALL time, not CPU time. That is why libc-bench gets numbers # where 84-dhrystone.test cannot: dhrystone self-times with times(2) and a # wasm32-hwjs task's CPU time settles only at exit (brintos/linux#38), whereas # CLOCK_REALTIME is host-backed and reads correctly mid-run. Every timing this # suite asserts on is therefore a real measurement. # # 21 OF 29 RUN; THE 8 THAT DO NOT ARE EVERY BENCHMARK THAT CREATES A THREAD. Both # b_malloc_thread_* and all six b_pthread_* die in glibc's nptl with "The futex # facility returned an unexpected error code." -- brintos/hardwarejs#12 (the futex # BITSET ops return ENOSYS in the Worker fast-path) with the thread-Worker traps of # brintos/linux-in-the-browser#85 behind it. That is the finding this port exists # to surface, and it is asserted as ONE red below rather than eight, because all # eight are the same defect seen eight times. # # STDOUT MUST BE A FILE, AND THAT IS WHY THE MISSING EIGHT LEAVE NO TRACE. A child # puts(label) BEFORE running its workload. Onto a tty that flushes at the newline, # so a console run shows all 29 labels and only 21 time lines; into a file glibc # fully buffers, so a child that aborts loses its label too and the file holds # exactly the 21 survivors. Both were observed. The file form is the one asserted # here because "the label is present iff the benchmark completed" is a much # sharper predicate than counting orphaned labels. # THE FUNCTIONAL RUN IS OPT-IN, AND THAT IS THE POINT OF THIS BLOCK. # # Running libc-bench under this runner wedges the whole GUEST about two times in # five -- measured, 4 of 10 runs: the kernel traps in __wasm32_alloc_process_task # by way of __hwjs_clear_parked on the printk path, after which no process can be # created and the console is gone. The runner's own wall-clock bound cannot save # the run, because the watchdog is starved with everything else; only the host # harness's timeout ends it. # # A suite whose failure mode is to destroy the guest running it must not be on by # default in a shared sweep, because the cost lands on OTHER packages' coverage # rather than on this one: brintos_local_verify --batch gives each slice a fresh # guest, so a wedge here forfeits the rest of that slice's suites, not just this # file. So the default is a skip that names the blocker, and the run is one env # var away for anyone who wants it: # # BST_LIBC_BENCH_RUN=1 brintos-selftest 84-libc-bench.test # # The 11 cases below are what that produces on a run that survives -- 9 pass, # 2 attributed reds, ~32 s, identical across all 6 clean runs. # The port itself is NOT blocked by any of this: libc-bench run standalone on the # target is clean and repeatable (3 of 3 runs, exit 0, 21 of 29 benchmarks timed # in 6-9 s), which is the evidence R3 rests on. It is specifically the # background-and-busy-poll supervision this runner is obliged to use that # libc-bench's fork-and-thread storm cannot survive. # # BRINTOS-WORKAROUND: brintos/linux-in-the-browser#85 remove — both the opt-in guard below and the setsid on the run itself exist only for that race. Without setsid the wedge is every run (2 of 2); with it, 4 of 10. Delete the guard, the env var and the setsid together once #85 lands, and this becomes an ordinary suite. if [ "${BST_LIBC_BENCH_RUN:-0}" != "1" ]; then bst_skip libc-bench/run "wedges the guest ~2 runs in 5 under this runner (brintos/linux-in-the-browser#85); re-run with BST_LIBC_BENCH_RUN=1 to exercise it" exit 0 fi require setsid libc-bench/setsid-present W=${TMPDIR:-/tmp}/bst-libc-bench.$$ rm -rf "$W"; mkdir -p "$W" setsid libc-bench > "$W/out" 2> "$W/err" rc=$? # The parent survives all 29 forks even though 8 children die -- run_bench() only # wait()s on the status it is handed and never propagates it, so a non-zero rc # here would mean the harness itself broke, not that a benchmark failed. if [ "$rc" -eq 0 ]; then bst_ok libc-bench/exit-0 else bst_notok libc-bench/exit-0 "rc=$rc $(_bst_1line "$(head -3 "$W/err")")"; fi # fam — every label in the file must have completed. fam() { _miss="" while IFS= read -r _l; do [ -n "$_l" ] || continue grep -qF "$_l" "$W/out" || _miss="$_miss; $_l" done < "$2" if [ -z "$_miss" ]; then bst_ok "$1" else bst_notok "$1" "no output for:$(_bst_1line "$_miss")"; fi } # --- the six malloc benchmarks that do not spawn a thread --------------------- # sparse/bubble walk a large allocation set; tiny1/tiny2 hammer the small-bin # fast path; big1/big2 cross the mmap threshold. Together they are the closest # thing this image has to an allocator regression net. cat > "$W/w" <<'EOF' b_malloc_sparse (0) b_malloc_bubble (0) b_malloc_tiny1 (0) b_malloc_tiny2 (0) b_malloc_big1 (0) b_malloc_big2 (0) EOF fam libc-bench/malloc "$W/w" # --- string: the five strstr shapes plus memset/strchr/strlen ----------------- # The five needles are chosen by upstream to hit distinct strstr paths -- a plain # alphabet, a shuffled one, a repeated-prefix pair that forces backtracking, and # two long runs of 'a' that exercise the Two-Way skip table. A codegen fault in # the string routines shows up here before it shows up anywhere else on the image. cat > "$W/w" <<'EOF' b_string_strstr ("abcdefghijklmnopqrstuvwxyz") b_string_strstr ("azbycxdwevfugthsirjqkplomn") b_string_strstr ("aaaaaaaaaaaaaacccccccccccc") b_string_strstr ("aaaaaaaaaaaaaaaaaaaaaaaaac") b_string_strstr ("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaac") b_string_memset (0) b_string_strchr (0) b_string_strlen (0) EOF fam libc-bench/string "$W/w" cat > "$W/w" <<'EOF' b_utf8_bigbuf (0) b_utf8_onebyone (0) EOF fam libc-bench/utf8 "$W/w" # putcgetc drives the locked stdio path (one lock/unlock per byte); the _unlocked # variant drives the same bytes with the lock elided, so the pair is also a check # that the single-threaded stdio lock is not a no-op on this arm. cat > "$W/w" <<'EOF' b_stdio_putcgetc (0) b_stdio_putcgetc_unlocked (0) EOF fam libc-bench/stdio "$W/w" cat > "$W/w" <<'EOF' b_regex_compile ("(a|b|c)*d*b") b_regex_search ("(a|b|c)*d*b") b_regex_search ("a{25}b") EOF fam libc-bench/regex "$W/w" # --- the whole-run count ------------------------------------------------------ # The per-family cases name WHICH benchmark vanished; this one is the backstop # that catches a 30th benchmark appearing, or a 22nd survivor, on a future SRCREV # or a fixed runtime -- either of which should be noticed rather than absorbed. done_n=$(grep -c '^ time:' "$W/out") check_eq libc-bench/completed-21 21 "$done_n" # --- every reported time is a real measurement -------------------------------- # The point of contrast with dhrystone. A zero here would mean CLOCK_REALTIME had # gone the way of times(2) and the whole suite had quietly stopped measuring # anything, while every case above still passed. zeros=$(awk '/^ time:/ { t=$2; sub(/,$/,"",t); if (t+0 <= 0) print t }' "$W/out") if [ -z "$zeros" ]; then bst_ok libc-bench/timings-nonzero else bst_notok libc-bench/timings-nonzero "$(_bst_1line "non-positive: $zeros")"; fi # --- EXPECTED RED 1: every thread-creating benchmark is gone ------------------ # brintos/hardwarejs#12 (futex BITSET ENOSYS in the Worker fast-path), with the # thread-Worker park race of brintos/linux-in-the-browser#85 under it. Asserted # positively -- "these eight completed" -- so that the day the futex ops land this # case goes green on its own and nobody has to remember to delete an exclusion. cat > "$W/w" <<'EOF' b_malloc_thread_stress (0) b_malloc_thread_local (0) b_pthread_createjoin_serial1 (0) b_pthread_createjoin_serial2 (0) b_pthread_create_serial1 (0) b_pthread_uselesslock (0) b_pthread_createjoin_minimal1 (0) b_pthread_createjoin_minimal2 (0) EOF fam libc-bench/threads "$W/w" # The red above says the eight are missing; this says WHY, from the target's own # stderr rather than from this comment. If the eight ever vanish for some new # reason, this case goes red too and the pair stops agreeing -- which is the # signal that the attribution needs re-deriving. if grep -q 'futex facility returned an unexpected error' "$W/err"; then bst_ok libc-bench/futex-diagnosed else bst_notok libc-bench/futex-diagnosed "no futex diagnosis on stderr: $(_bst_1line "$(head -2 "$W/err")")" fi # --- EXPECTED RED 2: the memory half of every result is zero ------------------ # print_stats() reads /proc/self/smaps to report virt/res/dirty per benchmark. On # this arm /proc/self/smaps AND /proc/self/maps are both EMPTY (0 lines, no error # -- the files open and read clean), and /proc/self/status carries no VmSize or # VmRSS, so the loop matches nothing and all three columns stay 0. libc-bench is # half a memory-efficiency benchmark, so this is not cosmetic: the allocator # footprint comparison the tool exists for is unavailable on wasm32-hwjs. # brintos/linux#49. if [ "$(cat /proc/self/smaps 2>/dev/null | wc -l)" -gt 0 ]; then bst_ok libc-bench/heap-footprint else nz=$(grep -c 'virt: [1-9]' "$W/out") bst_notok libc-bench/heap-footprint "/proc/self/smaps is empty; $nz of $done_n results report a non-zero footprint" fi rm -rf "$W"