# BST-TIMEOUT: 600 # 47 serial regressions actually execute (the other 54 are # # reported without running -- see the header), plus ~60 # # file and manifest cases. Re-set from the ==BST-SUITE # # line, which is the measurement. . "$BST_LIB" # Concurrency Kit 0.7.0+git (meta-oe recipes-devtools/concurrencykit) is the # lock-free and non-blocking structure library: bounded rings, FIFOs, stacks, # hash sets and tables, epoch-based reclamation, hazard pointers, eight spinlock # variants and the cohort/rwcohort composites -- all of them over ck_pr, an # atomic-primitive abstraction covering CAS, fetch-and-add, fetch-and-store, the # bit-test-and-{set,reset,complement} family, unary ops and eight fence flavours # across every width. # # --------------------------------------------------------------------------- # WHY THIS SUITE RUNS UPSTREAM'S REGRESSIONS INSTEAD OF A PROBE # # The distro is static-only, so the package installs ~40 headers, libck.a and # ck.pc -- and NOT ONE executable. Forty suites in this tree answer that with a # hand-written probe cross-linked against the archive. This one does not need to: # upstream ships 109 regression programs whose `check:` targets define 101 # invocations, each driving one structure and exiting non-zero on a violated # invariant, all non-interactive. They ARE the package's acceptance, and a probe # written here would be a worse copy of them. # # The recipe builds them, installs the validate tree under /usr/libexec/ck-tests # one directory per structure, and beside each one a `cases` file: the # directory's own `check:` recipe with $(CORES), $(HALF) and $(SIZE) expanded and # its for-loops unrolled, generated at build time by running the recipe against # argv-echoing stubs. So what runs below is upstream's definition of its own # acceptance, not a transcription of it -- which matters because a wrong # argument count makes a regression print usage and exit non-zero, and that # reads exactly like the platform failure this port exists to measure. # # --------------------------------------------------------------------------- # THE SPLIT THAT DECIDES WHAT IS A PASS: 47 SERIAL, 54 THREADED # # data/concurrencykit-serial.txt names the 47 cases that create no thread. They # are the pass criterion: ck_pr's 23 primitive validators -- every atomic this # arm has, exercised directly -- plus the single-threaded exercisers for # ck_array, ck_backoff, ck_bitmap, ck_cc, ck_hs, ck_ht, ck_rhs, ck_ec's seven # property tests, and the serial builds inside ck_fifo, ck_hp and ck_stack. # A red among those 47 is a defect in this port and nothing else. # # THE LIST IS GENERATED, BY src/concurrencykit_mkserial.sh, AND NOT BY # DIRECTORY. Splitting by directory is the obvious thing and it is wrong by 16 # cases, all in the direction that hides real assertions behind an excuse: # ck_fifo_mpmc creates no thread while ck_fifo_spsc does, from the same # directory; six of ck_stack's twelve cases are serial builds of the same four # .c files whose siblings are not. The generator asks each BINARY, via `nm -u` # on a native build -- a question that cannot be put to the wasm32 build, where # static linking DEFINES pthread_create in every module whether it is called or # not. # # --------------------------------------------------------------------------- # THE 54 THREADED CASES ARE REPORTED WITHOUT BEING RUN, AND THAT IS A # MEASUREMENT RATHER THAN A PRECAUTION # # The refusal is explicit, deliberate and catchable. The guest kernel prints it # in full, once per attempt: # # wasm32: REFUSING thread-mode clone (CLONE_VM, pid=122, flags=0x3d0f00) with # EAGAIN - the coro substrate's runtime protocol state is single-lane per # process; a second Worker lane over the same user_memory is a structural # protocol violation, not a schedulable thread. pthread_create() gets EAGAIN # (catchable std::system_error). Opt-in: HWJS_EXPERIMENTAL_THREAD_CLONE=1. # # So this is NOT the "parks the coroutinized thread-start frame and traps at the # next park" mechanism that 78-liburcu and 95-squashfs-tools record for the same # issue. It is a clean EAGAIN from clone(2), and most of CK checks it -- the # shared driver the spinlock, rwlock and cohort families are built on does # # if (pthread_create(&threads[i], NULL, thread, NULL)) { # ck_error("ERROR: Could not create thread %" PRIu64 "\n", i); # exit(EXIT_FAILURE); # } # # and those cases fail in one line. If that were the whole story these 54 would # run here and ship red-but-honest. # # IT IS NOT THE WHOLE STORY, AND WHAT MAKES THE DIFFERENCE IS NOT THE THREAD GAP # BUT THE ABSENCE OF PREEMPTION UNDER IT. Some of these regressions do not check # pthread_create -- ck_ec_smoke_test.c starts a waiter and discards the return at # four call sites (319, 337, 365, 383), then sits in ck_ec_wait() on an event # count nothing will ever post to. On a host with threads that is invisible. # Here it is fatal to the whole run: # # hwjs-worker-stall cpu=6 loop unavailable 5330.7ms (threshold 1000ms) # residents=[0:runnable] occurrences=1 # # A SPINNING GUEST PROCESS IS NOT PREEMPTABLE. One runnable resident holds the # substrate's single lane, so a timeout(1) wrapper around the case never gets to # run either -- and neither does the console bridge, so nothing already reported # escapes. (That wrapper is gone now, for a second reason; see below.) Measured twice on hz1, 2026-08-19: the first run # wedged after nine EAGAINs and produced no BST line in eighteen minutes, # losing the 47 serial results that had already passed; a second run with # ck_ec_smoke_test excluded by name got one case further and wedged again, on a # different binary (the kernel logged two clone attempts from the same pid, then # silence). So it is a property of the class and not of one test, and finding # the full set costs one lost run per member. # # The 54 are therefore reported not ok, unexecuted, each naming the issue. # BST_CK_THREADED=1 runs them, which is how both measurements above were taken # and how they should be re-taken; expect to lose the run. # # WHAT IS LOST BY NOT RUNNING THEM IS SMALL AND WORTH SAYING. Every one of them # would report the same EAGAIN for the same reason, so the 54 results carry one # bit between them -- and that bit is already established, in the kernel's own # words, quoted above. What they would add is the confirmation that CK's # structures behave under concurrency, and no arrangement of this platform can # supply that today. # # BRINTOS-WORKAROUND: linux-in-the-browser#85 remove -- these 54 are MASKED and # will NOT flip green by themselves when threads arrive. Delete the # BST_CK_THREADED gate below, and this marker with it, once clone(2) stops # answering EAGAIN. R="${BST_CK_ROOT:-}" LIBDIR=$R/usr/lib INCDIR=$R/usr/include PCDIR=$R/usr/lib/pkgconfig TESTS=${BST_CK_TESTS:-$R/usr/libexec/ck-tests} SERIAL=$BST_DATA/concurrencykit-serial.txt if [ ! -f "$LIBDIR/libck.a" ]; then bst_skip concurrencykit/present "concurrencykit not installed on this image" exit 0 fi bst_ok concurrencykit/present # --- the package is whole --------------------------------------------------- check concurrencykit/archive test -s "$LIBDIR/libck.a" check concurrencykit/pkgconfig test -s "$PCDIR/ck.pc" # Shape, not absence: this arm converts to shared libraries per recipe # (BRINTOS-1740), so a .so here is a conversion, not a policy breach. bst_lib_shape concurrencykit/no-shared "$LIBDIR/libck.so" # One header per subsystem the library ships. A port that dropped one would look # complete to anyone only linking libck.a, because most of CK is header-only -- # every spinlock, every queue macro, ck_ring, ck_bitmap and ck_pr itself compile # entirely from the headers, and src/ holds only twelve objects. for h in ck_pr.h ck_cc.h ck_md.h ck_ring.h ck_fifo.h ck_stack.h ck_queue.h \ ck_array.h ck_bitmap.h ck_hs.h ck_ht.h ck_rhs.h ck_epoch.h ck_hp.h \ ck_barrier.h ck_brlock.h ck_bytelock.h ck_rwlock.h ck_swlock.h \ ck_tflock.h ck_pflock.h ck_sequence.h ck_backoff.h ck_cohort.h \ ck_rwcohort.h ck_ec.h ck_spinlock.h ck_limits.h ck_stdbool.h \ ck_stdint.h; do check "concurrencykit/header-$h" test -s "$INCDIR/$h" done # The spinlock family is eight separate headers under spinlock/, pulled in by # ck_spinlock.h; they are the largest single block of the package and the one a # partial install would silently drop. for h in anderson.h cas.h clh.h dec.h fas.h hclh.h mcs.h ticket.h; do check "concurrencykit/spinlock-$h" test -s "$INCDIR/spinlock/$h" done # --- the port's defining facts ---------------------------------------------- # gcc/ck_pr.h is the generic primitive set over the compiler's __sync_* # builtins. On x86_64 it is a supplement to gcc/x86_64/ck_pr.h; here it is the # WHOLE implementation, because ck_pr.h's per-architecture chain matches nothing # and falls through to it. If it were not installed, the -dev package would be # unusable by any consumer on this arm while libck.a still linked fine. check concurrencykit/gcc-generic-pr test -s "$INCDIR/gcc/ck_pr.h" check concurrencykit/gcc-generic-cc test -s "$INCDIR/gcc/ck_cc.h" # There is no include/gcc/wasm32 and this port did not add one -- the whole # claim of the port is that the generic path is correct here. A directory # appearing would mean somebody wrote per-architecture assembly for a target # that has no assembler. check concurrencykit/no-wasm32-archdir test ! -d "$INCDIR/gcc/wasm32" # ck.pc is what a downstream consumer compiles against, and these two fields are # the port. -D__unknown__ names configure's unknown-platform profile; # -DCK_USE_CC_BUILTINS=1 tells the consumer's ck_pr.h to take the builtins # implementation directly rather than reaching the same place by falling off the # end of the arch chain. check_out concurrencykit/pc-cc-builtins "-DCK_USE_CC_BUILTINS=1" cat "$PCDIR/ck.pc" check_out concurrencykit/pc-profile "-D__unknown__" cat "$PCDIR/ck.pc" check_out concurrencykit/pc-libs "-lck" cat "$PCDIR/ck.pc" # ck_md.h is generated by configure and records what it decided about the # machine. CK_MD_RMO is the weakest of the three memory models, so every # ck_pr_fence_* emits a real barrier instead of being elided as it is under TSO # -- the conservative and correct choice for a target whose ordering is the wasm # memory model. The other two are the honest "we do not know" answers: pointer # packing off (no VMA layout to exploit) and VMA bits unknown. check_out concurrencykit/md-rmo "CK_MD_RMO" cat "$INCDIR/ck_md.h" check_out concurrencykit/md-no-ptr-pack "CK_MD_POINTER_PACK_DISABLE" cat "$INCDIR/ck_md.h" check_out concurrencykit/md-vma-unknown "CK_MD_VMA_BITS_UNKNOWN" cat "$INCDIR/ck_md.h" check_out concurrencykit/md-no-rtm "CK_MD_RTM_DISABLE" cat "$INCDIR/ck_md.h" # --- the regressions -------------------------------------------------------- if [ ! -d "$TESTS" ]; then bst_notok concurrencykit/tests-present "$TESTS is absent -- concurrencykit-tests is not on this image, so nothing below ran" exit 0 fi bst_ok concurrencykit/tests-present if [ ! -s "$SERIAL" ]; then bst_notok concurrencykit/serial-list "$SERIAL is absent or empty -- every regression below would be scored as thread-blocked, including the atomics" exit 0 fi bst_ok concurrencykit/serial-list # The list's own size is pinned. If a moved SRCREV changes which cases create # threads, the counts below still balance to 101 while scoring different cases # as the pass criterion -- a silent re-classification, invisible in the totals. check_eq concurrencykit/serial-list-size 47 "$(grep -cv '^#' "$SERIAL")" # THE DETAIL STRINGS ARE TERSE ON PURPOSE, AND THAT IS A CONSOLE LIMIT RATHER # than a style choice. These two are repeated across 54 result lines, and the # guest console is a finite pipe: the first version of this file carried the # whole explanation inline at ~330 characters a line, and the run reported 64 of # its 161 cases with "the console dropped the rest" -- a suite that looked small # and green instead of large and mostly-masked. The reasoning belongs in this # header, which costs the console nothing. _bst_1line caps ad-hoc details at 100 # characters for the same reason; a fixed string has to keep to it by hand. B85="clone(2) EAGAIN, single-lane substrate -- brintos/linux-in-the-browser#85" NOTRUN="masked (BST_CK_THREADED=1), $B85" # --------------------------------------------------------------------------- # NO timeout(1), AND NO LIVE fd ACROSS THE FORKS. BOTH COST A RUN TO LEARN. # # The obvious shape here -- `while read ... done < "$cf"` with each case run # under `timeout 10` -- reports 64 of its 161 cases on this guest and then # stops. What the console shows at the cut is not a dropped line but this: # # glibc-wasm32: RESTART-SENTINEL-LEAK val=513 nr=33 site=error_label hits=1 # (kernel restart sentinel reached a glibc errno return path) # 78-concurrencykit.test: line 239: Unknown error 513 # # 513 is ERESTARTNOINTR, an interval the kernel is supposed to consume and # never show anyone, and line 239 was the `read` -- not the case being run. A # signal arriving while the shell blocked in read(2) on the cases file returned # the internal restart sentinel through glibc's errno path instead of # restarting the call, the shell treated it as a fatal read error, and the # suite ended mid-file. The runner then reported a clean `==BST-END==` over 64 # cases, so the failure presents as a SMALL GREEN SUITE rather than as an # error: 58 pass, 6 fail, and the 97 cases that never ran leave no trace. # # So two things change, and each would have been enough on its own: # # - The bound is gone. It was there against a slow case and it never protected # against the real hazard (the header explains why -- an unpreemptable spin # starves the lane the bound would have to run in), so all it contributed was # the SIGALRM/SIGCHLD traffic that leaked the sentinel. Every case that # actually runs here is single-threaded and short. # # - The cases file is slurped before the loop instead of being read from an fd # held open across 47 fork/exec pairs. Nothing then blocks in read(2) while a # child is signalling, which is the state the leak needs. # # This is a platform defect and is reported as one; it is not specific to # Concurrency Kit, and any suite that runs many children while reading a file # can hit it. nser=0; nthr=0 for cf in "$TESTS"/*/cases; do [ -f "$cf" ] || continue dir=$(dirname "$cf") grp=$(basename "$dir") _all=$(cat "$cf") _oifs=$IFS IFS=' ' set -- $_all IFS=$_oifs for line in "$@"; do bin=${line%% *} args=${line#"$bin"} case "$bin" in ''|'#'*) continue ;; esac id="concurrencykit/$grp-$bin" if grep -qxF "$grp/$bin" "$SERIAL" 2>/dev/null; then nser=$((nser+1)) out=$(cd "$dir" && ./"$bin" $args &1); rc=$? if [ "$rc" -eq 0 ]; then bst_ok "$id" else bst_notok "$id" "serial regression failed rc=$rc: $(_bst_1line "$out")" fi continue fi nthr=$((nthr+1)) if [ -z "${BST_CK_THREADED:-}" ]; then bst_notok "$id" "$NOTRUN" continue fi out=$(cd "$dir" && ./"$bin" $args &1); rc=$? if [ "$rc" -eq 0 ]; then bst_ok "$id" else bst_notok "$id" "$B85 (rc=$rc) $(_bst_1line "$out")" fi done done # The counts are an assertion, not a footnote. A cases file that failed to # generate, or a -tests package that installed one directory short, would leave # this suite reporting a clean green over a fraction of the regressions -- the # one failure mode a per-case report cannot show, because what is missing emits # no line at all. check_eq concurrencykit/count-serial 47 "$nser" check_eq concurrencykit/count-threaded 54 "$nthr"