brintos

brintos / hwjs-cc public Read only

0
0

F162: GNU make 4.4.1 traps at startup — set-over-set "parked slot not empty at a new park" (coroutinized frame parked under a NATIVE caller), free ← variable_expand_string #6

Closed jdbrinton opened this issue · 1 comment
jdbrinton jdbrinton commented

Found while fixing linux-in-the-browser#77 (make exit 2 = the musl posix_spawn gap, fixed libc-side). Rebuilding GNU make 4.4.1 on the current F162 drop (4278a49) produces a binary that traps before doing ANY work — even make --version — so the #77 image remedy (rebuild make against the fixed musl) is blocked on this.

Witness (identical across every build variant)

hwjs-user-rt VIOLATION: set-over-set: parked slot not empty at a new park
(orphaned handle 0x005420d0 owner-word=0x00000017 id-via-frame-header[h+4]=0x0000009a;
incoming handle 0x00542210 owner-word=0x0000001f)
— a coroutinized frame parked under a NATIVE caller and the witness reached the next park

Trap stack (O2 build, from onUserError):

make.free (wasm-function[159])
make.variable_expand_string (wasm-function[330])
make.variable_expand.resume (wasm-function[266])
make.hwjs_resume / hwjs_drive_loop / _start

The host reaps it as a SIGILL death ($? = 132, tty stays live — the #83 reap belt works), but make is unusable.

Build recipe

CC=toolchains/hwjs-cc/bin/wasmcc
CFLAGS='-O2 -include alloca.h -D_GNU_SOURCE'
configure --host=wasm32-hwjs-linux-musl --build=x86_64-pc-linux-gnu \
  --disable-nls --disable-dependency-tracking
env: WASMLD_HWJS=1 WASMLD_ASYNCIFY=0, sysroot = distros/common/build/musl-1.2.5-build
(one generated-Makefile fixup: strip the polluted "typedef unsigned int size_t; void *alloca(size_t);"
prefix from lib/Makefile GL_CFLAG_GNULIB_WARNINGS — same as the findutils #83 build)

Ruled out (all variants trap IDENTICALLY, same handles/owner-words)

  • -O2 vs -O1: both trap → not the memory-effect-lie fold class (F160 strip is on).
  • --disable-load --without-guile (drops -Wl,--export-dynamic + dlopen machinery): still traps.
  • musl pre/post the #77 posix_spawn/pclose overlays: both trap → not interaction with the new libc suspenders.
  • -O0: does not link — the pass REFUSES glob/f_mtime/ar_glob with vla/dynamic-alloca-address-escape-opaque-sink (loud, correct never-silent). At O1/O2 those shapes are DS-LOWERED instead and the link succeeds — so a believed-sound lowering (or a distinct orphan source) is producing a chain whose caller is native at runtime.
  • Not a #77 regression: the kent/brintos-dev image's make (older drop, 1.7 MB) runs --version and parses Makefiles fine (it only failed at posix_spawn, now fixed in musl).

Note src/expand.c uses dynamic alloca in exactly the trapping neighborhood (variable_expand_string / patsubst paths), and make is a heavy qsort_r/function-pointer consumer — both active axes (the #73 gqsort/VLA work).

Repro artifacts

  • make --version alone reproduces (no Makefile needed) under the musl rig or the brintos-dev rootfs headless harness.
  • Build dirs + logs preserved in the #77 session scratchpad (make-build{,2,4,5}, drain/dmesg logs); the O2 binary is 1.63 MB.

Blocking: linux-in-the-browser#77 end-to-end (make -f /tmp/mk printing MK_OK on a current-toolchain make). The libc half of #77 is landed and gate-proven (run-posix-spawn-gate.mjs); this is the remaining half.

jdbrinton jdbrinton commented

Fixed on joel-dev — hwjs-cc d3aa993, superproject pin 2be7e2c (F163). GNU make 4.4.1 rebuilt on the fixed toolchain now links with zero refusals and runs clean.

Root cause (two layers)

1. The binary should never have shipped: wasmld's never-silent REFUSE gate had a pipefail/SIGPIPE hole. The gate was grep 'REFUSE fn=' | grep -qv <benign> under set -o pipefail. grep -q exits at first match and closes the pipe; when the refusal lines span more than one stdio buffer (~4 KiB) and the left grep keeps scanning after the reader exits, the left grep dies with SIGPIPE (141), pipefail makes the whole condition false, and the gate is silently skipped — precisely on refusal-heavy links. make's merged module had 16 refused functions and shipped anyway. The refused variable_expand_string stayed NATIVE while its callees (xstrdup — frame owner 0x17, destroyer 0x9a; free — owner 0x1f) were coroutinized, so the callee ramp's park was never taken and the next park hit set-over-set: parked slot not empty — the exact ticket frame free ← variable_expand_string ← variable_expand.resume at make --version. The "-O0 refuses loudly vs -O2 traps" contrast was this gate race plus different refusal volumes (26 vs 16 refusals), not an optimizer-fold class: F160's strip was active, behavior was identical at -O1/-O2, and the F162 memo and libc interaction were ruled out with IR diffs.

2. Six real classifier gaps produced the 16 refusals (all vla/dynamic-alloca-address-escape-opaque-sink, all lifted in pass/coroutinize/CoroutinizePass.cpp):

  • Canonicalized pointer diffs: instcombine rewrites pct - pat - 1 into add(ptrtoint p, xor(ptrtoint q, -1)) — the Sub-only ptrDiffConfinedVal walk missed it. Now a sign-tracked walk over the add/sub/xor(-1) spine (linearSingleTerm + ptrtointFreeTree) confines any linear two-pointer difference. (make: variable_expand_string/patsubst_expand_pat.)
  • Reassociated diffs: h + (cur - key) reassociated to sub(cur, add(h, key)) — same lift covers it. (make: jhash-style hash tails.)
  • Registration-helper field attribution: fn ptrs stored through hash_init(ht, h1, h2, cmp)-shape helpers weren't attributed to the table fields, so the alloca'd lookup key dispatched through those fields hit an empty candidate set → wild. New regHelperParamStoreOffsets arm + intra-element initializer attribution. (make: the whole hash.c family — strcache_add_len, f_mtime, file_*.)
  • GlobalAlias name-uses joined candidate sets as opaque uses → net-wild poison. Aliases now pass through.
  • Field-FATE consult: a call through a struct-field fn ptr now meets classifyEscape over the closed-world candidate set (in-memory Memory-class escapes allowed) instead of refusing outright; plus a param-rooted callback devirt arm (paramActualFns, exact actuals, cycles fixpoint) for ar_scan-style dispatch.
  • Sort-site comparator: a dynamic-alloca array passed to qsort/qsort_r with a constant defined comparator (make's ar_glob → musl wrapper_cmp union dispatch) is now classified by the comparator's own params with a Memory floor.

The wasmld gate is rewritten SIGPIPE-free (capture with || true, filter, test non-empty, print the surviving lines before the FATAL), same for the §8b native-caller intersection gate. Refusals that genuinely can't be handled still refuse — now guaranteed loudly: a refused merged module fails the link and ships nothing.

RED → GREEN evidence

  • RED (pre-fix): rebuilt make traps at startup: hwjs-user-rt VIOLATION: set-over-set: parked slot not empty, RuntimeError: unreachable at make.free <- make.variable_expand_string <- make.variable_expand.resume; link log shows 16 REFUSE fn= lines and the gate demonstrably skipped (left grep rc=141 under pipefail, pinned by test/refuse-gate-sigpipe.sh's RED arm: refusals split around 200k filler lines deterministically SIGPIPE the old pipeline; the new form fires).
  • GREEN (final artifact, headless, zero userErrors): make --version prints GNU Make 4.4.1 and exits 0; printf 'all:;@echo MK_OK\n' | make -f -MK_OK; a tab-recipe shell command runs; a 2-target dependency build orders correctly; $(shell echo ...) expands.
  • Gates landed (all GREEN with genuinely flipping RED arms): pass/coroutinize/f163_make_shapes.mjs (7 shapes, 7 RED knob flips), test/refuse-gate-sigpipe.sh, and the superproject distro gate distros/linux-6.12-glibc-bash-coreutils/tools/run-hwjscc6-make-shapes-gate.mjs (whole-program musl link of the exact make function family: 0 refusals, parks>0, balanced; RED knobs → loud link FATAL with named refusals, no artifact).
  • Full pass suite green (every FINDINGS gate incl. F158 dtor, F159 gqsort, F160 gstrerror, F161 fpcast, F162 memo); four control fixtures (glob_altdirfunc, gcd_subdiv_hook, hashcb_table, fwritex_writefield) sharpened to decl-sink captures because the new meets legitimately lower their old Memory-class shapes.
  • All distros rebuilt green on the re-derived plugin (provenance commit=d3aa993 dirty=0): kernel, linux-6.12-musl-busybox, linux-6.12-musl-bash-coreutils, linux-6.12-glibc-bash-coreutils, wabuntu-24.04. Substrate/glibc regression gates (r6, malloc-churn/fatal, dlopen, F158–F161) all pass. parked_abi/drive_abi unchanged.

SHAs

  • hwjs-cc: d3aa993 (joel-dev, pushed)
  • superproject: 2be7e2c (joel-dev, pushed; pin + distro gate)

Deploy is deferred to the next nightly per batch policy; the headless gate on the final artifact is authoritative. Known future lift (documented in FINDINGS F163): general correlated-context threading beyond the qsort name/shape contract.

jdbrinton closed this as completed