brintos

brintos / hwjs-cc public Read only

0
0

coroutinize: e8-registry escape-walk blowup on the GMP mpn shape (hgcd_hook) survives F162's round memo — coreutils factor unlinkable even at a 2h budget #9

Open dmitry opened this issue · 1 comment
D dmitry commented

Component: hwjs-cc / HwjsCoroutinize, e8-registry phase Found: 2026-07-15, coreutils 9.4, Yocto glibc arm (core-image-brintos-emcraft, MACHINE=brintos-wasm32, TCLIBC=glibc) Toolchain: hwjs-cc d3aa993 (F163), superproject ecb8f63. Both d51b4e6 (the #16 classify memo) and 4278a49 (F162, the #1 e8-registry round memo) are ancestors of this HEAD — verified with git merge-base --is-ancestor.

Symptom. Linking coreutils src/factor (which links GMP's mpn code), the e8-registry escape walk never converges. Two runs, identical except the budget:

BUDGET REFUSE phase=e8-registry fn=hgcd_hook exceeded=wall-clock elapsed_ms=600016  budget_ms=600000  work_units=129056768  work_budget=0
BUDGET REFUSE phase=e8-registry fn=hgcd_hook exceeded=wall-clock elapsed_ms=7200062 budget_ms=7200000 work_units=1501134848 work_budget=0

12x the wall-clock bought 11.6x the work and no convergence — a flat ~210k units/s with no sign of a fixpoint. That is #1's "work-units unconverged, still climbing" signature, but on GMP's hgcd_hook rather than clang's visitExpr: F162's round-scoped memo bounds the clang SCC shape and does not bound GMP's recursive mpn web in this phase.

Why this is not the existing issues.

  • linux-in-the-browser#16 (closed) was the classify-phase blowup; d51b4e6 fixed it and is present — classify is not where we spin.
  • hwjs-cc#1 (closed, F162) fixed e8-registry for the clang visitExpr shape. Its close note says: "If it still budget-refuses, post the attributed line and I'll take it from there." This is that line, for a different shape.
  • linux-in-the-browser#25 (closed) is the GMP TMP_ALLOC escape refusal set — a fast, loud refuse, not a spin.

Repro. MACHINE=brintos-wasm32 TCLIBC=glibc bitbake coreutils with the emcraft tool set (no --enable-no-install-program); watch the src/factor link. Every other coreutils tool links in ~1s on the same host — the pass's own BUDGET-OK telemetry shows ~0.9-1.1s / ~0.4-0.5M units per function there.

On the ops knob. The refuse text recommends raising HWJS_CORO_BUDGET_SECONDS. For this shape that advice is a multi-hour dead end — we spent 2h of a shared build host to learn it buys nothing. It may be worth having the attributed refuse say so when work-units are climbing linearly, rather than pointing at a budget that cannot help.

Impact. Blocks core-image-brintos-emcraft (glibc): 1916 tasks, 924 from sstate, exactly one failure — this link. The --enable-no-install-program=expr,factor workaround sanctioned in #16/#25 remains the way out and is what we will carry.

No candidate fix. The defect is in the pass's escape-walk memoization, upstream in hwjs-cc; we have no diagnosis of the pass internals to offer and therefore no branch and no patch — reporting evidence only.

D dmitry commented

Confirmed fix for this shape: the per-target escape-walk work cap from Alexander Dyachenko's fix/e8-round-memo (d7ad555) resolves it. Grafting only that knob (HWJS_E8_TARGET_WORK_CAP) onto F163 (d3aa993) makes coreutils link. The round memo from that branch is already in F163 as e8RoundMemoOn (F162) and — as this issue notes — does not bound the GMP hgcd_hook web; the work cap does.

Result (glibc arm, MACHINE=brintos-wasm32, TCLIBC=glibc):

  • bitbake -c compile coreutilssrc/factor links, 0 BUDGET REFUSE (was phase=e8-registry fn=hgcd_hook, unconverged at 600 s and 7200 s).
  • bitbake core-image-brintos-emcraft1932 tasks, all succeed; the image ships dpkg and is browser-verified.
  • Cap value: HWJS_E8_TARGET_WORK_CAP=2000000. A normal per-target walk is ~0.5 M steps in the pass's own BUDGET-OK telemetry; hgcd_hook climbs past 129 M — so 2 M bails the runaway to a conservative Opaque and leaves every normal target untouched.

The graft is +13 lines on F163 (helpers e8TargetWork/e8TargetWorkCap, the tick check in the classifyEscape worklist, and the per-target reset in e8PerTargetNocapture), default-off (cap=0).

Proposed fix

Branch: bst52-e8-target-work-cap Commit: 906deb3

The cap is a hard step ceiling that bails to a conservative Opaque (sound over-approximation, never a miscompile), so it is safe to enable globally — a capped target is de-optimized, never mis-lowered. Interim until the escape-walk memoization is fixed properly.

Proposed fix targets brintos/hwjs-cc only — no distro submodule pins moved. All distros share one lineage, so pin bumps are coordinated on merge, not per-engineer.