. "$BST_LIB" # BST-TIMEOUT: 1200 # TWO python3 starts, and they are the whole cost. Every # # other case below reads the filesystem or dpkg with shell # # builtins, or greps output the first start already # # produced. # # # # The two are: the probe, which takes ONE start for all 50 # # measurements, and the console script, which takes one # # more to show that /usr/bin/yappi behaves the same way the # # module does. # # # # 1200 is 600s per start, ten times the ~60s a python3 # # start has measured at here and the same per-start budget # # 99-python3-annotated-types gives its two. Nothing below # # is quadratic in anything. # # # # A suite-timeout red is a batch-wide cost, so if the two # # starts prove slower than that on target the RIGHT fix is # # this number, not fewer cases. # # # # NOT MEASURED ON TARGET YET: this port's first on-target # # run is the batch verify. # # python3-yappi 1.6.0 (meta-openembedded/meta-python # recipes-devtools/python/python3-yappi_1.6.0.bb, `inherit pypi # python_setuptools_build_meta ptest', PyPI sdist) on the wasm32-hwjs glibc # image. # # ============================================================================ # THE ONE QUESTION THIS SUITE ANSWERS, AND WHY IT IS OPEN. # ============================================================================ # # yappi is a deterministic, thread-aware CPU profiler implemented as a C # extension. `yappi.py' LINE 6 IS A BARE, UNGUARDED `import _yappi' -- not a # `try'/`except ImportError' around an optional accelerator, which is the shape # cbor2, simplejson and bitstruct have and which is why those three are usable # here with their extensions absent. Every public callable is a thin wrapper # over the extension, so THE WHOLE PACKAGE STANDS OR FALLS ON ONE LOAD, and # whether that load succeeds is the only interesting question about this port. # # THE LOADER CAN BIND IT -- MEASURED, on this image's own provider view. # # `_yappi.cpython-312.so' is a genuine wasm SIDE MODULE: `\0asm' version 1, # a `dylink.0' custom section and this arm's `hwjs.abi' section. Its # `dylink.0' carries only a MEM_INFO subsection and declares NO NEEDED # entries, so it asks for no other shared library. # # `tools/brintos_dlopen_census.py' composes this image's merged provider view # -- the rt module out of `vmlinux.wasm', then `libc.so', then the interpreter # -- against the module's import section, in the order the loader does it: # # providers: main 1715, libc.so 3812, rt 114 # OK usr/lib/python3.12/site-packages/_yappi.cpython-312.so # 91 env imports, all bound # 0 module(s) would fail to dlopen # # THE ORDER OF THOSE PROVIDERS IS THE WHOLE POINT, and it is where the recipe # phase's first reading of this port went wrong. Scored against the # INTERPRETER ALONE the module looks unloadable: it imports six libc FUNCTIONS # -- `fprintf', `getrusage', `gettimeofday', `memset', `strcmp', `time' -- and # the libc DATA symbol `stderr' as a `GOT.mem' import, and the interpreter # exports 1680 CPython names, 16 hwjs names and 19 wasm/lld names and NO libc # whatever. But a dlopen'd module does NOT resolve against the main's export # section: it resolves against rt + libc.so + main merged, and `libc.so' # publishes 3812 symbols including all seven. Reading only the main is how a # port talks itself into predicting a refusal that will not happen. # # AND THE INTERPRETER WILL ATTEMPT THE LOAD. `/usr/bin/python3.12' carries the # `.cpython-312.so' suffix string, `dlopen' and # `_PyImport_LoadDynamicModuleWithSpec', so the suffix is registered and # `_yappi' is a candidate -- not the empty-EXTENSION_SUFFIXES world that # `99-python3-cbor2' measured `ModuleNotFoundError: No module named "_cbor2"' # in. BRINTOS-2046, which stopped psutil, lz4, xxhash, crc32c and schedutils # at `unresolved import env.Py...', closed on 2026-09-02 by publishing CPython's # `PyAPI_FUNC'/`PyAPI_DATA' surface; `main 1715' above is that fix measured. # python3-gpiod (BRINTOS-2422) is the first port here to import a compiled # extension green on target, measured 2026-09-09. # # WHAT THE CENSUS DOES NOT SAY is what happens once the module INIT runs. It # answers whether the loader can BIND the module and nothing more, so the # import is still the thing being measured here rather than assumed. # # SO THE SUITE IS BUILT TO BE INFORMATIVE EITHER WAY, on the shape # 99-python3-crc32c uses. `import-ok' is ONE case and carries the verdict. If # the load fails, every behavioural case is SKIPPED with the reason attached # rather than reded -- twenty-five reds all naming one platform fact would bury # it rather than report it. If the load succeeds, the whole functional battery # the ticket asks for runs and is scored. # # ============================================================================ # MEASURED vs PREDICTED -- which is which, case by case. # ============================================================================ # # A recipe-phase lane builds no image and boots no guest, so NOTHING here has # run on target. But not every expectation below is a guess, and the two are # labelled separately at each case: # # * MEASURED -- read off THIS BUILD's own artifacts on the build host: the # `.deb' payload and metadata, the sizes, the wasm header and `dylink.0', # the shipped `yappi.py' source, and the import/export sections of both the # extension and the target interpreter. These are facts about what was # built, and a red is a real change. # # * PREDICTED -- derived from upstream source or from the platform reading # above. A red here is new information, not necessarily a defect. # # THREE OF THE TICKET'S PREMISES ARE NARROWED AGAINST THE SOURCE, recorded here # so nobody re-derives them: # # * "a clock type this machine cannot provide REFUSED with the documented # error" -- THERE IS NO SUCH REFUSAL. `set_clock_type()' accepts WALL and # CPU unconditionally and the package probes no capability anywhere. The # only two refusals that exist are for an unknown clock NAME and for # changing the clock while stats are held, and both are exercised below. # # * "a wasm32 runtime may offer no per-thread CPU clock at all" -- IT HAS # ONE. Measured on THIS BUILD's `.so': the CPU path compiled to # USE_CLOCK_TYPE_RUSAGE, so the binary carries `getrusage'/`1000ns' and # `gettimeofday'/`100ns' and NOT `clockgettime'/`1ns' -- setup.py's # `has_function timer_create' probe did not define LIB_RT_AVAILABLE on this # cross build. It is MICROSECOND-granular, which is why the duration cases # assert finite and non-negative rather than positive: a short call can # legitimately report 0.0. # # * "a run with a second `threading.Thread' asserted to report BOTH" -- # pthread_create does not produce a running thread on this arm # (linux-in-the-browser#85), which 99-python3.test's # `python3/thread-start-refused' asserts from the interpreter's own side. # The second thread is still started below and what it did is reported. # P=python3-yappi # The suite ran at all. Every case below is conditional on something, so this # one line is what separates "ran and found nothing" from "did not run". bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages DI=$SITE/yappi-1.6.0.dist-info PROBE=$BST_DATA/python3-yappi-probe.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-yappi.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ---------------------------------------------------------------- packaging -- # MEASURED. Where the packaging says the payload lands. These run before any # interpreter start, so a red here says the .deb is wrong rather than that the # import misbehaves. check $P/site-packages-dir test -d "$SITE" check $P/module-file test -s "$SITE/yappi.py" check $P/extension-file test -s "$SITE/_yappi.cpython-312.so" check $P/dist-info-dir test -d "$DI" check $P/dist-info-metadata test -s "$DI/METADATA" check $P/dist-info-record test -s "$DI/RECORD" # THE CONSOLE SCRIPT the recipe's entry_points produce. Present and executable # here; whether it RUNS is the last section, and the two are kept apart because # a missing file and a failing import are different defects. check $P/console-script test -x /usr/bin/yappi # ------------------------------------------------------------------- dpkg ---- if ! command -v dpkg-query >/dev/null 2>&1; then bst_skip $P/dpkg "no dpkg-query on this image" else # MEASURED against the .deb this build produced. check_eq $P/dpkg-version "1.6.0-r0" \ "$(dpkg-query -W -f '${Version}' python3-yappi 2>/dev/null)" # THE RUNTIME DEPENDENCY LINE, ASSERTED VERBATIM. meta-python declares # `RDEPENDS:${PN} += "python3-datetime python3-pickle python3-threading"' # and python3-core arrives from the pypi/setuptools class. All four are # sub-packages of this image's own python3 (BRINTOS-1476) and all four are # named directly on IMAGE_INSTALL, so the line is satisfiable as written # and this port needed no bbappend to make it so. check_eq $P/dpkg-depends \ "python3-core, python3-datetime, python3-pickle, python3-threading" \ "$(dpkg-query -W -f '${Depends}' python3-yappi 2>/dev/null)" # NO -ptest PACKAGE ON THIS IMAGE. The recipe does `inherit ptest' and its # RDEPENDS:${PN}-ptest names python3-gevent, python3-zopeinterface, # python3-pytest, python3-profile, python3-multiprocessing and # python3-unittest-automake-output -- none of which is ported. `ptest' is # not in DISTRO_FEATURES here, so do_install_ptest never ran and no such # .deb was produced; this case is what notices if that changes and drags # six unported packages into the image's dependency closure. check_eq $P/no-ptest-package "" \ "$(dpkg-query -W -f '${Version}' python3-yappi-ptest 2>/dev/null)" fi # ------------------------------------------------------------------ probe ----- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "probe missing at $PROBE" exit 0 fi bst_ok $P/probe-present # 600s for the one start that does all 50 measurements. On overrun bst_probe # names the last line the probe printed, which names the case it stopped at. # # The probe EXITS 0 whichever way the import went, so this case passing means # "the probe ran", not "the module works" -- `import-ok' below carries that. bst_probe $P/probe-run "$W/probe" 600 python3 "$PROBE" OUT="$W/probe" # One `grep' per value, split on the FIRST `=' so a value containing one is # safe. Every key the probe emits is [a-z0-9_]+. pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } # THE PROBE RAN TO THE END AND EMITTED EVERYTHING. Without these two, a probe # that died half way would report every case after the death as a mismatch # against an empty string, which reads as two dozen behavioural failures # instead of one truncated transcript. # # 50 is a CONSTANT of the probe and not a property of this image: every # functional key is emitted on both branches, as `skipped:no-import' when the # load failed, precisely so this count can be asserted without knowing which # way the load went. check_eq $P/probe-complete "yes" "$(pv probe_done)" check_eq $P/probe-count "50" "$(pv emitted_count)" # ---------------------------------------------------------------- identity -- # THE ARM. This is the one case that says the transcript came from the wasm32 # interpreter rather than from a host copy -- every other value below would # pass either way, and a suite that could be satisfied by an x86-64 python3 is # not measuring this port. 31 here, 63 on an x86-64 control. check_eq $P/py-maxsize-bits "31" "$(pv py_maxsize_bits)" check_eq $P/py-version "3.12" "$(pv py_version)" # ------------------------------------------------------------- the payload -- # MEASURED, all of it, on the artifacts this build produced. # # THE EXTENSION BUILT CORRECTLY, and this is kept strictly separate from # whether it LOADS: "the port built it" and "the platform can use it" are two # statements and a reader has to be able to tell which one changed. # # `so-dylink' is the one that says this is a wasm SIDE MODULE rather than a # plain object -- the thing that makes it loadable in principle at all. check_eq $P/so-present "yes" "$(pv so_present)" check_eq $P/so-magic "0061736d" "$(pv so_magic)" # Every relink of the extension moves its byte count, so only a non-empty # object is pinned here. so_size=$(pv so_size) check_eq $P/so-size "nonzero" "$([ "${so_size:-0}" -gt 0 ] 2>/dev/null && echo nonzero || echo "${so_size:-empty}")" check_eq $P/so-dylink "yes" "$(pv so_dylink)" # yappi.py IS THE SDIST'S, UNMODIFIED -- 46304 bytes from the sdist whose # sha256 the recipe pins, so whatever happens below is upstream's code on this # image and not something the packaging did to it. check_eq $P/py-present "yes" "$(pv py_present)" check_eq $P/py-size "46304" "$(pv py_size)" # THE UNGUARDED IMPORT, READ OUT OF THE SHIPPED SOURCE rather than asserted # about it from memory. This is what makes the package all-or-nothing: line 6 # is a bare `import _yappi' with no `try' above it, so there is no fallback to # reach. A release that grew one would change this, and this port would need # re-judging. check_eq $P/py-bare-import-line "6" "$(pv py_bare_import_line)" # --------------------------------------------------------- the distribution -- # MEASURED. The metadata path needs NO import, so these answer on either # branch -- which is worth having: they prove the wheel was INSTALLED rather # than merely unpacked, and they are the one clause of the ticket's "reports # the version the recipe pins" that survives a failed load. check_eq $P/dist-version "1.6.0" "$(pv dist_version)" check_eq $P/dist-name "yappi" "$(pv dist_name)" # BOTH top-level names are declared, including the extension. This is the # packaging's own statement that `_yappi' is part of the distribution rather # than a build artifact that leaked in. check_eq $P/dist-top-level "_yappi,yappi" "$(pv dist_top_level)" check_eq $P/dist-entry-points "yappi=yappi:main" "$(pv dist_entry_points)" # ------------------------------------------------- the extension boundary --- # PREDICTED. These three say WHICH mechanism is in play, and they are kept # apart from `import-ok' below on purpose: the verdict does not depend on them. # # `ext-suffixes-nonempty' is the interpreter's own answer to "can I load a .so # at all". Predicted YES -- the binary carries the `.cpython-312.so' suffix # string and `_PyImport_LoadDynamicModuleWithSpec' -- which is the OPPOSITE of # what an empty lib-dynload would suggest, and is why this port's refusal (if # it refuses) is expected to be a LINK failure rather than the # `ModuleNotFoundError' 99-python3-cbor2 measured for `_cbor2'. _esc=$(pv ext_suffixes_count) case "$_esc" in ""|0) _esn=no ;; *) _esn=yes ;; esac check_eq $P/ext-suffixes-nonempty "yes" "$_esn" # `find_spec' locates a module WITHOUT executing it, so this pair separates "the # file is installed and is a candidate" from "importing it works". Both # predicted `found': yappi.py is ordinary Python, and `_yappi' is a candidate # exactly when a suffix is registered. check_eq $P/find-spec-yappi "found" "$(pv find_spec_yappi)" check_eq $P/find-spec-underscore "found" "$(pv find_spec_underscore_yappi)" # `_yappi' is third-party, so it could only ever arrive as a file -- it is not # and cannot be one of the interpreter's builtin modules. check_eq $P/no-builtin-underscore "no" "$(pv builtin_has_underscore_yappi)" # ------------------------------------------------------------ THE VERDICT --- # ONE case for the platform fact, carrying the exception if there is one. # # EXPECTED `yes', on the dlopen census in the header: all 91 of this module's # `env' imports bind against the merged rt + libc.so + main view, and the # interpreter registers the `.cpython-312.so' suffix so `_yappi' is a # candidate. That is measured about the BIND; the import also has to survive # `PyInit__yappi', which nothing off-target can check, so this case is a real # measurement rather than a formality. # # A RED HERE IS THE FINDING, and it would carry its own diagnosis: the class # and message below say whether the loader refused a symbol, or the module # init did. _imp=$(pv import_yappi_ok) check_eq $P/import-ok "yes" "$_imp" # THE TWO IMPORT PATHS MUST AGREE. `import _yappi' directly and `import yappi' # (which reaches it via line 6) are the same load through two doors, so a # DISAGREEMENT is a real defect however the verdict went -- it would mean the # extension is reachable one way and not the other, which no mechanism above # explains. Asserted as a relation rather than a value, so this case says the # same thing on either branch. check_eq $P/import-paths-agree "$_imp" "$(pv import_underscore_ok)" if [ "$_imp" = "yes" ]; then bst_skip $P/fails-on-extension "the extension loaded; nothing refused" else # ...AND WHICH HALF FAILED. On a refusal `import yappi' dies on line 6 and # the message names `_yappi'; a `no' here would mean the pure-Python half had # acquired a second, independent problem hiding behind the same red. check_eq $P/fails-on-extension "yes" "$(pv yappi_fails_on_extension)" fi # ============================================================================ # THE FUNCTIONAL BATTERY -- only meaningful if the load succeeded. # ============================================================================ FUNCTIONAL="clock-default clock-wall-readback clock-cpu-readback clock-info-cpu clock-info-wall clock-bad-name clock-change-with-stats stats-before-profiling stop-without-start start-twice is-running-before is-running-during is-running-after profile-outer-ncall profile-helper-ncall profile-names-present durations-finite durations-nonnegative thread-count thread-main-present second-thread-body thread-count-after-second pstat-roundtrip pstat-function-count clear-stats-empties" if [ "$_imp" != "yes" ]; then # The extension did not load. SKIP -- not red -- every behavioural case, each # carrying the reason: they are all one fact, and twenty-five reds would bury # it rather than report it. `import-ok' above is the one case that carries # the verdict, and the design doc attributes it. _why="_yappi.cpython-312.so did not load: $(pv import_underscore_class): $(pv import_underscore_msg)" for _c in $FUNCTIONAL; do bst_skip $P/$_c "$_why"; done else # ------------------------------------------------------------- clocks ---- # PREDICTED from upstream source and from THIS BUILD's `.so' strings. # `get_clock_type()' returns the lowercase name (both `wall' and `cpu' are # string literals in the binary); the default is CPU_CLOCK. check_eq $P/clock-default "cpu" "$(pv clock_default)" check_eq $P/clock-wall-readback "wall" "$(pv clock_wall_readback)" check_eq $P/clock-cpu-readback "cpu" "$(pv clock_cpu_readback)" # THE CLOCK BACKENDS THIS BUILD COMPILED TO, asserted as `api|resolution'. # USE_CLOCK_TYPE_RUSAGE, so cpu is getrusage at microsecond granularity -- # `clockgettime' and `1ns' are absent from the binary. check_eq $P/clock-info-cpu "getrusage|1000ns" "$(pv clock_info_cpu)" check_eq $P/clock-info-wall "gettimeofday|100ns" "$(pv clock_info_wall)" # THE FIRST OF THE TWO REFUSALS THAT EXIST: an unknown clock NAME. Raised in # yappi.py against the CLOCK_TYPES table BEFORE the extension is reached -- # and `set_clock_type' upcases its argument first, so the message carries # `MONOTONIC' and not `monotonic'. check_eq $P/clock-bad-name \ "YappiError: Invalid clock type:MONOTONIC" "$(pv clock_bad_name)" # THE SECOND: changing the clock while stats are held. This one DOES reach # the extension, so the class is `error' -- `_yappi.error', the exception # PyErr_NewException creates -- and not yappi.py's own `YappiError'. check_eq $P/clock-change-with-stats \ "error: clock type cannot be changed previous stats are available. clear the stats first." \ "$(pv clock_change_with_stats)" # -------------------------------------------------------- error paths ---- # The ticket's "error paths" are documented NO-OPS in 1.6.0 rather than # errors, and these say so. `stop()' un-started returns: this build's `.so' # carries no "profiler is not started yet" string at all. A second `start()' # returns without restarting. check_eq $P/stats-before-profiling "0" "$(pv stats_before_profiling)" check_eq $P/stop-without-start "returned" "$(pv stop_without_start)" check_eq $P/start-twice "returned" "$(pv start_twice)" check_eq $P/is-running-before "False" "$(pv is_running_before)" check_eq $P/is-running-during "True" "$(pv is_running_during)" check_eq $P/is-running-after "False" "$(pv is_running_after)" # ------------------------------------------- the committed call graph ---- # A deterministic profiler over a known graph has a REFERENCE ANSWER, and # these two are literals rather than machine-dependent: `_outer' is called # once and calls `_helper' exactly 7 times. check_eq $P/profile-outer-ncall "1" "$(pv profile_outer_ncall)" check_eq $P/profile-helper-ncall "7" "$(pv profile_helper_ncall)" check_eq $P/profile-names-present "yes" "$(pv profile_names_present)" # THE DURATIONS ARE THE MACHINE'S. Asserted only to be finite and # non-negative -- the cpu clock is microsecond-granular here, so a short call # can legitimately report 0.0 and a "positive" assertion would be flaky. check_eq $P/durations-finite "yes" "$(pv durations_finite)" check_eq $P/durations-nonnegative "yes" "$(pv durations_nonnegative)" # ------------------------------------------------------ thread stats ---- # ONE context: this arm has no second thread (linux-in-the-browser#85), so # `get_thread_stats()' keeps seeing exactly one. MEASURED 2026-09-10, and not # what this suite first predicted: the kernel does not let the thread start # and then decline to schedule it -- it refuses the clone outright, printing # `wasm32: REFUSING thread-mode clone (CLONE_VM, flags=0x3d0f00) with EAGAIN' # to the console, so `Thread.start()' raises rather than returning. That is a # deliberate refusal with an opt-in (`HWJS_EXPERIMENTAL_THREAD_CLONE=1'), not # a crash. `never-ran' -- a thread that starts and never runs -- is therefore # unreachable here, and asserting it scored the platform's documented answer # as a defect of the port. If this arm gains working threads these three go # red together, which is the right shape of red. check_eq $P/thread-count "1" "$(pv thread_count)" check_eq $P/thread-main-present "yes" "$(pv thread_main_present)" check_eq $P/second-thread-body "refused:RuntimeError" "$(pv second_thread_body)" check_eq $P/thread-count-after-second "1" "$(pv thread_count_after_second)" # -------------------------------------------------- pstat round-trip ---- # `save(type="pstat")' writes a file the stdlib `pstats' reads back, which is # the interchange the ticket asks about. Written under TMPDIR: a write to the # virtiofs root is what brintos/linux#73 is about. check_eq $P/pstat-roundtrip "yes" "$(pv pstat_roundtrip)" check_eq $P/pstat-function-count "2" "$(pv pstat_function_count)" # `clear_stats()' empties the table. check_eq $P/clear-stats-empties "0" "$(pv clear_stats_empties)" fi # --------------------------------------------------------- console script ---- # THE ENTRY POINT BEHAVES THE SAME WAY THE MODULE DOES. `/usr/bin/yappi' is a # stub that does `from yappi import main', so it can only start if the # extension loaded. Asserted against the SAME verdict as `import-ok' rather # than against a fixed expectation, so the two cannot disagree: a script that # started while the module would not import -- or the reverse -- is a real # defect and is exactly what this case catches. # CALLED WITH NO ARGUMENTS. yappi 1.6.0 registers no `--version' -- its # `main()' builds an `optparse' parser carrying only -b/-c/-o/-f/-s -- so the # first form of this case asked for an option upstream does not have and scored # upstream's own `exit 2' as a defect of the port. With no arguments `main()' # takes its documented `parser.print_usage(); sys.exit(2)' branch, which is # reached ONLY once `import yappi' -> `import _yappi' has succeeded and the # parser has been built from the module's own CLOCK_TYPES. So rc=2 plus that # usage line is proof the extension loaded, and an ImportError traceback naming # `_yappi' is the refusal shape -- which is the verdict this case exists to # hold against `import-ok'. The CLI's profiling path is not exercised here; the # functional battery above covers profiling through the module. yout=$(/usr/bin/yappi 2>&1); yrc=$? if [ "$_imp" = "yes" ]; then if [ "$yrc" -eq 2 ]; then # The usage line, and not a traceback: only yappi's own parser prints both # of these metavars, and it is built after the extension has loaded. case "$yout" in *Traceback*) bst_notok $P/console-script-agrees \ "module imported but /usr/bin/yappi traceback'd: $(_bst_1line "$yout")" ;; *clock_type*output_file*) bst_ok $P/console-script-agrees ;; *) bst_notok $P/console-script-agrees \ "rc=2 but not yappi's own usage line: $(_bst_1line "$yout")" ;; esac else bst_notok $P/console-script-agrees \ "module imported but /usr/bin/yappi exited $yrc: $(_bst_1line "$yout")" fi bst_skip $P/console-script-names-extension "the extension loaded; no traceback" else if [ "$yrc" -ne 0 ]; then bst_ok $P/console-script-agrees else bst_notok $P/console-script-agrees \ "rc=0 -- /usr/bin/yappi started, so _yappi imported after all: $(_bst_1line "$yout")" fi # ...AND IT NAMES THE EXTENSION. One grep over the traceback the call above # already captured -- no extra interpreter start. case "$yout" in *_yappi*) bst_ok $P/console-script-names-extension ;; *) bst_notok $P/console-script-names-extension \ "traceback does not mention _yappi: $(_bst_1line "$yout")" ;; esac fi rm -rf "$W"