# BST-TIMEOUT: 300 # measured 12s (==BST-SUITE, 2026-08-20, local guest); 25x headroom # # The marker is here rather than left at the 60s default because almost all of # those 12 seconds are ONE process -- an 11 MB probe module instantiated once, # which then writes 601 keys, scans the database six times and compacts it twice. # That is a single indivisible cost on a host having a bad day, not a hundred # small ones that average out. 300 is deliberately not larger: the bound exists to # protect the run's ==BST-END== sentinel from a suite that hangs, and the specific # hang this port could produce -- a compactor pump that never terminates -- would # sit on whatever bound it is given. . "$BST_LIB" # leveldb (meta-oe recipes-dbs/leveldb, 1.23 at SRCREV 068d5ee1) -- Google's # embedded ordered key-value store: an LSM tree in a library, the storage engine # underneath a long list of local databases. Eighth port of the C++ round # (BRINTOS-1094), after abseil-cpp, re2, gflags, glog, fmt, double-conversion and # snappy. # # --------------------------------------------------------------------------- # WHAT THIS PORT HAD TO CHANGE, AND WHY THE PROBE IS THE ONLY WITNESS FOR IT # # This target has ONE THREAD OF EXECUTION and brintos-libcxx is configured to # say so (LIBCXX_ENABLE_THREADS=OFF), so there is no std::mutex, no # std::condition_variable and no std::thread. leveldb assumes all three: its # compactor runs on a background thread and every "wait" in the database is a # wait for that thread. The port's three patches move the compactor onto the # calling thread and make port::CondVar::Wait() RUN one queued work item # instead of blocking. # # That is a change to WHEN COMPACTION HAPPENS, and nothing static can see it. # The headers are unchanged, the symbols are unchanged, the archive is # unchanged in shape -- a build with a broken scheduler ships an archive that # looks exactly like a working one and hangs, or silently loses writes, the # first time something opens a database. So the assertions that matter here are # the probe's, and specifically the ones on the far side of a close/reopen. # # THE PROBE IS ALSO THE ONLY CONSUMER. leveldb installs one program, # leveldbutil, and it is a dump tool for a single .ldb file -- it opens no # database. An undefined symbol in a static archive is not an error until # something links it, which is how glog (BRINTOS-1101) shipped green and # unlinkable and what fmt's __udivmodti4 turned out to be (BRINTOS-1102, # linux-in-the-browser#155). # # WHERE THE EXPECTED VALUES COME FROM, WHICH IS NOT FROM A RUN. # # data/leveldb-probe writes a corpus defined by a RULE, stated here and in the # probe's own header: # # key/%06d i in [0,500) -> "val/%06d/" + 64 copies of ('a' + i%26) # bat/%06d i in [0,100) -> "bval/%06d" (one WriteBatch) # zsync -> "sync-witness" (WriteOptions.sync) # # then deletes key/000100..key/000199 and reports FNV-1a/64 over the # "key=value\n" stream the database hands back, in iteration order. # # Both digests below were computed FROM THAT RULE, by a program that does not # link leveldb at all -- not captured from a passing run. A build that returned # a self-consistent but wrong set agrees with itself and disagrees with these. # That is the difference between an oracle and a snapshot, and it is the whole # reason the corpus is a rule rather than random bytes. # # 601 keys before the delete : 67a7a707e5407a88 # 501 keys after the delete : 224f371943091ddc # # THE SURVIVING SET IS ASSERTED FOUR TIMES, NOT ONCE, and the four are the point # of the exercise rather than repetition: # # live in the memtable, nothing written down yet # reopened after close + open -- the log replay path # compacted after CompactRange(nullptr, nullptr) rewrote every table # recompacted after a second close + open, now over compacted tables # # `reopened` is the case the ticket names: it is the one that depends on fsync # and rename semantics, which this arm has had gaps in before (BRINTOS-54). A # port whose writes never reach the filesystem passes `live` perfectly. # --------------------------------------------------------------------------- # A root prefix, empty on the target and set only when this suite is validated # natively before it ships. The runner never sets it. R="${BST_LEVELDB_ROOT:-}" ARCHIVE=$R/usr/lib/libleveldb.a INCDIR=$R/usr/include/leveldb UTIL=$R/usr/bin/leveldbutil PROBE=$BST_DATA/leveldb-probe # THE CANARY, which is not an assertion about leveldb at all. # # brintos_local_verify.py loses a suite's FIRST BST result line before the host # parser ever sees it -- BRINTOS-1149, a defect in our own host-side tooling and # not in the runner, the guest or any package. The harness reports the deficit # (`declared N case(s), N-1 reached the console`) but computes its pass count as # total-minus-failures, so THE DROPPED LINE IS COUNTED AS A PASS. A suite whose # first case is a genuine `not ok` therefore loses the one line that mattered, # silently. # # So the first line out of this file asserts something already true. Same idiom, # and the same reason, as 56-snappy, 82-double-conversion and 83-flatbuffers. # Remove this case when 1149 is fixed. bst_ok leveldb/suite-canary if [ ! -f "$ARCHIVE" ]; then bst_skip leveldb/present "leveldb not installed on this image" exit 0 fi bst_ok leveldb/present nl=' ' tmp=${TMPDIR:-/tmp} # has # A shell `case` rather than a grep: this suite makes ~90 content assertions and # an exec apiece is what killed the first draft of the abseil suite on target, # where every process is a fresh instantiation of a multi-megabyte module. has() { case "$2" in *"$nl$3$nl"*) bst_ok "$1" ;; *) bst_notok "$1" "missing: $3" ;; esac } # sub sub() { case "$2" in *"$3"*) bst_ok "$1" ;; *) bst_notok "$1" "missing: $3" ;; esac } # hasnt -- substring, for "must NOT be there" hasnt() { case "$2" in *"$3"*) bst_notok "$1" "present but must not be: $3" ;; *) bst_ok "$1" ;; esac } # --- the package is whole ---------------------------------------------------- check ldb/hdr-db test -f "$INCDIR/db.h" check ldb/hdr-options test -f "$INCDIR/options.h" check ldb/hdr-slice test -f "$INCDIR/slice.h" check ldb/hdr-status test -f "$INCDIR/status.h" check ldb/hdr-write-batch test -f "$INCDIR/write_batch.h" check ldb/hdr-iterator test -f "$INCDIR/iterator.h" check ldb/hdr-comparator test -f "$INCDIR/comparator.h" check ldb/hdr-cache test -f "$INCDIR/cache.h" check ldb/hdr-env test -f "$INCDIR/env.h" check ldb/hdr-filter test -f "$INCDIR/filter_policy.h" check ldb/hdr-table test -f "$INCDIR/table.h" check ldb/hdr-c-api test -f "$INCDIR/c.h" check ldb/hdr-export test -f "$INCDIR/export.h" check ldb/archive-present test -f "$ARCHIVE" # leveldbutil is the ONE program this port installs, and the stock recipe adds it # by hand (do_install:append) rather than through cmake's install rules -- so its # absence is a plausible failure that nothing else here would notice. check ldb/util-present test -f "$UTIL" # Static-only is distro policy and the stock meta-oe recipe asks for the opposite # EXPLICITLY (-DBUILD_SHARED_LIBS=ON), which the bbappend removes. A .so here # would mean that removal stopped matching -- the failure mode is silent, because # bitbake's :remove drops nothing when the token does not match and cmake would # then take whichever setting it read last. # An unmatched glob comes back unexpanded, which is the "no .so here" answer and # costs no process at all. bst_lib_shape ldb/no-shared-lib "$R/usr/lib/libleveldb.so*" # --- the header states the version ------------------------------------------- # # Three separate claims agree on 1.23 at SRCREV 068d5ee1: the recipe filename, # the source's CMake project() version, and these two constants. This is where a # header/archive mismatch becomes visible rather than inferred; the probe prints # the same pair back at runtime, from the archive's side of the boundary. DBH=$nl$(cat "$INCDIR/db.h" 2>/dev/null)$nl has ldb/ver-major "$DBH" "static const int kMajorVersion = 1;" has ldb/ver-minor "$DBH" "static const int kMinorVersion = 23;" # --- the declared API -------------------------------------------------------- sub ldb/api-open "$DBH" "static Status Open(const Options& options, const std::string& name," sub ldb/api-put "$DBH" "virtual Status Put(const WriteOptions& options, const Slice& key," sub ldb/api-delete "$DBH" "virtual Status Delete(const WriteOptions& options, const Slice& key) = 0;" sub ldb/api-write "$DBH" "virtual Status Write(const WriteOptions& options, WriteBatch* updates) = 0;" sub ldb/api-get "$DBH" "virtual Status Get(const ReadOptions& options, const Slice& key," sub ldb/api-iterator "$DBH" "virtual Iterator* NewIterator(const ReadOptions& options) = 0;" sub ldb/api-snapshot "$DBH" "virtual const Snapshot* GetSnapshot() = 0;" sub ldb/api-compact "$DBH" "virtual void CompactRange(const Slice* begin, const Slice* end) = 0;" sub ldb/api-destroy "$DBH" "LEVELDB_EXPORT Status DestroyDB(const std::string& name," sub ldb/api-repair "$DBH" "LEVELDB_EXPORT Status RepairDB(const std::string& dbname," WBH=$nl$(cat "$INCDIR/write_batch.h" 2>/dev/null)$nl sub ldb/api-batch-put "$WBH" "void Put(const Slice& key, const Slice& value);" sub ldb/api-batch-delete "$WBH" "void Delete(const Slice& key);" sub ldb/api-batch-handler "$WBH" "class LEVELDB_EXPORT Handler {" OPH=$nl$(cat "$INCDIR/options.h" 2>/dev/null)$nl sub ldb/api-create-if-missing "$OPH" "bool create_if_missing = false;" sub ldb/api-error-if-exists "$OPH" "bool error_if_exists = false;" sub ldb/api-write-sync "$OPH" "bool sync = false;" sub ldb/api-snappy-enum "$OPH" "kSnappyCompression = 0x1" # --- the archive is this arm's, and it carries the C++ arm's own symbols ----- # # Read through the ar SYMBOL INDEX -- member "/", the map of exported symbol to # member offset -- rather than over the whole body. Its size is read from the # header rather than guessed at, because leveldb's index is ~33 kB where snappy's # was ~5 kB and a fixed window sized for one is wrong for the other. check_out ldb/ar-magic "!" head -c 8 "$ARCHIVE" ARHDR=$(head -c 68 "$ARCHIVE" 2>/dev/null | tail -c 60) IDXSZ=$(printf '%s' "$ARHDR" | cut -c49-58 | tr -d ' ') if [ -n "$IDXSZ" ] && [ "$IDXSZ" -gt 0 ] 2>/dev/null; then bst_ok ldb/ar-index-sized idx=$tmp/bst-leveldb-idx.$$ head -c $((68 + IDXSZ)) "$ARCHIVE" | tr -c '\40-\176\n' ' ' > "$idx" 2>/dev/null IDX=$nl$(cat "$idx" 2>/dev/null)$nl rm -f "$idx" # THE VTABLES. leveldb is virtual-dispatch end to end -- DB, Iterator, Env, # Comparator and the three file classes are all abstract -- so these symbols # must be exported, and a C compile of these sources could not emit them. sub ldb/sym-vtable-comparator "$IDX" "_ZTVN7leveldb10ComparatorE" sub ldb/sym-vtable-writablefile "$IDX" "_ZTVN7leveldb12WritableFileE" sub ldb/sym-vtable-randomaccess "$IDX" "_ZTVN7leveldb16RandomAccessFileE" sub ldb/sym-vtable-internalcmp "$IDX" "_ZTVN7leveldb21InternalKeyComparatorE" # ...and NO typeinfo objects, because the recipe builds -fno-rtti. Asserted # deliberately rather than left unstated: _ZTV without _ZTI is exactly what # -fno-rtti produces, and a reader finding no _ZTI should be able to see that # it was expected rather than wonder whether the scan simply missed them. hasnt ldb/sym-no-typeinfo "$IDX" "_ZTIN7leveldb" sub ldb/sym-db-open "$IDX" "_ZN7leveldb2DB4OpenE" sub ldb/sym-batch-put "$IDX" "_ZN7leveldb10WriteBatch3PutERKNS_5SliceES3_" sub ldb/sym-batch-delete "$IDX" "_ZN7leveldb10WriteBatch6DeleteERKNS_5SliceE" sub ldb/sym-destroydb "$IDX" "_ZN7leveldb9DestroyDBE" # The C entry points are a separate symbol set in the same archive; a port can # link the C++ ones and still be missing these. sub ldb/sym-c-open "$IDX" "leveldb_open" sub ldb/sym-c-put "$IDX" "leveldb_put" sub ldb/sym-c-iter-seek "$IDX" "leveldb_iter_seek" sub ldb/sym-c-write "$IDX" "leveldb_write" # libc++, not some other standard library: the bbappend's whole C++ opt-in is # pointing CXX at wasmc++-glibc so brintos-libcxx resolves. _ZNSt3__1 is # libc++'s inline namespace and nothing else emits it. sub ldb/sym-libcxx "$IDX" "_ZNSt3__1" # NO std::thread, std::mutex or std::condition_variable ANYWHERE in the index. # # The regression these catch is specific and silent: the port's three patches # are guarded on _LIBCPP_HAS_THREADS, so if brintos-libcxx is ever rebuilt with # LIBCXX_ENABLE_THREADS=ON they stop applying THEMSELVES -- the #if takes the # stock branch, leveldb compiles and links against std::thread, and the archive # ships depending on a background compactor that this target still cannot run. # Nothing else here would notice: the headers, the symbols we assert and the # probe's output would all be unchanged, and the failure would appear as a # database that hangs the first time it compacts. # # They are cheap and they cannot fail while the build is configured as it is # today -- that is the point of writing them down rather than an argument that # they are load-bearing now. hasnt ldb/sym-no-std-thread "$IDX" "_ZNSt3__16thread" hasnt ldb/sym-no-std-mutex "$IDX" "_ZNSt3__15mutex" hasnt ldb/sym-no-std-condvar "$IDX" "_ZNSt3__118condition_variable" # The members are LLVM IR BITCODE, not native objects: this toolchain defers # codegen to link time, so every member opens with BC C0 DE (42 43 c0 de) -- # not the ELF magic a native x86-64 leveldb would carry, and not the \0asm of a # finished wasm module either. This is the ONE assertion in the file that a # native build fails, which makes it the arm witness the headers cannot # provide: the probe's output is deliberately identical on both arms, so the # probe cannot be the thing that says which arm ran. # # WALKED, NOT ASSUMED. After the symbol index "/" comes an OPTIONAL "//" member # holding the names too long for the 16-byte inline field -- leveldb has three # such (two_level_iterator.cc.o, table_builder.cc.o, filter_policy.cc.o) -- and # only then the first object. Skipping it unconditionally lands inside the name # table and reads its text as the member magic. OFF=$((68 + IDXSZ)) [ $((OFF % 2)) -eq 1 ] && OFF=$((OFF + 1)) NAME2=$(od -An -c -j "$OFF" -N 2 "$ARCHIVE" 2>/dev/null | tr -d ' \n') if [ "$NAME2" = "//" ]; then bst_ok ldb/ar-longname-member LNSZ=$(dd if="$ARCHIVE" bs=1 skip=$((OFF + 48)) count=10 2>/dev/null | tr -d ' ') OFF=$((OFF + 60 + LNSZ)) [ $((OFF % 2)) -eq 1 ] && OFF=$((OFF + 1)) else bst_notok ldb/ar-longname-member "expected a // long-name member, read '$NAME2'" fi # Assert the name field before the magic, so a wrong magic below cannot be # blamed on arithmetic that landed somewhere other than a member header. # builder.cc.o is first because the archive follows the CMakeLists source # order, not an alphabetical one it happens to agree with here. MNAME=$(od -An -c -j "$OFF" -N 13 "$ARCHIVE" 2>/dev/null | tr -d ' \n') check_eq ldb/ar-first-member "builder.cc.o/" "$MNAME" MAGIC=$(od -An -tx1 -j $((OFF + 60)) -N 4 "$ARCHIVE" 2>/dev/null | tr -d ' \n') check_eq ldb/ar-member-bitcode 4243c0de "$MAGIC" if [ "$MAGIC" = "7f454c46" ]; then bst_notok ldb/ar-not-elf "members are ELF -- this is a NATIVE build, not wasm32" else bst_ok ldb/ar-not-elf; fi if [ "$MAGIC" = "0061736d" ]; then bst_notok ldb/ar-not-wasm-module "members are finished wasm modules, not bitcode" else bst_ok ldb/ar-not-wasm-module; fi else bst_notok ldb/ar-index-sized "could not read the ar symbol-index size" fi # --- leveldbutil, the one program the port installs -------------------------- # # It refuses with its usage on stderr and exit 1, both with no command and with # an unknown one. Small, but it is the only evidence that this arm can LINK and # RUN something against the archive through cmake's own link line rather than # through the recipe's hand-written one. if [ -x "$UTIL" ]; then bst_ok ldb/util-exec check_rc ldb/util-no-args-rc 1 "$UTIL" check_rc ldb/util-bad-command-rc 1 "$UTIL" frobnicate # NOT check_out: that helper requires the command to EXIT 0 as well as to # print the needle, and refusing with a usage message is exit 1 by definition. # Captured and matched separately so the rc and the text are two answers. uout=$("$UTIL" 2>&1) sub ldb/util-usage "$nl$uout$nl" "Usage: leveldbutil command..." sub ldb/util-usage-dump "$nl$uout$nl" "dump files..." # A file that is not there is an error, not a crash and not a silent success. check_rc ldb/util-dump-missing-rc 1 "$UTIL" dump "$tmp/bst-leveldb-nosuch.$$" else bst_notok ldb/util-exec "$UTIL is not executable" fi # --- the probe: does the shipped archive RUN A DATABASE? --------------------- if [ ! -f "$PROBE" ]; then bst_notok ldb/probe-present "$PROBE missing -- the library cannot be proven to run" else bst_ok ldb/probe-present # do_install stages data/* at 0644 and the deployed rootfs is read-only at # browser runtime, so the payload cannot be executed where it lies. Stage an # executable copy under TMPDIR, as every probe-driven suite here does. pd=$tmp/bst-leveldb.$$ p=$pd/leveldb-probe mkdir -p "$pd" 2>/dev/null if cp "$PROBE" "$p" 2>/dev/null && chmod 0755 "$p" 2>/dev/null && [ -x "$p" ]; then bst_ok ldb/probe-exec # The database goes under TMPDIR, which is a mode=1777 tmpfs. The virtiofs # root is read-only at runtime, so it is not a candidate -- and that is worth # saying plainly rather than leaving the reader to infer that the ticket's # "exercise of the virtio-fs root" was skipped: what this exercises is the # SYSCALL surface leveldb needs (open/pwrite/fsync/rename/link/unlink/mmap # and directory sync), which is the part that has had gaps. out=$("$p" "$pd" 2>"$pd/run.err"); rc=$? errout=$(cat "$pd/run.err" 2>/dev/null) rm -rf "$pd" if [ "$rc" -eq 0 ]; then bst_ok ldb/probe-rc else bst_notok ldb/probe-rc "probe exited $rc"; fi # The library writes nothing to stderr on a successful run. Its failure # channels under -fno-exceptions are assert() -> __assert_fail and the two # deliberate std::abort()s the port's own patches add (a wait that nothing # can satisfy, and Env::StartThread) -- all of which print here. A port whose # single-threaded scheduler wedged would show up on this line even if the # output it had already produced still matched. if [ -z "$errout" ]; then bst_ok ldb/probe-silent-stderr else bst_notok ldb/probe-silent-stderr "$(_bst_1line "$errout")"; fi O=$nl$out$nl # The whole-stream gate. Kept separate from the anchored cases below, because # a probe that died partway would leave those it never reached with nothing # to disagree with and could look like a shorter clean run. case "$O" in *"${nl}leveldb-probe-end${nl}") bst_ok ldb/probe-complete ;; *) bst_notok ldb/probe-complete "probe did not reach its final line" ;; esac # The version the probe was COMPILED against (the recipe's PV) beside the one # the ARCHIVE carries (leveldb::kMajorVersion / kMinorVersion, read at # runtime from the library's side of the boundary). has ldb/probe-version "$O" "leveldb-probe 1.23 major=1 minor=23" # === 1. the database opens, and refuses what it should =================== has ldb/open-create "$O" "open/create|OK" has ldb/open-missing "$O" "open/missing-refused|OK" # === 2. the three write paths =========================================== # # Individual Put, one WriteBatch, and a Put with WriteOptions.sync set. The # third is the one the ticket names: it is the write that forces fsync, and # therefore the write whose survival across the reopen below says the # durability path works rather than that the memtable does. has ldb/put-count "$O" "put/count|500" has ldb/batch-write "$O" "batch/write|OK" has ldb/put-sync "$O" "put/sync|OK" # === 3. iteration, in both directions, and Seek ========================= # # THE DIGEST IS THE ASSERTION, and the count is beside it because a probe # that returned nothing would satisfy any digest-absence check perfectly -- # that is the vacuous-green shape this round's siblings kept finding. # 67a7a707e5407a88 is FNV-1a/64 over the 601 "key=value\n" pairs the corpus # rule at the top of this file defines, computed without leveldb. has ldb/iter-count "$O" "iter/count|601" has ldb/iter-ordered "$O" "iter/ordered|OK" has ldb/iter-hash "$O" "iter/hash|67a7a707e5407a88" has ldb/riter-count "$O" "riter/count|601" has ldb/riter-ordered "$O" "riter/ordered|OK" has ldb/seek-exact "$O" "seek/exact|OK" has ldb/seek-successor "$O" "seek/successor|OK" # === 4. a snapshot does not see later writes ============================ has ldb/snapshot-hidden "$O" "snapshot/hidden|OK" has ldb/snapshot-live-sees "$O" "snapshot/live-sees|OK" # === 5. the delete, and the surviving set, four times =================== # # `live` is the memtable; `reopened` is the log-replay path the ticket asks # for; `compacted` is after every table on disk has been rewritten; and # `recompacted` is a second close/open over those rewritten tables. The same # four numbers and the same digest at all four stages is the claim. # 224f371943091ddc is the same rule with key/000100..key/000199 removed. has ldb/del-write "$O" "del/write|OK" for stage in live reopened compacted recompacted; do has "ldb/$stage-count" "$O" "$stage/count|501" has "ldb/$stage-ordered" "$O" "$stage/ordered|OK" has "ldb/$stage-iter-status" "$O" "$stage/iter-status|OK" has "ldb/$stage-hash" "$O" "$stage/hash|224f371943091ddc" has "ldb/$stage-main-present" "$O" "$stage/main-present|400" has "ldb/$stage-main-absent" "$O" "$stage/main-absent|100" has "ldb/$stage-main-wrong-value" "$O" "$stage/main-wrong-value|0" has "ldb/$stage-main-error" "$O" "$stage/main-error|0" # NotFound is a different answer from an error, and a count of successful # Gets cannot tell them apart -- a port that returned IOError for a missing # key would read as "absent" to the line above and fail this one. has "ldb/$stage-deleted-notfound" "$O" "$stage/deleted-notfound|100" has "ldb/$stage-sync-witness" "$O" "$stage/sync-witness|OK" done # The transitions themselves, asserted separately from their results: a # reopen that failed and a reopen that succeeded but lost data are different # defects and want different first questions. has ldb/close-first "$O" "close/first|OK" has ldb/reopen-first "$O" "reopen/first|OK" has ldb/compact-done "$O" "compact/done|OK" has ldb/close-second "$O" "close/second|OK" has ldb/reopen-second "$O" "reopen/second|OK" # No stage may report a disordered scan, a bad iterator status, a wrong value # or a lost sync witness. Stated as absences as well as by the per-stage # cases above, so a stage the probe never reached cannot pass by omission -- # these fire on the token wherever it appears. hasnt ldb/no-disordered "$O" "DISORDERED" hasnt ldb/no-bad-iter "$O" "iter-status|BAD" hasnt ldb/no-lost-witness "$O" "sync-witness|LOST" hasnt ldb/no-open-fail "$O" "|FAIL" # === 6. teardown ========================================================= # # DestroyDB removes the tree, and a subsequent open with create_if_missing # false must then be refused -- which is also the last word on whether the # database was ever really on the filesystem rather than in memory. has ldb/property-stats "$O" "property/stats|OK" has ldb/destroy-status "$O" "destroy/status|OK" has ldb/destroy-refused "$O" "destroy/reopen-refused|OK" else bst_notok ldb/probe-exec "could not stage an executable copy of the probe under $tmp" fi fi