. "$BST_LIB" # libee 0.4.1 is Adiscon's "event expression library inspired by CEE" -- an # in-memory event object (tags + named multi-valued fields) with four encoders # and three decoders over it. It is the DIRECT CONSUMER of libestr # (81-libestr.test), which it pulls in through PKG_CHECK_MODULES and builds every # one of its output strings with, so this suite is also the first thing on the # image that exercises libestr's relocating es_str_t through a second library. # # Unlike its sibling it is NOT a -dev-only port: src/Makefile.am declares # sbin_PROGRAMS = libee-convert, so an actual binary ships at /usr/sbin and the # cli-* group drives it end to end -- stdin to each of the four encoders. The # library itself is measured by a probe, for the reason below. # # --------------------------------------------------------------------------- # WHY THIS SUITE RUNS A PROBE INSTEAD OF STOPPING AT PAYLOAD SHAPE # # BRINTOS-217 is why: oniguruma COMPILED and PACKAGED perfectly on this arm and # then returned silently wrong offsets to every caller, so a payload-only suite # would have gone green on the exact defect the port existed to find. # # libee's exposure has three parts, and data/ee-probe (source: src/ee_probe.c, # cross-linked with wasmcc-glibc against the very archive that ships) is aimed at # all three rather than at generic API surface: # # dec-* THE HIGHEST-VALUE GROUP. ee_intDec is a decoder driven entirely # by two CALLER FUNCTION POINTERS -- it calls back into user code # once per input line and once per completed event. Indirect calls # across a coroutinized boundary are exactly what the pass's # indirect_marked census is about, and nothing else on this image # drives one this densely. ee_setDebugCB adds a third. # parse-* Every primitivetype parser advances an es_size_t *offs IN PLACE # across a buffer -- the loop-carried-cursor shape hwjs-cc#24 broke # for oniguruma. The RESULTING OFFSET is asserted, not just the # token, because an off-by-one there is invisible in the value. # enc-* scale-* Every encoder appends into an es_str_t whose es_extendBuf # realloc MOVES the whole object. The exact bytes are asserted, # and the same path is driven 200 fields and 4000 bytes deep. # # EVERY EXPECTED VALUE IS THE VALUE A NATIVE x86_64 BUILD OF THE IDENTICAL # RELEASE PRODUCES. The probe prints its value on both the OK and the FAIL path, # so a target run can be diffed against the native control line by line rather # than compared by tally -- and that diff is asserted ON TARGET with cmp(1) # (native-byte-identical), because the console dump truncates long lines and an # off-target diff of this output is misleading. # # The probe's layout lines are prefixed EE@ instead of EE- because they are # POINTER-WIDTH DEPENDENT and must legitimately differ from the control. They are # excluded from the byte-diff and asserted separately, by the layout-* group. # # SEVEN UPSTREAM DEFECTS ARE PINNED AS MEASURED VALUES and are marked -defect. # All seven reproduce identically on the native x86_64 control, so none is this # arm's and none is worked around -- pinning what the code really does means a # platform-side change shows up as a CHANGED VALUE instead of hiding behind a # relaxed check, and it is why this port carries no BRINTOS-WORKAROUND: marker. # Two of the seven are memory-safety bugs and are measured WITHOUT being # executed, because executing them would end the run: # ee_deleteField's nVals>1 arm frees the same list node forever (the cursor is # never advanced) -- an infinite loop over freed memory, so any field with two # or more values, and any event holding one, is undeletable. # ee_getFieldValueAsStr implements only n==0; the n>=1 arm is assert(0), and the # shipped build compiles asserts out (NDEBUG), so it returns an UNINITIALISED # pointer rather than failing. # The other five: the three event formatters never assign r=0 on success, so # their return code is whatever the last CHKR() wrote and therefore depends on # the event's CONTENT; field NAMES are unescaped in JSON and XML output while # values are escaped; CSV escapes an embedded quote as \" instead of doubling it # per RFC 4180; ee_setFlags is documented as overriding and implemented as |=, so # no public API can clear a flag; and ee_parseNumber reports success on a leading # digit run, leaving offs mid-token for "12ab". # # The suite uses builtin `case` matching over variables read ONCE, never # `check_out ... cat` per case: on this runtime the forking is the cost rather # than the work, and the cjson suite next door was killed by the runner's # per-suite bound at 156 cases written the other way. This suite forks for the # two archive greps, the four CLI runs, the probe, and the on-target byte-diff. # --------------------------------------------------------------------------- incdir=/usr/include/libee lib=/usr/lib/libee.a pc=/usr/lib/pkgconfig/libee.pc cli=/usr/sbin/libee-convert nl=' ' # hasfile -- ok iff is a non-empty regular file. # has -- ok iff contains the literal . # hasline -- ok iff carries as a WHOLE line. hasfile() { if [ -s "$2" ]; then bst_ok "$1"; else bst_notok "$1" "absent or empty: $2"; fi; } has() { case "$3" in *"$2"*) bst_ok "$1" ;; *) bst_notok "$1" "not found: $2" ;; esac; } hasline() { case "$nl$3$nl" in *"$nl$2$nl"*) bst_ok "$1" ;; *) bst_notok "$1" "not in archive: $2" ;; esac; } # An image built without libee is a coverage gap, not a failure: skip once and # stop rather than reporting a hundred reds for an absent package. if [ ! -s "$lib" ]; then bst_skip libee/present "libee.a not installed on this image" exit 0 fi bst_ok libee/present # --- the shipped payload ----------------------------------------------------- # Unlike libestr, upstream installs its headers into a PER-PACKAGE subdirectory # (include/libee/Makefile.am), so the directory itself is part of the contract: # every consumer includes . hasfile libee/archive "$lib" hasfile libee/pkgconfig "$pc" if [ -d "$incdir" ]; then bst_ok libee/incdir; else bst_notok libee/incdir "absent: $incdir"; fi for h in libee.h ctx.h event.h field.h value.h tagbucket.h fieldbucket.h \ primitivetype.h int.h obj.h internal.h timestamp.h; do hasfile "libee/hdr-$h" "$incdir/$h" done # The binary half of the port. libee is the rare -dev package on this image that # also ships an executable, so an absent one is a packaging failure rather than a # coverage gap -- hence not ok, not skip. hasfile libee/cli-present "$cli" pctext=$(cat "$pc" 2>/dev/null) has libee/pc-name "Name: libee" "$pctext" has libee/pc-version "Version: 0.4.1" "$pctext" has libee/pc-libs "-lee" "$pctext" has libee/pc-libdir "libdir=/usr/lib" "$pctext" has libee/pc-includedir "includedir=/usr/include" "$pctext" # --- what the installed headers must carry ----------------------------------- # The object system is the ABI: every libee object begins with `unsigned objID`, # a magic cookie the library checks, and the derived struct embeds ee_obj (or # repeats objID) as its FIRST member. A header that moved objID off the front # would invalidate every layout case below. evtext=$(cat "$incdir/event.h" 2>/dev/null) has libee/hdr-event-objid-first "struct ee_event {${nl} unsigned objID;" "$evtext" has libee/hdr-event-tags "struct ee_tagbucket *tags" "$evtext" has libee/hdr-event-fields "struct ee_fieldbucket *fields" "$evtext" has libee/hdr-fmt-json "ee_fmtEventToJSON" "$evtext" has libee/hdr-fmt-xml "ee_fmtEventToXML" "$evtext" has libee/hdr-fmt-csv "ee_fmtEventToCSV" "$evtext" has libee/hdr-fmt-syslog "ee_fmtEventToRFC5424" "$evtext" fdtext=$(cat "$incdir/field.h" 2>/dev/null) has libee/hdr-field-nvals "unsigned char nVals" "$fdtext" has libee/hdr-field-valroot "struct ee_valnode *valroot" "$fdtext" cttext=$(cat "$incdir/ctx.h" 2>/dev/null) has libee/hdr-ctx-ultracompact "EE_CTX_FLAG_ENC_ULTRACOMPACT" "$cttext" has libee/hdr-ctx-flattags "EE_CTX_FLAG_INCLUDE_FLAT_TAGS" "$cttext" has libee/hdr-ctx-version "char *ee_version(void)" "$cttext" # --- symbols actually present in the archive --------------------------------- # One grep over the archive, then whole-line matching against it. These are the # entry points the probe drives; a partial compile or a visibility mistake fails # here rather than as a link error inside the probe. arcsyms=$(grep -ao 'ee_[A-Za-z0-9_]*' "$lib" 2>/dev/null | sort -u) for f in ee_initCtx ee_exitCtx ee_version ee_setDebugCB ee_dbgprintf \ ee_newEvent ee_deleteEvent ee_newEventFromJSON ee_addStrFieldToEvent \ ee_getEventField ee_getEventFieldAsString ee_addTagToEvent \ ee_EventHasTag ee_assignTagbucketToEvent \ ee_newField ee_deleteField ee_nameField ee_addStrValueToField \ ee_getNumFieldVals ee_getFieldValueAsStr ee_getFieldAsString \ ee_newValue ee_deleteValue ee_setStrValue \ ee_newTagbucket ee_deleteTagbucket ee_addRefTagbucket \ ee_addTagToBucket ee_TagbucketHasTag ee_TagbucketGetNextTag \ ee_newFieldbucket ee_addFieldToBucket ee_getBucketField \ ee_fmtEventToRFC5424 ee_fmtEventToJSON ee_fmtEventToXML ee_fmtEventToCSV \ ee_intDec ee_jsonDec ee_apacheDec \ ee_parseNumber ee_parseWord ee_parseIPv4 ee_parseQuotedString \ ee_parseRFC5424Date ee_parseRFC3164Date ee_parseTime24hr; do hasline "libee/sym-$f" "$f" "$arcsyms" done # libee VENDORS its own copy of cJSON (src/cjson/cjson.c) rather than linking the # cJSON package this image also ships (81-cjson.test). Asserting the vendored # symbol is in the archive records that the two are independent: a fix to one is # not a fix to the other, and a future de-vendoring would fail this case. # Its own grep, because $arcsyms above matches ee_* only and cJSON_Parse can # never appear in it -- the case is worthless unless it reads the right list. cjsyms=$(grep -ao 'cJSON_[A-Za-z0-9_]*' "$lib" 2>/dev/null | sort -u) hasline libee/sym-vendored-cjson cJSON_Parse "$cjsyms" # --- libee-convert, end to end ----------------------------------------------- # The internal ("int") format is line-typed: e: opens an event, f: starts a # field, v: gives it a value. One input drives all four encoders, so a # decoder fault and an encoder fault land in different cases. if [ -x "$cli" ]; then in=${TMPDIR:-/tmp}/bst-ee-in.$$ printf 'e:\nf:host\nv:server1\nf:msg\nv:hello world\n' > "$in" 2>/dev/null o=$("$cli" -e json -i "$in" 2>&1) check_eq libee/cli-json '{"host": "server1", "msg": "hello world"}' "$o" o=$("$cli" -e syslog -i "$in" 2>&1) check_eq libee/cli-syslog '[cee@115 host="server1" msg="hello world"]' "$o" o=$("$cli" -e csv -E 'host,msg' -i "$in" 2>&1) check_eq libee/cli-csv '"server1","hello world"' "$o" o=$("$cli" -e xml -i "$in" 2>&1) check_eq libee/cli-xml \ 'server1hello world' "$o" # stdin rather than -i, and the documented failure path: a line whose type # character is not one of # e f v is EE_INVLDFMT (-4) and exits 1. o=$(printf 'e:\nf:a\nv:1\n' | "$cli" -e json 2>&1) check_eq libee/cli-stdin '{"a": "1"}' "$o" o=$(printf 'bogus\n' | "$cli" -e json 2>&1); rc=$? has libee/cli-badinput "error -4 in decoding stage" "$o" check_eq libee/cli-badinput-rc 1 "$rc" rm -f "$in" else bst_notok libee/cli-exec "not executable: $cli" fi # --- the library RUNNING, via the cross-linked probe ------------------------- probe=${TMPDIR:-/tmp}/bst-ee-probe.$$ if cp "$BST_DATA/ee-probe" "$probe" 2>/dev/null && chmod +x "$probe" 2>/dev/null; then # Run ONCE and keep the exit status; every case below matches over $out rather # than re-running, because forking is the cost on this runtime. out=$("$probe" 2>&1); prc=$? has libee/run-banner "ee-probe libee 0.4.1" "$out" # context, and the flags defect has libee/run-ctx-init "EE-ctx-init OK" "$out" has libee/run-ctx-objid "EE-ctx-objid OK" "$out" has libee/run-ctx-flags-set "EE-ctx-flags-set OK" "$out" has libee/run-ctx-flags-clear-defect "EE-ctx-flags-clear-defect OK" "$out" has libee/run-ctx-flags-accum-defect "EE-ctx-flags-accumulate-defect OK" "$out" has libee/run-ctx-reinit "EE-ctx-reinit-flags OK" "$out" has libee/run-ctx-exit "EE-ctx-exit-rc OK" "$out" # the debug callback -- the first of the three indirect-call sites has libee/run-dbgcb-called "EE-ctx-dbgcb-called OK" "$out" has libee/run-dbgcb-cookie "EE-ctx-dbgcb-cookie OK" "$out" has libee/run-dbgcb-msg "EE-ctx-dbgcb-msg OK" "$out" # value + field, including the two memory-safety defects (measured, not run) has libee/run-value-syslog "EE-value-syslog OK" "$out" has libee/run-value-xml "EE-value-xml OK" "$out" has libee/run-field-name "EE-field-name OK" "$out" has libee/run-field-rename-refused "EE-field-rename-refused OK" "$out" has libee/run-field-val0 "EE-field-val0 OK" "$out" has libee/run-field-nvals-three "EE-field-nvals-three OK" "$out" has libee/run-field-accessor-defect "EE-field-multival-accessor-defect OK" "$out" has libee/run-field-destructor-defect "EE-field-multival-destructor-defect OK" "$out" has libee/run-field-delete-single "EE-field-delete-single OK" "$out" has libee/run-field-syslog "EE-field-syslog OK" "$out" has libee/run-field-xml "EE-field-xml OK" "$out" # event has libee/run-event-getfield "EE-event-getfield OK" "$out" has libee/run-event-getfield-miss "EE-event-getfield-missing OK" "$out" has libee/run-event-getstr "EE-event-getstr OK" "$out" has libee/run-event-getstr-appends "EE-event-getstr-appends OK" "$out" has libee/run-event-getstr-miss "EE-event-getstr-missing-untouched OK" "$out" # tags, incl. the cookie iterator (a loop-carried opaque pointer) has libee/run-tag-refcnt "EE-tag-bucket-refcnt OK" "$out" has libee/run-tag-has "EE-tag-has OK" "$out" has libee/run-tag-has-not "EE-tag-has-not OK" "$out" has libee/run-tag-iter-count "EE-tag-iter-count OK" "$out" has libee/run-tag-iter-order "EE-tag-iter-order OK" "$out" has libee/run-tag-addref "EE-tag-addref-refcnt OK" "$out" has libee/run-tag-unref "EE-tag-unref-refcnt OK" "$out" has libee/run-event-hastag "EE-event-hastag OK" "$out" # encoders: the exact bytes, and both arms of the return-code defect has libee/run-enc-syslog "EE-enc-syslog OK" "$out" has libee/run-enc-json "EE-enc-json OK" "$out" has libee/run-enc-xml "EE-enc-xml OK" "$out" has libee/run-enc-csv "EE-enc-csv OK" "$out" has libee/run-enc-syslog-rc-tags-defect "EE-enc-syslog-rc-withtags-defect OK" "$out" has libee/run-enc-xml-rc-tags-defect "EE-enc-xml-rc-withtags-defect OK" "$out" has libee/run-enc-json-rc-2f-defect "EE-enc-json-rc-2fields-defect OK" "$out" has libee/run-enc-syslog-rc-not-defect "EE-enc-syslog-rc-notags-defect OK" "$out" has libee/run-enc-xml-rc-not-defect "EE-enc-xml-rc-notags-defect OK" "$out" has libee/run-enc-json-rc-1f-defect "EE-enc-json-rc-1field-defect OK" "$out" has libee/run-enc-csv-rc-explicit "EE-enc-csv-rc-explicit OK" "$out" has libee/run-enc-json-escape "EE-enc-json-escape OK" "$out" has libee/run-enc-xml-escape "EE-enc-xml-escape OK" "$out" has libee/run-enc-syslog-escape "EE-enc-syslog-escape OK" "$out" has libee/run-enc-json-name-defect "EE-enc-json-name-unescaped-defect OK" "$out" has libee/run-enc-xml-name-defect "EE-enc-xml-name-unescaped-defect OK" "$out" has libee/run-enc-csv-quote-defect "EE-enc-csv-quote-escaping-defect OK" "$out" has libee/run-enc-csv-missing-defect "EE-enc-csv-missing-defect OK" "$out" has libee/run-enc-csv-sep-ok "EE-enc-csv-embedded-sep-ok OK" "$out" has libee/run-enc-json-empty "EE-enc-json-empty OK" "$out" has libee/run-enc-xml-empty "EE-enc-xml-empty OK" "$out" has libee/run-enc-syslog-empty "EE-enc-syslog-empty OK" "$out" # THE DECODER -- indirect calls back into probe code, once per line and once # per completed event. If the coroutinize pass mishandled a caller function # pointer, this is where it shows. has libee/run-dec-rc "EE-dec-int-rc OK" "$out" has libee/run-dec-events "EE-dec-int-events OK" "$out" has libee/run-dec-json "EE-dec-int-json OK" "$out" has libee/run-dec-2-rc "EE-dec-int2-rc OK" "$out" has libee/run-dec-2-events "EE-dec-int2-events OK" "$out" has libee/run-dec-2-json0 "EE-dec-int2-json0 OK" "$out" has libee/run-dec-2-json1 "EE-dec-int2-json1 OK" "$out" has libee/run-dec-comment "EE-dec-comment-rc OK" "$out" has libee/run-dec-comment-ev "EE-dec-comment-events OK" "$out" has libee/run-dec-bad-rc "EE-dec-bad-rc OK" "$out" has libee/run-dec-bad-events "EE-dec-bad-events OK" "$out" has libee/run-dec-orphan "EE-dec-orphan-rc OK" "$out" # the vendored cJSON, in and back out has libee/run-json-new "EE-json-new OK" "$out" has libee/run-json-field "EE-json-field OK" "$out" has libee/run-json-roundtrip "EE-json-roundtrip OK" "$out" has libee/run-json-bad "EE-json-bad OK" "$out" # the cursor parsers: the OFFSET is asserted alongside the token has libee/run-parse-num-offs "EE-parse-num-offs OK" "$out" has libee/run-parse-num-val "EE-parse-num-val OK" "$out" has libee/run-parse-num-defect "EE-parse-num-trailing-defect OK" "$out" has libee/run-parse-num-nondigit "EE-parse-num-nondigit-rc OK" "$out" has libee/run-parse-word-offs "EE-parse-word-offs OK" "$out" has libee/run-parse-word-val "EE-parse-word-val OK" "$out" has libee/run-parse-word-resume "EE-parse-word-resume-val OK" "$out" has libee/run-parse-word-resoffs "EE-parse-word-resume-offs OK" "$out" has libee/run-parse-ipv4-offs "EE-parse-ipv4-offs OK" "$out" has libee/run-parse-ipv4-val "EE-parse-ipv4-val OK" "$out" has libee/run-parse-ipv4-bad "EE-parse-ipv4-bad OK" "$out" has libee/run-parse-quoted-offs "EE-parse-quoted-offs OK" "$out" has libee/run-parse-rfc5424-offs "EE-parse-rfc5424-offs OK" "$out" has libee/run-parse-rfc3164-offs "EE-parse-rfc3164-offs OK" "$out" has libee/run-parse-time24-offs "EE-parse-time24-offs OK" "$out" # scale: 200 fields and a 4000-byte value through the relocating append path has libee/run-scale-lookup-first "EE-scale-lookup-first OK" "$out" has libee/run-scale-lookup-last "EE-scale-lookup-last OK" "$out" has libee/run-scale-json-head "EE-scale-json-head OK" "$out" has libee/run-scale-json-tail "EE-scale-json-tail OK" "$out" has libee/run-scale-json-len "EE-scale-json-len OK" "$out" has libee/run-scale-xml-len "EE-scale-xml-len OK" "$out" has libee/run-scale-big-len "EE-scale-big-len OK" "$out" has libee/run-scale-churn "EE-scale-churn OK" "$out" # --- layout, the lines that must differ from the control ------------------- # These are the EE@ lines: pointer-width dependent, excluded from the byte-diff # below, and asserted here against THIS arm's values. sizeof-ptr 4 is the whole # premise; the objID offsets must be 0 on any arm because the magic cookie the # library checks is the first member of every object; the rest are the wasm32 # ABI's packing of structs whose members are all 4-byte pointers. lay() { case "$nl$out$nl" in *"$nl$2$nl"*) bst_ok "$1" ;; *) bst_notok "$1" "want '$2', got '$(printf '%s\n' "$out" | grep "^${2% *} " | head -n 1)'" ;; esac; } lay libee/layout-ptr "EE@sizeof-ptr 4" lay libee/layout-event "EE@sizeof-event 16" lay libee/layout-field "EE@sizeof-field 28" lay libee/layout-value "EE@sizeof-value 16" lay libee/layout-tagbucket "EE@sizeof-tagbucket 20" lay libee/layout-ctx "EE@sizeof-ctx 28" lay libee/layout-timestamp "EE@sizeof-timestamp 8" lay libee/layout-ev-objid "EE@off-event-objid 0" lay libee/layout-fd-objid "EE@off-field-objid 0" lay libee/layout-ev-tags "EE@off-event-tags 8" lay libee/layout-ev-fields "EE@off-event-fields 12" lay libee/layout-fd-name "EE@off-field-name 8" lay libee/layout-fd-nvals "EE@off-field-nvals 12" lay libee/layout-fd-val "EE@off-field-val 16" # BYTE-IDENTITY AGAINST THE NATIVE CONTROL, asserted here ON TARGET rather than # by eyeballing two logs off-target. data/ee-native.txt is the output of this # same probe source built natively against a host autotools build of the # identical release, with the EE@ layout lines removed. Comparing it inside the # guest keeps any terminal capture out of the comparison -- the console dump # truncates long lines, which is exactly how an off-target diff of this output # misleads -- and turns "matches native" from a one-off manual check into a # permanent regression guard: a platform change in libc number conversion, # ctype, or the allocator's growth policy shows up as a CHANGED VALUE here even # if every verdict above still passes. ctl=$BST_DATA/ee-native.txt gotf=${TMPDIR:-/tmp}/bst-ee-got.$$ printf '%s\n' "$out" | grep -v '^EE@' > "$gotf" 2>/dev/null if [ ! -s "$ctl" ]; then bst_notok libee/native-byte-identical "control absent or empty: $ctl" elif cmp -s "$ctl" "$gotf"; then bst_ok libee/native-byte-identical else bst_notok libee/native-byte-identical \ "first diff: $(diff "$ctl" "$gotf" 2>/dev/null | sed -n '2,3p' | tr '\n' ' ' | cut -c1-90)" fi rm -f "$gotf" # The probe's own tally and its exit status. 148/148 is what the same source # scores natively against a host autotools build of the identical release. has libee/run-summary "ee-probe pass 148/148" "$out" check_eq libee/run-exit0 0 "$prc" rm -f "$probe" else bst_notok libee/run-stage "cannot stage $BST_DATA/ee-probe" fi