# BST-TIMEOUT: 1800  # 315 cases behind 119 mcpp forks and 4 tclsh spawns; the
#                    # forks are the cost, not the scoring. Measured on target
#                    # and recorded in the design doc.
. "$BST_LIB"

# mcpp 2.7.2 -- a portable, standards-VALIDATING C preprocessor (meta-oe
# recipes-devtools/mcpp). See mcpp_%.bbappend for what the port adds and why.
#
# ---------------------------------------------------------------------------
# WHAT THIS SUITE IS, AND WHY IT IS NOT A PROBE
#
# It closes on the PACKAGE'S OWN UPSTREAM SUITE. mcpp exists because its author
# wrote a conformance suite -- the "Validation Suite for Standard C Conformance
# of Preprocessing" -- and then wrote a preprocessor to pass it. The suite ships
# in the tarball; meta-oe packages none of it. The bbappend stages the arm that
# can be scored by machine (cpp-test/, 119 source files carrying their expected
# results as DejaGnu directives) and this file runs it under ptest-runner and
# reports one BST case per upstream assertion.
#
# Nothing here asserts a preprocessing behaviour of its own. Every one of the
# ~307 green cases below is upstream's: trigraph and digraph conversion, line
# splicing, the #if expression grammar and its overflow edges, macro
# re-definition rules, # and ## semantics, rescanning and recursion blocking,
# the translation limits of C90 §5.2.4.1, and the diagnostics required for each
# violation of a syntax rule or constraint.
#
# WHY THE cpp-test ARM AND NOT test-c. Upstream documents three ways to run the
# suite (doc/cpp-test.html §3.2) and two of them compile and execute C on the
# machine under test: §3.2.2's cpp_test drives `cc -o%s %s.c` over the 35
# self-checking programmes in test-c, and §3.2.1 is a human reading preprocessed
# output. This image ships no C compiler, so neither can run here -- that is a
# property of the image, not of the port, and it is reported as a skip below
# rather than quietly dropped. The arm that DOES run is the stronger one anyway:
# §3.2.3.1 -- "While the cpp_test tool in 3.2.2 can test only samples with n_* or
# i_* as a name, testsuite allows samples which require diagnostic messages such
# as e_*, w_*, and u_* to be tested automatically." And it is not a different
# corpus: §3.2.3.2 says cpp-test "is the one created by copying necessary files
# in each directory of test-t and test-l", so running it IS running test-t.
#
# ---------------------------------------------------------------------------
# THE TWO CASES THAT ARE EXPECTED NOT TO PASS, AND HOW THAT WAS ESTABLISHED
#
# Both were reproduced on an x86-64 NATIVE oracle -- same mcpp 2.7.2, same
# corpus, same scorer, same flags -- before this suite was written, so neither is
# a statement about wasm32. They are listed by name below and reported as skips
# carrying the reason; a case in that list that PASSES is reported red, because
# then the list is stale and the reason no longer holds.
#
#   test-t/n_cnvucn.C/dg-final   Conversion of multi-byte characters to UCN in
#       C++ translation phase 1. mcpp does not implement it -- its own manual
#       says so ("UCN ... is not recognized. Multi-byte characters in identifier
#       are not recognized") -- and under upstream's own harness the case passes
#       only because GCC's -finput-charset machinery does the conversion before
#       mcpp is reached. Standalone there is no driver to do it.
#
#   test-t/u_1_19.c/excess-errors  A macro that expands to `defined(...)` in a
#       #if. This is a u_* case: §4.2 UNDEFINED behaviour, where the suite
#       records what an implementation does rather than what it must do. mcpp
#       issues the expected warning AND a further "Bad defined syntax" error;
#       the extra diagnostic is what fails DejaGnu's excess-errors check.
#
# Everything else must be green. A red anywhere else is a real defect in the
# port, the arm, or the codegen beneath it, because the corpus is pure
# file-to-file text translation: no socket, no device, no clock, no float, and
# the only filesystem contact is reading the corpus and writing the .i files.
#
# ONE PLACE WHERE THE ARM LEGITIMATELY SHOWS THROUGH, and it is asserted rather
# than excluded: e_14_10.c expects `#if LONG_MAX - LONG_MIN` to be diagnosed as
# out of range, which is only true when the LONG_MAX mcpp reads and the `long`
# mcpp was compiled with agree. On wasm32 both are 32-bit, so they do -- but only
# because the bbappend stages THIS TARGET's headers beside the corpus and runs
# with -I-. Pointing the same corpus at 64-bit headers, or a 64-bit mcpp at these
# ones, silently drops all four of that file's cases to red. That is why the
# system headers travel with the payload.
#
# ---------------------------------------------------------------------------
# THE COUNTS ARE ASKED OF THE PAYLOAD, NOT ONLY PINNED HERE
#
# do_install_ptest writes a manifest recording how many test sources it staged
# per sub-corpus, counted from the tree rather than transcribed, and the cases
# below compare the run's own file count against it. want_total is pinned as
# well, because a manifest-vs-run agreement cannot see a payload that got
# SHORTER: a truncated corpus would stage less and run less, agreeing with itself
# the whole way down. The pinned numbers are upstream 2.7.2's, measured on the
# native oracle; a version bump that changes them is meant to be a visible
# decision here rather than a silent drift.

ptestdir=/usr/lib/mcpp/ptest
runner=/usr/bin/ptest-runner
bin=/usr/bin/mcpp

# Upstream 2.7.2: 111 sources in cpp-test/test-t and 8 in cpp-test/test-l, and
# the scorer derives 288 + 27 cases from the dg directives inside them.
want_files=119
want_total=315
want_fail=0

# Below this file's BST-TIMEOUT on purpose, so an overrun is reported by
# ptest-runner in its own voice instead of this whole file being killed around
# it -- the arrangement 77-nghttp2.test uses.
runner_t=1500

tmp=${TMPDIR:-/tmp}/bst-mcpp.$$
mkdir -p "$tmp" 2>/dev/null || tmp=/tmp

# ----------------------------------------------------------- what shipped ----
check_rc mcpp/bin        0 test -x "$bin"
check_rc mcpp/lib-hdr    0 test -s /usr/include/mcpp_lib.h
check_rc mcpp/out-hdr    0 test -s /usr/include/mcpp_out.h
check_rc mcpp/ar         0 test -s /usr/lib/libmcpp.a

# The arm is static-only, so -dev and -staticdev are the whole library and the
# shared object the recipe's libtool would otherwise build is never produced.
check_rc mcpp/no-so      1 test -e /usr/lib/libmcpp.so.0

# mcpp installs no pkg-config file at all -- upstream ships none and the recipe
# adds none. Asserted in the negative for the same reason 80-libmpc.test does:
# one arriving later should be a visible decision, not a surprise.
check_rc mcpp/no-pc      1 test -e /usr/lib/pkgconfig/mcpp.pc

# The binary answers for itself before the suite is trusted to. -v prints the
# banner and the include path; the banner names the build variant, and
# "compiler-independent-build" is the one the corpus's flag mapping assumes
# (a GNUC build would take gcc's own option spellings instead).
out=$("$bin" -v /dev/null 2>&1)
case "$out" in
  *"MCPP V.2.7.2"*) bst_ok mcpp/version ;;
  *) bst_notok mcpp/version "want MCPP V.2.7.2, got: $(printf '%s' "$out" | head -1)" ;;
esac
case "$out" in
  *compiler-independent-build*) bst_ok mcpp/build-variant ;;
  *) bst_notok mcpp/build-variant "not a compiler-independent build: $(printf '%s' "$out" | head -1)" ;;
esac

# The ptest payload the bbappend stages.
check_rc mcpp/ptest-dir     0 test -d "$ptestdir"
check_rc mcpp/ptest-script  0 test -x "$ptestdir/run-ptest"
check_rc mcpp/ptest-scorer  0 test -s "$ptestdir/mcpp-dg.tcl"
check_rc mcpp/ptest-corpus  0 test -d "$ptestdir/cpp-test/test-t"
check_rc mcpp/ptest-manifest 0 test -s "$ptestdir/manifest"

# The staged system headers. Their absence would not fail loudly -- it would
# quietly redden the six cases that #include one -- so it is asserted here where
# the reason is legible.
check_rc mcpp/ptest-sysinc  0 test -s "$ptestdir/sysinc/limits.h"

# tclsh is what evaluates upstream's dg-final fragments; without it the suite
# cannot run at all, and saying so once beats 315 unexplained reds.
require tclsh mcpp/tclsh-present

# THE ARM THAT CANNOT RUN HERE, said once and in its own voice. test-c's 35
# self-checking programmes are compiled and executed by upstream's cpp_test
# tool; no C compiler is on this image, so there is no subject.
if command -v cc >/dev/null 2>&1 || command -v gcc >/dev/null 2>&1; then
  bst_notok mcpp/test-c-arm "a C compiler appeared on this image -- upstream's cpp_test arm is now runnable and this suite does not run it"
else
  bst_skip mcpp/test-c-arm "upstream's test-c arm compiles and runs 35 programmes (doc/cpp-test.html 3.2.2); this image ships no C compiler"
fi

# What the manifest says was staged, read before the run so a payload that lost
# files is named as such rather than showing up as a short total.
man_t=$(sed -n 's/^sources test-t //p' "$ptestdir/manifest" 2>/dev/null)
man_l=$(sed -n 's/^sources test-l //p' "$ptestdir/manifest" 2>/dev/null)
check_eq mcpp/manifest-test-t 111 "$man_t"
check_eq mcpp/manifest-test-l 8   "$man_l"

# ------------------------------------------------------- run it, once ---------
# The harness is preferred wherever it exists, because this ticket closes on the
# suite run UNDER ptest-runner -- its discovery and its tally are part of what is
# asserted. The fallback runs the identical script directly and SKIPS only the
# cases that are about the harness itself.
out=$tmp/runner.out
if [ -x "$runner" ]; then
  bst_ok mcpp/ptest-runner-present
  "$runner" -t "$runner_t" mcpp > "$out" 2>&1
  rc=$?
  check_eq mcpp/ptest-runner-rc 0 "$rc"

  begin=; end=; rerr=
  while read -r k v; do
    case "$k" in
      BEGIN:) begin=$v ;;
      END:)   end=$v ;;
      ERROR:) rerr="$v" ;;
    esac
  done < "$out"
  check_eq mcpp/runner-begin    "$ptestdir" "$begin"
  check_eq mcpp/runner-end      "$ptestdir" "$end"
  check_eq mcpp/runner-no-error ""          "$rerr"
else
  bst_skip mcpp/ptest-runner-present "this image ships no ptest-runner"
  bst_skip mcpp/ptest-runner-rc      "no ptest-runner to run"
  for c in runner-begin runner-end runner-no-error; do
    bst_skip "mcpp/$c" "harness cases have no subject without ptest-runner"
  done
  ( cd "$ptestdir" && sh ./run-ptest ) > "$out" 2>&1
fi

# ------------------------------------- one case per upstream assertion --------
# A single `while read` over the capture: no fork per case, and every assertion
# upstream wrote gets its own BST case, so a red names the directive that broke
# rather than just the file it lived in.
npass=0
nfail=0
nskip=0
nfiles=0
nsub=0
xfail_hit=0
xfail_pass=0

# The known non-passes, by exact case name. See the block at the top of this
# file for how each was established; both reproduce on the x86-64 native oracle.
is_xfail() {
  case "$1" in
    "test-t/n_cnvucn.C/dg-final")     return 0 ;;
    "test-t/u_1_19.c/excess-errors")  return 0 ;;
    *)                                return 1 ;;
  esac
}
xfail_why() {
  case "$1" in
    "test-t/n_cnvucn.C/dg-final")
      echo "mcpp implements no multi-byte-to-UCN conversion (its manual says so); upstream's harness gets it from gcc -finput-charset, which standalone mcpp has no equivalent of -- same result on the x86-64 native oracle" ;;
    "test-t/u_1_19.c/excess-errors")
      echo "u_* is undefined behaviour (doc/cpp-test.html 4.2): mcpp issues the expected warning plus a further 'Bad defined syntax' error, which trips DejaGnu's excess-errors check -- same result on the x86-64 native oracle" ;;
  esac
}

while read -r verdict name rest; do
  case "$verdict" in
    PASS:)
      if is_xfail "$name"; then
        bst_notok "mcpp/$name" "expected non-pass now passes -- the known-non-pass list in this file is stale and should drop it"
        xfail_pass=$((xfail_pass + 1))
      else
        bst_ok "mcpp/$name"
      fi
      npass=$((npass + 1))
      ;;
    FAIL:)
      if is_xfail "$name"; then
        bst_skip "mcpp/$name" "$(xfail_why "$name")"
        xfail_hit=$((xfail_hit + 1))
      else
        bst_notok "mcpp/$name" "upstream assertion failed: $rest"
      fi
      nfail=$((nfail + 1))
      ;;
    SKIP:)
      bst_skip "mcpp/$name" "$rest"
      nskip=$((nskip + 1))
      ;;
    ERROR:)
      # run-ptest's own refusals (no mcpp, no corpus, no tclsh, unwritable
      # scratch). Distinct from an upstream assertion failing, and reported as
      # such rather than being read past on the way to a short total.
      bst_notok "mcpp/ptest-setup" "run-ptest refused: $name $rest"
      ;;
    INFO:)
      # INFO: mcpp-dg sub=S files=F pass=P fail=F skip=S total=T
      # INFO: mcpp-ptest files=F pass=P fail=F skip=S total=T
      case "$name" in
        mcpp-dg)    nsub=$((nsub + 1)) ;;
        mcpp-ptest)
          for kv in $rest; do
            case "$kv" in files=*) nfiles=${kv#files=} ;; esac
          done
          ;;
      esac
      ;;
  esac
done < "$out"

# Both sub-corpora reported. A run that died inside test-t never prints test-l's
# INFO line, and this is what notices.
check_eq mcpp/sub-corpora    2              "$nsub"
check_eq mcpp/files-run      "$want_files"  "$nfiles"

# The pinned totals. Without these, a run that produced three PASS lines and died
# would read as three green cases and nothing else.
check_eq mcpp/suite-total    "$want_total"  "$((npass + nfail + nskip))"

# Every non-pass accounted for: the two known ones hit, and nothing else red.
check_eq mcpp/known-nonpass  2              "$xfail_hit"
check_eq mcpp/stale-nonpass  0              "$xfail_pass"
check_eq mcpp/unexpected-fail "$want_fail"  "$((nfail - xfail_hit))"

rm -rf "$tmp"