Testing an integrated app

Testing and coverage gates covers the framework's own verification. This page is about the harder problem: an app wired to a real protocol, where the proofs still hold but say nothing about the adapter.

The governing idea is that different tiers produce different kinds of evidence, and collapsing them is how false confidence is manufactured.

The tiers, and what each one proves

In-memory green is not evidence

An in-process adapter that appends to a JavaScript object satisfies every Doc post-condition a workflow can state. That is exactly what makes it worthless as integration evidence: the checks pass whether or not a network exists.

State the tier every time. "52/52 workflows pass" is a claim about the in-memory tier. "52/52 pass with ENCAPP_LIVE=1 against spawned nodes" is a different, much stronger claim. They are one flag apart and they are not interchangeable.

# in-memory — fast, and NOT wire evidence
node ./test/test-sdk/encapp-smoke.mjs all
 
# live — real nodes, minutes not seconds
ENCAPP_LIVE=1 node ./test/test-sdk/encapp-smoke.mjs all

The real browser tier finds what the suite cannot

Every genuine fake found in the flagship project was found by using the app, not by the suite: a wallet chooser that was a stub, a profile pre-filled with a fixture name, a dev server that spawned no node, tests that quietly hit production.

They survived because each test tier supplied the missing piece itself. A headless pool that sets the app binding by hand cannot detect a build that fails to publish it. A Node runtime with a real node:fs cannot detect a browser bundle whose shim throws on load.

So the browser tier drives dist/index-live.html in real Chromium, with real typing and real clicks, and then asserts the result on the node with a raw client — no app code in the read path:

connect → onboard (type a name → Save) → post → assert it renders
       → assert the row is ON THE NODE via a raw adapter read

If the assertion goes through the app, it can be satisfied by the app's own local echo. Read the wire directly or the check proves nothing.

Probes isolate one property

A probe spawns a stack, exercises one protocol property, and asserts it — no UI, no corpus. They are where you prove that a delete really deletes for everyone, that two groups are separate compartments, that a kicked member cannot decrypt what follows.

Probes are also where a capability is established before it is used. Probe the mechanism first, then implement against it. Six theories about a node died to six experiments in one project; each experiment took minutes.

Checks that guard the seam itself

These are cheap, run in seconds, and each closes a hole that a passing suite would otherwise hide.

The bundle-freshness one is easy to underrate. Every probe and every workflow imports adapter source; a user loads a bundle. Edit the adapter, skip the rebuild, and the entire suite goes green while the shipped app runs old code.

Make every claim refutable

The discipline that made the flagship project's boundaries legible: keep a ledger in which every assertion about the repository carries the command that would refute it, and run the ledger as a gate.

### the app's seams are DECLARED, not hard-coded in the host
- status: VERIFIED
- verify: sh -c 'grep -q "\"deletes\":\[{" dist/index.html && …'

Three rules make it work:

1. A claim with no verify command is not a claim. Delete it or give it one.
2. A claim marked BLOCKED whose verify command succeeds fails the build. This mechanises the stale-rationale trap: a note asserting something is impossible must carry a command that re-attempts it, so the day it starts working the build goes red and the note gets corrected.
3. A green check that has never been red is indistinguishable from one that cannot fail. Red-prove every detector against the real bug before trusting it.

That third rule is not theoretical. Detectors in this stack reported clean before they worked — one counted payload atoms as effects, one treated a skip as a pass, one flagged whitespace inside colour values. A security check that routes through the node may be testing authorization when you believe it is testing cryptography; isolate the mechanism you claim to test.

Test against the real binding, never a stub

Any test that exercises host behaviour driven by the app's declaration must load the emitted binding out of the built artifact. A hand-written stub is a fiction — the app can declare one thing and the test assert another, and nothing notices.

Parse it by brace-matching, not with a regex. A regex that stops at the first ]} works only while every declared form is a flat array; the first nested array truncates the capture, and the resulting JSON.parse failure points nowhere near the change that caused it.

A practical order

1. lake build — proofs current.
2. Fast checks — freshness, binding honesty, emitted UI, error channel, no-production.
3. In-memory matrix — quick regression, labelled as such.
4. Probes — the protocol properties your app depends on.
5. Live matrix — the corpus against real nodes.
6. Real browser — the shipped bundle, driven like a user.
7. The ledger — every claim re-verified.

Steps 1–3 belong on every save. Steps 4–7 belong before you tell anyone it works.

Recap

• In-memory tests are not wire evidence: a local echo satisfies every Doc check whether or not a network exists; always state the tier
• Five tiers exist: proof (decidable, no adapter), in-memory (shared adapter, no wire), live (real nodes), real browser (shipped bundle), and probes (one property, no UI)
• Each tier supplies the missing piece of the tier above it, which is why a suite stays green while the shipped app is broken — headless pools set bindings by hand, Node runtimes have real node:fs
• Probes are where capability is established: a mechanism is proven before it is used; six separate divergence theories died to six experiments
• Six fast checks (bundle freshness, binding honesty, emitted UI, error channel, no-production, detection red-proof) close holes that passing suites hide
• The ledger discipline: every claim carries a command that would refute it; BLOCKED claims with passing verify commands fail the build; a green check that has never been red is indistinguishable from one that cannot fail

Next steps

• Testing and coverage gates — the four verification tiers and the axiom audit
• Host bindings — declaring policy so adapters are generic executors, not re-implementers
• ENC protocol integration — the adapter contract and checklist for a new integration