Conformance

24 suites, 1,660 cases. Every renderer replays them, ours in CI on every change, yours with the same files and the same verdict.

What it means

A renderer is Exponential UI conformant when its runner replays every suite of the manifest and the report passes: every suite present, every case run, nothing failed or skipped. Then a surface means the same thing on it as on ours. The same tree comes out of the reducer, every macro expands the same way, a theme resolves to the same colours, a control measures to the same box, and the host makes the same policy decisions.

The fixtures in fixtures/ are the contract. The TypeScript reference generated their expected values, and the Rust core replays them byte for byte. A renderer built on the Rust core gets the pure suites through its bindings, one built on the TypeScript package gets them from @exponential-at/ui, and anything else implements the reducer and host.json itself.

The suites

Generated from the fixtures into conformance/manifest.json (version 2), which drift-gates the case counts. A suite's unit says what one case is.

SuiteUnitCasesA runner checks, per case
catalogcase466Reduce the case's nested node with the core catalog: no issues. Paint it under the default theme: no crash, no Unknown placeholder, every visible node painted.
macroscase200Expand the case: the tree equals expanded (JSON-equal, recipes included).
basic-mapcase13Reduce the A2UI basic surface: the tree and the issues equal the expected ones.
extensioncase4Register the extension; reduce each case: tree + issues equal; its natives reach the extension painter (an Extension leaf of that kind).
theme-resolvedtheme3Load each built-in theme: the resolved theme equals the fixture.
theme-recipescase356Resolve the part's recipe for the case's theme, props, mode and states: the visual equals the fixture.
theme-extendscase3Load the theme over its parents: the chain and the probed styles equal the fixture.
theme-invalidcase24Validate the bad theme: the same issue paths, never a crash.
control-geometrytheme × control × case231Measure the control under the theme: the box (height, padding, gap, border, radius) equals the fixture.
layoutwidth × direction4Lay the surface out with the fixture's fixed measure: every node's frame equals the fixture (exact for the Rust core and native painters, within 1 px for a CSS renderer).
overlaycase11Place the overlay: side, flip and frame equal the fixture within tolerancePx.
replaytheme × mode6Paint the kitchen sink under each built-in theme and mode: the reduced tree equals kitchen-sink.expanded.json, no Unknown, the accessibility order is the pre-order of the painted nodes.
host-transportcase20Feed the chunks through ONE decoder (JSONL / SSE), or decode the MCP result, or build the MCP action call: messages + issues equal expected.
host-policycase57Decide the function / combine / url, build the media request, parse the source, negotiate the catalog ids: each equals expected.
host-routerpackage or flow8Validate each package; for each flow build ONE router with extensionIds, install packages (issues = installIssues), route every step: the ops equal expected.
bindcase26Expand the input; for every dataset run the bind pass: the bound tree, every set press and the per-row slot cells equal the fixture (reducer issues too for extra).
style-conditionscase7Flatten the style for every context: equal to expected (key order irrelevant).
code-tokenscase17Tokenize the code: the lines of tokens equal expected.
formatcall or value86Resolve each format call through the English fallback (zoned: at the case's offsetMinutes): the string equals expected exactly; display each value in a text prop: equals expected. A platform Formatter may run too, at en-US / UTC (zoned: timeZone), comparing after U+202F and U+00A0 → space.
template-itemscase22Key the items / rows, list the template instances (path, key, accumulated suffix), reduce the surfaces (templates lifted): equal to expected.
text-directionsurface direction2Reduce the tree; every node's resolved direction and physical text alignment equal expected.
resizablecase38Normalize / resize / key / measure / convert: the numbers equal expected within 1e-6.
virtual-listcase26Window the extents, compute the scrollToIndex offset (sectioned: data index → row), group the sections, pin the sticky header: equal to expected.
animationstheme × animation30Time each animation under the theme and sample its frames (and the reduced-motion frame): equal to expected within 1e-3; opacity: the painted opacity = the node's own × the frame's, within 1e-3.

Total: 1,660 cases in 24 suites.

Run it on your renderer

A third-party renderer (Flutter, Qt, a terminal UI) becomes conformant in four steps:

  1. Read the manifest. It lists each suite's fixture files, the unit and the case count. It ships in the npm package as @exponential-at/ui/conformance/manifest.json, next to the fixtures.
  2. Replay every case the way the suite's check says: reduce, expand, resolve, measure, lay out, place, paint, decode or decide, and compare with the fixture's expected value. Geometry suites take your painter's own measurements (controls within 0.5, a CSS layout within 1 px).
  3. Write the report (below), one entry per suite, naming every failed case.
  4. Check it. The checker compares the report with the manifest and prints the verdict:
shell
bun run --filter @exponential-at/ui conformance:check path/to/report.json
# CONFORMANT  my-renderer (flutter 0.3.0): 1660 cases passed in 24 suites

Outside the monorepo, checkReport() from @exponential-at/ui gives the same verdict in code. A report with a missing suite, a short case count or a skipped case is not conformant.

The report

One JSON object per run, validated by report.schema.json:

json
{
  "renderer": "my-renderer",
  "platform": "web",
  "version": "0.3.0",
  "conformanceVersion": 2,
  "suites": {
    "catalog": { "cases": 466, "passed": 466, "failed": [] },
    "macros":  { "cases": 200, "passed": 200, "failed": [] },
    …
  }
}
  • renderer, platform, version: who ran it, where.
  • conformanceVersion: the manifest's version the runner was written against.
  • suites.<id>: cases run, passed, failed (the failed cases' names with a short reason) and, optionally, skipped.

Our runners write it to $EXPONENTIAL_UI_CONFORMANCE_REPORT (default .conformance/<name>.json at the repo root).

Our runners and CI

The core and all four first-party renderers run the full suite:

RendererRunnerCommand
exponential-ui (Rust core)conformance.rscargo test -p exponential-ui --test conformance
@exponential-at/ui-reactsuites.test.tsbun run --filter @exponential-at/ui-react test:conformance
exponential-ui-gpuisuites.rscargo test -p exponential-ui-gpui --test suites
ExponentialUI (SwiftUI)ConformanceTests.swiftswift test
at.exponential:ui-composeConformanceTest.kt./gradlew :ui-compose:testDebugUnitTest

CI (exponential-ui.yml) runs all of them on every SDK pull request and fails unless every report is conformant. A release tag is only cut from a commit where that workflow is green.

The real-font harness

The suites lock the contract with a fixed fake measure. The real-font harness locks what real text does to it: the kitchen sink and the responsive cases, rendered by React (headless Chromium) and gpui with the same committed fonts (Inter, Geist, JetBrains Mono, Nunito, Fira Code), compared frame by frame against a committed web baseline.

StepWhat it checks
Frame dumpsEvery placed part of every case (fixture × theme × mode × width × direction): frame, text, line count.
ComparisonFrames within a pixel tolerance; a text may wrap one line more or less. Unexplained divergences are origins: the fix list.
Known divergencesconformance-known.json: per cause, which side is right and who fixes it; the ratchet only goes down.
Pixel stepThe web surface beside a real gpui window: pixel ratio and SSIM. Reported, never gating.
shell
bun run --filter @exponential-at/ui conformance -- --only kitchen-sink/playful/dark/390/ltr

Next: the comparison becomes suite real-font in the manifest, so conformance:check stays the one verdict.

A2UI listing

Once two platforms pass, Exponential UI gets an entry on the A2UI ecosystem's renderers page, as a pull request to google/A2UI:

Exponential UI (React, SwiftUI, Jetpack Compose, gpui): native renderers of the A2UI v0.9 basic catalog plus a core catalog, runtime themes, extension catalogs and a host API (transports, functions, bindings, policy). Apache-2.0. https://ui.exponential.at