Skip to main content

Frontend Quality Gate: Pattern-Owned Testing Ethos (v0.1)

This document defines a backend-grade testing ethos for the Evalium frontend. The goal is to make frontend changes feel as safe as backend changes by enforcing invariants through a pattern-owned test bundle.

This is deliberately “a lot” — the point is to prevent drift and regression in the pillars (drawer, inventory, authoring flows), including accessibility.

1) Core Principle​

Every shared UI pattern has an owner and a mandatory test bundle.

A change is not “done” until the pattern bundle passes locally and in CI (when CI exists).

Pattern examples:

  • DRW-01/DRW-02: Context Drawer + Coordinator (URL-as-state, shallow routing)
  • INV-01: Inventory Table (filter/sort/paging, selection, row actions)
  • IMP-01: Import Wizard (preview → commit)
  • BULK-01: Bulk Action Drawer (dryRun → apply)
  • VAL-01: Validation Sidekick (“go to” focus targets)

2) The Pattern-Owned Test Bundle (Mandatory)​

Each pattern must have all of the following:

A) Unit Tests (Vitest)​

Purpose: prove pure logic and contracts fast.

What belongs here:

  • URL/query parsing + serialization (round-trip)
  • Param preservation rules
  • Deterministic mapping functions (status → chip label, etc.)
  • Request/response normalization logic (SDK boundary helpers)
  • Any helper that does not require a browser

Exit criteria:

  • Fast (<1–2s per suite)
  • Covers edge cases (invalid inputs, empty inputs, unexpected params)
  • No network, no DOM required

B) Component Tests (Playwright Component Testing preferred)​

Purpose: prove accessibility and interactive semantics in a real browser engine.

What belongs here:

  • Dialog semantics for drawers/sheets:
    • focus trap
    • ESC closes
    • click-outside closes (when allowed)
    • focus restore to opener
    • proper role="dialog" + labeling
  • Menu semantics:
    • keyboard navigation (Arrow keys)
    • Enter/Space activation
    • focus returns correctly after close
  • Table semantics at component level:
    • row selection via keyboard
    • row action menu reachable without hover

Exit criteria:

  • Uses role-based locators (getByRole, getByLabelText) wherever possible
  • Includes keyboard-only coverage
  • No reliance on hover-only interactions

C) E2E Smoke Tests (Playwright Test)​

Purpose: prove end-to-end invariants: routing, history, and real-user flows.

Rules:

  • Keep E2E focused and minimal per pattern (typically 1–3 tests).
  • E2E tests should confirm the contract “end to end” rather than replicate unit tests.

Exit criteria:

  • Reliable and deterministic
  • Uses stable test fixtures (showroom routes are ideal)
  • Captures traces on failure

D) Automated Accessibility Gate (Axe + Manual Keyboard Assertions)​

Purpose: ensure accessibility is not an afterthought and doesn’t regress.

Minimum:

  • Automated scan gate (axe) on:
    • the showroom page for the pattern
    • at least one “real app” page once available
  • Manual keyboard assertions in Playwright:
    • tab order does not escape traps
    • focus is always visible and returns to a meaningful element
    • no keyboard dead-ends

Exit criteria:

  • No critical violations
  • No keyboard dead-ends in covered flows

3) Example: DRW (Context Drawer) Pattern Contract Tests​

DRW Unit Tests (Vitest)​

  • drawer-state parsing:
    • parses drawer=user:123
    • rejects invalid formats
  • drawerTab round-trip
  • add/remove drawer params:
    • preserves unrelated query params
    • idempotent behavior (removing twice doesn’t change output)

DRW Component Tests (Playwright CT)​

  • Drawer uses dialog semantics:
    • focus trap works (Tab stays inside)
    • ESC closes
    • close button works
    • focus returns to opener element
  • Drawer states:
    • loading
    • error + retry
    • forbidden
    • not found
  • Tabs:
    • tabs are reachable and switch content
    • switching tabs does not break focus model

DRW E2E Smoke (Playwright Test)​

  1. Click opens drawer / Back closes
  • visit showroom
  • click a link
  • assert drawer visible + URL includes drawer=...
  • go back
  • assert drawer not visible + URL no drawer=...
  1. Deep-link opens / ESC closes / focus restore
  • visit showroom with ?drawer=user:123
  • assert open
  • press ESC
  • assert closed
  • focus is sane (for deep-link there may be no “opener”; define deterministic fallback)

Important note: Back-close is only guaranteed when opening created a history entry. Deep-link load may not have a prior history entry.

4) Showroom Routes as Test Fixtures​

We maintain /dev/ui/* routes as deterministic fixtures:

  • controlled data
  • clear labels
  • stable selectors via accessibility roles/names
  • states: loading / ready / forbidden / notfound / error

Benefits:

  • tests validate shared components without requiring backend availability
  • faster iteration on UI pillars
  • reduces flakiness

Rules:

  • Showroom routes must be gated to dev builds.
  • Showroom routes should exercise edge states intentionally.

5) Guardrails (Pre-flight gates)​

These should run on every “terminal CI” and later in CI pipelines:

A) Typecheck + build sanity​

  • svelte-check (or equivalent)
  • tsc (if separate)
  • npm run build (when repo is stable)

B) Lint​

  • ESLint + Svelte rules
  • Custom rule ideas (optional but high value):
    • prevent internal navigation that ignores SvelteKit base path
    • discourage hover-only action access

C) Tests​

  • Unit: Vitest
  • Component: Playwright CT (pattern set)
  • E2E: Playwright smoke (pattern set)
  • A11y: axe + keyboard assertions in the E2E/CT suites

6) Trace / Debug Policy (Mandatory)​

  • Always record traces on failure.
  • Keep screenshots/videos for failing E2E tests.
  • A failing test must provide enough artifacts to diagnose without re-running locally.

7) Local “CI make” Equivalent​

We keep a lightweight local workflow (names illustrative):

  • make fe-check

    • lint
    • typecheck
    • unit tests

  • make fe-smoke

    • Playwright E2E smoke suite (showroom + key flows)
    • a11y gates included

  • make fe-ci

    • runs fe-check + fe-smoke

Rule: A PR is not “ready” unless make fe-ci passes.

8) Incremental Adoption Plan (So This Doesn’t Stall Delivery)​

Phase 1 (now)

  • Apply the full bundle to the pillar patterns first:
    • DRW (Context Drawer)
    • INV (Inventory Table)
    • IMP (Import Wizard) after INV
  • Use showroom fixtures to keep it deterministic.

Phase 2

  • Extend to authoring flows (publish impact summary, validation sidekick)
  • Add at least one real-page E2E per pattern once available.

Phase 3

  • Expand coverage breadth (more entities), but keep the pattern bundle principle:
    • strengthen contracts, avoid duplicative “page tests”.

9) Non-negotiables​

  • Pattern invariants must be tested, not assumed.
  • Accessibility is treated as a contract (keyboard-first).
  • Prefer role-based selectors in tests.
  • Showroom routes are first-class test fixtures.
  • Tests should fail loudly and provide artifacts (traces) for fast debugging.