Skip to main content

0. Terminology & Naming (Stability vs Product Language)

Evalium intentionally keeps stable internal primitives (DB/API naming) while allowing product-facing vocabulary to evolve.

  • Internal primitives (DB/API): evaluation → assignment → session → submission
  • Product vocabulary: template → task issue → execution window → ledger entry

Rule: The mechanics stay consistent; the framing reflects “Execution Ledger / Operational Defensibility”.

1. System Goals

Evalium’s architecture is driven by eight main goals:

  1. Secure multi-tenancy + internal silos Enforced boundaries between customers (tenants) and between internal org units inside a tenant (silos).

  2. Immutable versioned authoring Templates, checklists, rubrics, and scoring policies are versioned and cannot be mutated after publishing.

  3. Append-only execution ledger (forensic truth) Operational records are WORM where it matters. Corrections are additive (amend/void), never destructive.

  4. The Golden Thread Support long-running real-world work (months-long projects/engagements), not just single “sessions”.

  5. Client “glass box” transparency A read-only, RLS-enforced portal that shows the ledger in situ with history, not just exported PDFs.

  6. Contextual binding + verification levels Certain workflows require stronger evidence (time source, device identity, location), enforced server-side.

  7. Deep reporting and controlled disclosure Analytics and feedback derived from frozen snapshots and structured tags, with strict visibility controls.

  8. Backend-first development All backend flows are implemented and validated before UI exposure to prevent “UX-led architecture drift”.

2. Product Philosophy

Note on terminology: This document uses Evalium’s canonical, industry-agnostic language. Customer- or sector-specific terms are applied via an alias layer and are intentionally excluded from architectural definitions.

For the K/O/E domain model and terminology, see: docs/product/KOE-model.md

Principle of Progressive Disclosure​

The backend supports the complex case (verification levels, silos, ratification). The frontend defaults to the simple case. We never simplify the database to match a simple UI; we simplify the UI to mask a powerful database.

The “Save Button Lie” (Operational UX Reality)​

SMBs expect to “edit” operational records.

  • Frontend: “Edit / Save”
  • Backend: “Amend” (append-only), with full history and audit trail available when needed.

Section 2.2 — Communication Tiers & the UX of Defensibility​

Evalium deliberately separates Execution Truth from User Experience.

The backend operates using strict, precise concepts (e.g. append-only ledgers, immutable snapshots, RLS-enforced scopes). These guarantees ensure correctness, auditability, and long-term defensibility.

The presentation layer does not expose these mechanics directly.

Instead, the architecture supports a tiered communication model:

  • Tier 1 (Forensic / Internal):

    • Used by the system and internal services
    • Prioritises correctness and precision
    • Forms the canonical source of truth

  • **Tier 2 (Experiential / User-Facing):

    • Used in all normal user interactions
    • Prioritises clarity, calm, and speed
    • Masks forensic mechanics behind an alias layer

This decoupling allows Evalium to support a broad professional-services market with a simple interface, while retaining enterprise-grade guarantees that can be revealed only when situational seriousness requires it.

For examples of how this manifests in practice, see Normal Use & Depth Discovery and Language & Tone Guidelines.

3. High-Level Architecture

Evalium is a modular monolith: a single deployable backend with clear service boundaries and event-driven internal communication.

+------------------------------+        +------------------------------+
|     Admin Console (Web)      |        |    Client Portal (Web)       |
|  authoring / Ops / Reports   |        |   Read-only Ledger View      |
+--------------+---------------+        +--------------+---------------+
               \                              /
                \                            /
                 v                          v
+---------------------------------------------------------------------+
|                         API Handlers (Go + Chi)                     |
|        Auth middleware, request validation, DTO assembly            |
+------------------------------+--------------------------------------+
                               v
+---------------------------------------------------------------------+
|                         Service Layer (Modular)                     |
|  authoring | Assignment/Execution | Ledger | Evidence | Reporting    |
|  Identity/RBAC | Remediation | (future) Ratification/Hashing         |
+------------------------------+--------------------------------------+
                               v
+---------------------------------------------------------------------+
|                            TxManager                                |
|     Scoped transactions + RLS GUC injection (SET LOCAL tenant/org)   |
+------------------------------+--------------------------------------+
                               v
+---------------------------------------------------------------------+
|                         PostgreSQL (RLS)                             |
|  Tenancy + Silos | Versioning | Ledger | Snapshots | Reporting models|
+------------------------------+--------------------------------------+

+------------------------------+        +------------------------------+
| Evidence Storage (Files)     |        | Background Workers           |
| (object store or filesystem) |        | scoring, remediation, etc.   |
+------------------------------+        +------------------------------+

Note: Internal service boundaries follow ADR guidance (event-driven dispatchers/listeners where appropriate).

4. Core Domains (Three Worlds)

Evalium is easiest to reason about as three worlds with strict rules between them.

4.1 authoring World (Versioned Templates)​

What a tenant defines before work happens:

  • Evaluations/templates (checklists, rubrics, “licence to operate” gates)
  • Questions/items (knowledge checks, prompts, rubric criteria)
  • Policies (verification level requirements, feedback disclosure rules)
  • Structured tags (topic/skill/difficulty, or domain-specific equivalents)

Invariant: once published/versioned, authoring content is immutable.

4.2 Execution World (The Ledger)​

What happens in the real world:

  • Task issued (assignment)
  • Work performed (session/execution window)
  • Ledger entries recorded (submission/ledger entry + items)
  • Evidence captured (uploads, attachments, metadata)
  • Reviews and sign-offs (verifier actions)
  • Amendments and voids (additive corrections)
  • (future) State ratification and milestone hashes

WORM scope (important clarification)​

WORM / append-only applies to the execution ledger itself: ledger entries and their audit events. Operational scaffolding (sessions, assignments, projections, sync state) may be mutable by design.

Invariant: execution ledger entries are append-only and reconstructable.

Enforcement details: docs/architecture/LEDGER-BOUNDARY-AND-ENFORCEMENT.md.

Evidence Core Contract (Ledger Primitive)​

Evidence is a first-class ledger primitive in the Execution World. It is not a file type or a UI-only concept. All evidence flows through one core contract, regardless of entry point.

Core requirements:

  • Asset metadata: asset ID, storage reference, cryptographic hash + algorithm, media type
  • Capture context: captured_at, actor_id, device_id
  • Optional context: GPS, trusted time, network context (policy-driven)
  • Association link: exactly one of submission_id or submission_item_id
  • Decision ledger: append-only events (pending, approved, rejected, void)
  • Verification proof: step-up identity proof when policy requires it

Entry points (same core, different association):

  • Standalone Evidence: submission with zero items (implemented)
  • Inline Evidence: evidence linked to submission_item_id (planned)
  • Mixed Evidence: answer + artefact together (planned)

Evidence events appear in the Client Portal Glass Box as ledger-derived history, not as mutable files.

4.3 Reporting World (Projections)​

Views and read models built from the ledger:

  • Summary tables for analytics
  • Exports (CSV/BI)
  • Client portal projections
  • Audit timelines
  • (future) xAPI / LRS integration

Invariant: reporting must not depend on live authoring data; it uses frozen snapshots.

Replica seam: reporting reads must go through a store interface that can switch between primary and replica pools without handler changes (default is primary).

4.4 Query Routing Policy (Primary vs Projection)​

Primary (Execution World)

  • short, scoped transactions only
  • insert-only for ledger tables
  • minimal joins; never used for analytics screens

Projection (Reporting World)

  • denormalised/flattened for read performance
  • used for dashboards, exports, cohort slicing
  • still RLS-enforced via TxManager scope

authoring World

  • standard CRUD with strict versioning rules
  • immutable once published

5. Multi-Tenancy & Org Units (Internal Silos)

Evalium supports two-layer isolation:

5.1 Tenants​

A tenant is an Evalium customer. Every row belongs to exactly one tenant.

5.2 Org Units​

Optional subdivisions within a tenant:

  • regions
  • departments
  • teams
  • client accounts (if the customer uses Evalium to service multiple end-clients)

Hierarchy note: org units may be arranged in a user-defined hierarchy (future), but Evalium only enforces tenant + org unit boundaries. It does not enforce semantic meaning of the hierarchy.

5.3 Enforcement via RLS​

PostgreSQL Row-Level Security (RLS) is the final arbiter.

Each request sets scope variables via SET LOCAL within TxManager:

  • app.tenant_id
  • app.org_unit_id
  • app.org_scope_all (true only for privileged global admins)

RLS policies enforce tenant + org scoping even if an application query is missing filters.

5.4 Developer Checklist (RLS)​

When adding a new tenant/org scoped table:

  1. Add tenant_id and org_unit_id (unless derived via parent)
  2. Enable RLS for the table
  3. Add tenant isolation policy
  4. Add org scoping policy (or parent-EXISTS policy for join tables)

6. authoring Architecture (Versioning Model)

6.1 Questions / Items​

  • questions: identity (stable)
  • question_versions: immutable content (payload, tags, rubric/scoring metadata)

6.2 Evaluations / Templates​

  • evaluations: identity (stable)
  • evaluation_versions: immutable definition (sections, items, navigation rules, scoring rules, disclosure rules)

6.3 Preview Pipeline​

Preview endpoints assemble draft/active versions for UI authoring. Only one version per evaluation may be active.

7. The Golden Thread: Engagement / Project Wrapper

Professional services work is not always a single “session”. Evalium introduces an Engagement (or Project) as the long-running thread container.

7.1 Hierarchy​

Engagement / Project (Thread container)
→ Assignments (Tasks / Gates / Field Work)
→ Sessions (Execution Windows)
→ Ledger Entries (Submissions + Items + Evidence + Sign-offs)

7.2 What Engagement is (and is not)​

  • Is: a grouping container that threads long-running work and enables timeline + milestone proof.
  • Is not: a replacement for Assignment/Session/Submission; it references them.

Implementation note: Engagement can be introduced as a wrapper without breaking the existing lifecycle; it becomes an optional parent reference for assignments and ledger entries.

8. Execution Lifecycle (Assignment → Session → Ledger Entry)

Evalium retains a clean lifecycle, but reframes intent from “test delivery” to “task execution logging”.

8.1 Assignments (Task Issue)​

Assignments define who may perform which task/template:

  • user_id or invited email
  • evaluation_version_id
  • attempt limits (where relevant)
  • frozen org scope
  • frozen disclosure config
  • cohort/run labels (for analytics)
  • (future) availability windows / schedule

Assignments represent author intent and the boundaries of allowed work.

8.2 Sessions (Execution Window)​

Created when an operator launches an assignment. Tracks:

  • start time
  • timing and navigation state
  • org scope inheritance
  • one active attempt per session

8.3 Ledger Entries (Submissions)​

Created when execution is recorded/completed.

Contains:

  • frozen version_snapshot
  • item responses/observations
  • scoring/outcome (where applicable)
  • derived tags/feedback tokens (per disclosure policy)
  • evidence links (metadata)
  • verification level + context metadata (when required)

Ledger entries are immutable.

8.4 Universal Review / Approval Layer (Maker–Checker)​

Review and approval is a platform capability, not a “feature of Observations”.

  • Applies to K/O/E depending on policy.
  • Transitions (e.g., pending_review → approved/rejected) are ledger events.
  • The underlying recorded work remains immutable; only the approval state evolves via additive events.

9. version_snapshot (Reconstruction Layer)

A ledger entry stores a version_snapshot that freezes everything required to reconstruct “what the operator saw and did”:

  • sections + items
  • question versions
  • structured tags
  • evaluation metadata
  • disclosure config (feedback mode + keys)
  • navigation config
  • tenant + org scope
  • assignment linkage

Rule: reporting and reconstruction must use the snapshot as canonical, not re-join live authoring tables.

10. Contextual Binding & Verification Levels (Server-Enforced)

Certain workflows require stronger proof that an event happened under specific conditions.

10.1 Context Metadata​

Every ledger write accepts a context_metadata payload (shape defined in FOUNDATION.md). Policy decides what’s required.

10.2 Verification Levels​

Verification levels (L1–L4) define trust strength, for example:

  • L1 self-verified entry
  • L2 peer-verified sign-off
  • L3 system-verified constraints satisfied
  • L4 context-verified (location/time/device + step-up auth, if required)

10.3 Where enforcement happens​

Enforcement occurs at the write boundary for relevant events:

  • record answer
  • attach evidence
  • submit / complete
  • approve / reject
  • ratify milestone (engagement + submission scopes currently live)

10.4 What is stored​

Verification is recorded on each relevant ledger event (and can be summarized on the submission):

  • required verification level
  • verification evidence (what was checked, by whom/system, outcome)
  • context metadata (where applicable)

11. Amendment / Void Model (Ledger Truth vs UX)

Execution ledger entries are never deleted or overwritten. Corrections are represented as additional ledger events.

  • Amend: append a new event referencing the original record
  • Void: append a void event with reason + actor + timestamp (and step-up proof if required)

Reporting and “current state” views are computed from the amendment chain, while history views show the full timeline.

12. Client Portal (“Glass Box” Architecture)

The Client Portal is a read-only projection surface (a "Truth Projector"), not a collaboration tool. It allows stakeholders to view the ledger state in situ, eliminating the need for static, mutable exports like PDFs.

12.1 The Three Lenses​

The Glass Box projects truth through three distinct lenses, each answering a specific hostile question:

  1. Engagement Glass Box (The "Golden Thread")

    • Question: "Did the job happen?"
    • Scope: Timeline of execution events, completion status, and milestone ratification.
    • Use Case: Project management and delivery confidence.

  2. Submission Glass Box (The "Forensic Truth")

    • Question: "Prove this task was done correctly."
    • Scope: A frozen render of the version_snapshot. Shows specific answers, L4 verification badges (time/loc/bio), and forensic evidence metadata.
    • Invariant: Must show the history of amendments (crossed out), not just the final state.

  3. Programme Glass Box (The "Compliance Passport")

    • Question: "Is this person qualified right now?"
    • Scope: Live enrolment status (Valid/Expired), requirement progress, and certificate validity.
    • Use Case: Site access control and licence-to-operate verification.

Access is granted via Link Principals (Tier 1 Identity), not shared accounts.

  • Mechanism: Secure, scoped, revocable magic links.
  • Audit: Every view is an auditable event (who viewed what at when).
  • RLS: Strictly enforced by tenant_id + org_unit_id + scope_id.

13. Hashing & State Ratification (Milestone Proof)

Evalium supports generating cryptographic proofs of ledger state at milestones.

13.1 Hashing engine responsibilities​

  • Compute a state fingerprint/hash from canonical ledger data + snapshot references at a point in time
  • Store fingerprint as an immutable ledger event tied to an engagement milestone
  • Ensure repeatable computation (same inputs → same fingerprint)

13.2 Ratification​

Ratification is an event where a permitted stakeholder signs off a milestone:

  • references the state fingerprint/hash
  • records signer identity + timestamp
  • records step-up method (if required)

Current implementation includes engagement and submission ratification events. Additional scopes can be introduced incrementally.

14. Evidence Capture Architecture

Evidence is not “just file uploads”; it is part of chain-of-custody.

14.1 Evidence is orthogonal (K/O/E)​

Evidence (E) can be used:

  • inside Knowledge (K): the subject uploads artefacts about themselves (self-evidence)
  • inside Observation (O): the observer/inspector uploads artefacts about a person/thing being observed
  • as Evidence-first evaluations: a standalone evidence task step

In all cases, evidence is ledger-bound, inherits verification/context policy, and is stored with defensible metadata.

14.2 Flow​

  1. Evidence is captured/uploaded (web or mobile)
  2. File is stored (object storage or filesystem, deployment-dependent)
  3. Metadata is appended to the ledger (who, when, context, linkage)
  4. (future/optional) file hashing stored alongside metadata to enable integrity checks

14.3 Storage separation​

Keep file storage separate from the ledger DB:

  • DB stores metadata + references
  • storage holds bytes

This keeps the ledger lean and queryable.

15. Reporting & Analytics Architecture

Reporting is derived from the ledger and snapshots.

15.1 Pillars​

  1. version_snapshot is canonical for historical reconstruction
  2. Write-time metrics captured at execution
  3. Structured tags frozen at execution time

15.2 Disclosure controls​

Feedback visibility (admin vs operator vs client) is policy-driven and enforced server-side.

15.3 Reporting read models​

  • Flattened reporting tables (for speed and BI exports)
  • Cohort/team analysis
  • Item analytics

15.4 Exports and defensibility​

Exports (CSV/PDF) are derivative artifacts. Where applicable, exports should reference the ledger state they were produced from (snapshot ID / state fingerprint) to preserve defensibility.

15.5 Future integration​

  • xAPI statements
  • LRS integration
  • psychometrics / reliability analysis (domain-dependent)

16. Offline Delivery & Sync (Planned Subsystem)

Evalium supports offline-first patterns where appropriate, without weakening the ledger.

16.1 Goals​

  • Capture offline, submit online (server remains source of truth)
  • Reliable evidence upload lifecycle (resumable, stateful)
  • Policy gating: offline enable/disable enforced server-side (tenant/org/programme/evaluation as configured)

16.2 Key components (planned)​

  • Batch sync endpoint(s): upload queued events/answers efficiently (not chatty replay)
  • Asset lifecycle API: request upload URL, confirm completion, track sync state
  • Policy checks at session creation: prevent offline launches for workflows marked online-only (integrity)

17. Authentication & Authorization (High-Level)

See: docs/security/roles-and-access-control.md

17.1 Authentication​

  • Passwordless magic links for baseline access
  • Supports client portal links
  • Supports (future) step-up auth for high-stakes actions

17.2 Authorization​

Capability-based:

User → Roles → Capabilities → Permissions

Backend enforces permissions. RLS enforces data boundaries regardless of roles.

18. Frontend Surfaces

Evalium typically exposes three primary UI surfaces (deployment may vary):

  1. Admin Console

    • authoring
    • assignments/tasks
    • reporting
    • remediation workflows (void/amend, review)

  2. Operator Execution UI

    • task execution (sessions)
    • evidence capture
    • offline-friendly patterns (future) where needed

  3. Client Portal

    • read-only ledger visibility
    • milestone hash/ratification (future/optional)

Frontend should contain no permission logic beyond hiding UI elements; backend remains authoritative.

19. Non-Functional Requirements (NFRs)

19.1 Performance​

  • Core CRUD and read endpoints should be fast (<100ms typical for indexed reads)
  • Execution UI should feel instantaneous after load
  • Long tasks must not block request threads (use workers)

19.2 Accessibility​

  • WCAG 2.2 AA minimum
  • ATAG 2.0 for authoring tools (accessible authoring UI + support for accessible authored output)
  • Keyboard navigation, ARIA correctness, focus management, robust error messaging

19.3 Reliability​

  • All writes are atomic (TxManager required)
  • Keyed idempotency is required for durable/admin mutating endpoints (Idempotency-Key), with replay-safe semantics on retry
  • Background tasks are idempotent and restart-safe
  • Archive-first for destructive operations (authoring world)
  • No destructive ops in the execution ledger world

Idempotency scope, exemptions, and implementation details: ➡️ docs/architecture/IDEMPOTENCY-AND-RETRY-POLICY.md

19.4 Security​

  • RLS is the ultimate enforcement layer
  • TxManager is the only DB access path (CI guardrail enforced)
  • SET LOCAL only (no SET SESSION)
  • JWT stores role IDs, not capability lists
  • Magic links must be strong, single-use, short-lived

19.5 Observability​

  • Trace/request ID propagation
  • Structured logs for critical ledger events
  • No PII leakage in logs
  • Slow query logging
  • Optional OpenTelemetry

19.6 Scalability​

  • Queries scale linearly via RLS + indexing
  • Nothing assumes single tenant or single org
  • Reporting read models prevent heavy ad-hoc joins over ledger history

19.7 Localization Contract​

  • Localized authored content uses canonical envelope shape: defaultLocale + locales
  • Locale tags are normalized and validated as well-formed BCP-47
  • Passage reads without lang return canonical localized envelope; with lang, content is projected and locale indicates resolved locale/fallback
  • Request payloads may accept plain single-locale content, but persistence/response behavior is normalized to the canonical contract
  • OpenAPI contract guardrail enforces this shape (scripts/check_openapi_localization.sh)

20. Development Workflow (Backend-First Pipeline)

  1. Schema + SQLC queries
  2. Service logic
  3. HTTP handlers
  4. Shell regression scripts
  5. Go integration tests
  6. API cookbook examples
  7. Frontend wiring
  8. Incremental release

This prevents “UI-first” shortcuts that undermine defensibility invariants.

21. Key Implementation Pointers (Current + Planned)

This section maps architectural concepts to code locations where they exist today, and marks new components as planned.

ConceptLocationNotes
TxManagerbackend/internal/pg/tx.goAll DB access goes through TxManager (scoped transactions).
Scope Injectionbackend/internal/pg/scope.goHelpers for tenant/org scope in context.
RLS Policiesbackend/internal/db/schema/Migrations define RLS policies (150/160 series).
SQLC Queriesbackend/internal/db/queries/All queries rely on RLS scope.
Event Dispatcherbackend/internal/services/remediation/dispatcher.goExample event-driven internal messaging.
Event Listenerbackend/internal/services/programme_progress_listener.goExample listener pattern.
Auth Middlewarebackend/internal/http/middleware/JWT parsing and context wiring.
Dev/Test Scope Middlewarebackend/internal/http/middleware/set_scope.goDev/test only scope mapping; must not be enabled in production builds.
Engagement/Project servicebackend/internal/services/engagements_service.goGolden-thread grouping + timeline + hash + ratification (engagement scope).
Hashing/Ratification servicebackend/internal/services/results_service_hashing.goSubmission state hashing + submission ratification.
(Planned) Client Portal projectionTBDRead-only RLS projection layer + UI.
(Planned) Offline sync endpointsTBDBatch sync + asset lifecycle API + policy enforcement at launch.

22. Summary

Evalium’s architecture provides:

  • Strong tenant + org-unit isolation via RLS
  • Immutable, versioned authoring
  • Append-only execution ledger for ledger entries + audit events (amend/void, no destructive edits)
  • Keyed idempotency on durable writes with deterministic retry behavior
  • Deterministic localized-content contracts with BCP-47 normalization
  • Frozen snapshots for defensible reconstruction
  • A “Golden Thread” engagement wrapper for long-running work
  • A client “glass box” portal for transparency
  • Contextual binding + verification level enforcement at write boundaries
  • A universal review/approval layer across K/O/E (policy-driven)
  • A path to milestone hashing and state ratification
  • A planned offline + sync subsystem built around server-truth and robust asset management
  • Backend-first discipline that keeps the system honest under scrutiny

For engineering invariants and the ledger rules: ➡️ docs/architecture/FOUNDATION.md