Skip to main content

📘 UX-OPERATIONS.md

This document specifies the UX patterns for the Operations category. This is the "Live Pulse" of Evalium, where administrative intent (Assignments) meets real-time execution (Command Centre) and human judgement (Grading Queue).

Operations UX must stay calm and actionable for SMBs, while still supporting enterprise-grade workflows via progressive disclosure and capability gating.

Capability Baseline (Validated 2026-02-25)​

This Operations plan includes both live backend capability and target-state UX.

Backend-live now:

  • Assignment issuance, redemption, monitoring, overrides, and schedules are implemented.
  • Delivery session runtime APIs are implemented for create/answer/submit plus event/probe telemetry.
  • Observation review/finding flows are implemented (approve/reject/request-changes/reopen with findings enforcement).
  • Defensibility readiness/exception projections are implemented for submission, engagement, and programme rollups.

Backend gaps to treat as feature-flagged/pending:

  • No first-class persisted assignment draft entity yet (wizard draft is currently a projection/local workflow concern).
  • Explicit first-class proctor command endpoints (pause / resume / terminate) are not yet a normalized backend action surface.
  • A dedicated subjective grading queue projection API is not yet a first-class backend surface.

0. Operations Doctrine (MUST)​

  • On-demand by default: Live monitoring is manual refresh first. Optional auto-refresh is available but off by default.
  • Smallest sufficient context: Tables drive flow; drill-ins happen in drawers; full pages are reserved for primary assets.
  • Ledger principle applies: Session/proctor events, grading/review decisions, and overrides are append-only and must record who/when/why.
  • Operations → Insights handoff is explicit: Live actions must be visible later in durable history (Submission / Assignment Activity).
  • Security is visible: When an assignment/session is configured as high-security (e.g., lockdown), the UI must reflect a High Stakes mode with a persistent Security Level badge.

0.1 Operations Interface Guardrails (MUST)​

  • Assignment monitor labels must remain canonical and deterministic:
    • invited|started|completed|expired -> Not started|In progress|Completed|Expired
  • Queue and monitor actions must be keyboard-complete and touch-usable.
  • Refreshing live tables must preserve focus and selection context.
  • High-stakes actions (override/proctor/review decision) require explicit reason capture and durable audit linkage.
  • Mobile/iPad supports quick triage and monitoring actions; dense investigation flows may be desktop-optimized with explicit handoff.
  • Status meaning must never rely on color only; chips must include textual semantics.

1. Assignments & Schedules​

Assignments define who may take an assessment and under what conditions. For SMBs, the focus is on frictionless issuance with safe defaults.

1.1 The Issue Wizard​

When creating a new assignment, the UI defaults to the simplest path to avoid "enterprise anxiety".

Wizard Steps​

  1. Step 1: Who?
  • Select individual Users, Emails, or Groups.
  1. Step 2: What?
  • Select the Evaluation or Programme to issue.
  1. Step 3: When & Limits (Simple Defaults)
  • When (simple):
    • Start: Now (default) or Schedule start…
    • Deadline: optional (“No deadline” default)
  • Standard Limits toggle (default ON):
    • UI displays the resolved default limits inline (read-only), e.g.:
      • “Standard: 45m, 1 attempt”
    • “Where do standards come from?” link opens a drawer explaining the source (tenant defaults / evaluation defaults).
  1. Step 4: Policy (The "Power Drawer")

  • Security Level Badge (MUST): The drawer header displays a badge derived from the chosen security_config:

    • Standard (default) / Elevated / High Stakes

  • High Stakes Mode (SHOULD): If High Stakes is selected, the wizard enters a visibly distinct mode:

    • Displays a short banner: “High security enabled. Proctoring and audit requirements are stricter.”
    • Shows a compact checklist of implications (e.g., “Lockdown required”, “Stricter integrity flags”).
    • Keeps advanced security fields visible within the drawer (still collapsed by default, but the security subsection is expanded once a high-stakes config is chosen).

  • Advanced Section: collapsed by default; contains:

    • security_config (Lockdown Browser requirements)
    • assignment_overrides (policy exceptions template)
    • specific availability windows (if not covered by simple start/deadline)
    • other advanced issuance rules

Primary / Secondary Actions​

  • Primary CTA: Send Now
  • Secondary CTA: Save Draft (feature-flagged until persisted draft backend state exists)

Until persisted assignment draft state exists, draft behavior must be explicitly labeled as non-authoritative (local/projection), and must not be presented as an immutable backend status.

Trust & Preview (SHOULD)​

Before sending, show a compact summary panel:

  • Recipients count (and sample list)
  • Asset issued (Evaluation/Programme)
  • When (start/deadline)
  • Limits (time/attempts)
  • Security requirements (if any) + Security Level badge
  • Results grouping label (see Run Labels)

If Security Level is High Stakes, the summary must include a short “What this means” hint.

1.2 Run Labels (Results Grouping)​

To help SMBs understand reporting grouping without exposing data-model complexity:

  • The UI auto-generates a run_label (e.g., Compliance_Q1_2025) on send.
  • The label is visible in the wizard summary and Assignment Hub.
  • Provide a safe inline action: Edit label (simple validation).
  • Helper text: “This groups results in Insights.”

1.3 Schedules (Automated Issuance)​

For recurring compliance needs, the UI uses the "Set and Forget" pattern.

Pattern​

A simplified version of the Issue Wizard that includes:

  • Frequency Picker (e.g., “Every 12 months”)
  • Next Runs Preview (SHOULD): show next 3 planned run dates and their generated labels
  • “Last run” and “Next run” indicators for trust

All mutating schedule actions must use idempotency keys and show deterministic retry messaging.

Run Labels (Automated)​

  • The UI generates a run label per run (e.g., Compliance_Q1_2025).
  • Labels are editable (safe) and visible in the schedule detail.

2. The Command Centre (Live Monitoring)​

The Command Centre is a real-time view of active delivery_sessions. It transitions from Operational monitoring (live) to Durable history (completed submissions).

2.1 The Monitor Table​

Refresh Doctrine (MUST)​

  • Default interaction is on-demand refresh via a persistent Refresh button.
  • The UI must display a Freshness Indicator (e.g., “Updated 12s ago”).
  • Optional Auto-refresh is available but OFF by default.
    • When enabled, show cadence options: 10s / 30s / 60s.

Performance Safeguards (MUST)​

  • Auto-refresh pauses when the browser tab is hidden.
  • Auto-refresh pauses when the Session Drill-in drawer is open and the user is interacting, to avoid UI jumping.
  • Prefer “cheap refresh” patterns where supported (ETag/304, cursor-based deltas, or a lightweight summary endpoint).

Glanceable Status (MUST)​

Candidates are shown with status chips. Status taxonomy must be consistent:

  • Live (actively answering / recently active)
  • Idle (no activity for N minutes; define threshold in implementation)
  • Paused
  • Submitted
  • Expired
  • Terminated

Assignment Monitor Bucket Labels (MUST)​

For assignment-target cohort views (for example GET /api/v1/assignments/\{id\}/monitor), frontend labels must use this exact mapping:

  • backend invited -> UI Not started
  • backend started -> UI In progress
  • backend completed -> UI Completed
  • backend expired -> UI Expired

Rules:

  • Do not show backend terms directly in primary UI.
  • Do not mix synonymous labels in one screen (for example Started and In progress together).
  • If a filter chip is shown, it must use the UI label and map back to canonical backend value.

Security Level (SHOULD)​

  • Each row shows a compact Security Level badge (Standard / Elevated / High Stakes), derived from the assignment/session policy.
  • Provide a quick filter: “High Stakes only”.

Pulse Telemetry (SHOULD)​

If a candidate loses browser focus, a pulse indicator appears next to their name.

  • Tooltip shows: “Focus lost X times, last at HH:MM”
  • Clicking opens the Session Drill-in drawer focused on telemetry events.
  • In High Stakes mode, telemetry indicators should be more prominent (without becoming noisy), and tooltips must include counts + last occurrence time.

Error / Stale State (MUST)​

If refresh fails:

  • Show: “Connection issue — last updated Xm ago”
  • Keep Refresh available at all times.
  • Display a Stale badge if data age exceeds a defined threshold (e.g., >2–5 minutes).

2.2 The Session Drill-in (Right Drawer)​

Clicking a candidate opens a side-panel for "Proctor Actions".

Drawer Sections​

  • Summary: candidate, assignment/run label, current section, time remaining (if applicable), Security Level badge
  • Timeline: vertical view of session events (started, section changed, focus lost, reconnects)
  • Telemetry: focus events, suspicious indicators (capability gated)
  • Live Actions: capability-gated proctor controls
    • Current: event-driven controls where backend supports session/proctor event writes.
    • Future: dedicated Pause / Resume / Terminate commands once normalized endpoints are available.

Proctor Notes & Reasons (Ledger UX) (MUST)​

  • High-risk proctor actions should require a reason when supported by backend policy.
  • Lower-risk actions may allow optional notes for speed.
  • All persisted proctor/session events are visible later on:
    • Submission Activity (if a submission exists)
    • Assignment Activity (always)

Security Probe (Progressive Disclosure) (MUST)​

Display security posture in human language first:

  • Lockdown: ✅ Verified / ⚠️ Not detected / ⛔ Tampered (if supported)

Technical signals are revealed only behind “Show technical details” (capability gated), e.g. headers, fingerprint signals.

Example user-facing message:

  • “Lockdown verified” (expand to “SEB Header Present” only for privileged users)

2.3 Live → History Transition (MUST)​

Command Centre must clearly separate:

  • Live Monitoring: delivery_sessions and real-time signals (operational)
  • History: submissions as durable truth (audit/reporting)

When a session reaches a terminal state (Submitted/Expired/Terminated):

  • The row’s primary action becomes View Submission (full page) where available.
  • The drawer includes a durable summary and links to the Submission ledger.

3. The Grading Queue (Subjective Workflow)​

The Grading Queue handles the assessment of Essays, Observations, and Evidence uploads. It follows the Ledger Principle: every grade is a durable record and requires provenance.

This section is target-state. Current backend already supports observation findings + approval transitions, but a dedicated queue/read-model surface should be treated as incremental UX work.

3.1 The Queue Hub​

Queue Item Definition (MUST)​

Default SMB model:

  • A queue item represents a Submission with one or more subjective items pending.
  • The row shows: “2 items pending” and opens the grading view with an item navigator.

Filter-First (MUST)​

Assessors can filter by:

  • Org Unit (hidden globally if tenant has only one org unit)
  • Evaluation / Programme
  • Status
  • “Urgency” (e.g., “Due in 2 days”)

Status Tags (MUST)​

Items are marked as:

  • Awaiting Review
  • In Progress (claimed by a reviewer)
  • Requires Second Opinion

Ownership / Claiming (SHOULD)​

To avoid collisions:

  • Assessors can Claim an item.
  • “In Progress” displays the reviewer.
  • Filters: Assigned to me, Unclaimed.

3.2 The Split-Screen Grading Interface​

To maximise efficiency, the reviewer never leaves the page to see the response.

Layout​

  • Left Pane (The Evidence):
    • Candidate work:
      • Rendered Markdown for essays
      • Document/video player for evidence uploads
  • Right Pane (The Rubric):
    • Displays the versioned Rubric from the version_snapshot
    • Assessors click levels (e.g., Pass / Merit / Distinction) rather than input raw numbers
    • Shows:
      • rubric version scope chip (“Rubric vX” / “Evaluation vY”)
      • Security Level badge (Standard/Elevated/High Stakes)

Item Navigator (MUST for multi-item submissions)​

  • A compact list to switch between subjective items (Essay 1, Evidence 2, etc.)
  • Each item has its own draft/final state.

Draft vs Finalise (MUST)​

  • Reviewers can Save Draft without finalising.
  • Finalise Grade is the high-stakes action.

Mandatory Rationale (Ledger) (MUST)​

Finalising a grade requires a mandatory Rationale comment.

  • Stored as a durable ledger record on the submission.
  • Visible later in Submission Activity / audit views.

Second Opinion (SHOULD)​

If “Requires Second Opinion” is used:

  • The UI must show why it was triggered (manual flag / rule threshold / policy)
  • The second reviewer sees the first reviewer’s rationale (capability gated if needed).

4. High-Stakes Accommodations (Overrides)​

Evalium supports making "at-the-moment" changes to live assignments without re-issuing the assessment.

4.1 Override Drawer​

  • Accessible from:
    • Assignment Hub
    • Command Centre (candidate drill-in drawer)

4.2 Supported Actions​

  • Extend time_limit for a specific user
  • Add an extra_attempt for a specific user

4.3 Preview + Ledger Rules (MUST)​

Before applying, the UI shows a compact preview:

  • “Effective time limit: 60m → 75m”
  • “Attempts: 1 → 2”
  • “Applies immediately”

Reason is mandatory.
Every override is an immutable row in the assignment_overrides table.

The UI must display effective totals:

  • “Effective Limit: 60m (Base) + 15m (Override)”

If Security Level is High Stakes, the Apply flow adds an additional confirmation line:

  • “This accommodation will be recorded for audit and may affect compliance.”

4.4 Default Scope (SHOULD)​

Overrides default to:

  • This assignment run only
  • This user only Optional advanced control:
  • “Valid until” (time-bound override)

5. Summary of Navigation Contracts​

Operational TaskPatternPrimary CTALink Contract (Peek + Full Page)
Issue TestWizardSend NowUser Profile (Who)
Monitor SessionsTable + DrawerRefreshSession Timeline & Telemetry
Grade Submission (Future)Split-ScreenFinalise GradeRubric Definition (version-scoped)
Apply OverridesDrawerApply ChangeOverrides Ledger (and link to impacted Submission)
Triage DefensibilityQueue Table + RollupsAcknowledge / Suppress / ResolveSubmission/Engagement/Programme details

6. Operations → Insights Handoff (MUST)​

To preserve trust and auditability, operational actions must surface in durable history and derived readiness views:

  • Session/Proctor Events (including pause/resume/terminate once dedicated commands exist)

    • Visible in Assignment Activity
    • Visible in Submission Activity when a submission exists

  • Grading Finalisation

    • Rationale stored as a submission ledger record
    • Visible in Submission Activity and audit exports

  • Overrides

    • Visible in Assignment Activity
    • Effective totals displayed in Submission summary (where relevant)

  • Defensibility State

    • Submission outcomes must flow into proof-readiness/KOE projections.
    • Exceptions should appear in defensibility queues (submission, engagement, programme requirement) for triage.

7. Evidence Integrity Operations (MUST)​

Operations must expose a first-class evidence workflow for review teams without forcing them into raw ledger JSON.

7.1 Evidence Action Surface​

Submission-scope endpoints:

  • POST /api/v1/submissions/\{id\}/evidence/record
  • POST /api/v1/submissions/\{id\}/evidence/amend
  • POST /api/v1/submissions/\{id\}/evidence/approve
  • POST /api/v1/submissions/\{id\}/evidence/reject
  • POST /api/v1/submissions/\{id\}/evidence/verify
  • POST /api/v1/submissions/\{id\}/evidence/void

Item-scope endpoints:

  • POST /api/v1/submissions/items/\{itemId\}/evidence/record
  • POST /api/v1/submissions/items/\{itemId\}/evidence/amend
  • POST /api/v1/submissions/items/\{itemId\}/evidence/approve
  • POST /api/v1/submissions/items/\{itemId\}/evidence/reject
  • POST /api/v1/submissions/items/\{itemId\}/evidence/verify
  • POST /api/v1/submissions/items/\{itemId\}/evidence/void

7.2 Evidence Panel UX​

Required elements:

  • scope switch (Submission vs Item)
  • integrity indicator (queued / verified / failed / unknown)
  • latest decision indicator (approved / rejected / voided / none)
  • action rail with explicit idempotent retry behavior

Rules:

  • all mutating evidence actions include Idempotency-Key.
  • verify action is async intent; UI must show queued state and refresh affordance.
  • amend/void actions require explicit reason capture and show lineage to prior event when available.

7.3 Error and Safety Behavior​

  • 409 conflicts must be rendered as state drift, not generic failure.
  • 404 should be presented as scope-hidden or missing evidence target.
  • evidence action failures must not clear currently visible evidence history.

8. Programme and Enrolment Operations (MUST)​

Operations must include a programme-focused control plane because programme progress and defensibility rollups are now backend-live.

8.1 Programme Hub Screens​

Required screens:

  1. Programme list and detail
  2. Programme requirements manager
  3. Enrolment list and enrolment detail
  4. Per-enrolment progress view
  5. User cross-programme enrolments view

Primary APIs:

  • GET/POST /api/v1/programmes
  • GET/PATCH/DELETE /api/v1/programmes/\{id\}
  • POST /api/v1/programmes/\{id\}/requirements
  • PATCH/DELETE /api/v1/programmes/\{id\}/requirements/\{reqId\}
  • GET/POST /api/v1/programmes/\{id\}/enrolments
  • GET /api/v1/programmes/\{id\}/enrolments/\{enrolId\}
  • GET /api/v1/programmes/\{id\}/enrolments/\{enrolId\}/progress
  • GET /api/v1/programmes/\{id\}/enrolments/user/\{userId\}
  • GET /api/v1/programmes/enrolments/user/\{userId\}

8.2 Required Programme UX Blocks​

  • requirement list with evaluationVersionId visibility
  • enrolment status card (in_progress, completed, certified, etc.)
  • requirement progress rows with linked submissionId when present
  • engagement linkage visibility where engagementId exists
  • rollup handoff link to programme-requirement defensibility exceptions

8.3 Certificate Verification Entry​

Public/operations verification route:

  • GET /api/v1/certs/verify/\{token\}

UX behavior:

  • provide a lightweight verify screen with clear valid/invalid state.
  • do not expose internal identifiers when verification fails.

9. Accessibility & Performance Notes (Operations)​

  • All tables, drawers, and split-screen grading interfaces must support keyboard-only use.
  • Drawer focus management and escape behaviour is consistent across Operations.
  • Monitor table refresh must not cause focus loss or disorienting reflows.
  • Grading actions must be resilient to network interruption (draft autosave where feasible).
  • Operations actions follow task-tier policy: quick monitor/triage tasks are mobile-safe, dense investigation screens are desktop-optimized with explicit handoff.

reference:

  • /docs/ux/UX-ACCESSIBILITY-STANDARDS-AND-GUARDRAILS.md
  • /docs/ux/UX-MOBILE-AND-TASK-TIER-POLICY.md