Skip to main content

Evaluation Deletion, Archival, and Results Snapshots

This plan aligns deletion/archival behaviour with Evalium’s assessment lifecycle and moves us to a snapshot-based results model (Option B). Goal: safe cleanup for authors while preserving submitted results and keeping runtime/session data decoupled from authoring tables.

Current Pain​

  • DELETE /evaluations/{id} fails (409) once submissions/sessions exist because submissions and delivery_sessions FK evaluation_versions.
  • Runtime results depend on live authoring tables, so hard delete would orphan data.
  • Tests/scripts rely on DB truncation to clean up because the API can’t delete “in-use” evaluations.

Principles​

  • authoring (evaluations/versions/sections/items/buckets) is immutable once used.
  • Sessions are runtime artefacts; submissions/results are long-lived.
  • Deletion with existing results should default to archive, not cascade.
  • Reporting must survive authoring edits/deletes via snapshots.

Target Behaviour​

  • DELETE /evaluations/{id}: archive when sessions/submissions/assignments exist; hard-delete only if no runtime or results exist.
  • Archive: set status to archived/inactive, close active sessions, block new sessions/assignments.
  • Submissions: store a frozen snapshot of the evaluation version (sections/items/questions/weights/answers). Results rendering reads snapshot, not live authoring tables.
  • Usage visibility: provide counts of submissions/active sessions/assignments in responses (409 or dedicated usage endpoint) to guide UX.

Data Model Changes​

  • Add submissions.version_snapshot jsonb (or a reporting table) containing:
    • section order, items, navigation, time limits
    • question stems/options/answers, weights/scoring config
    • bucket resolutions if applicable
  • Keep evaluation_version_id for traceability, but results rendering uses the snapshot.
  • Ensure evaluation_versions includes attempt constraints (max_attempts_per_user, cooloff_seconds) as schema columns (already added).

API/Behaviour Changes​

  • DELETE /evaluations/{id}: archive-by-default when in use; hard delete only when clean.
  • GET /evaluations/{id}/usage (or enriched 409 body): { submissions, activeSessions, assignments }.
  • Session creation: auto-close/deny when evaluation is archived/disabled.
  • Submission creation: populate version_snapshot at submit time (using resolved evaluation graph).

Roadmap​

  • Phase 1 (now):
    • Ship snapshot on submission; wire results rendering to snapshot.
    • Add usage info to delete conflicts; keep delete-as-archive semantics when in use.
    • Implementation plan:
      • Schema/sqlc: add submissions.version_snapshot jsonb.
      • ResultsService: in SubmitSession, resolve evaluation graph (version → sections → items → question versions) and persist into version_snapshot.
      • EvaluationService delete: return structured conflict/usage data and archive when in use; only hard-delete when clean.
      • Session gating: deny session creation when evaluation is archived/disabled.
      • Usage endpoint: expose GET /evaluations/{id}/usage (or enrich 409) with counts of submissions, active sessions, assignments placeholder.
      • Results rendering: prefer version_snapshot when present; fallback to live authoring only if snapshot missing.
      • Tests: integration coverage for snapshot presence, delete-conflict usage payload, and snapshot-backed results (use DB truncation for cleanup).
  • Phase 2: Enforce archive-only for in-use evals; auto-close active sessions on archive; block new sessions/assignments when archived/disabled.
  • Phase 3: Add explicit assignments/schedules layer to gate session creation. Add retention knobs (auto-purge sessions; keep submissions).
  • Phase 4: Reporting projection/export (optional): denormalize submissions into reporting_submissions or warehouse feed for analytics.

Testing Plan​

  • Extend backend/tests/delivery_session.sh and Go integration tests to:
    • submit a session, assert version_snapshot present
    • attempt delete, expect archive/409 with usage data
    • archive eval, confirm sessions closed and new session creation blocked
  • Keep DB truncation for test isolation; no cascade deletes in APIs.

UX Guidance​

  • On delete: show counts and offer “Archive instead”. Only allow hard delete when clean.
  • On archive: show that sessions are closed and new attempts blocked; results remain viewable.
  • Results pages load from snapshot to ensure fidelity even after authoring changes or delete.

Why This Matches Evalium​

  • SMB-friendly: safe defaults, clear UX, minimal data loss risk.
  • Audit-friendly: results survive authoring changes/deletes via snapshot.
  • Scalable: decouples runtime/reporting from authoring tables; avoids FK-driven lock-in.