Skip to main content

Results & Scoring Roadmap

This checklist tracks the backend-first milestones for Evalium’s scoring, feedback, and results experience. Each milestone ships with a dedicated shell test that exercises the relevant API using realistic questions/evaluations so we keep parity with real-world usage.

✅ Foundations (current status)​

  • Section/bucket/passage authoring + preview (backend/tests/evaluations_preview.sh)
  • Bucket dry-run endpoint (backend/tests/evaluations_buckets.sh)
  • Evaluation validation + publish gating (backend/tests/evaluations_validate.sh)

🚧 Milestones​

1. Runtime Session & Submission Model​

  • Add delivery_sessions, submissions, and submission_items tables with timestamps, seed, device/IP metadata.
  • Create ingestion APIs: POST /sessions, POST /sessions/\{id\}/answers, POST /sessions/\{id\}/submit.
  • Shell test: backend/tests/delivery_session.sh simulates a full candidate flow (start session, answer items, submit).

2. Scoring Engine & Configs​

  • Attach scoringConfig, feedbackPolicy, and exposurePolicy fields to evaluation_versions (JSONB or tables).
  • Implement pluggable scorers per question type; “check answer” endpoint uses same engine.
  • Shell test: backend/tests/scoring_check_answer.sh creates an eval in practice mode, runs check-answer, verifies feedback policy.

3. Submission Results API​

  • Compute canonical results on submission (overall, per section, per tag, per item).
  • Expose /submissions/\{id\}/results with stakeholder-aware projections (candidate view vs. admin view).
  • Shell test: backend/tests/submission_results.sh submits answers and asserts scores & tag breakdowns.

4. Feedback Blocks & Routing​

  • Introduce feedback_blocks (evaluation-level, tag-level, question-level) with visibility rules.
  • Wire results API to attach feedback blocks according to policies; add “next step” routing hooks.
  • Shell test: backend/tests/feedback_blocks.sh configures feedback + verifies candidate results.

5. Certificates & Outcome Automation​

  • Add certificate templates + issuance rules to evaluation config.
  • Implement POST /submissions/\{id\}/certificate (auto-triggered on passing results).
  • Shell test: backend/tests/certificates.sh publishes an evaluation with certificate rules, submits passing attempt, validates certificate issuance.

6. Analytics & Psychometrics (stretch)​

  • Persist per-item stats (difficulty, discrimination, distractor counts) derived from submission_items.
  • Expose admin analytics endpoints/dashboards.
  • Shell/ETL test: backend/tests/item_stats.sh seeds historical submissions and validates computed metrics.

Testing Guidelines​

  • Keep each .sh script self-contained: create real questions/evaluations, run the workflow, and clean up resources.
  • Prefer representative data (multiple tags, sections, buckets) so tests approximate real usage.
  • Scripts live under backend/tests/ and double as runnable documentation for the API surface.

This roadmap ensures scoring, feedback, and results evolve systematically while preserving our backend-first contract. The frontend can light up each capability as the corresponding milestone lands.