Skip to main content

Checklist: Evaluations API (MVP)

This checklist outlines the step-by-step implementation of the MVP for the Evaluations API. Each step builds upon the last and includes the creation of a corresponding .sh file to test the new functionality.

Step 0: Preparation & Refactoring​

  • Task: Remove the //go:build ignore tag from backend/internal/http/handler/evaluation.go to enable the API.
  • Task: Refactor the existing evaluation.go code. Create a new services/evaluation_service.go and repositories/evaluation_repository.go, following the same clean architecture as the Questions feature.
  • Test: The backend server should compile and run without errors after this refactoring.

Step 1: Core Evaluation CRUD​

  • Task: Implement the basic Create, Read, Update, Delete endpoints for the parent evaluations object.
    • POST /api/v1/evaluations
    • GET /api/v1/evaluations
    • GET /api/v1/evaluations/\{id\}
    • PATCH /api/v1/evaluations/\{id\}
    • DELETE /api/v1/evaluations/\{id\}
  • Test Script: Create backend/evaluations_crud.sh to test these core endpoints.

Step 2: Evaluation Versioning​

  • Task: Implement endpoints to manage versions of an evaluation. The evaluation's actual content will be tied to a version.
    • POST /api/v1/evaluations/\{id\}/versions
    • GET /api/v1/evaluations/\{id\}/versions
    • POST /api/v1/evaluations/\{id\}/publish (to set the active version).
  • Test Script: Extend backend/evaluations_crud.sh to test creating and publishing versions.

Step 3: Evaluation Sections​

  • Task: Implement endpoints to add, view, and manage ordered sections within an evaluation version.
    • POST /api/v1/evaluation-versions/\{versionId\}/sections
    • GET /api/v1/evaluation-versions/\{versionId\}/sections
    • PATCH /api/v1/sections/{sectionId}
    • DELETE /api/v1/sections/{sectionId}
  • Test Script: Create backend/evaluations_sections.sh to test section management.

Step 4: Adding Questions to Sections​

  • Task: Implement the ability to add specific questions to a section.
    • POST /api/v1/sections/{sectionId}/items (to add a question_version_id to a section).
    • GET /api/v1/sections/{sectionId}/items
    • PATCH /api/v1/items/\{itemId\} (to reorder).
    • DELETE /api/v1/items/\{itemId\}.
  • Test Script: Extend backend/evaluations_sections.sh to test adding and reordering questions.

Step 5: Smart Buckets​

  • Task: Implement the backend logic for creating and managing "smart buckets".
    • POST /api/v1/buckets (to create a bucket with tag filters, selection mode, etc.).
    • GET /api/v1/buckets/{bucketId}
    • PATCH /api/v1/buckets/{bucketId}
    • Update POST /api/v1/sections/{sectionId}/items to also support adding a bucket_id.
  • Test Script: Create backend/evaluations_buckets.sh to test creating buckets and adding them to sections.

Step 6: Evaluation Preview Endpoint​

  • Task: Implement the POST /api/v1/evaluations/\{id\}/preview endpoint. This is a crucial, read-only endpoint that will resolve all the smart buckets into a concrete list of questions, simulating what a candidate would see.
  • Test Script: Create backend/tests/evaluations_preview.sh to test that the preview logic correctly assembles an evaluation based on its rules.

Step 7: Passage Blocks & Bucket Dry Run​

  • Task: Expose passage CRUD plus /passages/\{id\}/questions so sections can add passage blocks that bring their ordered question list.
  • Task: Add POST /api/v1/buckets/\{id\}/preview to dry-run a bucket draw (seeded, warnings/shortfall surfaced).
  • Test Script: Extend backend/tests/evaluations_preview.sh and evaluations_buckets.sh to cover passage items and bucket dry-run responses.

Step 8: Evaluation Preflight Validation​

  • Task: Add POST /api/v1/evaluations/\{id\}/validate to aggregate bucket/passage warnings before publish and return a structured report.
  • Task: Block POST /publish unless validation returns zero warnings (422 with warning list).
  • Test Script: Create backend/tests/evaluations_validate.sh to exercise the validation endpoint end-to-end.