Skip to main content

Evalium Skills & Competency Engine (Backend Design + BFF API Ideas)

1) Purpose and scope​

Build a flexible, audit-proof Skills/Competency Engine that can:

  • infer “competence” from immutable submission snapshots (Knowledge now; Observation/Evidence later),
  • support retroactive mapping + recalculation when SMBs start tagging late,
  • provide UI-ready read models so the frontend doesn’t do heavy lifting.

This stays aligned with the KOE concept: competence is a view layer over Evaluations + Programmes, not a separate HR module.

2) Architectural constraints we should lean on (already in Evalium)​

Security & scoping​

  • PostgreSQL RLS is the security boundary, set via TxManager SET LOCAL GUCs (tenant_id, org_unit_id, org_scope_all).
  • All DB access must go through TxManager; no pool-level queries.

Audit-proof measurement​

  • Everything versioned and immutable, and version_snapshot is canonical for historic reporting: never re-join live authoring tables when interpreting past attempts.
  • Submissions freeze tags, scoring settings, navigation, evaluation metadata, org scope, etc.

Known gap we can “pull forward”​

  • Aggregated reporting is not implemented yet and is planned via projection/materialized view layer. The Skills engine can be the first “real” projection consumer.

3) Core model (hybrid + retroactive mapping)​

Key objects​

  1. Skill A global capability you track (“Breathing”, “Safe Lifting”, “Medication Handling”).

  2. Competency-trusted tagging (governed) Separate from free author tags. Your matrix already notes tag conventions exist but validation is light—this is where you add guardrails.

  3. Mapping Set (versioned) A governed “interpretation lens” that defines how Skills are inferred from evidence. Crucially: this is how you support “we forgot to tag; recalc old data” without mutating immutable versions/snapshots.

  4. Evidence Facts (projection) A flattened, query-friendly table generated from submission snapshots + mapping sets (and later observation/evidence states).

4) Data model proposal (backend-focused)​

This is intentionally aligned with Evalium’s versioning philosophy.

A) Skills​

  • skills (identity)

  • skill_versions (immutable definition)

    • name, description, optional level scheme, status (draft/published)
    • (optional) “framework affinity” metadata if you want domain grouping

B) Competency tag dictionary (trusted taxonomy)​

  • competency_tag_keys (e.g., skill, level, framework)
  • competency_tag_values (e.g., breathing, safe_lifting, level:3, framework:healthcare_2005)
  • This enables permission-gated creation/editing of trusted values while still allowing free tags on questions.

C) Mapping sets (the hybrid selector engine)​

  • skill_mapping_sets (identity)

  • skill_mapping_set_versions (immutable ruleset)

  • skill_mapping_rules (rows under a version)

    • rule_type enum:

      • tag_predicate (matches frozen structured tags in version_snapshot)
      • evaluation / evaluation_version (strictness knob)
      • question_version (precision backfill knob)
      • framework (stable governance knob; recommended)

    • selector_json (predicate payload; keep it explicit and parseable)

    • target_skill_id, contribution strategy (“score threshold”, “average”, “best-of N”, “pass/fail”, etc.)

    • precedence: specific beats general (question_version > evaluation_version > evaluation/framework > tag_predicate)

D) Projection tables (read models)​

You’ll likely want two layers:

  1. Skill Evidence Facts (atomic, auditable)
  • keyed by: tenant/org/user/submission + skill + “evidence unit” (item/tag aggregation)
  • fields like: contribution value, pass/fail, weight, evidence source (K/O/E), mapping_set_version_id, computed_at
  1. Skill Rollups (UI fast path)

  • per user per skill:

    • status (not started / in progress / achieved)
    • last updated time
    • confidence/coverage
    • last evidence summary

  • per cohort/team:

    • distribution counts and gap lists

These sit naturally under the “reporting projection layer” you already planned.

E) Jobs (recalc/backfill)​

A small job table similar in spirit to your existing worker patterns (reaper/privacy/remediation).

  • skill_projection_jobs: type recalc, date range, mapping_set_version_id, org scope, status/progress
  • store “result summary” for UX (rows affected, coverage delta, etc.)

5) Projection & consistency rules​

When to (re)compute​

  1. On submission finalisation (new evidence arrives)
  • parse version_snapshot (canonical) and generate facts for active mapping set versions.
  1. On remediation apply/revert
  • remediation can change scores/outcomes; derived skill facts must be updated accordingly. Your remediation is idempotent and score history is append-only—good foundation for deterministic reproject.
  1. On mapping set publish

  • triggers a targeted backfill job:

    • “recalc last 12 months”
    • “recalc for evaluation X”
    • “recalc for org unit Y”

  • this is the main SMB “we started late” unlock.

Audit posture​

Every derived record should reference:

  • submission_id (and optionally submission_score_version_id)
  • mapping_set_version_id
  • “evidence basis” (which selector/rule matched)

That preserves defensibility without rewriting immutable snapshots.

6) Backend-for-Frontend API ideas (so UI does minimal work)​

Your frontend shouldn’t be joining tables, computing rollups, or implementing rule logic. The backend should ship view-model endpoints.

Below are BFF-style endpoints (names are illustrative) with the shape the UI needs.

A) Skills Library (admin view)​

GET /api/v1/skills Returns a UI-ready list:

  • skill name, status, last computed
  • evidence coverage (% of submissions/items mapped)
  • “needs recalculation” flag (mapping published since last compute)
  • counts: users covered, users achieved, users missing evidence

Why: this becomes the Skills landing page without extra queries.

B) Skill detail (admin view)​

GET /api/v1/skills/{skillId}

  • skill definition (current version)
  • active mapping set version summary
  • coverage breakdown by evidence type (K/O/E) (future-proof to KOE)
  • top cohorts/org units impacted

GET /api/v1/skills/{skillId}/evidence-sources

  • resolved mapping rules in priority order
  • “effective selector” preview (human readable + machine form)

C) Mapping set editor (governed workflow)​

POST /api/v1/skill-mapping-sets/\{id\}/versions (create draft version)

POST /api/v1/skill-mapping-sets/\{id\}/versions/\{versionId\}/validate Return:

  • rule validation errors (unknown competency tags, invalid selectors, conflicting rules)
  • warnings (broad tag predicate may overcount; missing scope in regulated mode)

This aligns with your existing pre-publish validation pattern on evaluations.

POST /api/v1/skill-mapping-sets/\{id\}/versions/\{versionId\}/impact-preview Return a “dry run” summary so the UI can show:

  • submissions affected
  • users newly “achieved”
  • users who lose evidence (if you ever allow downgrades)
  • coverage delta

(Your feature matrix already calls out “impact preview” as a deferred-but-known concept in other workflows; this is a similar UX value.)

POST /api/v1/skill-mapping-sets/\{id\}/versions/\{versionId\}/publish

  • publishes the mapping version and triggers recalc job options.

D) Recalculation jobs (progress UX)​

POST /api/v1/skills/recalculate Body includes:

  • scope: org unit(s), evaluation/evaluation_version, date range, mapping_set_version Returns:
  • job id + estimated affected partitions (no heavy client polling logic)

GET /api/v1/skills/recalculate/jobs/{jobId}

  • status, progress, summary metrics

E) Person competence profile (consumption view)​

GET /api/v1/users/\{userId\}/skills-profile Return a complete profile card model:

  • skill list (status, last evidence date, score/confidence/coverage)
  • “missing evidence” reasons (e.g., “no mapped items in last 12 months”)
  • quick drill links (submissions that contributed)

This matches the KOE doc’s “competence profile view” intent.

GET /api/v1/users/\{userId\}/skills-profile/{skillId}/evidence

  • timeline of evidence facts with friendly labels (evaluation title, run_label, etc.) Assignments already carry run_label to support cohort reporting semantics.

F) Cohort/team skill overview (later, but design now)​

GET /api/v1/skills/overview with filters:

  • org unit, group/cohort, date window, job role, etc. Returns:
  • heatmap-ready matrix (skill × group) with counts + percentages
  • “top gaps” list

Your reporting matrix explicitly ties cohort/group views to flattened reporting tables—this endpoint is the BFF face of that.

G) authoring helpers (so authors see “competency coverage” instantly)​

GET /api/v1/evaluations/\{id\}/versions/\{versionId\}/competency-coverage Return:

  • % items with competency-trusted tags
  • which skills appear (from tags)
  • warnings: missing competency tags for items that otherwise have free-form tags

This keeps authoring UX “smart” without pushing logic into the client (and matches your “workflow intelligence in authoring” goal).

7) Permissions & enforcement​

Introduce capabilities consistent with your existing model (atomic caps + Require(...) per route).
Suggested caps:

  • skills.read (view profiles/rollups)
  • skills.manage (create/edit skill definitions)
  • skills.mapping.manage (create/publish mapping sets)
  • skills.taxonomy.manage (manage competency tag dictionary)

And keep org scoping consistent with the platform’s silo rules.

8) Compliance & retention (don’t accidentally break your GDPR posture)​

You already have strong compliance workflows (forget/hybrid scrub, DSAR export, retention incidents). Derived skill facts are derived personal data, so:

  • forget/retention jobs must also remove/anonymise rows in skill_evidence_facts and rollups,
  • DSAR export should include competence profile (or explicitly omit it by policy).

This is important because your compliance subsystem is already “real” (jobs + ledger + worker + receipts).

9) Testing strategy (backend confidence without UI)​

  • Invariant tests: computing skills from version_snapshot does not query live authoring tables.
  • RLS tests: skill projections respect tenant/org scope the same way submissions/reporting do.
  • Remediation propagation tests: apply/revert updates skill facts deterministically (idempotency-friendly).
  • Job idempotency tests: repeated recalc with same parameters results in stable outputs.