6432 lines
505 KiB
Markdown
6432 lines
505 KiB
Markdown
# 03 — Database Design
|
||
|
||
## Status / Scope of this document
|
||
|
||
**Status:** Design, ready to implement. This is the physical data model for the greenfield
|
||
Utopia Brands TalentFlow backend. It is binding for migration authors and for anyone writing
|
||
a query against this database.
|
||
|
||
**Scope.** The complete PostgreSQL schema: engine choice, modelling principles, identifier and
|
||
convention rules, 21 business areas with per-table documentation to the assignment §11 standard,
|
||
search design, query patterns, PII and retention policy, soft-delete and audit policy, and an
|
||
ordered migration sequence.
|
||
|
||
**What this document is built from.** Three sources and nothing else:
|
||
|
||
| Source | Role here |
|
||
|---|---|
|
||
| `docs/architecture/_repo-findings.md` | Verified repository facts. Cited inline as `js/data.js:123` etc. |
|
||
| `docs/architecture/_decisions.md` Part 2 | Binding database decisions. This document is the implementation of Part 2, not a re-litigation of it. |
|
||
| `docs/TalentFlow-ATS-Business-Requirements-v1.0.docx` | The requirements source. Cited as BRD §n. |
|
||
|
||
**There is no meeting transcript.** None exists in the repository or on disk
|
||
(`_repo-findings.md` §B). The BRD plus the assignment prompt are the authoritative requirements
|
||
source, and where neither settles a question this document states an **assumption** in bold.
|
||
|
||
**What does not exist yet.** No backend, no database, no migration directory, no ORM, no driver,
|
||
no `.env`, no Docker, no tests (`_repo-findings.md` §B). Every table below is new. The only
|
||
assets carried forward are `css/styles.css`, `js/ui.js` and `js/charts.js` (§G), which are a
|
||
rendering concern and appear here only as the reason certain display columns exist
|
||
(`order_index`, `colour_token`, `label`).
|
||
|
||
**Deviations.** Section 30 lists every place this document consolidated, split or renamed a
|
||
table the assignment named, with the relational reasoning. Section 32 lists the points where
|
||
`_decisions.md` Part 1 and Part 2 need explicit reconciliation.
|
||
|
||
**Reading order for implementers.** Sections 1–5 are the rules; you cannot correctly read the
|
||
per-table sections without them. Sections 6–26 are the schema. Section 33 is the migration order
|
||
and is the file you actually work from.
|
||
|
||
---
|
||
|
||
## 1. Database choice and justification
|
||
|
||
**Decision: one managed PostgreSQL 16+ instance (target 17), one logical database, five
|
||
application schemas plus the library-owned `queue` schema.**
|
||
This restates `_decisions.md` Part 2 and adds the physical layout. The five schemas this document
|
||
designs are `ref`, `app`, `ai`, `audit` and `staging`; `queue` is created and owned by
|
||
`procrastinate`'s own migrations and is listed in §1.1 only so nobody assumes it appears by
|
||
accident.
|
||
|
||
The repository dictates nothing — there is no driver, ORM, migration directory, Dockerfile or
|
||
env file (`_repo-findings.md` §B) — so this is a free capability-fit choice, and it was made by
|
||
listing the hard requirements and asking which engine expresses each one *in the database* rather
|
||
than in application code.
|
||
|
||
| Requirement (source) | PostgreSQL feature that satisfies it | What we would write instead without it |
|
||
|---|---|---|
|
||
| Raw email/webhook envelopes and parser output stored losslessly (BRD §8.1) | `jsonb` + GIN | A blob column and a parallel parse table |
|
||
| Candidate free-text search with ranking (BRD §5.2) | `tsvector`, `ts_rank_cd`, GIN | LIKE scans or a search cluster |
|
||
| Typo-tolerant name/employer matching, shared with duplicate detection | `pg_trgm` GIN, `similarity()` | An in-memory fuzzy match in the app |
|
||
| Exact money across 6+ currencies (BRD §9.1 `salary`) | `numeric(14,2)` + FK to `ref.currency` | Float rounding, or integer minor units everyone must remember to divide |
|
||
| Interview double-booking prevention (BRD §5, interviews) | `tstzrange` + `EXCLUDE USING gist` (`btree_gist`) | An application-level check that races |
|
||
| Raw-intake-before-candidate as a physical law | `NOT NULL` FK, non-deferrable | A service-layer convention that imports bypass |
|
||
| "At least one contact channel" across two tables | `DEFERRABLE INITIALLY DEFERRED` constraint trigger | Nothing — a CHECK cannot span tables |
|
||
| Soft-delete-tolerant uniqueness, merge-tolerant uniqueness | Partial unique indexes with `WHERE` | Application-side uniqueness, i.e. none |
|
||
| Append-only score and audit tables | Column-level `GRANT`, `REVOKE UPDATE` | A trigger allowlist that can be wrong |
|
||
| Audit growth to tens of millions of rows | Declarative `PARTITION BY RANGE` | A mass `DELETE` and an autovacuum problem |
|
||
| Semantic matching in Phase 2 without a new datastore (constraint: no separate AI service) | `pgvector` + HNSW in the same database | A vector database and a second consistency problem |
|
||
| Later chatbot isolation without new infrastructure | Row-level security, column-level privileges | Prompt engineering as an access control |
|
||
|
||
Every row lands in one engine. That is the argument: not "Postgres is good", but that the
|
||
load-bearing invariants of this design are *expressible* here and would otherwise all migrate
|
||
into application code — which is precisely the failure mode the prototype already demonstrates,
|
||
where a random integer stands in for a score (`js/data.js:123`) and an RBAC matrix gates nothing
|
||
(`js/rbac.js:78`).
|
||
|
||
**Rejected.** MySQL 8 — no trigram similarity, no partial indexes, no exclusion constraints, no
|
||
deferrable constraint triggers, no vector type; duplicate detection and the scheduling invariant
|
||
would all become application code. MongoDB or any document store — the entire brief is relational
|
||
invariants (raw intake before candidate, per-application scores, reversible merge, version
|
||
pinning); in a document store those become conventions. SQL Server — licence cost, no capability
|
||
gain. SQLite — no partitioning, weak concurrency. Multiple or per-region databases — forbidden by
|
||
constraint and unjustified at 66 named seats (BRD §4). Elasticsearch or a vector DB in Phase 1 —
|
||
forbidden, and absurd against a prototype holding 100 generated rows (`js/data.js:112`).
|
||
|
||
### 1.1 Physical layout
|
||
|
||
| Schema | Contents | Grant posture for the application role |
|
||
|---|---|---|
|
||
| `ref` | Reference and controlled-vocabulary tables. Slow-changing, small, cached. | `SELECT` only; writes via the admin/migration role |
|
||
| `app` | All operational domain tables. | `SELECT, INSERT, UPDATE, DELETE` except where narrowed per table |
|
||
| `ai` | AI capability registry, prompt/model versions, run ledger, suggestions, conversations. | `SELECT, INSERT`; `UPDATE` only on review columns |
|
||
| `audit` | `audit_event` (partitioned) and governance tables. | `INSERT, SELECT` only. `UPDATE`, `DELETE`, `TRUNCATE` revoked |
|
||
| `staging` | Bulk-import landing tables and non-production anonymisation scratch. Never read by the application. | Per-job role only |
|
||
| `queue` | Owned by the `procrastinate` library (`_decisions.md` Part 1: Postgres-backed queue). **Not designed in this document** — it is created by the library's own migrations. | Library-managed |
|
||
|
||
Extensions, Phase 1: `pg_trgm`, `unaccent`, `btree_gist`, `pgcrypto`. Phase 2: `pgvector`, and
|
||
`pg_partman` or a small scheduled SQL function for audit partitions. Server and application role
|
||
set `timezone = 'UTC'`. Managed hosting with PITR and automated backups; no self-managed
|
||
Postgres.
|
||
|
||
Separate roles, not one. The three that carry the argument: `talentflow_migrate` (owns DDL),
|
||
`talentflow_app` (the runtime role, grants per the table above), `talentflow_readonly` (analytics and
|
||
ad-hoc psql). The separation is what
|
||
makes the append-only guarantees on `audit.audit_event`, `app.ats_result` and `app.job_version`
|
||
real rather than aspirational — the runtime role physically lacks the privilege. Three more exist for
|
||
single narrow purposes and hold nothing else — `talentflow_sealer` (the audit chain, §28.1),
|
||
`talentflow_redactor` (the two-person audit redaction path, §28.2) and `talentflow_worker` (the
|
||
scoring job, which deliberately cannot write a review verdict or an override, §20.4, §20.7) — and
|
||
three read-only roles are provisioned inert for Phase 2+ defence in depth (§28.6). Migration `001`
|
||
creates all nine (§33).
|
||
|
||
---
|
||
|
||
## 2. Modelling principles
|
||
|
||
Nine rules. Each one closes a specific verified gap in the prototype rather than being generic
|
||
good practice, and every table below is checkable against them.
|
||
|
||
| # | Principle | Why (evidence) |
|
||
|---|---|---|
|
||
| P1 | **Identity is separate from participation.** A person (`candidate`) exists independently of any job they applied to (`job_application`). Nothing about an application ever lives on the candidate. | One flat array carries `jobId`, `stage`, `aiScore` and `recruiter` on the candidate (`js/data.js:117-127`), so one person cannot hold two applications. Highest-value correction in the model. |
|
||
| P2 | **Raw arrival precedes identity.** Nothing enters the system as a candidate. Everything enters as a `raw_intake` row, and `candidate.created_from_raw_intake_id` / `job_application.raw_intake_id` are `NOT NULL` against non-deferrable FKs. | The prototype inbox is pre-resolved — each row already carries name, email, `jobId`, `atsScore` and recruiter (`js/data.js:284-300`) — so "arrived but cannot become a candidate" has no representation. |
|
||
| P3 | **Decisions that drive numbers are immutable versions.** Jobs, requirements, scoring configs, pipeline configs, prompt templates, scorecard and assessment templates, offers: entity row + immutable version rows + a `current_version_id` pointer. Downstream rows pin the version they used. | Jobs are mutable single records (`js/data.js:85-108`); a requirement edit today silently rewrites the inputs of every score already computed. |
|
||
| P4 | **Derived results are append-only snapshots.** A score, a parse, an AI run, a sent message is never updated. Recompute inserts a new row and marks the old one superseded. | `aiScore: int(52,98)` (`js/data.js:123`) has no components, evidence, model or version; there is nothing to preserve and no way to reproduce it. |
|
||
| P5 | **Current state is a denormalised column; history is a typed table.** Both exist. History rows are written by a database trigger, carry `valid_from`/`valid_to` intervals, and are protected from overlap by a GiST `EXCLUDE`. | There are no history tables at all (`_repo-findings.md` §F). Deriving current state from history on every list read would be the wrong default for the 23 list screens (`js/app.js:7-16`). |
|
||
| P6 | **Enumerable things are rows; unstable things are JSONB; nothing enforceable depends on JSONB.** Promotion out of JSONB is one-way and happens on write, and the original JSONB is never edited. | Stage lists, statuses, sources and a skills pool are hardcoded JS arrays (`js/data.js:20-46`); all are things a recruiting lead will change without a deploy. |
|
||
| P7 | **Polymorphism is expressed as typed nullable columns plus a CHECK, never as `(subject_type, subject_id)`.** A polymorphic FK is unenforceable by the database, which is the exact class of defect this model exists to eliminate. | Consistent with `_decisions.md`'s rejection of a single polymorphic assignment table. Applied to `access_scope`, `approval_request`, `ai_suggestion`, `task`, `note` subjects. |
|
||
| P8 | **Erasure is pseudonymisation, not deletion.** Retention purges replace personal columns with deterministic tokens and delete blobs; skeleton rows, scores and history survive. | The brief requires permanent history and BRD §7.4 requires honouring deletion requests. Row deletion would tear holes in funnel metrics, break FKs from audit and history, and make merge reversal unreplayable. |
|
||
| P9 | **Every invariant that can be a constraint is a constraint.** Regexes, sums, ranges, exclusivity, ordering and privilege are enforced in the database. Application validation is a UX affordance layered on top, never the guarantee. | Imports, integrations and psql fixes will all happen in the first month, and none of them run the application's validators. |
|
||
|
||
Two rules about the rules:
|
||
|
||
- **P6 has teeth.** No CHECK, unique index, FK, generated column or partial-index predicate may
|
||
reference JSONB content anywhere in this schema. If a field needs enforcement, it gets promoted
|
||
to a column. Grep for `->>` inside a constraint definition is a valid CI gate.
|
||
- **P9 has a documented cost.** The volume of PL/pgSQL here (history triggers, deferrable
|
||
contactability and weight-sum triggers, immutability triggers, audit hash chaining, search index
|
||
maintenance) is a real skills risk for a two-person team with a junior. `_decisions.md` already
|
||
assigns it: **Talha owns all trigger and constraint code; Ahmed Mujtaba owns migrations,
|
||
reference data, the `pii_classification` CI completeness check, search index tuning and the
|
||
constraint test suite.** Every trigger gets a test that attempts the forbidden write and asserts
|
||
the exception. That test suite is a genuinely varied, independently demonstrable junior
|
||
workstream and is treated as a deliverable, not as optional hygiene.
|
||
|
||
---
|
||
|
||
## 3. Identifier strategy
|
||
|
||
**Three identifiers per entity, each with exactly one job.**
|
||
|
||
| Identifier | Type | Where | Purpose | Never used for |
|
||
|---|---|---|---|---|
|
||
| `id` | `bigint GENERATED ALWAYS AS IDENTITY` | Every table | Primary key. Every foreign key in the schema is `bigint`. | Never serialised outside the database — not in URLs, API payloads, emails or exports |
|
||
| `public_id` | `uuid NOT NULL UNIQUE` (UUIDv7) | Externally addressable entities only | The only identifier in HTTP paths, API responses, email links and exports | Never an FK target; never treated as a capability (§3.2) |
|
||
| `reference_code` | `text NOT NULL UNIQUE` | `candidate`, `job`, `job_application` only | Human-readable code recruiters say out loud: `CAN-5001`, `JOB-1001`, `APP-30001`, preserving the prototype's shape (`js/data.js:118`, `js/data.js:95`, `js/data.js:298`) | **Internal only.** Enumerable by construction. Must never appear in a candidate-facing URL or email |
|
||
|
||
`public_id` appears on: `candidate`, `job`, `job_version`, `job_posting`, `job_application`,
|
||
`raw_intake`, `ats_result`, `candidate_document`, `interview`, `offer`, `offer_version`,
|
||
`candidate_merge`, `app_user`, `talent_pool`, `assessment_assignment`, `outbound_message`,
|
||
`conversation`, `task`. Tables that are only ever reached through a parent (all `*_criterion`,
|
||
all `*_history`, all child rows of candidate) do not get one — an unnecessary UUID column is 16
|
||
bytes plus an index on every row for an address nobody uses.
|
||
|
||
**Why the split.** The two requirements pull in opposite directions and both have to be met.
|
||
Internally `bigint` wins on every axis that matters at join time: 8 bytes instead of 16 in every
|
||
index and every FK, monotonic insertion so B-tree pages stay dense and WAL churn stays low,
|
||
readable `EXPLAIN` output, and ad-hoc queries a junior can type. Externally, a sequential id in a
|
||
candidate-facing URL such as `/applications/30042/status` discloses the entire candidate base by
|
||
enumeration, and application ids *will* appear in candidate status emails. UUIDv7 supplies ~74
|
||
random bits, which defeats enumeration, while remaining time-ordered — a minor bonus here, since
|
||
`public_id` is only ever an indexed lookup column and never an FK, so its clustering benefit is
|
||
not the reason for the choice.
|
||
|
||
**Accepted leak, stated:** UUIDv7 embeds a millisecond creation timestamp, so a candidate-facing
|
||
id reveals when the record was created. Low severity for an internal HR system, and it is the
|
||
price of not paying for a 16-byte FK in every index.
|
||
|
||
**Generation.** UUIDv7 is generated application-side, or by a small PL/pgSQL `uuidv7()` shim used
|
||
as the column `DEFAULT`. **This must be one explicit decision at provisioning time**, because
|
||
PostgreSQL 18 has a native `uuidv7()` and mixing sources between environments produces columns
|
||
with different defaults. `_decisions.md` flags this as an open provisioning item; it is not
|
||
resolved here.
|
||
|
||
**Rejected.** UUIDv4 as primary key everywhere — doubles the size of every index and FK and
|
||
destroys insert locality on the high-volume tables (`audit_event`, `ats_result_criterion`, all
|
||
history tables). UUIDv7 as primary key everywhere — fixes locality, still doubles index width,
|
||
and buys nothing `public_id` does not already buy. `bigint` alone with no `public_id` — the
|
||
enumeration exposure above. Natural keys such as email as candidate PK — emails change, get
|
||
merged, and are the thing duplicate detection exists to reconcile.
|
||
|
||
### 3.1 Reference code allocation
|
||
|
||
Three dedicated sequences (`app.candidate_ref_seq`, `app.job_ref_seq`,
|
||
`app.job_application_ref_seq`) with the code assigned by a `BEFORE INSERT` trigger:
|
||
`'CAN-' || nextval(...)`. Sequences, not `max()+1`, because concurrent intake will otherwise
|
||
collide. Gaps from rolled-back transactions are accepted and documented — a recruiter does not
|
||
care that `CAN-5107` never existed, and closing gaps requires a lock that intake cannot afford.
|
||
|
||
### 3.2 URL-guessability is not authorization
|
||
|
||
`public_id` is an identifier, never a capability. Every candidate-facing surface — application
|
||
status page, document upload link, interview confirmation, offer acceptance, assessment invite —
|
||
is gated by `app.candidate_access_token` (§6.8): `token_hash bytea` (the hash, never the token),
|
||
a typed subject reference, scope, `issued_at`, `expires_at`, `consumed_at`, `revoked_at`,
|
||
`issued_by_user_id`. Internal recruiter access is gated by the authorization layer against
|
||
`app_user`, never by knowledge of a `public_id`.
|
||
|
||
Unguessable ids drift into being treated as secrets the moment a link is emailed, and emailed
|
||
links leak through forwarding, mail archives and support tickets. A token can expire and be
|
||
revoked; an entity id cannot. Storing only the hash means a database read does not hand over live
|
||
access. This also keeps candidate-facing access entirely outside the recruiter permission model,
|
||
which matters because the prototype has no authorization of any kind — the RBAC matrix is a
|
||
display widget with no `can()` function anywhere (`js/rbac.js:78`, `js/rbac.js:111-112`).
|
||
|
||
---
|
||
|
||
## 4. Conventions
|
||
|
||
These are stated once and apply to every table in sections 6–26. The per-table entries below do
|
||
**not** repeat them; they only note departures.
|
||
|
||
### 4.1 Naming
|
||
|
||
| Rule | Form | Example |
|
||
|---|---|---|
|
||
| Tables | `snake_case`, **singular** | `job_application`, not `job_applications` |
|
||
| Schema qualification | Always written in migrations and in this document | `app.candidate`, `ref.currency` |
|
||
| Foreign key column | `<referenced_table>_id`; role-qualified when a table is referenced twice | `job_id`, `created_by_user_id`, `surviving_candidate_id` |
|
||
| Instant | `*_at`, always `timestamptz` | `applied_at`, `submitted_at` |
|
||
| Calendar date | `*_on` or `*_date`, `date` type | `retention_due_on`, `as_of_date`, `start_date` |
|
||
| Boolean | `is_*` / `has_*`, `NOT NULL`, always with a default | `is_mandatory`, `is_current` |
|
||
| Money | `<name>_amount` + `<name>_currency_code` pair | `base_salary_amount`, `base_salary_currency_code` |
|
||
| Enumerated FK | `*_id` to a `ref` table | `status_id`, `stage_id`, `source_channel_id` |
|
||
| Technical closed set | `text` + `CHECK`, no `_id` | `actor_kind`, `op_kind`, `outcome` |
|
||
| Immutable version table | `<entity>_version` with `version_no` | `job_version`, `offer_version` |
|
||
| History table | `<entity>_<dimension>_history` | `job_application_stage_history` |
|
||
| Join/child table | `<parent>_<thing>` | `candidate_email`, `role_permission` |
|
||
| Index | `ix_<table>_<cols>`; unique `uq_`; exclusion `ex_`; check `ck_<table>_<rule>`; FK `fk_` | `uq_candidate_email_normalised` |
|
||
| Views | `v_<name>`; live-row views `v_<entity>_live` | `v_candidate_live` |
|
||
|
||
Plural table names, `tbl_` prefixes, and abbreviations other than the three reference-code
|
||
prefixes are prohibited. One naming argument settled once is worth more than the marginal
|
||
prettiness of any alternative.
|
||
|
||
### 4.2 Audit columns
|
||
|
||
Every `app` and `ai` table carries:
|
||
|
||
```sql
|
||
created_at timestamptz NOT NULL DEFAULT now(),
|
||
updated_at timestamptz NOT NULL DEFAULT now(), -- maintained by trigger, not by the app
|
||
created_by_user_id bigint NULL REFERENCES app.app_user(id),
|
||
updated_by_user_id bigint NULL REFERENCES app.app_user(id)
|
||
```
|
||
|
||
with these exceptions, stated once:
|
||
|
||
- **Append-only tables have no `updated_at` / `updated_by_user_id`.** A column that can never
|
||
change must not exist: `ats_result` (except its narrow review columns), all `*_version`,
|
||
all `*_history`, `raw_intake`, `candidate_merge_operation`,
|
||
`candidate_consent`, `outbound_message` (except delivery-status columns),
|
||
`access_grant` (except its three revocation columns, §7.7), `ats_result_override` (except
|
||
`is_current`/`superseded_by_id`, §20.7), `audit.audit_event`.
|
||
- **Three append-only tables have a named *closing* write, and no other write.** These are rows
|
||
that are opened before the work they describe finishes, so they must be closable exactly once
|
||
and immutable thereafter. The grant is column-scoped and the immutability trigger fires once the
|
||
row is closed, so "append-only" here means *write-once-then-frozen*, not *insert-only*:
|
||
|
||
| Table | Columns the closing `UPDATE` may touch | What freezes it | § |
|
||
|---|---|---|---|
|
||
| `ai.ai_model_invocation` | `status`, `response`, `error`, `input_tokens`, `output_tokens`, `total_tokens`, `cost_amount`, `cost_currency_code`, `latency_ms`, `finished_at` | `BEFORE UPDATE` trigger refusing any write when `OLD.status <> 'running'` | §27.2 |
|
||
| `app.intake_parse_attempt` | `status`, `parsed`, `error`, `confidence`, `field_count`, `text_char_count`, `injection_signal`, `injection_signal_codes`, `finished_at`, `duration_ms`, `candidate_document_id` | `BEFORE UPDATE` trigger refusing any write when `OLD.finished_at IS NOT NULL`, and refusing to re-point `candidate_document_id` once set | §18.1 |
|
||
| `audit.audit_event` | `before`, `after`, `redacted_at`, `pre_redaction_row_hash` — **and `prev_hash`/`chain_seq` for the sealing job only** | the redaction path is granted to `talentflow_redactor` alone and requires a matching `audit_event_redaction` row in the same transaction; the sealing job writes `prev_hash`/`chain_seq` once, and only while they are NULL | §28.1, §28.2 |
|
||
|
||
Nothing else in this schema may update an append-only table. `app.ats_result` keeps its narrow
|
||
`is_current`/`superseded_by_id`/review-column grant (§20.4) and remains otherwise frozen.
|
||
- **`created_by_user_id` is `NOT NULL` where a human must have acted**: `candidate_merge`,
|
||
`intake_resolution` when `decision_mode = 'human'`, `approval_decision`, `offer_version`,
|
||
`scorecard`. Elsewhere it is nullable because integrations and background jobs legitimately
|
||
create rows with no human actor, and a fake system user id would be worse than a NULL.
|
||
- `ref` tables carry `created_at`, `updated_at`, `is_active` and no user columns — they are
|
||
administered through the back-office, whose changes are captured in `audit.audit_event`.
|
||
|
||
`updated_at` is maintained by one shared trigger function
|
||
(`app.tg_set_updated_at()`), attached in every table's migration. Application code never writes
|
||
it: an application-maintained `updated_at` is wrong the first time someone runs an `UPDATE` in
|
||
psql, and being wrong there is exactly when it matters.
|
||
|
||
### 4.3 Time
|
||
|
||
- **Every instant is `timestamptz`, stored UTC.** `timestamp without time zone` is **banned**
|
||
with exactly one documented exception: `app.interview.local_start_wall`, the organiser's chosen
|
||
wall-clock time.
|
||
- Dates that are genuinely calendar-only are `date`: `retention_due_on`, `fx_rate.as_of_date`,
|
||
`candidate_employment.start_date`/`end_date` (CVs frequently give month precision only),
|
||
`offer_version.start_date`.
|
||
- **Any `date` derived from a `timestamptz` must name its zone explicitly in the expression.**
|
||
`created_at::date` and `cast(created_at as date)` depend on the session `TimeZone` GUC and are
|
||
therefore **not immutable**, so they cannot appear in a `CHECK`, a generated column or an index
|
||
expression — Postgres 16 rejects
|
||
`GENERATED ALWAYS AS ((created_at)::date) STORED` with
|
||
`ERROR: generation expression is not immutable`. The idiom is
|
||
`GENERATED ALWAYS AS (((created_at AT TIME ZONE 'UTC'))::date) STORED`, which is immutable
|
||
because the zone is a literal. Consequence to state to the business rather than discover later:
|
||
every derived calendar date in this schema is a **UTC** date. An offer created at 23:30
|
||
Asia/Karachi (18:30 UTC) has a `created_on` of the same UTC day, but one created at 05:30
|
||
Karachi (00:30 UTC) does not share the recruiter's calendar day. This matters only for the
|
||
same-day boundary on `offer_version` date CHECKs (§25.2); nothing in the product renders a UTC
|
||
date to a user.
|
||
- Timezones are stored as **IANA zone names** validated against `pg_timezone_names`. A UTC offset
|
||
such as `+05:00` is never stored as a timezone: offsets do not survive DST or zone-rule changes,
|
||
and integrations will send `PST`, `IST` and `Asia/Calcutta`-style values that otherwise resolve
|
||
silently to the wrong or a deprecated zone.
|
||
- **Anything a user scheduled stores both the instant and the intent.** `interview` carries
|
||
`starts_at`/`ends_at` (`timestamptz`), `scheduling_timezone` (IANA), and `local_start_wall`
|
||
(naked `timestamp`). For a single one-off interview `starts_at` alone would do; both are stored
|
||
because the moment there are panel slots, availability windows, or a reschedule across a DST
|
||
boundary, intent and instant diverge, and recomputing intent from UTC after a tzdata release can
|
||
produce a different wall-clock answer. Storing intent makes a tzdata update a re-resolution job
|
||
over a queryable set of rows instead of silent data loss.
|
||
- **Availability and working hours are wall-clock rules, never instants**: `(user_id, timezone,
|
||
weekday, local_start_time time, local_end_time time)`. "Available 09:00–17:00 local" converted
|
||
to UTC becomes wrong twice a year.
|
||
- The prototype has no timezone discipline at all — plain JS `Date`, `toLocaleDateString` for
|
||
display, and a hardcoded "today" of `2026-07-09` (`js/data.js:54`, `js/data.js:237`,
|
||
`js/candidates.js:18`). None of that carries forward.
|
||
|
||
### 4.4 Money
|
||
|
||
Every monetary value is an **adjacent column pair**:
|
||
|
||
```sql
|
||
<name>_amount numeric(14,2),
|
||
<name>_currency_code char(3) REFERENCES ref.currency(code),
|
||
CONSTRAINT ck_<table>_<name>_money
|
||
CHECK ((<name>_amount IS NULL) = (<name>_currency_code IS NULL))
|
||
```
|
||
|
||
Rules:
|
||
|
||
1. **Ranges** additionally get `CHECK (max_amount >= min_amount)` and
|
||
`CHECK (min_currency_code = max_currency_code)`.
|
||
2. **Minor units are validated.** A `DEFERRABLE` constraint trigger checks
|
||
`amount = round(amount, c.minor_unit)` against `ref.currency`, so `JPY 500000.50` cannot be
|
||
stored.
|
||
3. **The original is authoritative and immutable.** Conversions live in `app.fx_rate` and are
|
||
recorded on the row as a denormalised reporting pair *plus the pinned* `fx_rate_id`:
|
||
`base_salary_reporting_amount`, `base_salary_reporting_currency_code`, `fx_rate_id`, with
|
||
`CHECK ((fx_rate_id IS NULL) = (base_salary_reporting_amount IS NULL))`.
|
||
4. **Compensation is classified `sensitive_personal`** (§30), not ordinary personal data.
|
||
5. **A period-bearing amount must store its period, and comparisons must be annualised.** A bare
|
||
number is ambiguous, and so is a number whose period lives only in the reader's head.
|
||
`offer_version.base_salary_period` and `job_version.salary_period` both carry
|
||
`CHECK (… IN ('annual','monthly','hourly','daily'))`, and any index or report that ranks or
|
||
bands compensation reads a stored annualised column, never the raw amount (§9.2, §25.2).
|
||
Converting currency does not normalise period: without this rule an hourly rate and an annual
|
||
salary sort against each other in the same banding index.
|
||
6. **One documented exception to `numeric(14,2)` and to rule 2: metered AI cost.**
|
||
`ai.ai_model_invocation.cost_amount` is `numeric(18,8)` and is **exempt from the minor-unit
|
||
rounding trigger**. A single model invocation costs fractions of a cent; at `numeric(14,2)`
|
||
every row would round to `0.00` and the cost-per-capability rollup (§27.2) would return zeros
|
||
across ~1.2M rows, while any attempt to store a genuine `USD 0.00420000` would be rejected by
|
||
the rounding trigger. The currency FK is retained, and the rollup rounds to minor units **only
|
||
at presentation**. This exception applies to metered internal consumption and to nothing else —
|
||
`job_posting_metric.spend_amount` is a payable amount invoiced by an external platform, so it
|
||
stays `numeric(14,2)` under rules 1–3. The two must not be given a shared rule.
|
||
|
||
The column-pair CHECK is the highest-value money constraint in the schema: an amount without its
|
||
currency is unusable, and worse, silently assumed to be the local currency by whoever reads it
|
||
next. The prototype has bare integers and no currency field anywhere (`js/data.js:126`,
|
||
`js/data.js:99`) and offer validation checks only `salary > 0` (`js/offers.js:129`), so this is a
|
||
real gap, not a formality. `numeric` rather than integer minor units because every query, report
|
||
and CSV export stays readable without a division everyone must remember, and Postgres `numeric`
|
||
arithmetic is exact; the rounding trigger recovers the one guarantee integer minor units give for
|
||
free. Storing the converted value with `fx_rate_id` rather than converting at read time is the
|
||
same philosophy as ATS version pinning — a board report re-run next quarter must show the same
|
||
number.
|
||
|
||
**Rejected:** the Postgres `money` type (locale-dependent output, implicit single-currency
|
||
assumption), `float`/`double precision` (rounding errors in compensation are indefensible),
|
||
integer minor units (correct but every ad-hoc query needs a mental division and this team will
|
||
write many), a single generic money table joined everywhere (an extra join per offer read for no
|
||
invariant gain).
|
||
|
||
### 4.5 Soft delete
|
||
|
||
`deleted_at timestamptz NULL` + `deleted_by_user_id bigint NULL` with
|
||
`CHECK ((deleted_at IS NULL) = (deleted_by_user_id IS NULL))`, on **recruiter-removable entities
|
||
only**:
|
||
|
||
`candidate`, `job`, `job_application`, `candidate_note`, `candidate_document`, `candidate_tag`,
|
||
`candidate_link`, `candidate_email`, `candidate_phone`, `task`, `saved_search`, `saved_report`,
|
||
`talent_pool`, `app_user`, `message_template`, `scorecard_template`, `assessment_template`,
|
||
`pipeline_config`, `scoring_config`, `matching_config`, `interview` (cancellation is a status;
|
||
deletion is a mistake being hidden), `offer` (a draft created in error).
|
||
|
||
`talent_pool_member` is deliberately **not** in that list — its `removed_at` column already carries
|
||
the same meaning, and two columns for one concept is how they drift apart. The complete policy,
|
||
including the forbidden list and the cascade and restore rules, is §31.3.
|
||
|
||
**Never on append-only tables:** `audit.audit_event`, `ats_result`, `ats_result_criterion`, every
|
||
`*_history`, every `*_version`, `raw_intake`, `raw_intake_attachment`, `intake_parse_attempt`,
|
||
`intake_resolution`, `candidate_merge`, `candidate_merge_operation`, `candidate_consent`,
|
||
`outbound_message`, `ai.ai_model_invocation`.
|
||
|
||
Three consequences that are part of the rule:
|
||
|
||
- Every uniqueness rule that must tolerate re-creation is a **partial index** with
|
||
`WHERE deleted_at IS NULL`.
|
||
- Default application reads go through per-entity live views (`v_candidate_live`,
|
||
`v_job_application_live`, `v_job_live`), so seeing deleted rows requires deliberately querying
|
||
the base table.
|
||
- **Soft delete is not erasure and must never be presented as such.** Erasure is §31.
|
||
|
||
`is_deleted boolean` is rejected: it loses *when*, and therefore cannot drive retention.
|
||
|
||
### 4.6 Status handling
|
||
|
||
Three tiers, applied without exception (this is `_decisions.md`'s enum strategy, made concrete):
|
||
|
||
**Tier 1 — `ref` table FK.** Anything the business will edit, or that needs display metadata
|
||
(`label`, `order_index`, `is_terminal`, `colour_token`). All lifecycle statuses, pipeline stages,
|
||
rejection reasons, source channels, currencies, departments, business units, grades, employment
|
||
types, skills, education levels, locations, assignment roles.
|
||
|
||
**Tier 2 — `text` + `CHECK`.** Small closed technical sets only engineers change:
|
||
`actor_kind`, `decision_mode`, `op_kind`, `outcome`, `raw_intake.state`, `scope_type`,
|
||
`resolution_kind`, `virus_scan_status`, `match_kind`, `suggestion_status`. A `ref` table for
|
||
these would be ceremony plus a join on the hottest append paths.
|
||
|
||
**Tier 3 — native Postgres `enum` types: NOT USED, anywhere.** Adding a value is easy but
|
||
reordering or removing one requires a type rewrite, they cannot carry `order_index`,
|
||
`is_terminal` or a colour token, and they force a migration for what should be a row insert. The
|
||
prototype hardcodes stage lists, statuses, sources and a skills pool as JS arrays
|
||
(`js/data.js:20-46`) and every one of those is something a recruiting lead will want to change
|
||
without a deployment — which is the definition of reference data.
|
||
|
||
**Generated state columns are derived from trigger-denormalised `ref` attributes, not from a
|
||
hardcoded key list**, so an index predicate can never disagree with the status it is derived from
|
||
*and* Tier 1's promise survives. The trigger that denormalises `status_key` onto a statused table
|
||
also denormalises `status_is_terminal` and `status_is_negative` from `ref.lifecycle_status`, and
|
||
the generated column reads those booleans:
|
||
|
||
```sql
|
||
status_is_terminal boolean NOT NULL, -- maintained by trigger from ref.lifecycle_status
|
||
state text NOT NULL GENERATED ALWAYS AS
|
||
(CASE WHEN status_is_terminal THEN 'terminal' ELSE 'active' END) STORED
|
||
```
|
||
|
||
This matters because the alternative — `CASE WHEN status_key IN ('hired','rejected',…)` — puts the
|
||
terminal set in DDL, and adding "On Hold" or a new terminal status then requires
|
||
`ALTER TABLE … ALTER COLUMN state` plus a full rewrite of every row, which is exactly the
|
||
deployment Tier 1 exists to avoid. With the boolean denormalised, adding a terminal status is an
|
||
`INSERT` into `ref.lifecycle_status` plus a bounded backfill of the affected applications.
|
||
See `job_application.state` (§15.1) and `offer.status_is_terminal` (§25.1).
|
||
|
||
### 4.7 Enum-vs-lookup-table policy, and the two consolidations
|
||
|
||
The tier rules above leave one question: how many lookup tables? Two consolidations, both
|
||
deliberate, both justified here because they are the largest structural deviations from a
|
||
table-per-vocabulary reading of the assignment.
|
||
|
||
**Consolidation 1 — `ref.lifecycle_status` replaces six near-identical status tables.**
|
||
|
||
`job_status`, `candidate_status`, `job_application_status`, `offer_status`, `interview_status` and
|
||
`assessment_status` have *identical* shape and identical metadata needs: label, ordering,
|
||
`is_terminal`, `is_negative`, `requires_reason`, colour. Six tables differing only in their name
|
||
is duplication, not modelling.
|
||
|
||
```sql
|
||
CREATE TABLE ref.lifecycle_status (
|
||
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
|
||
domain text NOT NULL CHECK (domain IN
|
||
('job','candidate','job_application','offer','interview',
|
||
'assessment_assignment','approval_request','raw_intake_review')),
|
||
key text NOT NULL CHECK (key ~ '^[a-z][a-z0-9_]{1,40}$'),
|
||
label text NOT NULL,
|
||
order_index int NOT NULL,
|
||
is_terminal boolean NOT NULL DEFAULT false,
|
||
is_negative boolean NOT NULL DEFAULT false,
|
||
requires_reason boolean NOT NULL DEFAULT false,
|
||
colour_token text NULL, -- a css/styles.css var name, e.g. 'b-green'
|
||
is_active boolean NOT NULL DEFAULT true,
|
||
CONSTRAINT uq_lifecycle_status_domain_key UNIQUE (domain, key),
|
||
CONSTRAINT uq_lifecycle_status_id_domain UNIQUE (id, domain), -- enables the composite FK
|
||
CONSTRAINT uq_lifecycle_status_domain_order UNIQUE (domain, order_index)
|
||
);
|
||
```
|
||
|
||
The consolidation would normally cost type safety — an `offer.status_id` could point at a
|
||
`candidate` status. It does not here, because of `uq_lifecycle_status_id_domain` and a
|
||
**composite FK against a constant generated column**:
|
||
|
||
```sql
|
||
ALTER TABLE app.offer
|
||
ADD COLUMN status_domain text NOT NULL
|
||
GENERATED ALWAYS AS ('offer') STORED,
|
||
ADD CONSTRAINT fk_offer_status
|
||
FOREIGN KEY (status_id, status_domain)
|
||
REFERENCES ref.lifecycle_status (id, domain);
|
||
```
|
||
|
||
The generated constant column is 8 bytes of storage per row and buys back exactly the domain
|
||
safety a dedicated table would have given. This pattern is applied at every status FK in the
|
||
schema and is written once in migration `002` as a documented idiom.
|
||
|
||
**Tradeoff, stated:** one more column per statused table and one non-obvious idiom a junior must
|
||
be taught, against six fewer tables, one back-office screen instead of six, and one place to add
|
||
"On Hold" to a lifecycle. Chosen because the idiom is taught once and the duplication would be
|
||
paid forever.
|
||
|
||
**Consolidation 2 — `ref.vocabulary_value` for pure-label vocabularies.**
|
||
|
||
Vocabularies with no structural columns and no relationships beyond "something references this
|
||
label" live in one table: `interview_type` (7 values, BRD §9.2), `interview_mode` (3),
|
||
`assessment_type` (6), `document_type`, `note_kind`, `task_kind`, `consent_purpose`,
|
||
`cost_band` (BRD §8.2), `channel_type`, `link_type`, `participant_role`, `parse_issue_severity`.
|
||
|
||
```sql
|
||
CREATE TABLE ref.vocabulary_value (
|
||
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
|
||
vocabulary_key text NOT NULL CHECK (vocabulary_key ~ '^[a-z][a-z0-9_]{2,40}$'),
|
||
value_key text NOT NULL CHECK (value_key ~ '^[a-z][a-z0-9_]{1,40}$'),
|
||
label text NOT NULL,
|
||
order_index int NOT NULL,
|
||
colour_token text NULL,
|
||
metadata jsonb NOT NULL DEFAULT '{}'::jsonb, -- display only; P6 applies
|
||
is_active boolean NOT NULL DEFAULT true,
|
||
CONSTRAINT uq_vocab_value UNIQUE (vocabulary_key, value_key),
|
||
CONSTRAINT uq_vocab_id_key UNIQUE (id, vocabulary_key)
|
||
);
|
||
```
|
||
|
||
Referenced by the same composite-FK-plus-generated-constant idiom. Twelve tables become one, and
|
||
adding a new vocabulary becomes an `INSERT` rather than a migration.
|
||
|
||
**What stays a dedicated `ref` table, and why:** `pipeline_stage` (carries `order_index`,
|
||
`is_terminal`, department scoping, and is referenced by transition rules and three history
|
||
tables), `skill` + `skill_alias` (a taxonomy with aliases and a trigram index),
|
||
`rejection_reason` (carries `is_negative`, cooling-off override and reporting rollup),
|
||
`source_channel` (11 values with per-channel attribution reporting and a distinct
|
||
`intake_channel` connection table hanging off it), `currency` (carries `minor_unit`, referenced by
|
||
CHECKs), `department`, `business_unit`, `location`, `region`, `grade`, `employment_type`,
|
||
`education_level` (all organisational dimensions used as report group-bys and access-scope
|
||
targets), `assignment_role`, `publish_platform`, `app_module`, `permission_action`, `tag`.
|
||
|
||
### 4.8 Constraint idioms used throughout
|
||
|
||
| Idiom | Written as | Used for |
|
||
|---|---|---|
|
||
| Soft-delete-tolerant uniqueness | `CREATE UNIQUE INDEX … WHERE deleted_at IS NULL` | every re-creatable natural key |
|
||
| Merge-tolerant uniqueness | `… WHERE deleted_at IS NULL AND suppressed_by_merge_id IS NULL` | `candidate_email`, `candidate_phone`, `candidate_link` |
|
||
| One current row | `CREATE UNIQUE INDEX … WHERE is_current` / `WHERE valid_to IS NULL` | `ats_result`, `job_assignment` primary recruiter |
|
||
| Non-overlapping intervals | `EXCLUDE USING gist (subject WITH =, tstzrange(valid_from, valid_to) WITH &&)` | every history and assignment table |
|
||
| Cross-table invariant | `CONSTRAINT TRIGGER … DEFERRABLE INITIALLY DEFERRED` | contactability, requirement weight sum, money minor units |
|
||
| Immutability | column-level `GRANT` **and** a trigger pair — a conditional `BEFORE UPDATE` plus an unconditional `BEFORE DELETE` (never one combined `BEFORE UPDATE OR DELETE` with a `WHEN` clause; see the fourth syntax fact) | all `*_version`, `ats_result`, `audit_event` |
|
||
| Typed polymorphism (P7) | N nullable FK columns + `CHECK (num_nonnulls(...) = 1)` + a `*_kind` discriminator with a CHECK tying kind to column | `access_scope`, `approval_request`, `ai_suggestion`, `task`, `candidate_access_token` |
|
||
| Derived predicate | `GENERATED ALWAYS AS (...) STORED` joined through a `key` | `job_application.state`, `interview.slot`, `candidate.name_normalised` |
|
||
|
||
Immutability is enforced **twice** on purpose: a `GRANT` can be misconfigured during environment
|
||
setup, while a trigger travels with the schema and documents intent in place. That doubling only
|
||
pays if the trigger is complete, so **immutability triggers allow-list the mutable columns rather
|
||
than deny-listing the immutable ones** — see §9.2 for the idiom, which compares
|
||
`to_jsonb(OLD) - '<mutable cols>'` against `to_jsonb(NEW)` minus the **same** keys. Both sides are
|
||
part of the idiom: the subtraction alone is not a predicate, and the `NEW` half is exactly what
|
||
forces the trigger split named in the fourth syntax fact below. A column added by a later migration
|
||
must not be able to open a hole in a trigger written today.
|
||
|
||
**Four syntax facts about these idioms, stated once because getting any of them wrong costs a
|
||
migration:**
|
||
|
||
- **Partial uniqueness is an *index*, never a table constraint.**
|
||
`CONSTRAINT uq_x UNIQUE (a, b) WHERE …` is not valid PostgreSQL — it fails with
|
||
`ERROR: syntax error at or near "WHERE"`. Every partial unique rule in this document is written
|
||
as a separate `CREATE UNIQUE INDEX <name> ON <table> (<cols>) WHERE <predicate>;` statement after
|
||
the `CREATE TABLE`, with no `CONSTRAINT` keyword.
|
||
- **A partial unique index cannot be `DEFERRABLE`.** Only a true table constraint can be deferred,
|
||
and `UNIQUE (…) WHERE …` cannot be a table constraint, so there is no way to make a partial
|
||
unique rule evaluate at COMMIT. Any invariant that genuinely needs end-of-transaction evaluation
|
||
must therefore be a **deferred constraint trigger**, not a partial unique index. This is
|
||
load-bearing, not pedantry: it is why merge reversal cannot rely on `uq_application_live`
|
||
deferring and must instead replay in dependency order (§19.5).
|
||
- **A constraint trigger may only be `AFTER ROW`.**
|
||
`CREATE CONSTRAINT TRIGGER … BEFORE INSERT` is a syntax error
|
||
(`ERROR: syntax error at or near "BEFORE"`). A check that must run immediately and can read only
|
||
already-committed rows is a plain `BEFORE INSERT` **row** trigger; a check that must run at
|
||
COMMIT is `CREATE CONSTRAINT TRIGGER … AFTER … DEFERRABLE INITIALLY DEFERRED`.
|
||
- **An immutability trigger is a *pair*, because a `WHEN` clause can see neither `TG_OP` nor, on a
|
||
combined `UPDATE OR DELETE` trigger, `NEW`.** Two independent errors, and fixing the first exposes
|
||
the second. `TG_OP` is a PL/pgSQL variable that exists only inside the trigger *function* body; in
|
||
a `WHEN` clause PostgreSQL parses it as a column and fails with
|
||
`ERROR: column "tg_op" does not exist`. And a `WHEN` clause on a trigger declared
|
||
`BEFORE UPDATE OR DELETE` may not reference `NEW` at all —
|
||
`ERROR: DELETE trigger's WHEN condition cannot reference NEW values` — because the `DELETE` case
|
||
has no `NEW` row, which makes the `to_jsonb(OLD) … IS DISTINCT FROM to_jsonb(NEW)` allow-list
|
||
above structurally impossible to express in one combined trigger. The working shape, used at every
|
||
`WHEN`-clause immutability trigger in this document, is therefore **two triggers on the same
|
||
table sharing one function**: `BEFORE UPDATE … FOR EACH ROW WHEN (<the to_jsonb comparison, plus
|
||
any OLD-only predicates>)`, and an unconditional `BEFORE DELETE … FOR EACH ROW` with no `WHEN`
|
||
clause at all. `OLD` is legal in either. The only table that keeps a single combined
|
||
`BEFORE UPDATE OR DELETE` trigger is one whose logic lives entirely in the function body and has
|
||
no `WHEN` clause to begin with — `audit.audit_event` (§28.2) and `app.intake_parse_attempt`
|
||
(§18.1), both of which branch on `TG_OP` inside PL/pgSQL, where it is available.
|
||
|
||
### 4.9 Nullable vs required — the default
|
||
|
||
`NOT NULL` is the default and a nullable column needs a reason. Three reasons are accepted and
|
||
each per-table entry states which applies:
|
||
|
||
1. **Not yet known** — `job_application.terminal_at` before the application ends.
|
||
2. **Legitimately absent** — `candidate_employment.end_date` (`NULL` = current role);
|
||
`candidate_skill.skill_id` (`NULL` = parser produced a label that maps to no taxonomy entry,
|
||
which must still be storable and reviewable, never dropped).
|
||
3. **Actor genuinely absent** — `created_by_user_id` on rows created by an integration.
|
||
|
||
"The form does not collect it yet" is not a reason; that is a `DEFAULT`.
|
||
|
||
### 4.10 Retention, PII and audit defaults
|
||
|
||
Stated once; per-table entries state only the class and any departure.
|
||
|
||
- **Every column on a candidate-touching table has a row in `audit.pii_classification`**, and a
|
||
CI check fails the build if one is missing. Classes: `internal`, `personal`,
|
||
`sensitive_personal`, `special_category`. **No `special_category` data is stored in Phase 1** —
|
||
no diversity, health or accommodation data. That is a decision, not an omission: it needs
|
||
separate access control, aggregate-only reads and a distinct lawful basis, none of which is in
|
||
scope.
|
||
- **Default audit requirement:** every table classified `personal` or above gets the generic
|
||
data-change audit trigger. Access events (profile viewed, export run, chatbot answer) are
|
||
written explicitly by the application, because no trigger can observe a read.
|
||
- **Default retention:** driven by `candidate.retention_due_on` and the `retention_policy` table;
|
||
the purge action is pseudonymisation (§31).
|
||
|
||
---
|
||
|
||
## 5. Table map
|
||
|
||
**159 tables — and that figure is the sum of the `Tables` column below, deliberately.** An earlier
|
||
revision opened "144" while its area rows summed to 147, which `08-requirements-traceability.md` §6
|
||
caught: with the headline below its own column sum, nothing in this map could be shown exhaustive in
|
||
either direction. The headline is now derived from the column, so a reader can verify it on the page.
|
||
Two things that follow from that and are stated rather than absorbed: a few rows count an area's
|
||
objects slightly generously (`08` §6 recounts the individually-documented tables and derives a lower
|
||
figure from §§6–28 directly), and **`08` §6 is the authoritative per-table enumeration** for the
|
||
reverse "does every table trace to a requirement" check. This map is the index; sections 6–26 are the
|
||
detail.
|
||
|
||
| Area | § | Schema | Tables | Phase |
|
||
|---|---|---|---|---|
|
||
| Reference data and vocabularies | 6 | `ref` | 23 | 1 |
|
||
| Identity, organization and access scope | 7 | `app` | 11 | 1 |
|
||
| Files | 8 | `app` | 1 | 1 |
|
||
| Job requisitions, versions and requirements | 9 | `app` | 7 | 1 |
|
||
| Approval workflow | 10 | `app` | 4 | **1–2** — `approval_request` + `approval_decision` are **Phase 1**, single-approver only (REQ-JOB-07 is confirmed at Phase 1 in `00` §2.4, and Phase 1 is where `job_version` publishing first exists, so publishing must not ship ungated — `07` T-17b). The Phase 1 UI is `07` A-16b (approve/reject, `job.approve`-gated) and the acceptance criterion is `07` §4.5 #13. `approval_route` + `approval_route_step`, ordered multi-step chains and the OBD-12 grade/threshold branching are **Phase 2** (`07` §4.2 excludes only *multi-step* chains). Migration split accordingly: `017a` Phase 1, `017b` Phase 2 (§33). Resolves `08` GAP-01 |
|
||
| Job postings and publishing | 11 | `app` | 2 | 1 (`job_posting`, migration 006a) / 4 (`job_posting_metric`) |
|
||
| Recruitment intake and inbox | 12 | `app` | 10 | 1 |
|
||
| Candidates | 13 | `app` | 12 | 1 |
|
||
| Candidate documents | 14 | `app` | 2 | 1 |
|
||
| Applications | 15 | `app` | 4 | 1 |
|
||
| Recruiter assignments | 16 | `app` | 2 | 1 |
|
||
| Pipeline configuration | 17 | `app` | 4 | 1 |
|
||
| CV parsing runs | 18 | `app` | 2 | 1 |
|
||
| Duplicate detection and merge | 19 | `app` | 5 | 1–2 |
|
||
| ATS scoring snapshots | 20 | `app` | 8 | 1 |
|
||
| Fairness evaluation | 21 | `app` | 3 | 3 |
|
||
| Communications | 22 | `app` | 8 | 1 minimal send / 2 full pipeline (§22) |
|
||
| Interviews, feedback and scorecards | 23 | `app` | 11 | 2 |
|
||
| Assessments | 24 | `app` | 4 | 3 |
|
||
| Offers | 25 | `app` | 4 | 3 |
|
||
| Talent pool and saved searches | 26 | `app` | 3 | 3 |
|
||
| Currency and FX | 26.2 | `app`/`ref` | 1 | 1 |
|
||
| Search ranking configuration | 29.4 | `app` | 2 | 1 |
|
||
| AI orchestration and chatbot | 27 | `ai` | 13 | 1–2 |
|
||
| Audit, governance and activity history | 28.1–28.3, 28.5 | `audit` / `app` | 8 | 1 (`audit.*`) / 3 (`app.candidate_erasure_request`, §28.5) |
|
||
| Operational surfaces (tasks, settings, reports, idempotency) | 28.4 | `app` | 5 | 1–2 |
|
||
|
||
Sections after the schema: **29** search and query design, **30** deviations from the named entity
|
||
list, **31** governance policies (PII, retention, soft delete, audit), **32** reconciliation with
|
||
`_decisions.md` and risks, **33** migration sequence.
|
||
|
||
**The four tables that took the count from 155 to 159, and why they are here rather than in `05`.**
|
||
`05-security-rbac-ai-governance.md` §9.2 declares nine additive schema objects as security and
|
||
explainability controls, and `adr/0009-permission-enforcement-strategy.md` — status **Accepted** —
|
||
specifies an enforcement strategy that resolves scope from two of them. None existed in this
|
||
document, which `08` records as **GAP-27**: an accepted ADR depending on schema nothing defines is a
|
||
defect, not a documentation gap. All nine are now adopted here, four of them as tables — `ref.region`
|
||
(§6), `app.access_grant` (§7.7), `app.ats_result_override` (§20.7) and
|
||
`app.candidate_erasure_request` (§28.5) — and five as columns or roles:
|
||
`app.access_scope.region_id` + `ref.location.region_id` (§6, §7.3),
|
||
`app.ats_result_criterion.match_state` (§20.5), `audit.audit_event.source_service` (§28.1),
|
||
`app.intake_parse_attempt.injection_signal` (§18.1), `audit.pii_classification.data_subject_kind`
|
||
(§28.2), and the three read-only database roles (§28.6, documented as roles with their `GRANT`s
|
||
rather than as tables, because that is what they are). §32.1 row 5 records the reconciliation and
|
||
§33 names the migration each lands in. **One item on `05` §9.2's list is deliberately not adopted:**
|
||
scope columns on `role_assignment` for business unit / department / region, which `05` §9.2 itself
|
||
**withdraws** — `role_assignment.access_scope_id` → `access_scope` already carries them (§7.4), and
|
||
parallel columns would give one enum two homes. `08` GAP-27's remediation text still lists them and
|
||
is stale on that single point.
|
||
|
||
**Per-table documentation format.** Significant tables get: a purpose line, a column table, then
|
||
four labelled lines — **Constraints**, **Indexes**, **Profile** (status handling ·
|
||
nullable-vs-required · soft delete · retention · PII · audit · expected volume) and
|
||
**Queries**. Uniform-profile tables (pure reference tables, history tables, immutable child rows)
|
||
are documented in a per-area roll-up table, because their §11 attributes are constant and are
|
||
stated once in §4: `bigint` identity PK, no soft delete, audited via the parent, retention follows
|
||
the parent, PII as marked.
|
||
|
||
**Volume basis — labelled assumption.** No production figures exist. Assume 40,000 applications
|
||
and 25,000 new candidate identities per year, 250 inbound documents per peak day, 66 named seats
|
||
(BRD §4), and a five-year horizon. All "expected volume" figures below are five-year totals on
|
||
that basis and should be re-derived, not inherited, if the platform is ever pointed at bulk
|
||
job-board feeds (`_decisions.md` flags this risk explicitly).
|
||
|
||
---
|
||
|
||
## 6. Reference data and vocabularies (`ref`)
|
||
|
||
All 23 tables share the uniform profile: `bigint` identity PK, `key text NOT NULL` with a
|
||
`^[a-z][a-z0-9_]{1,40}$` CHECK, `label text NOT NULL`, `is_active boolean NOT NULL DEFAULT true`,
|
||
`created_at`/`updated_at`, no soft delete (deactivate instead — a deleted reference value orphans
|
||
history), no retention obligation, PII class `internal`, audited (the back-office is the only
|
||
writer and its changes matter), volume in the tens to low thousands of rows, and cached in the
|
||
application with a short TTL.
|
||
|
||
**Deactivation, not deletion, is the rule.** `is_active = false` hides a value from new pickers
|
||
while leaving every historical FK valid. Nothing in `ref` is ever deleted; a migration that
|
||
`DELETE`s a `ref` row is a bug.
|
||
|
||
| Table | Purpose | Distinguishing columns | Seed volume | Source |
|
||
|---|---|---|---|---|
|
||
| `ref.lifecycle_status` | All entity lifecycle statuses across 8 domains (§4.7) | `domain`, `order_index`, `is_terminal`, `is_negative`, `requires_reason`, `colour_token` | ~45 | BRD §9.2 (job 4, offer 6, interview 4); `js/data.js:25,169` |
|
||
| `ref.vocabulary_value` | 12 pure-label vocabularies (§4.7) | `vocabulary_key`, `order_index`, `metadata` | ~60 | BRD §9.2; `js/data.js:131,132` |
|
||
| `ref.pipeline_stage` | The 7 canonical stages, orderable, per-department-scopable | `order_index`, `is_terminal`, `is_negative`, `department_id NULL`, `colour_token` | 7 → ~20 | BRD §9.2; `js/data.js:27` |
|
||
| `ref.rejection_reason` | Why an application ended negatively | `is_candidate_visible`, `cooling_off_days NULL` (overrides the default), `rollup_key` | ~15 | BRD §5; new |
|
||
| `ref.source_channel` | The 11 inbound channels | `channel_type` (FK to vocabulary), `icon_token`, `colour_token`, `is_agency`, `is_identifying` | 11 | BRD §8.1; `js/data.js:268-282` |
|
||
| `ref.publish_platform` | The 8 outbound platforms | `cost_band_id`, `supports_unpublish`, `api_kind` | 8 | BRD §8.2; `js/jobboard.js` |
|
||
| `ref.department` | 10 departments | `business_unit_id NULL`, `head_user_id NULL`, `code` | 10 | BRD §9.2; `js/data.js:20` |
|
||
| `ref.business_unit` | 5 business units (the Utopia brand dimension) | `code` | 5 | BRD §9.2; `js/data.js:21` |
|
||
| `ref.location` | Named work locations | `country_code char(2)`, **`region_id NULL`** (FK `ref.region` — replaces the free-text `region` column, see the note below), `city`, `timezone` (IANA), `is_remote` | ~20 | `js/data.js:22` |
|
||
| `ref.region` | The recruiting-region grouping over `ref.location`: the routing dimension regional desks are organised on, and the fifth access dimension in `05` §2.3 | `code`, `order_index` | ~6 | `05` §2.3 dimension 5; ADR-0009 ("Scope — eight dimensions"); `06` §1.15 |
|
||
| `ref.grade` | L2–L7 | `order_index`, `band_min_amount`/`band_min_currency_code`, `band_max_*` | 6 | BRD §9.2; `js/data.js:24` |
|
||
| `ref.employment_type` | Full-time, Part-time, Contract, Internship | — | 4 | BRD §9.2; `js/data.js:23` |
|
||
| `ref.education_level` | Highest education, ordered | `order_index` | 5 | `js/data.js:26` |
|
||
| `ref.skill` | Canonical skill taxonomy | `category`, `name_normalised` (generated, trigram-indexed), `is_reviewed` | ~500 | `js/data.js:46` (`skillsPool`) |
|
||
| `ref.skill_alias` | Alias → canonical skill | `skill_id`, `alias_normalised` UNIQUE, `source` | ~2,000 | new |
|
||
| `ref.currency` | ISO 4217 currencies. **PK is `code char(3)`, not `id`** — see the note below | `minor_unit`, `symbol`, `is_reporting_currency` | ~20 | ISO 4217; `js/offers.js:129` |
|
||
| `ref.assignment_role` | primary_recruiter, supporting_recruiter, sourcer, coordinator, hiring_manager, interviewer | `is_exclusive` (drives the partial unique index), `applies_to` CHECK in (`job`,`job_application`,`both`) | 6 | `_decisions.md` |
|
||
| `ref.app_module` | The 25 permission-controlled modules — one row per module boundary in `_decisions.md` Part 1 | `order_index`, `route_key` **nullable** (matches `js/app.js:7-16` where the module has a screen; `identity`, `audit`, `document_parsing` and the two `integrations_*` modules have no standalone route yet) | 25 | `_decisions.md` Part 1 module table; `05` §2.9 |
|
||
| `ref.permission_action` | The 10 permission verbs (`view`, `create`, `edit`, `transition`, `approve`, `assign`, `configure`, `export`, `delete`, `administer`) | `order_index` | 10 | `05` §2.2; ADR-0009 |
|
||
| `ref.tag` | Free recruiter tags on candidates | `colour_token`, `created_by_user_id` | ~100 | `js/candidates.js` |
|
||
| `ref.non_identifying_contact` | Addresses, domains and phone numbers that must never act as an identity key — agency mailboxes, shared household numbers, `info@`-style generics (§12.5 layer 4) | `kind` CHECK in (`email_address`,`email_domain`,`phone_e164`,`phone_prefix`), `value_normalised` UNIQUE with `kind`, `reason`, `added_by_user_id` | ~50 | §12.5 |
|
||
| `ref.retention_subject` | The vocabulary of things that can carry a retention period. `audit.retention_policy.subject_id` FKs here instead of holding a hardcoded CHECK list (§28.2, §31.2) | `default_trigger_event` CHECK in (`last_activity`,`created_at`,`first_seen_at`,`sent_at`,`finished_at`), `is_subject_driven boolean` (false = purged on a pure time sweep because the row has no reachable data subject) | 15 | §31.2 |
|
||
| `ref.consent_purpose` | *Folded into `ref.vocabulary_value`* — listed here so the §10.x reader can find it | — | 4 | — |
|
||
|
||
**`ref.currency` uses `code char(3)` as its primary key**, the single exception to §3's `bigint`
|
||
rule. Justification: the currency code is the value everyone reads, it is stable by international
|
||
standard, it is 3 bytes rather than 8, and `CHECK` constraints on money columns reference it
|
||
directly (§4.4). A surrogate id would mean every monetary row carries both an id and the code
|
||
anyway, since the code must be readable in reports.
|
||
|
||
**`ref.department.business_unit_id` is nullable — assumption.** The BRD lists 10 departments and
|
||
5 business units as independent vocabularies and jobs carry both (`js/data.js:95-96`), so it is
|
||
not established that a department belongs to exactly one business unit. The FK is nullable and
|
||
optional. If the business confirms a strict hierarchy, tighten it to `NOT NULL` in a later
|
||
migration; the reverse (loosening) would require rewriting every report group-by.
|
||
|
||
**`ref.region` replaces `ref.location.region`, and it is not a topology region.** Two things need
|
||
saying, in that order.
|
||
|
||
*First, why it is a table.* `adr/0009-permission-enforcement-strategy.md` is **Accepted**, it lists
|
||
`region` as the fourth of eight scope dimensions, and its `job_application` predicate reads
|
||
`job__current_version__location__region_id IN region_ids`. An earlier revision of this document gave
|
||
`ref.location` a plain `region **text**` column, so that predicate referenced a column that did not
|
||
exist and `ref.region` referenced a table that did not exist — `08` GAP-27, and the reason an
|
||
accepted ADR was resting on undefined schema. `ref.region` and `ref.location.region_id` therefore
|
||
land in migration `002`, where `ref.location` is created, and adr/0009's resolution path
|
||
(`job_version.location_id` → `ref.location.region_id` → `ref.region`) is expressible against real
|
||
columns. The free-text column is **replaced**, not kept alongside: two spellings of one grouping is
|
||
the drift R4 exists to warn about, and because `002` is not yet written there is no data to migrate —
|
||
this is a column definition, not a backfill. Consequence to carry into `06`: §2.5 currently describes
|
||
`/regions` as a derived distinct-value list over the text column, which becomes a read of `ref.region`
|
||
like every other `ref` collection.
|
||
|
||
*Second, what it is not.* `02-system-architecture.md` §1 states "no tenant column, no region column"
|
||
and excludes a `tenant`/`region` module. That constraint is about **topology** — one database, one
|
||
cloud region, residency and erasure per record, never per deployment — and `ref.region` does not
|
||
touch it. This is an organisational grouping of work locations, in the same class as
|
||
`ref.business_unit` and `ref.department`: a report group-by and an access-scope target, on one
|
||
database, with no data partitioned by it. Anyone reading `ref.region` as a re-entry point for
|
||
per-region databases is reading it wrong, and `02` §10 remains binding.
|
||
|
||
*Third, what is still open.* Whether `region` becomes a *grantable* `scope_type` is
|
||
`_open-items.md` **OPEN-05** (owner: Talent Lead + Talha) and is **not** decided here. §7.3 states
|
||
exactly which part of the region shape is in force and which single line OPEN-05 unblocks.
|
||
|
||
---
|
||
|
||
## 7. Identity, organization and access scope
|
||
|
||
### ERD 1 — Identity and organization
|
||
|
||
```mermaid
|
||
erDiagram
|
||
business_unit ||--o{ department : "contains"
|
||
department ||--o{ app_user : "employs"
|
||
app_user ||--o{ role_assignment : "granted"
|
||
role ||--o{ role_assignment : "granted as"
|
||
access_scope ||--o{ role_assignment : "scoped to"
|
||
role ||--o{ role_permission : "has"
|
||
permission ||--o{ role_permission : "in"
|
||
app_module ||--o{ permission : "module of"
|
||
permission_action ||--o{ permission : "action of"
|
||
app_user ||--o{ user_session : "authenticates"
|
||
app_user ||--o{ candidate_access_token : "issued"
|
||
business_unit ||--o{ access_scope : "target of"
|
||
department ||--o{ access_scope : "target of"
|
||
region ||--o{ location : "groups"
|
||
region ||--o{ access_scope : "target of (pending OPEN-05)"
|
||
app_user ||--o{ access_grant : "grantee of"
|
||
permission ||--o{ access_grant : "granted"
|
||
|
||
app_user {
|
||
bigint id PK
|
||
uuid public_id
|
||
text email_normalised
|
||
bigint department_id FK
|
||
bigint business_unit_id FK
|
||
bigint manager_user_id FK
|
||
text timezone
|
||
text status
|
||
}
|
||
role {
|
||
bigint id PK
|
||
text key
|
||
text access_level_key
|
||
boolean is_system
|
||
}
|
||
permission {
|
||
bigint id PK
|
||
bigint app_module_id FK
|
||
bigint permission_action_id FK
|
||
text key
|
||
}
|
||
role_permission {
|
||
bigint role_id FK
|
||
bigint permission_id FK
|
||
}
|
||
access_scope {
|
||
bigint id PK
|
||
text scope_type
|
||
bigint business_unit_id FK
|
||
bigint department_id FK
|
||
bigint region_id FK
|
||
bigint job_id FK
|
||
text scope_key
|
||
}
|
||
region {
|
||
bigint id PK
|
||
text key
|
||
text code
|
||
}
|
||
access_grant {
|
||
bigint id PK
|
||
uuid public_id
|
||
bigint grantee_user_id FK
|
||
bigint permission_id FK
|
||
text subject_table
|
||
timestamptz expires_at
|
||
timestamptz revoked_at
|
||
}
|
||
role_assignment {
|
||
bigint id PK
|
||
bigint user_id FK
|
||
bigint role_id FK
|
||
bigint access_scope_id FK
|
||
timestamptz valid_from
|
||
timestamptz valid_to
|
||
}
|
||
user_session {
|
||
bigint id PK
|
||
bigint user_id FK
|
||
bytea token_hash
|
||
timestamptz expires_at
|
||
}
|
||
candidate_access_token {
|
||
bigint id PK
|
||
bytea token_hash
|
||
text scope
|
||
timestamptz expires_at
|
||
}
|
||
```
|
||
|
||
### 7.1 `app.app_user`
|
||
|
||
*Purpose.* Every internal person who can sign in: the 66 named seats in BRD §4. Not candidates —
|
||
candidates are `app.candidate` and never authenticate (BRD §3 explicitly scopes the candidate
|
||
portal out; the RBAC role exists with zero users, `js/data.js:432`).
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | PK |
|
||
| `public_id` | `uuid` | no | UUIDv7, unique |
|
||
| `email_normalised` | `text` | no | `lower()` of the work address; the login identifier |
|
||
| `email_original` | `text` | no | Exactly as provisioned |
|
||
| `full_name` | `text` | no | |
|
||
| `display_name` | `text` | no | Defaults to `full_name` |
|
||
| `employee_ref` | `text` | yes | HRIS reference. Nullable — no HRIS integration is in scope |
|
||
| `job_title` | `text` | yes | |
|
||
| `department_id` | `bigint` | yes | FK `ref.department` |
|
||
| `business_unit_id` | `bigint` | yes | FK `ref.business_unit` |
|
||
| `manager_user_id` | `bigint` | yes | Self-FK; nullable at the top of the tree |
|
||
| `timezone` | `text` | no | IANA, default `'UTC'`, validated against `pg_timezone_names` |
|
||
| `locale` | `text` | no | Default `'en'` |
|
||
| `theme_preference` | `text` | yes | `light`/`dark`/`system`. Carries forward the one thing the prototype persists (`js/app.js:64,193`) |
|
||
| `status` | `text` | no | `active` / `invited` / `suspended` / `offboarded`, CHECK. Tier 2 (§4.6) — this set is not business-editable |
|
||
| `sso_subject` | `text` | yes | External IdP subject. Unique when present |
|
||
| `password_hash` | `text` | yes | Argon2id. NULL when SSO-only |
|
||
| `mfa_enrolled_at` | `timestamptz` | yes | |
|
||
| `last_login_at` | `timestamptz` | yes | |
|
||
| `avatar_colour_token` | `text` | yes | Preserves the deterministic avatar colouring `js/ui.js` already implements |
|
||
| audit + soft delete | | | §4.2, §4.5 |
|
||
|
||
**Constraints.** `uq_app_user_public_id UNIQUE (public_id)`. `CREATE UNIQUE INDEX
|
||
uq_app_user_email ON app.app_user (email_normalised) WHERE deleted_at IS NULL`.
|
||
`CREATE UNIQUE INDEX uq_app_user_sso ON app.app_user (sso_subject) WHERE sso_subject IS NOT NULL
|
||
AND deleted_at IS NULL`. `ck_app_user_email_shape CHECK (email_normalised = lower(email_normalised)
|
||
AND email_normalised ~ '^[^@[:space:]]+@[^@[:space:]]+[.][^@[:space:]]+$')`.
|
||
`ck_app_user_auth CHECK (password_hash IS NOT NULL OR sso_subject IS NOT NULL)` — an account with
|
||
neither cannot authenticate and is a provisioning bug.
|
||
`ck_app_user_manager_not_self CHECK (manager_user_id <> id)`.
|
||
|
||
**Indexes.** `ix_app_user_department (department_id) WHERE deleted_at IS NULL`;
|
||
`ix_app_user_status (status) WHERE deleted_at IS NULL`; trigram GIN on `full_name` for the user
|
||
picker.
|
||
|
||
**Profile.** *Status:* `text` + CHECK, Tier 2. *Required:* everything except the nullable columns
|
||
above, each justified by §4.9 reason 1 or 2. *Soft delete:* yes — an offboarded recruiter's name
|
||
must still render on the requisitions and scorecards they own, so the row can never be removed;
|
||
`status = 'offboarded'` is the operational state and `deleted_at` is reserved for a provisioning
|
||
mistake. *Retention:* employee data retained for the employment relationship plus 7 years
|
||
(**assumption**, policy `staff_record`); not subject to candidate erasure. *PII:* `personal`
|
||
(name, email, department); `internal` for ids and preferences. *Audit:* yes, plus explicit access
|
||
audit on any read of another user's profile. *Volume:* ~120 rows over 5 years (66 seats plus
|
||
churn). *Queries:* login by `email_normalised`; the user picker; recruiter workload rollups;
|
||
`scopes_for(user)` (§7.5).
|
||
|
||
### 7.2 `app.role`, `app.permission`, `app.role_permission`
|
||
|
||
*Purpose.* The real RBAC matrix, which the prototype only draws: **7 roles × 25 modules × 10 verbs**,
|
||
fully enumerated in `05` §2.9. In the prototype the matrix is a display widget — clicking a cell
|
||
mutates an in-memory array and nothing reads it (`js/rbac.js:78`, `js/rbac.js:83-85`,
|
||
`js/rbac.js:111-112`), and there is no `can()` anywhere. Here it is the input to the single
|
||
authorization decision point.
|
||
|
||
**On the prototype's 8 × 13 × 8.** `js/data.js:425-446` renders 8 roles, 13 modules and 8
|
||
permission types, and findings §D records that those are demo data derived from a single `level`
|
||
cutoff index — not a specification. BRD §4 is a **seat allocation** (66 named seats across the
|
||
recruiting population), not a permission catalogue. The catalogue this schema seeds is therefore
|
||
the one in `05` §2.1/§2.2: 7 roles, the 25 module boundaries from `_decisions.md` Part 1, and the
|
||
10 verbs from ADR-0009. The role, module and verb sets are seeded **configuration** in `ref`/`app`
|
||
tables, not structure — adding a role or a module is an INSERT plus grants, not a migration of the
|
||
permission model.
|
||
|
||
| Table | Columns | Notes |
|
||
|---|---|---|
|
||
| `app.role` | `id`, `key` UNIQUE, `label`, `description`, `access_level_key` (`view`/`edit`/`approve`/`manage`/`administrator`, CHECK), `is_system boolean`, `colour_token`, `order_index`, audit, soft delete | 7 seeded rows, one per role in `05` §2.1 (`system_admin`, `hr_admin`, `recruiter`, `director`, `hiring_manager`, `interviewer`, `management_viewer`). `is_system = true` rows cannot be deleted or have their `key` changed (trigger). `access_level_key` is documentation and UI grouping only — it is **not** used to derive permissions, unlike `buildMatrix(level)` at `js/data.js:439-444`, which derives the whole matrix from a single cutoff index and is why the prototype matrix is uniform nonsense |
|
||
| `app.permission` | `id`, `app_module_id` FK, `permission_action_id` FK, `key` generated `module_key \|\| '.' \|\| action_key`, UNIQUE `(app_module_id, permission_action_id)`, UNIQUE `(key)` | 250 rows — the full 25 modules × 10 verbs cross product, enumerated in migration `003`. Seeding the complete cross product (rather than only the pairs some role holds) is deliberate: `app.permission` is the *vocabulary*, `app.role_permission` is the *policy*, so a new grant is always an INSERT into one table against an existing FK, and a typo in an action key fails the FK instead of silently granting nothing. The generated `key` is what application code names (`can(actor, 'candidate.export', resource)`) |
|
||
| `app.role_permission` | `role_id` FK, `permission_id` FK, PK `(role_id, permission_id)`, `granted_at`, `granted_by_user_id` | Presence = allowed. **No `effect` column and no deny rows** — see below |
|
||
|
||
**Allow-only, no deny rows.** A deny-overrides model needs precedence rules, and precedence rules
|
||
in an authorization system with two developers and no existing implementation is where subtle
|
||
"why can this person see that" bugs live. If a genuine exception appears, the answer is a narrower
|
||
role, not a deny row. Stated as a tradeoff: we lose the ability to express "Recruiter, but not
|
||
export" without minting a role; we gain an authorization decision that is a single set-membership
|
||
test.
|
||
|
||
**Profile (all three).** *Status:* n/a. *Soft delete:* `role` only. *Retention:* permanent
|
||
configuration. *PII:* `internal`. *Audit:* yes — a permission grant change is one of the highest-value
|
||
audit events in the system. *Volume:* 7 / 250 / ~265 — the last figure is the count of populated
|
||
verb cells in the `05` §2.9 matrix, i.e. ~15% of the 1,750 `(role, module, verb)` triples the
|
||
capability test in `05` §2.11 walks. *Queries:* the RBAC matrix screen (one query
|
||
per role); `can()` resolution, which reads a cached `role_id → permission_key[]` map refreshed on
|
||
`role_permission` change.
|
||
|
||
### 7.3 `app.access_scope` — the generalized scope model
|
||
|
||
*Purpose.* One canonical, deduplicated registry of *what a grant applies to*. This is the table
|
||
that replaces the pile of role-specific mapping tables an ATS usually accretes —
|
||
`user_department`, `recruiter_job`, `hiring_manager_department`, `interviewer_application`,
|
||
`department_head_business_unit`, `pool_viewer_pool` — with a single dimension.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | PK |
|
||
| `scope_type` | `text` | no | CHECK in (`global`, `business_unit`, `department`, `location`, `job`, `job_application`, `talent_pool`) — **seven values in force.** `region` is the pending eighth; see "The `region` branch" below |
|
||
| `business_unit_id` | `bigint` | yes | FK `ref.business_unit` |
|
||
| `department_id` | `bigint` | yes | FK `ref.department` |
|
||
| `location_id` | `bigint` | yes | FK `ref.location` |
|
||
| `region_id` | `bigint` | yes | FK `ref.region`. Structurally provisioned, not yet grantable — no row can carry it until `scope_type`'s CHECK admits `'region'` |
|
||
| `job_id` | `bigint` | yes | FK `app.job` |
|
||
| `job_application_id` | `bigint` | yes | FK `app.job_application` |
|
||
| `talent_pool_id` | `bigint` | yes | FK `app.talent_pool` |
|
||
| `scope_key` | `text` | no | `GENERATED ALWAYS AS (scope_type \|\| ':' \|\| coalesce(business_unit_id, department_id, location_id, region_id, job_id, job_application_id, talent_pool_id, 0)::text) STORED` |
|
||
| `created_at` | `timestamptz` | no | |
|
||
|
||
**Constraints.**
|
||
|
||
```sql
|
||
CONSTRAINT ck_access_scope_target CHECK (
|
||
(scope_type = 'global' AND num_nonnulls(business_unit_id, department_id, location_id,
|
||
region_id, job_id, job_application_id, talent_pool_id) = 0)
|
||
OR (scope_type = 'business_unit' AND business_unit_id IS NOT NULL AND num_nonnulls(department_id, location_id, region_id, job_id, job_application_id, talent_pool_id) = 0)
|
||
OR (scope_type = 'department' AND department_id IS NOT NULL AND num_nonnulls(business_unit_id, location_id, region_id, job_id, job_application_id, talent_pool_id) = 0)
|
||
OR (scope_type = 'location' AND location_id IS NOT NULL AND num_nonnulls(business_unit_id, department_id, region_id, job_id, job_application_id, talent_pool_id) = 0)
|
||
-- the region branch: unreachable while scope_type's own CHECK omits 'region' (see below)
|
||
OR (scope_type = 'region' AND region_id IS NOT NULL AND num_nonnulls(business_unit_id, department_id, location_id, job_id, job_application_id, talent_pool_id) = 0)
|
||
OR (scope_type = 'job' AND job_id IS NOT NULL AND num_nonnulls(business_unit_id, department_id, location_id, region_id, job_application_id, talent_pool_id) = 0)
|
||
OR (scope_type = 'job_application' AND job_application_id IS NOT NULL AND num_nonnulls(business_unit_id, department_id, location_id, region_id, job_id, talent_pool_id) = 0)
|
||
OR (scope_type = 'talent_pool' AND talent_pool_id IS NOT NULL AND num_nonnulls(business_unit_id, department_id, location_id, region_id, job_id, job_application_id) = 0)
|
||
),
|
||
CONSTRAINT uq_access_scope_key UNIQUE (scope_key)
|
||
```
|
||
|
||
**Indexes.** `uq_access_scope_key` is the working index (every lookup is by scope identity).
|
||
Partial indexes on `job_id` and `department_id` for the reverse question "who has access to this
|
||
job".
|
||
|
||
**The `region` branch — what is in force, what is provisioned, and the one line OPEN-05 unblocks.**
|
||
`05` §2.3, `06` §1.15 and `08` GAP-27 all record the same three coordinated edits that adopting
|
||
`region` requires, and all three warn that the third is the one people miss: the `scope_type` CHECK,
|
||
a nullable `region_id` with its branch in the exclusive-arc CHECK, **and** the `scope_key` generated
|
||
column's `coalesce` list. Miss the third and two different scopes — say department 7 and region 7 —
|
||
collide on one `uq_access_scope_key` value, which is a **silent authorization defect**, not a
|
||
migration error: the second grant is rejected as a duplicate and the operator sees an unexplained
|
||
failure, or worse, an existing row is reused and a departmental grant silently becomes a regional
|
||
one. The resolution here is to make that class of mistake structurally impossible while leaving the
|
||
business question untouched:
|
||
|
||
| Edit | Where | Status |
|
||
|---|---|---|
|
||
| `region_id bigint NULL REFERENCES ref.region(id)` | migration `011`, with the table | **Done.** A nullable `bigint` on a ~2,000-row table |
|
||
| `region_id` added to **every** other branch's `num_nonnulls(...) = 0` list, plus the `region` branch itself | migration `011`, with the table | **Done.** This is the half that cannot be added safely later — it tightens an existing constraint |
|
||
| `region_id` added to the `scope_key` `coalesce` list | migration `011`, with the table | **Done.** A generated column cannot be redefined in place; changing it later is `DROP COLUMN` + `ADD COLUMN` + a rebuild of `uq_access_scope_key` on a table every request reads |
|
||
| `'region'` added to the `scope_type` CHECK | **pending `_open-items.md` OPEN-05** | **Not done, deliberately.** One `DROP CONSTRAINT` / `ADD CONSTRAINT` pair on a CHECK |
|
||
|
||
So `region` is a **pending eighth `scope_type`** exactly as `05` §2.3 and `06` §1.15 state it — not a
|
||
value in force, `identity` rejects it, and no `access_scope` row can carry one. What has changed is
|
||
that adopting it is now one line instead of three coordinated edits, and the collision hazard is
|
||
gone in both worlds. **This is not a decision on OPEN-05:** if the answer is *no*, `ref.region` stays
|
||
a reporting vocabulary, regional desks are granted several `location` rows (dimension 4, already in
|
||
the enum), and the cost is one unused nullable column and one unreachable CHECK branch. If the answer
|
||
is *yes*, the branch and the key are already right.
|
||
|
||
**Where this leaves adr/0009, stated so the two documents agree rather than each asserting.**
|
||
adr/0009 resolves scope into a struct containing `region_ids` and appends
|
||
`job__current_version__location__region_id IN region_ids` to the `job_application` predicate. Both are
|
||
correct against this schema and both are **empty by construction** until the CHECK admits `'region'`:
|
||
`region_ids` is populated from `access_scope` rows that cannot exist, so the predicate branch
|
||
evaluates against an empty array. That is the honest reading of the ADR against a seven-value enum —
|
||
the branch exists, it is written once, it is exercised by a test that asserts an empty `region_ids`
|
||
contributes no rows, and it starts returning rows the day OPEN-05 answers yes. `05` §2.3's warning
|
||
that "an unexercised authorization branch is worse than a missing feature" is satisfied by that test
|
||
rather than by omitting the column, which is the version that left the ADR pointing at nothing.
|
||
|
||
**Why generalized, and why not polymorphic.** Three arguments, in order of weight.
|
||
|
||
1. **The alternative does not close.** `05` §2.1 defines 7 roles; the access dimensions a real ATS
|
||
grants along are business unit, department, location, requisition, individual application and
|
||
talent pool — 6 dimensions. Role-specific mapping tables give up to 7 × 6 = 42 possible tables
|
||
and, in practice, a new one each time a role gains a scope. Each carries its own uniqueness
|
||
rules, its own history columns, and its own bug. `can(actor, action, resource)` would have to
|
||
know all of them; adding a scope would mean touching the authorization core.
|
||
2. **One table means one authorization query.** `_decisions.md` Part 1 makes `iam.can()` the
|
||
single chokepoint and makes it the same code path the chatbot uses, because "the chatbot must
|
||
never bypass access controls" is only real if there is exactly one implementation to review.
|
||
That is achievable with one scope dimension and not achievable with 42.
|
||
3. **It is generalized without being polymorphic.** The obvious generic design —
|
||
`(scope_type text, scope_id bigint)` — is what P7 forbids, because Postgres cannot enforce it:
|
||
a `scope_id` of 999999 pointing at a deleted department is indistinguishable from a valid row,
|
||
and `_decisions.md` rejects exactly this shape for assignments ("a polymorphic FK cannot be
|
||
enforced by the database at all"). Typed nullable columns plus `num_nonnulls` give a genuine FK
|
||
per dimension *and* one table. The cost is six mostly-null `bigint` columns — 48 bytes per row
|
||
on a table of a few hundred rows, which is nothing — and a verbose CHECK written once.
|
||
|
||
**Deduplication matters.** `scope_key` being unique means "the Engineering department" is one row
|
||
referenced by many grants, so revoking access to a department is a query over
|
||
`role_assignment WHERE access_scope_id = ?` rather than a scan of six tables.
|
||
|
||
**Profile.** *Status:* n/a. *Required:* `scope_type` and exactly one target. *Soft delete:* no —
|
||
a scope row is referenced by historical grants and must stay resolvable. *Retention:* permanent.
|
||
*PII:* `internal`. *Audit:* creation only (the grant carries the interesting event).
|
||
*Volume:* ~2,000 (bounded by distinct departments, jobs and pools ever scoped). *Queries:*
|
||
`scopes_for(user)`; "who can see this requisition"; the access-review export.
|
||
|
||
### 7.4 `app.role_assignment`
|
||
|
||
*Purpose.* Who holds which role over which scope, **and when** — the authorization grant. Interval
|
||
form, so an access review can answer "who could approve offers in Engineering last March".
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `user_id` | `bigint` | no | FK `app.app_user` |
|
||
| `role_id` | `bigint` | no | FK `app.role` |
|
||
| `access_scope_id` | `bigint` | no | FK `app.access_scope` |
|
||
| `valid_from` | `timestamptz` | no | Default `now()` |
|
||
| `valid_to` | `timestamptz` | yes | `NULL` = current |
|
||
| `granted_by_user_id` | `bigint` | no | |
|
||
| `reason` | `text` | yes | Required by application policy for `global` scope grants |
|
||
| `created_at` | `timestamptz` | no | |
|
||
|
||
**Constraints.** `ck_role_assignment_interval CHECK (valid_to IS NULL OR valid_to > valid_from)`.
|
||
```sql
|
||
CONSTRAINT ex_role_assignment_overlap EXCLUDE USING gist (
|
||
user_id WITH =, role_id WITH =, access_scope_id WITH =,
|
||
tstzrange(valid_from, valid_to) WITH &&
|
||
)
|
||
```
|
||
so the same grant cannot be duplicated across overlapping periods — which is how "revoked" grants
|
||
quietly come back.
|
||
|
||
**"Current" is a time predicate, not `valid_to IS NULL`.** This is the single most important
|
||
sentence in §7. A grant issued with an end date — temporary cover during leave, a contractor until
|
||
31 December, the delegated approval authority the approval engine (§10) assumes — has
|
||
`valid_to IS NOT NULL` from the moment it is created. If "current" is spelled `valid_to IS NULL`,
|
||
every such grant is invisible to the effective-scope view and the user silently has **no** access;
|
||
symmetrically, a grant with `valid_from` in the future is live immediately. Both are authorization
|
||
defects in the one module every decision in this design routes through, so the predicate is fixed
|
||
once, here, and reused verbatim:
|
||
|
||
```sql
|
||
-- app.is_live(valid_from, valid_to) — an IMMUTABLE-shaped inline predicate, written out
|
||
-- rather than wrapped in a function so the planner can still use the indexes below.
|
||
ra.valid_from <= now() AND (ra.valid_to IS NULL OR ra.valid_to > now())
|
||
```
|
||
|
||
**Indexes.**
|
||
|
||
| Index | Purpose |
|
||
|---|---|
|
||
| `ix_role_assignment_user_period (user_id, valid_from, valid_to)` | The correct hot path: it serves the time predicate above for both open-ended and bounded grants. |
|
||
| `ix_role_assignment_open (user_id) WHERE valid_to IS NULL` | Retained as a narrow fast path only, because open-ended grants are the large majority. It is **never** the sole source of "current" — the query always also unions/scans the bounded grants via the index above. |
|
||
| `ix_role_assignment_scope (access_scope_id, valid_from, valid_to)` | The reverse question, "who has access to this scope", same predicate. |
|
||
|
||
**Permission test required (A-22 suite).** Grant a role with `valid_to = now() + interval '1 day'`
|
||
and assert the user can act **today**; grant one with `valid_from = now() + interval '1 day'` and
|
||
assert the user cannot act today. Those two assertions are the regression guard for this entire
|
||
section, and their absence is what would let the wrong predicate ship.
|
||
|
||
**Profile.** *Status:* interval, not a status column. *Soft delete:* no — revocation is
|
||
`valid_to`, and the row is the audit evidence. *Retention:* permanent (access-review obligation).
|
||
*PII:* `internal`. *Audit:* yes, high value. *Volume:* ~1,500 over 5 years. *Queries:*
|
||
`scopes_for(user)` on every authenticated request (cached for the request lifetime only, per
|
||
adr/0009 and §7.5 — not per session);
|
||
quarterly access review; "who had `offers.approve` on 2026-03-14".
|
||
|
||
### 7.5 The resolved-scope view, and why assignment is a separate concept
|
||
|
||
Authorization scope is **not** the same thing as operational ownership, and conflating them is a
|
||
common ATS modelling error. `role_assignment` says *what a person is permitted to see and do*.
|
||
`job_assignment` / `job_application_assignment` (§16) say *who is working this requisition*. They
|
||
differ in cardinality (a coordinator can be assigned without being granted anything extra), in
|
||
who writes them (HR-admin vs recruiting lead), and in lifetime.
|
||
|
||
They do interact, and the interaction is expressed as one view rather than as denormalised grants:
|
||
|
||
```sql
|
||
CREATE VIEW app.v_user_effective_scope AS
|
||
-- 1. role assignments (origin = role_grant). "Explicit grant" is branch 4, not this one.
|
||
SELECT ra.user_id, s.scope_type, s.business_unit_id, s.department_id, s.location_id,
|
||
s.region_id, s.job_id, s.job_application_id, s.talent_pool_id,
|
||
NULL::bigint AS candidate_id,
|
||
'role_grant'::text AS origin, r.key AS role_key, NULL::text AS permission_key
|
||
FROM app.role_assignment ra
|
||
JOIN app.access_scope s ON s.id = ra.access_scope_id
|
||
JOIN app.role r ON r.id = ra.role_id
|
||
WHERE ra.valid_from <= now()
|
||
AND (ra.valid_to IS NULL OR ra.valid_to > now())
|
||
UNION ALL
|
||
-- 2. operational ownership of a requisition implies scope over it
|
||
SELECT ja.user_id, 'job', NULL, NULL, NULL, NULL, ja.job_id, NULL, NULL, NULL,
|
||
'job_assignment', ar.key, NULL
|
||
FROM app.job_assignment ja JOIN ref.assignment_role ar ON ar.id = ja.role_id
|
||
WHERE ja.valid_from <= now()
|
||
AND (ja.valid_to IS NULL OR ja.valid_to > now())
|
||
UNION ALL
|
||
-- 3. interview participation implies scope over that application only
|
||
SELECT ip.user_id, 'job_application', NULL, NULL, NULL, NULL, NULL, i.job_application_id, NULL,
|
||
NULL, 'interview_participation', 'interviewer', NULL
|
||
FROM app.interview_participant ip
|
||
JOIN app.interview i ON i.id = ip.interview_id
|
||
WHERE ip.user_id IS NOT NULL AND i.deleted_at IS NULL
|
||
UNION ALL
|
||
-- 4. a live, unrevoked access_grant (§7.7). origin = access_grant; carries a permission, not a role.
|
||
SELECT g.grantee_user_id,
|
||
CASE g.subject_table
|
||
WHEN 'candidate' THEN 'candidate' -- see the note: view-only scope_type
|
||
WHEN 'job' THEN 'job'
|
||
WHEN 'job_application' THEN 'job_application'
|
||
WHEN 'interview' THEN 'job_application' -- resolved through interview.job_application_id
|
||
WHEN 'offer' THEN 'job_application' -- resolved through offer.job_application_id
|
||
END,
|
||
NULL, NULL, NULL, NULL,
|
||
g.job_id,
|
||
coalesce(g.job_application_id, i.job_application_id, o.job_application_id),
|
||
NULL,
|
||
g.candidate_id,
|
||
'access_grant', NULL, p.key
|
||
FROM app.access_grant g
|
||
JOIN app.permission p ON p.id = g.permission_id
|
||
LEFT JOIN app.interview i ON i.id = g.interview_id
|
||
LEFT JOIN app.offer o ON o.id = g.offer_id
|
||
WHERE g.granted_at <= now() AND g.expires_at > now() AND g.revoked_at IS NULL;
|
||
```
|
||
|
||
**All four branches use the time predicate from §7.4, not `valid_to IS NULL`.** A scheduled
|
||
recruiter handover — Ahmed covers requisition 412 from Monday to Friday — is expressed as a
|
||
bounded `job_assignment` row, and branch 2 must see it on Tuesday and stop seeing it on Saturday.
|
||
Branch 3 has no interval of its own: panel membership is current by existence, and removal is a
|
||
delete of the participant row or a cancellation of the interview. Branch 4 is the same predicate with
|
||
the nullable half removed, because `access_grant.expires_at` is `NOT NULL` (§7.7): there is no
|
||
open-ended grant to express, which is the entire point of that column.
|
||
|
||
**Branch 4 adds three columns, and each one is load-bearing rather than convenience.**
|
||
|
||
| Column | Why the view needs it |
|
||
|---|---|
|
||
| `region_id` | Branches 1's `access_scope` gained it (§7.3). Projecting it is what makes adr/0009's `region_ids` bucket readable from one place the day OPEN-05 answers yes, instead of a second edit to the view |
|
||
| `candidate_id` | A grant over a **candidate** has no `access_scope` counterpart, and it is the case the design exists for: `05` §9.3 makes `access_grant` the sanctioned path for a `system_admin` who needs candidate data, precisely so the alternative — a direct database connection — does not become the norm. Every other branch resolves a candidate *through* an application, an interview or a pool (`05` §2.6); this one does not, so it needs its own column |
|
||
| `permission_key` | A grant is per-`(module, verb)`; a role assignment is per-role. Without this column the two are indistinguishable in the view, and a grant of `candidate.view` would read as a grant of everything the role holds at that scope |
|
||
|
||
**`scope_type = 'candidate'` is emitted by the view and storable by nothing.** It appears in branch 4
|
||
only. `app.access_scope.scope_type` has no `candidate` value and must not gain one — a stored
|
||
candidate scope would be a permanent grant of direct candidate visibility, which is the thing `05`
|
||
§2.6 forbids ("a candidate is never visible directly except to `hr_admin`"). This is the concrete
|
||
reason `05` §2.3 makes `explicit_grant` a fourth **`origin`** rather than a tenth `scope_type`: the
|
||
grant table deliberately carries subjects the stored scope vocabulary does not, and the view is where
|
||
the two vocabularies meet. A reader who diffs the view's `scope_type` domain against the table's CHECK
|
||
and finds `candidate` in one and not the other has found the design, not a defect.
|
||
|
||
**How `scopes_for(user)` consumes branch 4 — and why it does not merge it.** `05` §2.6 and adr/0009
|
||
both resolve scope into a struct with one key per `scope_type` *plus a separate* `grants` map, and
|
||
that separation is not redundancy. `scopes_for` reads all four branches from this view in one query,
|
||
lands branches 1–3 in `job_ids` / `application_ids` / `department_ids` / `business_unit_ids` /
|
||
`location_ids` / `talent_pool_ids` / `region_ids`, and lands branch 4 in
|
||
`grants[<subject>][<permission_key>]` — **never** in the positional buckets. Merging a grant into
|
||
`application_ids` would widen a time-boxed grant of one verb into every verb the user already holds
|
||
at application scope, which silently converts the narrowest object in the authorization model into
|
||
the broadest. adr/0009's predicate spells the two halves separately for the same reason:
|
||
`… OR id IN application_ids OR id IN grants[job_application]`. The view is the declarative union
|
||
used by the reverse query ("who can see this requisition"), the quarterly access review, and
|
||
`scopes_for(user, ts)` reconstruction for audit; the struct is the per-request materialisation. Same
|
||
rows, two readings, one definition.
|
||
|
||
**Migration placement.** This view reads `app.role_assignment`, `app.access_scope`,
|
||
`app.job_assignment`, `app.access_grant`, `app.permission`, `app.interview_participant`,
|
||
`app.interview` and `app.offer`, so it cannot be created before all of them exist. Branches 1–2 and
|
||
**4** are created in migration **011** (the first migration in which `access_scope`, `access_grant`,
|
||
`job_assignment` and `job_application` all exist), with branch 4's `interview` and `offer` joins and
|
||
their `coalesce` terms absent at that point. Branch 3 **and** branch 4's `interview` resolution are
|
||
added by `CREATE OR REPLACE VIEW` in migration **019** when `interview_participant` and `interview`
|
||
arrive; branch 4's `offer` resolution is added by a further `CREATE OR REPLACE VIEW` in **023**. All
|
||
three steps only change the view body and never its column list, which is what `CREATE OR REPLACE
|
||
VIEW` permits — a column-list change would require `DROP VIEW`, and the view is depended on. §33
|
||
records all three steps; creating the view in 003 as an earlier draft of this document did is a
|
||
forward reference and would fail.
|
||
|
||
Branch 3 is the mechanism behind BRD §4's "Interviewer — access assigned interviews and submit
|
||
scorecards" and `_decisions.md`'s "interviewer sees only candidates on assigned interviews". It is
|
||
derived, never granted, so removing someone from an interview panel removes their access in the
|
||
same statement — which a stored grant would not.
|
||
|
||
The view is not queried per row. `iam.can()` materialises a compact scope set
|
||
(`{global: false, departments: [3,7], jobs: [412, 509], applications: [...], grants: {…}}`) and every
|
||
list query is filtered by it. **Assumption:** at 66 seats the largest scope set is a few hundred ids,
|
||
which is small enough to pass as a parameter array; if a user's application-level scope set ever
|
||
exceeds ~2,000 ids, switch that branch to an `EXISTS` subquery rather than an `IN` list.
|
||
|
||
**The materialised set is cached for the request lifetime, never across requests** — adr/0009 pins
|
||
this and branch 4 is why it matters enough to restate here. A revoked `access_grant` and a closed
|
||
`role_assignment` must both take effect on the **next** request, so the maximum staleness window is a
|
||
single in-flight request. A per-session cache would keep a revoked grant alive for as long as the
|
||
session lasts, and a revocation that does not revoke is worse than no grant mechanism at all.
|
||
|
||
### 7.6 Remaining identity tables
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.user_session` | Active sessions and refresh tokens. Replaces the inert session-timeout select at `js/settings.js:148-154` | `user_id`, `token_hash bytea` UNIQUE, `kind` (`web`/`api`), `issued_at`, `last_seen_at`, `expires_at`, `revoked_at`, `revoked_reason`, `ip inet`, `user_agent` | `ck` expiry > issue; `ix_user_session_user (user_id) WHERE revoked_at IS NULL AND expires_at > now()` | *PII:* `personal` (IP, user agent). *Retention:* 90 days after expiry, then deleted — this is the one table where hard delete is correct, since a dead session has no historical value. *Audit:* login/logout as access events. *Volume:* ~200k over 5 years, pruned |
|
||
| `app.candidate_access_token` | Capability tokens for candidate-facing surfaces (§3.2) | `token_hash bytea` UNIQUE, `purpose` (`status_page`/`document_upload`/`interview_confirm`/`offer_response`/`assessment_invite`, CHECK), `candidate_id`, `job_application_id`, `interview_id`, `offer_id`, `assessment_assignment_id` (typed nullable + `num_nonnulls = 1`), `issued_at`, `expires_at`, `consumed_at`, `revoked_at`, `issued_by_user_id`, `use_count` | `ck_token_subject`; `ck_expiry`; `ix` on `(expires_at) WHERE consumed_at IS NULL AND revoked_at IS NULL` for the sweep | *PII:* `internal` (the hash is not personal, the linkage is). *Retention:* deleted 30 days after expiry. *Audit:* issue, use and revoke, all three. *Volume:* ~400k over 5 years |
|
||
| `app.user_availability_rule` | Recurring working hours per user, wall-clock (§4.3) | `user_id`, `timezone`, `weekday smallint` CHECK 0–6, `local_start_time time`, `local_end_time time`, `valid_from date`, `valid_to date` | `ck_time_order CHECK (local_end_time > local_start_time)`; `ex` on `(user_id, weekday, daterange(valid_from, valid_to))` overlap | *PII:* `personal`. *Volume:* ~2,000. Detailed in §23 with the rest of scheduling |
|
||
| `app.user_availability_exception` | One-off leave or blocked periods | `user_id`, `starts_at`, `ends_at`, `kind` (`leave`/`blocked`/`extra`), `note` | `ck ends_at > starts_at`; GiST index on `tstzrange(starts_at, ends_at)` | *PII:* `personal` (leave is inferable). *Volume:* ~10,000 |
|
||
|
||
### 7.7 `app.access_grant` — the time-boxed exception
|
||
|
||
*Purpose.* One explicit, per-resource, time-bounded, reason-required grant of one `(module, verb)`
|
||
to one user. It is the sanctioned exception path for the cases the scope model deliberately cannot
|
||
express: a `system_admin` who needs to see one candidate to debug a data issue (`05` §9.3), a
|
||
recruiter loaned one application outside their desk, an interviewer who must submit a scorecard after
|
||
the interview reached a terminal status and their derived scope ended (adr/0009 R7). It **supplements**
|
||
`access_scope` and replaces nothing: `access_scope` is the deduplicated registry of *standing* scope
|
||
targets, and a grant is the opposite — singular, expiring, and attributable to the person who issued it.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | PK |
|
||
| `public_id` | `uuid` | no | UUIDv7, unique. The grant is an administered object with its own screen, so it needs an external id |
|
||
| `grantee_user_id` | `bigint` | no | FK `app.app_user` — who receives it |
|
||
| `permission_id` | `bigint` | no | FK `app.permission` — the `(module, verb)` pair, so a grant of `candidate.view` is not a grant of `candidate.export` |
|
||
| `subject_table` | `text` | no | The P7 discriminator. CHECK in (`candidate`, `job`, `job_application`, `interview`, `offer`) |
|
||
| `candidate_id` | `bigint` | yes | FK `app.candidate` |
|
||
| `job_id` | `bigint` | yes | FK `app.job` |
|
||
| `job_application_id` | `bigint` | yes | FK `app.job_application` |
|
||
| `interview_id` | `bigint` | yes | FK `app.interview`. Added by `ALTER` in migration `019` (§33) |
|
||
| `offer_id` | `bigint` | yes | FK `app.offer`. Added by `ALTER` in migration `023` (§33) |
|
||
| `reason` | `text` | no | Free text, mandatory, shown in the audit trail and in the weekly active-grant report. Not a vocabulary — see the note below |
|
||
| `granted_by_user_id` | `bigint` | no | FK `app.app_user` — who issued it |
|
||
| `granted_at` | `timestamptz` | no | Default `now()` |
|
||
| `expires_at` | `timestamptz` | **no** | **The single most important column on this table.** `NOT NULL`, with a 30-day ceiling |
|
||
| `revoked_at` | `timestamptz` | yes | Early revocation |
|
||
| `revoked_by_user_id` | `bigint` | yes | FK `app.app_user` |
|
||
| `revoke_reason` | `text` | yes | |
|
||
| `created_at` | `timestamptz` | no | Append-only — no `updated_at`, no `updated_by_user_id` (§4.2) |
|
||
|
||
**No separate `created_by_user_id`.** §4.2 makes it `NOT NULL` on tables where a human must have acted,
|
||
and this is one — but `granted_by_user_id` **is** that column, named for what it means. Carrying both
|
||
would give two columns that must always agree and a CHECK to make them agree, which is a constraint
|
||
whose only job is to police a duplication this table does not need.
|
||
|
||
**Constraints.**
|
||
|
||
```sql
|
||
-- Final form, after 023. At 011 the subject CHECK has three branches and no interview_id/offer_id;
|
||
-- 019 and 023 each add their column and re-declare it (§33, and the candidate_access_token pattern).
|
||
CONSTRAINT ck_access_grant_subject_table
|
||
CHECK (subject_table IN ('candidate','job','job_application','interview','offer')),
|
||
CONSTRAINT ck_access_grant_window CHECK (expires_at > granted_at),
|
||
-- duration-form, not `expires_at <= granted_at + interval '30 days'`: see the note below
|
||
CONSTRAINT ck_access_grant_ceiling CHECK (expires_at - granted_at <= interval '30 days'),
|
||
CONSTRAINT ck_access_grant_no_self CHECK (grantee_user_id <> granted_by_user_id),
|
||
CONSTRAINT ck_access_grant_revoked CHECK (num_nonnulls(revoked_at, revoked_by_user_id,
|
||
revoke_reason) IN (0, 3)),
|
||
CONSTRAINT ck_access_grant_revoked_order CHECK (revoked_at IS NULL OR revoked_at >= granted_at),
|
||
CONSTRAINT ck_access_grant_subject CHECK (
|
||
(subject_table = 'candidate' AND candidate_id IS NOT NULL AND num_nonnulls(job_id, job_application_id, interview_id, offer_id) = 0)
|
||
OR (subject_table = 'job' AND job_id IS NOT NULL AND num_nonnulls(candidate_id, job_application_id, interview_id, offer_id) = 0)
|
||
OR (subject_table = 'job_application' AND job_application_id IS NOT NULL AND num_nonnulls(candidate_id, job_id, interview_id, offer_id) = 0)
|
||
OR (subject_table = 'interview' AND interview_id IS NOT NULL AND num_nonnulls(candidate_id, job_id, job_application_id, offer_id) = 0)
|
||
OR (subject_table = 'offer' AND offer_id IS NOT NULL AND num_nonnulls(candidate_id, job_id, job_application_id, interview_id) = 0)
|
||
)
|
||
```
|
||
|
||
**P7, applied — and `05` §2.3's sketch is corrected here, not copied.** `05` §2.3 sketches this table
|
||
with `subject_table text NOT NULL` plus `subject_id bigint NOT NULL`, which is exactly the shape P7
|
||
(§2, §7.3) forbids: a `subject_id` of 999999 pointing at a deleted application is indistinguishable
|
||
from a valid row, and Postgres cannot enforce it at all. The correction is the idiom already used by
|
||
`access_scope`, `candidate_access_token`, `task` and `approval_request` — five typed nullable FKs,
|
||
`num_nonnulls(...) = 1` expressed as the exclusive-arc CHECK above, and `subject_table` retained as
|
||
the discriminator with each branch tying the discriminator to its column (§4.8). The prose intent of
|
||
`05` §2.3 is unchanged; only the enforceability is.
|
||
|
||
**`interview` and `offer` are storable only from `019` and `023`, and the CHECK says so by itself.**
|
||
`subject_table`'s own CHECK carries all five values from `011`, but the exclusive-arc CHECK is
|
||
re-declared by `019` and `023` as each column is added (§33, and the same pattern
|
||
`candidate_access_token` and `outbound_message` use). Between `011` and `019` a row with
|
||
`subject_table = 'interview'` matches no branch and is rejected by `ck_access_grant_subject` — a
|
||
constraint violation naming the constraint, which is the right failure. No application code can be
|
||
written against a subject whose column does not exist yet, and nothing silently accepts one.
|
||
|
||
**The 30-day ceiling is written as a duration, and the reason is §4.3.** adr/0009 and `05` §2.3 both
|
||
write it `expires_at <= granted_at + interval '30 days'`. That compiles, but `timestamptz + interval`
|
||
is **stable, not immutable** — adding `30 days` crosses a DST boundary differently depending on the
|
||
session `TimeZone`, so the same row can satisfy the CHECK under one session and be re-validated
|
||
differently under another (a `VALIDATE CONSTRAINT`, a table rewrite, a restore under a different
|
||
GUC). `expires_at - granted_at` uses `timestamptz - timestamptz`, which **is** immutable, and
|
||
`interval <= interval` compares at 24-hour days, so the constraint bounds the grant at exactly 720
|
||
hours in every timezone. Same ceiling, no session dependency. This is the same class of correction
|
||
§4.3 makes for `created_at::date`, applied to the other end of the same problem.
|
||
|
||
**Append-only plus revocation, enforced as privilege *and* trigger (§4.2, §4.8).** A grant is
|
||
evidence: the row records that a named person gave a named person access to a named record for a
|
||
named reason, and nothing about that may be edited afterwards. Revocation is a write to three columns
|
||
and nothing else.
|
||
|
||
```sql
|
||
REVOKE UPDATE, DELETE ON app.access_grant FROM talentflow_app;
|
||
GRANT SELECT, INSERT ON app.access_grant TO talentflow_app;
|
||
GRANT UPDATE (revoked_at, revoked_by_user_id, revoke_reason)
|
||
ON app.access_grant TO talentflow_app;
|
||
|
||
CREATE TRIGGER tg_access_grant_immutable
|
||
BEFORE UPDATE ON app.access_grant
|
||
FOR EACH ROW
|
||
WHEN (OLD.revoked_at IS NOT NULL
|
||
OR (to_jsonb(OLD) - 'revoked_at' - 'revoked_by_user_id' - 'revoke_reason')
|
||
IS DISTINCT FROM
|
||
(to_jsonb(NEW) - 'revoked_at' - 'revoked_by_user_id' - 'revoke_reason'))
|
||
EXECUTE FUNCTION app.tg_raise_immutable();
|
||
|
||
CREATE TRIGGER tg_access_grant_no_delete
|
||
BEFORE DELETE ON app.access_grant
|
||
FOR EACH ROW EXECUTE FUNCTION app.tg_raise_immutable();
|
||
```
|
||
|
||
Three rules across the pair, and both halves of the shape are deliberate. The split into an `UPDATE`
|
||
trigger and a `DELETE` trigger is **forced, not stylistic** — a `WHEN` clause on a combined
|
||
`BEFORE UPDATE OR DELETE` trigger can reference neither `TG_OP` nor `NEW` (§4.8, fourth syntax fact),
|
||
and the allow-list needs `NEW`. The `to_jsonb` allow-list form itself is deliberate per §4.8 — a
|
||
column added by a later migration is protected the moment it exists rather than the moment someone
|
||
remembers it:
|
||
|
||
| Rule | Raises when |
|
||
|---|---|
|
||
| A revoked grant is frozen | `OLD.revoked_at IS NOT NULL` on any `UPDATE` — a grant is revoked **once**, and "un-revoking" is a new grant with its own reason and its own audit event |
|
||
| Only the three revocation columns may ever change | the `to_jsonb` difference is non-empty after subtracting them |
|
||
| Nothing is ever deleted | any `DELETE` — `tg_access_grant_no_delete` carries no `WHEN` clause, so it fires unconditionally |
|
||
|
||
**No usage counters on this row, deliberately.** `candidate_access_token` (§7.6) carries `use_count`
|
||
and `consumed_at`; this table does not. Grant *usage* is an access event and belongs in
|
||
`audit.audit_event` — `05` §7.3 audits "`access_grant` issued/used/revoked/expired" — where it carries
|
||
the request id, the actor, the entity and the outcome. A counter here would be a second, driftable
|
||
record of the same fact, and the interesting question ("was this grant ever actually used, and for
|
||
what") needs the audit row anyway.
|
||
|
||
**`reason` is free text and `revoke_reason` is free text, unlike `ats_result_override.override_reason_id`
|
||
(§20.7).** The asymmetry is intentional and worth stating because the two tables otherwise look
|
||
alike. An override reason must be *enumerable* because override rates are aggregated as a fairness
|
||
signal (`05` §5.2). A grant reason is read by a human in a weekly report and in an audit
|
||
investigation, is never aggregated, and a vocabulary would produce `other` on most rows — which is a
|
||
vocabulary that has learned nothing and a free-text field with extra steps.
|
||
|
||
**Indexes.**
|
||
|
||
| Index | Purpose |
|
||
|---|---|
|
||
| `ix_access_grant_live (grantee_user_id) WHERE revoked_at IS NULL` | The hot path: `scopes_for(user)` branch 4. Narrow because expiry is a time predicate the planner applies after, and a partial index on `expires_at > now()` is not possible — `now()` is not immutable, so it cannot appear in an index predicate (§4.3, §4.8) |
|
||
| `ix_access_grant_expiry (expires_at) WHERE revoked_at IS NULL` | The expiry sweep that writes the `access_grant.expired` audit event, and the "expiring in the next 3 days" admin view |
|
||
| `ix_access_grant_grantor (granted_by_user_id, granted_at DESC)` | adr/0009 R4's weekly report of active grants **by grantor** — the control that detects the grant path becoming the normal path |
|
||
| `ix_access_grant_permission (permission_id, granted_at DESC)` | "more than ~5 grants a week for the same `(module, verb)`" — adr/0009 R4's threshold, which is a signal the *scope model* is wrong, not that more grants are needed |
|
||
| `ix_access_grant_candidate (candidate_id) WHERE candidate_id IS NOT NULL` | The reverse question on the highest-sensitivity subject: "who has been granted access to this candidate, and by whom" |
|
||
|
||
**Two permission tests this table exists to make possible (A-22 suite), both mandatory.** They are
|
||
the analogue of §7.4's interval assertions, and their absence is what would let a broken grant path
|
||
ship: (1) issue a grant with `expires_at = now() + interval '1 hour'`, assert the grantee can act,
|
||
then move the clock past expiry and assert the grantee **cannot** — the expiry must be a live
|
||
predicate, not a nightly sweep; (2) revoke a live grant and assert the grantee cannot act **on the
|
||
next request**, which is the request-lifetime cache assertion from §7.5 stated as a test.
|
||
|
||
**Profile.** *Status:* no status column — the state is `(expires_at, revoked_at)` against `now()`,
|
||
per §7.4's rule that "current" is a time predicate. *Required:* grantee, permission, subject,
|
||
reason, grantor, `granted_at`, `expires_at`; the three revocation columns are nullable under §4.9
|
||
reason 1 (not yet known) and are all-or-nothing. *Soft delete:* no — a grant row is the evidence that
|
||
an exception was made, so it can never be removed; revocation is `revoked_at`. *Retention:* permanent
|
||
(access-review obligation), and **not** subject to candidate erasure even when `candidate_id` is set —
|
||
the row records what *staff* did, which §31.2 and `05` §7.5 keep on a separate lawful basis from the
|
||
candidate's own data. *PII:* `internal` — the ids are internal and `reason` must not quote candidate
|
||
content, which is a reviewable rule for the admin screen's help text rather than a constraint.
|
||
*Audit:* yes, and this is one of the highest-value audit subjects in the system: issue, use, revoke
|
||
and expiry are four distinct events (`05` §7.3), and a grant change is on the real-time alert list
|
||
(`05` §4, "role/grant changes"). *Volume:* ~2,000 over 5 years — bounded by the 30-day ceiling and by
|
||
adr/0009 R4's threshold; if it materially exceeds this, the scope model has a gap and the right
|
||
response is a scope change, not a bigger table. *Queries:* `scopes_for(user)` branch 4 (§7.5); the
|
||
weekly active-grant report by grantor; the expiry sweep; "who can see this candidate"; the quarterly
|
||
access review, which must show grants alongside role assignments or it is not an access review.
|
||
|
||
**Why a table and not `role_assignment` with a short `valid_to`.** Four reasons, and the fourth is
|
||
the one that settles it. (1) A grant is per-`permission`, a role assignment is per-`role`; expressing
|
||
"see this one candidate" as a role assignment would require minting a single-permission role per
|
||
exception. (2) `role_assignment` has no subject beyond `access_scope`, and `access_scope` cannot
|
||
target a candidate (§7.5). (3) `role_assignment` has no mandatory `reason` and no self-grant CHECK,
|
||
and adding them there would impose them on the 1,500 ordinary assignments where they do not belong.
|
||
(4) `expires_at NOT NULL` is a *different guarantee* from `valid_to NULL`-able: on
|
||
`role_assignment`, `valid_to IS NULL` is the normal, correct state for a permanent grant, so the
|
||
column cannot carry a ceiling. Every long-lived exception in every access model started life as a
|
||
temporary grant nobody revoked; the `NOT NULL` plus the 30-day CHECK is the whole mechanism, and it
|
||
is only expressible on a table where no row is ever permanent.
|
||
|
||
---
|
||
|
||
## 8. Files
|
||
|
||
### 8.1 `app.stored_file`
|
||
|
||
*Purpose.* One registry for every blob the system holds: CV attachments, generated offer letters,
|
||
assessment reports, exports. Content-addressed, so the same CV arriving twice is one object.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `sha256` | `bytea` | no | 32 bytes. The content address |
|
||
| `byte_size` | `bigint` | no | CHECK > 0 |
|
||
| `mime_type` | `text` | no | Sniffed server-side, never trusted from the upload |
|
||
| `original_filename` | `text` | yes | As supplied. **Attacker-controlled** — see the XSS note below |
|
||
| `storage_backend` | `text` | no | CHECK in (`s3`, `azure_blob`, `local`) |
|
||
| `storage_key` | `text` | no | Object key. UNIQUE |
|
||
| `virus_scan_status` | `text` | no | CHECK in (`pending`, `clean`, `infected`, `error`, `skipped`), default `pending` |
|
||
| `virus_scan_at` | `timestamptz` | yes | |
|
||
| `virus_scanner_version` | `text` | yes | |
|
||
| `retention_class` | `text` | no | CHECK in (`candidate_document`, `generated_letter`, `assessment_report`, `export`, `system`) |
|
||
| `uploaded_by_user_id` | `bigint` | yes | NULL for integration-sourced files (§4.9 reason 3) |
|
||
| `deleted_from_store_at` | `timestamptz` | yes | Set when the purge removes the object. The row survives; the bytes do not |
|
||
| `created_at` | `timestamptz` | no | |
|
||
|
||
**Constraints.** `uq_stored_file_sha256 UNIQUE (sha256)` — deduplication is a hard rule, not an
|
||
optimisation. `uq_stored_file_key UNIQUE (storage_key)`. `ck_stored_file_scan CHECK
|
||
((virus_scan_status IN ('clean','infected','error')) = (virus_scan_at IS NOT NULL))`.
|
||
|
||
**Indexes.** `ix_stored_file_scan_pending (created_at) WHERE virus_scan_status = 'pending'` — the
|
||
worker's queue view. `ix_stored_file_retention (retention_class) WHERE deleted_from_store_at IS
|
||
NULL` — the purge's working set.
|
||
|
||
**Profile.** *Status:* `virus_scan_status`, Tier 2. *Required:* everything but the four nullables.
|
||
*Soft delete:* no — `deleted_from_store_at` is a different and stronger thing: the bytes are gone
|
||
and the row records that. *Retention:* the strictest policy of any referencing row wins; the purge
|
||
deletes the object and stamps `deleted_from_store_at`. *PII:* `sensitive_personal` — the content is
|
||
a CV, and `original_filename` routinely contains the candidate's name (`js/data.js:301`:
|
||
`attachment: name.split(' ')[0] + '_Resume.pdf'`). *Audit:* creation, download (an explicit access
|
||
event, since download is the read that matters most), and purge. *Volume:* ~350,000 objects,
|
||
~700 GB (**assumption**: one 250 KB CV plus a 1.5 MB generated letter per hired candidate).
|
||
*Queries:* signed-URL issue by id; the scan queue; the retention purge's blob list; dedupe on
|
||
upload by `sha256`.
|
||
|
||
**Why a separate registry rather than an `object_store_key` column on each owning table.**
|
||
`_decisions.md` Part 2 describes the key and hash inline on `raw_intake_attachment` and
|
||
`candidate_document`; Part 1's module list gives `files` a `StoredFile` entity with
|
||
`delete_for_subject()`. This design takes Part 1's shape and keeps Part 2's columns as the owning
|
||
rows' domain data, because three obligations attach to bytes rather than to domain rows:
|
||
(1) erasure must delete every blob for a subject in one place — with keys scattered across four
|
||
tables, a missed table is an unfulfilled deletion request (BRD §7.4); (2) virus scanning is a
|
||
single pipeline over all uploads regardless of who owns them; (3) content-addressed dedupe is only
|
||
possible with a single unique index on the hash, and the same CV genuinely does arrive through
|
||
Outlook and the careers portal. Owning tables keep `stored_file_id` plus their own `sha256`
|
||
denormalised copy for query convenience. **Flagged as a Part 1 / Part 2 reconciliation in §32.**
|
||
|
||
**One row per *content*, not per occurrence — and what that means for erasure.** `04` §9.1 #4 asked
|
||
for a "single content-addressed registry keyed on `sha256`"; this is that registry, named
|
||
`app.stored_file` in the `app` schema (there is no `files` schema — §1.1 lists five application
|
||
schemas). Content-addressing is adopted in its strong form, so the consequences must be written
|
||
down rather than left implicit:
|
||
|
||
| Question | Answer |
|
||
|---|---|
|
||
| Two candidates submit byte-identical CVs (an agency sending the same file twice, a template CV) | **One** `stored_file` row, two `candidate_document` rows pointing at it. `uq_stored_file_sha256` makes any other outcome impossible. |
|
||
| How many virus scans? | One, on first insert. `virus_scan_status` is a property of the bytes. |
|
||
| Candidate A is purged; candidate B still references the same bytes | The blob is **not** deleted and `deleted_from_store_at` stays NULL. `files.delete_for_subject()` deletes an object only when **no** non-purged row references it: `NOT EXISTS (SELECT 1 FROM app.candidate_document cd JOIN app.candidate c ON c.id = cd.candidate_id WHERE cd.stored_file_id = sf.id AND c.pseudonymised_at IS NULL)` and the equivalent for the other owning tables. What is always deleted for candidate A is A's `candidate_document` row's extracted text and A's own personal columns. |
|
||
| Does that weaken erasure? | No, and it is worth being precise. The retained bytes are not A's personal data uniquely — they are B's identical document, which B has an independent basis for. Deleting them would be erasing B's data on A's request. §31.2 records this as the one place where a purge leaves a blob standing, with the reference test as the stated condition. |
|
||
| Reference counting column? | **No.** A stored `ref_count` would drift the first time a psql fix moves a document. The `NOT EXISTS` test is evaluated at purge time against live rows, which is slower and correct. |
|
||
|
||
**XSS interaction, stated once and referenced later.** `original_filename` is deliberately stored
|
||
exactly as supplied, as are `candidate.full_name_original`, `candidate_email.address_original`,
|
||
`raw_intake.payload` and `intake_parse_attempt.parsed`. That is correct for storage — the
|
||
preserve-the-original rule is what makes re-parsing a recomputation rather than a recovery — and
|
||
it guarantees that attacker-controlled strings reach the rendering layer. The prototype has no
|
||
HTML escaping anywhere and 34 `innerHTML` sites (`_repo-findings.md` §E). **Storage-side
|
||
preservation must be paired with output-side escaping. Sanitising on write would violate the
|
||
preserve-the-original rule and is the wrong fix.**
|
||
|
||
---
|
||
|
||
## 9. Job requisitions, versions and requirements
|
||
|
||
### ERD 2 — Jobs, requisitions and publishing
|
||
|
||
```mermaid
|
||
erDiagram
|
||
job ||--|{ job_version : "has versions"
|
||
job ||--o| job_version : "current_version_id"
|
||
job_version ||--|{ job_requirement : "weighted requirements"
|
||
job_version ||--o{ job_posting : "published as"
|
||
job_posting ||--o{ job_posting_metric : "daily metrics"
|
||
job ||--o{ job_status_history : "status over time"
|
||
job ||--o{ job_vacancy : "vacancy slots"
|
||
job ||--o{ job_assignment : "owned by"
|
||
job ||--o{ job_scoring_assignment : "scored with"
|
||
job ||--o{ job_pipeline_assignment: "staged with"
|
||
job_version |o--o{ approval_request : "approved via"
|
||
approval_route ||--o{ approval_route_step : "steps"
|
||
approval_route ||--o{ approval_request : "routed by"
|
||
approval_request ||--o{ approval_decision : "decisions"
|
||
skill ||--o{ job_requirement : "requires"
|
||
|
||
job {
|
||
bigint id PK
|
||
uuid public_id
|
||
text reference_code
|
||
bigint department_id FK
|
||
bigint business_unit_id FK
|
||
bigint current_version_id FK
|
||
bigint status_id FK
|
||
bigint current_primary_recruiter_id FK
|
||
}
|
||
job_version {
|
||
bigint id PK
|
||
uuid public_id
|
||
bigint job_id FK
|
||
int version_no
|
||
text title
|
||
bigint grade_id FK
|
||
numeric salary_min_amount
|
||
char salary_min_currency_code
|
||
text salary_period
|
||
bytea content_hash
|
||
timestamptz effective_from
|
||
}
|
||
job_vacancy {
|
||
bigint id PK
|
||
bigint job_id FK
|
||
int seq
|
||
bigint filled_by_application_id FK
|
||
timestamptz filled_at
|
||
}
|
||
job_requirement {
|
||
bigint id PK
|
||
bigint job_version_id FK
|
||
text kind
|
||
bigint skill_id FK
|
||
boolean is_mandatory
|
||
numeric weight
|
||
}
|
||
job_posting {
|
||
bigint id PK
|
||
uuid public_id
|
||
bigint job_version_id FK
|
||
bigint publish_platform_id FK
|
||
text external_id
|
||
text state
|
||
}
|
||
approval_request {
|
||
bigint id PK
|
||
bigint job_version_id FK
|
||
bigint offer_version_id FK
|
||
bigint route_id FK
|
||
bigint status_id FK
|
||
}
|
||
approval_decision {
|
||
bigint id PK
|
||
bigint approval_request_id FK
|
||
int step_no
|
||
bigint approver_user_id FK
|
||
text decision
|
||
}
|
||
```
|
||
|
||
**Terminology, settled here.** The assignment names "job requisitions" and "jobs" as separate
|
||
areas. They are **one aggregate**, not two: a requisition *is* the job record of record, and
|
||
splitting them would create two identities for one thing and force every application to choose
|
||
which to reference. The vocabulary maps as: `job` = the stable requisition identity (what
|
||
`JOB-1001` names, what recruiters own, what applications are unique against); `job_version` = the
|
||
requisition *as approved and published at a point in time* (the thing an approval approves and a
|
||
posting posts); `job_requirement` = version-scoped, weighted criteria. BRD §9.1 supports this —
|
||
"Job: Requisition of record". **Recorded as a deviation in §30.**
|
||
|
||
### 9.1 `app.job`
|
||
|
||
*Purpose.* Stable requisition identity. Holds nothing that can be edited as content — every
|
||
editable field lives on a version.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `public_id` | `uuid` | no | UNIQUE |
|
||
| `reference_code` | `text` | no | UNIQUE, `JOB-1001` shape (`js/data.js:95`) |
|
||
| `department_id` | `bigint` | no | FK `ref.department`. On the identity, not the version: moving a requisition between departments is a new requisition, and department drives access scope |
|
||
| `business_unit_id` | `bigint` | no | FK `ref.business_unit` |
|
||
| `status_id` | `bigint` | no | FK `ref.lifecycle_status` domain `job` (Open / On Hold / Closed / Draft — BRD §9.2, `js/data.js:25`) |
|
||
| `status_domain` | `text` gen. | no | Constant `'job'`, part of the composite FK (§4.7) |
|
||
| `current_version_id` | `bigint` | yes | FK `app.job_version`, `DEFERRABLE INITIALLY DEFERRED`. NULL only between the `job` insert and its first version insert inside one transaction |
|
||
| `current_primary_recruiter_id` | `bigint` | yes | FK `app.app_user`. Denormalised from `job_assignment`, maintained by trigger |
|
||
| `opened_at` | `timestamptz` | yes | First transition to Open. **Reporting must not use this as the time-to-fill origin** — see the KPI note below |
|
||
| `closed_at` | `timestamptz` | yes | |
|
||
| `close_reason_id` | `bigint` | yes | FK `ref.vocabulary_value`, vocabulary `job_close_reason` (`filled` / `cancelled` / `headcount_withdrawn` / `merged`). Populated exactly when `closed_at` is |
|
||
| `target_start_date` | `date` | yes | |
|
||
| `vacancies_filled` | `int` | no | Default 0. Maintained by trigger from hired applications |
|
||
| audit + soft delete | | | |
|
||
|
||
**Constraints.** `uq_job_public_id`, `uq_job_reference_code`. `ck_job_current_version` — a
|
||
`DEFERRABLE` constraint trigger asserting `current_version_id IS NOT NULL` at COMMIT, so a job can
|
||
never persist without a version. `ck_job_closed CHECK (closed_at IS NULL OR opened_at IS NOT
|
||
NULL)`. `ck_job_close_reason CHECK ((closed_at IS NULL) = (close_reason_id IS NULL))`.
|
||
`ck_job_vacancies_filled CHECK (vacancies_filled >= 0)`.
|
||
|
||
**Time to Fill — the definition, and why these columns are not enough on their own.**
|
||
BRD REQ-ANL-01 names Time to Fill as a headline KPI, and three things about the columns above would
|
||
each produce a different number if the formula were left to whoever writes the report:
|
||
|
||
1. **`closed_at` alone conflates outcomes.** A requisition closed because headcount was pulled has
|
||
a `closed_at` and was never filled. Including it deflates the metric; excluding it by guessing
|
||
from `vacancies_filled` is fragile. `close_reason_id` is the fix, and time-to-fill restricts to
|
||
`close_reason = 'filled'`.
|
||
2. **`opened_at` is the *first* Open transition.** A requisition opened in January, put On Hold in
|
||
February and reopened in May would span its dormant period. The origin is therefore the
|
||
**start of the latest open interval** from `app.job_status_history`, not `opened_at`.
|
||
`opened_at` is retained for "when did this requisition first exist as work" and for the
|
||
requisition-age display.
|
||
3. **`vacancies_filled` is a bare counter with no timestamp.** For `job_version.vacancies > 1`
|
||
"the fill instant" is undefined. The fill instant is the **hired transition**, read from
|
||
`app.job_application_status_history`: for a single-vacancy requisition it is
|
||
`min(valid_from)` where the status key is `hired`; for N vacancies it is the Nth such
|
||
transition. Where per-vacancy reporting is actually required, `app.job_vacancy` (§9.6) carries
|
||
one row per slot with `filled_by_application_id` and `filled_at`.
|
||
|
||
**Both KPI formulas are written as SQL once, in migration 021, and nowhere else** — see §9.6. Time
|
||
to Hire is the well-behaved one (`job_application.applied_at` → the `hired` transition, exactly
|
||
recoverable from `job_application_status_history`); Time to Fill is the one that needs all three
|
||
corrections above. Writing them as views in a migration rather than in report code is the only way
|
||
"why does the dashboard disagree with the board pack" stops being a recurring question.
|
||
**08-requirements-traceability.md carries a GAP row for this alongside GAP-02/GAP-03.**
|
||
|
||
**Indexes.** `ix_job_open (department_id, business_unit_id) WHERE deleted_at IS NULL` filtered by
|
||
the open status in the query; `ix_job_recruiter (current_primary_recruiter_id) WHERE deleted_at IS
|
||
NULL` — every list screen in the 23-route IA filters by recruiter (`js/app.js:7-16`), which is
|
||
exactly why the denormalised column exists; `ix_job_status (status_id)`.
|
||
|
||
**Profile.** *Status:* `ref.lifecycle_status`, domain `job`, with `job_status_history`.
|
||
*Required:* department, business unit and status; the two nullables are §4.9 reason 1.
|
||
*Soft delete:* yes. *Retention:* permanent — requisition records are the defensibility evidence for
|
||
every hiring decision made against them. *PII:* `internal`. *Audit:* yes. *Volume:* ~1,200 over 5
|
||
years (**assumption**: ~250 requisitions/year). *Queries:* the Jobs list filtered by department,
|
||
status and recruiter; open-requisition counts per department for the dashboard
|
||
(`js/data.js:230`); "requisitions awaiting my approval".
|
||
|
||
### 9.2 `app.job_version`
|
||
|
||
*Purpose.* The immutable requisition content. Minting a new version is the only way to change a
|
||
requisition, which is the mechanism that makes score drift impossible.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `public_id` | `uuid` | no | UNIQUE. Externally addressable — a posting links to a version |
|
||
| `job_id` | `bigint` | no | FK `app.job` |
|
||
| `version_no` | `int` | no | 1-based, UNIQUE with `job_id` |
|
||
| `title` | `text` | no | |
|
||
| `description` | `text` | no | Markdown. May be AI-drafted — see `ai_suggestion_id` |
|
||
| `responsibilities` | `text` | yes | |
|
||
| `employment_type_id` | `bigint` | no | FK `ref.employment_type` |
|
||
| `location_id` | `bigint` | yes | FK `ref.location`. NULL for fully remote with no anchor |
|
||
| `is_remote` | `boolean` | no | Default false |
|
||
| `grade_id` | `bigint` | no | FK `ref.grade` |
|
||
| `vacancies` | `int` | no | CHECK >= 1 |
|
||
| `salary_min_amount` | `numeric(14,2)` | yes | |
|
||
| `salary_min_currency_code` | `char(3)` | yes | |
|
||
| `salary_max_amount` | `numeric(14,2)` | yes | |
|
||
| `salary_max_currency_code` | `char(3)` | yes | |
|
||
| `salary_period` | `text` | yes | CHECK in (`annual`, `monthly`, `hourly`, `daily`). Populated exactly when a range is. **Same reasoning §25.2 gives for the offer: a bare number is ambiguous** |
|
||
| `annualisation_hours_per_year` | `int` | yes | Default 2080 when `salary_period = 'hourly'`, else NULL. The assumption is pinned on the row, not implied |
|
||
| `annualisation_days_per_year` | `int` | yes | Default 260 when `salary_period = 'daily'`, else NULL |
|
||
| `salary_min_annualised_amount` | `numeric(14,2)` | yes | Derived at write time from `salary_min_amount`, `salary_period` and the annualisation basis. The **only** column the variance and banding queries read |
|
||
| `salary_max_annualised_amount` | `numeric(14,2)` | yes | Same |
|
||
| `is_salary_public` | `boolean` | no | Default false. Drives what a posting may show |
|
||
| `hiring_manager_user_id` | `bigint` | yes | The approver of record for this version |
|
||
| `min_experience_months` | `int` | yes | |
|
||
| `min_education_level_id` | `bigint` | yes | FK `ref.education_level` |
|
||
| `custom_fields` | `jsonb` | no | Default `'{}'`. Legitimate JSONB per P6 — display-only, no constraint may reference it |
|
||
| `content_hash` | `bytea` | no | sha256 over the normalised content. Feeds `ats_result.input_fingerprint` |
|
||
| `effective_from` | `timestamptz` | no | |
|
||
| `superseded_at` | `timestamptz` | yes | Set by trigger when a later version becomes current |
|
||
| `change_reason` | `text` | yes | Required by application policy for `version_no > 1` |
|
||
| `approval_request_id` | `bigint` | yes | The approval that cleared this version, if the route required one |
|
||
| `ai_suggestion_id` | `bigint` | yes | FK `ai.ai_suggestion` — set when the description came from the JD Generator (BRD AI-5). Makes AI provenance queryable |
|
||
| `created_at`, `created_by_user_id` | | no | No `updated_*` — the row is immutable (§4.2) |
|
||
|
||
**Constraints.** `uq_job_version_no UNIQUE (job_id, version_no)`. `uq_job_version_public_id`.
|
||
Money pair CHECKs per §4.4 plus `ck_job_version_salary_range CHECK (salary_max_amount IS NULL OR
|
||
salary_min_amount IS NULL OR salary_max_amount >= salary_min_amount)` and
|
||
`ck_job_version_salary_same_ccy CHECK (salary_min_currency_code IS NOT DISTINCT FROM
|
||
salary_max_currency_code)`. `ck_job_version_vacancies CHECK (vacancies >= 1)`.
|
||
`ck_job_version_salary_period CHECK ((salary_min_amount IS NULL) = (salary_period IS NULL))` and
|
||
`ck_job_version_annualised CHECK ((salary_min_amount IS NULL) = (salary_min_annualised_amount IS
|
||
NULL))`.
|
||
|
||
**Immutability, enforced twice — and the triggers allow-list, they do not deny-list.**
|
||
|
||
```sql
|
||
REVOKE UPDATE, DELETE ON app.job_version FROM talentflow_app;
|
||
GRANT SELECT, INSERT ON app.job_version TO talentflow_app;
|
||
GRANT UPDATE (superseded_at) ON app.job_version TO talentflow_app;
|
||
|
||
CREATE TRIGGER tg_job_version_immutable
|
||
BEFORE UPDATE ON app.job_version
|
||
FOR EACH ROW
|
||
WHEN ((to_jsonb(OLD) - 'superseded_at') IS DISTINCT FROM (to_jsonb(NEW) - 'superseded_at'))
|
||
EXECUTE FUNCTION app.tg_raise_immutable();
|
||
|
||
CREATE TRIGGER tg_job_version_no_delete
|
||
BEFORE DELETE ON app.job_version
|
||
FOR EACH ROW EXECUTE FUNCTION app.tg_raise_immutable();
|
||
```
|
||
|
||
**Why a pair and not one `BEFORE UPDATE OR DELETE` trigger.** The `WHEN` clause is the whole point of
|
||
this shape, and a `WHEN` clause on a combined `UPDATE OR DELETE` trigger can reference neither `TG_OP`
|
||
(a PL/pgSQL variable, not a column) nor `NEW` (the `DELETE` case has none) — §4.8's fourth syntax fact
|
||
gives both error messages. Since the allow-list *is* a comparison against `NEW`, the `DELETE` half
|
||
must be its own trigger, and it needs no `WHEN` clause because nothing may ever delete this row.
|
||
|
||
**Why the `to_jsonb` form and not `BEFORE UPDATE OF <column list>`.** An enumerated column list is a
|
||
deny-list, and a deny-list on an immutable table is wrong by construction: it protects the columns
|
||
the author remembered and silently permits every column added by a later migration. That is not
|
||
theoretical here — a column-list trigger written against the original 20 columns would have omitted
|
||
`employment_type_id`, `location_id`, `is_remote`, `min_experience_months`,
|
||
`min_education_level_id`, the two salary currency codes, `is_salary_public`,
|
||
`hiring_manager_user_id`, `responsibilities` and `custom_fields`, two of which
|
||
(`min_experience_months`, `min_education_level_id`) are **scoring inputs** and all of which are
|
||
covered by `content_hash` — so an update would desynchronise the hash that
|
||
`ats_result.input_fingerprint` depends on and score drift would return through the back door. The
|
||
column-level `GRANT` does block `talentflow_app`, but §4.8 states the whole reason immutability is
|
||
doubled: a GRANT can be misconfigured, and the **table owner** — migrations, and the psql data
|
||
fixes §9.5 concedes will happen — bypasses GRANTs entirely, leaving the trigger as the only guard.
|
||
Subtracting the one mutable key from `to_jsonb(OLD)`/`to_jsonb(NEW)` inverts the default: a new
|
||
column is protected the moment it exists.
|
||
|
||
**This rewrite applies to every immutability trigger in this document** — each written as the same
|
||
`BEFORE UPDATE` / `BEFORE DELETE` pair, each allow-listing only its
|
||
own mutable columns: `offer_version` (`superseded_at`), `scoring_config_version`
|
||
(`activated_at`, `deactivated_at`, `activation_evaluation_run_id`), `pipeline_config_version`,
|
||
`scorecard_template_version`, `assessment_template_version`, `matching_config_version`
|
||
(all `superseded_at`), and `ats_result` (`is_current`, `superseded_by_id`, and the three review
|
||
columns). **CI test (junior workstream, §2 P9):** in a scratch transaction,
|
||
`ALTER TABLE app.job_version ADD COLUMN ci_probe text`, attempt an `UPDATE … SET ci_probe = 'x'`,
|
||
assert the exception, roll back. Repeat per versioned table. That test is what makes the guarantee
|
||
survive the next twelve migrations.
|
||
|
||
**Deferrable weight-sum check — on the parent *and* the child.** Two constraint triggers, because
|
||
one of them alone cannot see the case that matters:
|
||
|
||
```sql
|
||
-- child side: any requirement write re-validates its version
|
||
CREATE CONSTRAINT TRIGGER tg_job_requirement_weights
|
||
AFTER INSERT OR UPDATE OR DELETE ON app.job_requirement
|
||
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW
|
||
EXECUTE FUNCTION app.tg_assert_requirement_weights();
|
||
|
||
-- parent side: a version is validated even when it has NO requirement rows at all
|
||
CREATE CONSTRAINT TRIGGER tg_job_version_weights
|
||
AFTER INSERT OR UPDATE ON app.job_version
|
||
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW
|
||
EXECUTE FUNCTION app.tg_assert_requirement_weights();
|
||
```
|
||
|
||
Both call the same function, which asserts for the version in question:
|
||
`count(*) >= 1 AND abs(sum(weight) - 1.0) <= 0.0001`.
|
||
|
||
**Why the parent trigger is not redundant.** A row trigger on the child fires only when a child row
|
||
is written, so a `job_version` published with **zero** `job_requirement` rows never triggers the
|
||
check: the weights sum to 0, scoring runs against a version with no criteria, and it produces an
|
||
`ats_result` with no `ats_result_criterion` rows and an `overall_score` derived from nothing. That
|
||
is the failure the check exists to prevent, and only the parent trigger catches it.
|
||
|
||
**Is a requirement-less version legal? No — decided explicitly.** Drafts do need to exist before
|
||
their requirements do, but that is a *transaction* concern, not a *row* concern, and the deferred
|
||
trigger already covers it: a draft version and its requirements are inserted in one transaction and
|
||
validated at COMMIT. A version that is persisted with no criteria is a bug in every case, so the
|
||
assertion is unconditional rather than gated on `job.current_version_id` or `effective_from`.
|
||
Tradeoff accepted: a multi-step "save my half-finished requisition" UI cannot spread one version
|
||
across two transactions and must either hold a client-side draft or mint the version on final save.
|
||
That is the cheaper side of the trade, because the alternative — an ungated version with no
|
||
criteria reaching the scorer — is invisible in the UI and expensive to find. **ERD 2 states this as
|
||
`job_version ||--|{ job_requirement`, which is now true rather than aspirational.**
|
||
|
||
**Indexes.** `ix_job_version_job (job_id, version_no DESC)`; `ix_job_version_hash (content_hash)`
|
||
for the "has anything changed" check; `ix_job_version_title_trgm` GIN trigram on `title` for the
|
||
job search box.
|
||
|
||
**Profile.** *Status:* none — a version is not statused; the *job* is, and the approval state lives
|
||
on `approval_request`. *Required:* title, description, employment type, grade, vacancies,
|
||
`content_hash`, `effective_from`. *Soft delete:* **no** — append-only. *Retention:* permanent.
|
||
*PII:* `internal`; salary range is `sensitive_personal` when populated. *Audit:* insert only (there
|
||
is nothing else to capture). *Volume:* ~4,000 (**assumption**: 3.3 versions per requisition).
|
||
*Queries:* current version by `job.current_version_id`; `version_at(job, ts)` for reconstructing
|
||
what a candidate applied against; the version diff view; batch rescore trigger on publish.
|
||
|
||
### 9.3 `app.job_requirement`
|
||
|
||
*Purpose.* The weighted, version-scoped criteria that scoring actually consumes. The prototype has
|
||
requirements as a plain skills array on a mutable job (`js/data.js:100`), so editing one silently
|
||
rewrites the inputs of every score already computed.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `job_version_id` | `bigint` | no | FK. **Never `job_id`** — this is the whole point |
|
||
| `kind` | `text` | no | CHECK in (`skill`, `experience_years`, `education_level`, `certification`, `language`, `location`, `custom`) |
|
||
| `skill_id` | `bigint` | yes | FK `ref.skill`. Required when `kind = 'skill'` |
|
||
| `label` | `text` | no | Display text, always present so the UI never needs a join to render |
|
||
| `operator` | `text` | yes | CHECK in (`>=`, `<=`, `=`, `contains`, `present`) |
|
||
| `threshold_value` | `numeric(12,2)` | yes | |
|
||
| `unit` | `text` | yes | e.g. `months`, `years` |
|
||
| `is_mandatory` | `boolean` | no | Default false. A failed mandatory requirement is a gate, not a deduction |
|
||
| `weight` | `numeric(6,4)` | no | CHECK `weight >= 0 AND weight <= 1` |
|
||
| `display_order` | `int` | no | |
|
||
| `created_at` | | no | Immutable; no `updated_*` |
|
||
|
||
**Constraints.** `ck_job_requirement_skill CHECK (kind <> 'skill' OR skill_id IS NOT NULL)`.
|
||
`ck_job_requirement_threshold CHECK (kind NOT IN ('experience_years','education_level') OR
|
||
threshold_value IS NOT NULL)`. The same skill cannot be required twice in one version, expressed as
|
||
a partial unique **index** per §4.8 (the `CONSTRAINT … UNIQUE … WHERE` form is not valid SQL):
|
||
|
||
```sql
|
||
CREATE UNIQUE INDEX uq_job_requirement_skill
|
||
ON app.job_requirement (job_version_id, skill_id)
|
||
WHERE skill_id IS NOT NULL;
|
||
```
|
||
|
||
Plus **both** deferred weight-sum triggers above (child and parent). Same INSERT/SELECT-only grants
|
||
and the same `to_jsonb`-form immutability trigger as `job_version`.
|
||
|
||
**Indexes.** `ix_job_requirement_version (job_version_id, display_order)`;
|
||
`ix_job_requirement_skill (skill_id)` for "which open requisitions want PostgreSQL".
|
||
|
||
**Profile.** *Soft delete:* no. *Retention:* permanent. *PII:* `internal`. *Audit:* insert only.
|
||
*Volume:* ~28,000 (~7 requirements per version, matching the prototype's `pickN(skillsPool,
|
||
int(4,7))` at `js/data.js:100`). *Queries:* render the requirement list for a version; the scoring
|
||
job's input set; skill-demand reporting across open requisitions; the skill-gap analysis
|
||
(BRD AI-12).
|
||
|
||
### 9.4 Job configuration bindings
|
||
|
||
Two tables with identical shape, both keeping configuration binding **orthogonal to job
|
||
versioning** so that a scoring or pipeline tweak does not fabricate a fake requisition revision.
|
||
|
||
| Table | Purpose | Columns | Constraints |
|
||
|---|---|---|---|
|
||
| `app.job_scoring_assignment` | Which scoring config version scores this job, and when | `job_id`, `scoring_config_version_id`, `valid_from`, `valid_to`, `assigned_by_user_id`, `reason` | `ex_job_scoring_overlap EXCLUDE USING gist (job_id WITH =, tstzrange(valid_from, valid_to) WITH &&)` — at most one active binding per job at any instant |
|
||
| `app.job_pipeline_assignment` | Which pipeline config version governs this job's stages | `job_id`, `pipeline_config_version_id`, `valid_from`, `valid_to`, `assigned_by_user_id`, `reason` | same EXCLUDE shape |
|
||
|
||
**Why not a column on `job_version`.** Simpler to query, but it forces a new job version on every
|
||
config change, so "what changed in this requisition" becomes unanswerable — every scoring tweak
|
||
appears as a requisition revision. Orthogonality is safe precisely because `ats_result` pins both
|
||
versions independently (§20.4), so provenance never depends on querying a binding table
|
||
temporally.
|
||
|
||
**Profile (both).** *Soft delete:* no. *Retention:* permanent. *PII:* `internal`. *Audit:* yes.
|
||
*Volume:* ~2,500 each. *Queries:* resolve the active config when creating an application or
|
||
running a batch rescore; "which jobs used config v4".
|
||
|
||
### 9.5 `app.job_status_history`
|
||
|
||
*Purpose.* Requisition status over time. Written **only** by a trigger on `app.job`.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `job_id` | `bigint` | no | FK |
|
||
| `status_id` | `bigint` | no | FK `ref.lifecycle_status` domain `job` |
|
||
| `valid_from` | `timestamptz` | no | |
|
||
| `valid_to` | `timestamptz` | yes | NULL = current |
|
||
| `actor_user_id` | `bigint` | yes | From `current_setting('app.actor_user_id', true)` |
|
||
| `actor_kind` | `text` | no | CHECK in (`user`, `system`, `integration`, `ai_agent`) |
|
||
| `actor_unknown` | `boolean` | no | Default false. `true` when the middleware failed to set the actor |
|
||
| `change_reason` | `text` | yes | From `current_setting('app.change_reason', true)` |
|
||
| `request_id` | `uuid` | yes | Correlates to `audit.audit_event` |
|
||
|
||
**Constraints.** `ex_job_status_history_overlap EXCLUDE USING gist (job_id WITH =,
|
||
tstzrange(valid_from, valid_to) WITH &&)`. `ck` interval order. Append-only grants; the trigger
|
||
runs as the table owner to close the previous row.
|
||
|
||
**Indexes.** `ix_job_status_history_current (job_id) WHERE valid_to IS NULL`;
|
||
`ix_job_status_history_from (valid_from)` for period reporting.
|
||
|
||
**Profile.** *Soft delete:* no. *Retention:* permanent. *PII:* `internal`. *Audit:* this *is* the
|
||
domain history; `audit_event` additionally records the change (the two serve different readers —
|
||
§31.4). *Volume:* ~5,000. *Queries:* "how long was this on hold"; requisition ageing; open-req count
|
||
as at a date.
|
||
|
||
**Why a trigger writes it.** The tradeoff is real and worth restating: application-enforced history
|
||
is more testable and needs no PL/pgSQL from the junior, but any path that forgets — a psql data
|
||
fix, a migration, a bulk import, the merge routine — loses history silently. With two developers
|
||
doing early ad-hoc data work, that will happen. Triggers cannot be bypassed, including by a manual
|
||
`UPDATE`, which is exactly the property the requirement needs. The `SET LOCAL` bridge answers the
|
||
usual objection that triggers cannot see who or why: they can, if the transaction tells them, and
|
||
`actor_unknown = true` converts a forgotten `SET LOCAL` into a visible data-quality signal rather
|
||
than a wrong attribution. **A dashboard query on `actor_unknown` counts is a required deliverable,
|
||
not optional** — without it the attribution gap is invisible.
|
||
|
||
### 9.6 `app.job_vacancy`, and the two hiring-cycle KPI definitions
|
||
|
||
*Purpose.* One row per **slot** on a requisition, so a 3-vacancy requisition has a fill instant per
|
||
vacancy rather than a bare counter. Created by trigger when a version's `vacancies` count rises;
|
||
never destroyed when it falls (an unfilled surplus slot is closed, not deleted).
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `job_id` | `bigint` | no | FK `app.job` |
|
||
| `seq` | `int` | no | 1-based. UNIQUE with `job_id` |
|
||
| `filled_by_application_id` | `bigint` | yes | FK `app.job_application`. §4.9 reason 1 |
|
||
| `filled_at` | `timestamptz` | yes | The `hired` transition instant, copied from `job_application_status_history` by the same trigger that increments `job.vacancies_filled` |
|
||
| `closed_without_fill_at` | `timestamptz` | yes | Set when the requisition closes with this slot unfilled, or when a later version reduces `vacancies` |
|
||
| `created_at` | `timestamptz` | no | |
|
||
|
||
**Constraints.** `uq_job_vacancy UNIQUE (job_id, seq)`.
|
||
`ck_job_vacancy_fill CHECK ((filled_by_application_id IS NULL) = (filled_at IS NULL))`.
|
||
`ck_job_vacancy_outcome CHECK (NOT (filled_at IS NOT NULL AND closed_without_fill_at IS NOT NULL))`.
|
||
A slot is filled, closed unfilled, or open — never two of those.
|
||
`CREATE UNIQUE INDEX uq_job_vacancy_application ON app.job_vacancy (filled_by_application_id) WHERE filled_by_application_id IS NOT NULL;`
|
||
so one hire cannot fill two slots.
|
||
|
||
**Profile.** *Soft delete:* no. *Retention:* permanent. *PII:* `internal`. *Audit:* yes.
|
||
*Volume:* ~1,600 (**assumption**: 1.3 vacancies per requisition). *Queries:* per-vacancy time to
|
||
fill; "which slots on this requisition are still open"; headcount reconciliation against Finance.
|
||
|
||
**The two KPI formulas live in migration 021 as views, and nowhere else.** REQ-ANL-01 names both.
|
||
Writing them once in a reviewable migration is the whole point — every report, dashboard tile and
|
||
board pack reads these views rather than re-deriving the arithmetic.
|
||
|
||
```sql
|
||
-- Time to Hire: applied → hired. Well-behaved; recoverable exactly from application history.
|
||
CREATE VIEW app.v_kpi_time_to_hire AS
|
||
SELECT ja.id AS job_application_id,
|
||
ja.job_id,
|
||
ja.applied_at,
|
||
jash.valid_from AS hired_at,
|
||
jash.valid_from - ja.applied_at AS time_to_hire
|
||
FROM app.job_application ja
|
||
JOIN app.job_application_status_history jash ON jash.job_application_id = ja.id
|
||
JOIN ref.lifecycle_status ls ON ls.id = jash.status_id
|
||
WHERE ls.domain = 'job_application' AND ls.key = 'hired'
|
||
AND ja.deleted_at IS NULL;
|
||
|
||
-- Time to Fill: latest OPEN interval → the Nth hired transition, filled requisitions only.
|
||
CREATE VIEW app.v_kpi_time_to_fill AS
|
||
WITH latest_open AS ( -- correction 2: not job.opened_at
|
||
SELECT jsh.job_id, max(jsh.valid_from) AS reopened_at
|
||
FROM app.job_status_history jsh
|
||
JOIN ref.lifecycle_status ls ON ls.id = jsh.status_id
|
||
WHERE ls.domain = 'job' AND ls.key = 'open'
|
||
GROUP BY jsh.job_id
|
||
)
|
||
SELECT j.id AS job_id,
|
||
lo.reopened_at,
|
||
jv.filled_at,
|
||
jv.seq,
|
||
jv.filled_at - lo.reopened_at AS time_to_fill
|
||
FROM app.job j
|
||
JOIN latest_open lo ON lo.job_id = j.id
|
||
JOIN app.job_vacancy jv ON jv.job_id = j.id AND jv.filled_at IS NOT NULL
|
||
JOIN ref.vocabulary_value cr ON cr.id = j.close_reason_id
|
||
WHERE cr.value_key = 'filled' -- correction 1: exclude cancelled / headcount_withdrawn / merged
|
||
AND j.deleted_at IS NULL;
|
||
```
|
||
|
||
Report-level aggregation (median, per-department, per-quarter) sits on top of these; the *definition*
|
||
is not repeated. A requisition still open reports no time to fill, which is correct — the honest
|
||
answer to "how long did it take to fill" for an unfilled requisition is "it hasn't been".
|
||
|
||
---
|
||
|
||
## 10. Approval workflow
|
||
|
||
**One approval engine, two subjects.** Requisition versions and offer versions both need
|
||
route-driven, multi-step, human sign-off with the same UI, the same reminder logic and the same
|
||
"who approved what when" query. Their approval shape is *identical*; only the subject differs.
|
||
Four tables serve both, using the typed-nullable-subject idiom (P7).
|
||
|
||
**Tension with `_decisions.md` Part 1, stated.** Part 1 rejects "a generic `workflow_engine`
|
||
module driving all state machines... until at least three modules demonstrably need the same
|
||
engine". This is not that. A state machine encodes per-aggregate transition rules and belongs to
|
||
the aggregate (which is why `pipeline_transition_rule` is separate and application-specific).
|
||
Approval routing is data-driven sequential sign-off with no aggregate-specific logic at all: the
|
||
route says who signs in what order, and the only outcomes are approve, reject and delegate. Sharing
|
||
it is normalisation, not premature abstraction. **Recorded in §30 and §32.**
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.approval_route` | A named, reusable chain of approvers, selected by subject kind and org attributes | `key` UNIQUE, `name`, `subject_kind` CHECK in (`job_version`,`offer_version`), `business_unit_id` NULL, `department_id` NULL, `grade_id_min` NULL, `amount_threshold_amount`/`_currency_code` NULL, `is_default`, `is_active`, audit | `CREATE UNIQUE INDEX uq_approval_route_default ON app.approval_route (subject_kind) WHERE is_default AND is_active;` (index form per §4.8); money pair CHECK. `amount_threshold_amount` is compared against the offer's **annualised reporting** amount (§25.2), never the raw `base_salary_amount` — an hourly rate tested against an annual threshold would route a senior offer through no approval at all. Route selection is most-specific-match, resolved in the service and **stored** on the request so a later route edit cannot rewrite history | *Soft delete:* no (deactivate). *Retention:* permanent. *PII:* `internal`. *Audit:* yes. *Volume:* ~20 |
|
||
| `app.approval_route_step` | One step of a route | `route_id`, `step_no`, `approver_role_id` NULL, `approver_user_id` NULL, `resolve_from` CHECK in (`role_in_scope`,`named_user`,`job_hiring_manager`,`department_head`,`requester_manager`), `is_optional`, `sla_hours` | `uq (route_id, step_no)`; `ck` exactly one of role/user/`resolve_from` dynamic | *Volume:* ~60 |
|
||
| `app.approval_request` | One approval in flight or completed, against one subject | `id`, `public_id`, `subject_kind` CHECK, `job_version_id` NULL FK, `offer_version_id` NULL FK, `route_id`, `route_snapshot jsonb` (the resolved chain as at open time), `status_id` FK `ref.lifecycle_status` domain `approval_request` (`open`/`approved`/`rejected`/`withdrawn`/`expired`), `current_step_no`, `opened_at`, `opened_by_user_id`, `closed_at`, `outcome` CHECK in (`approved`,`rejected`,`withdrawn`,`expired`) NULL, `sla_due_at` | `ck_approval_subject CHECK (num_nonnulls(job_version_id, offer_version_id) = 1 AND (subject_kind = 'job_version') = (job_version_id IS NOT NULL))`; two partial unique **indexes** per §4.8, `CREATE UNIQUE INDEX uq_approval_open_job ON app.approval_request (job_version_id) WHERE closed_at IS NULL AND job_version_id IS NOT NULL;` and the same shape for `offer_version_id`; `ix (status_id, sla_due_at)` for the overdue sweep | *Status:* Tier 1 + `outcome` Tier 2. *Soft delete:* no. *Retention:* permanent. *PII:* `internal`. *Audit:* yes. *Volume:* ~5,000 |
|
||
| `app.approval_decision` | An individual approver's act. Append-only | `approval_request_id`, `step_no`, `approver_user_id` **NOT NULL**, `decision` CHECK in (`approved`,`rejected`,`delegated`,`abstained`), `decided_at`, `comment`, `delegated_to_user_id` NULL, `on_behalf_of_user_id` NULL | `uq (approval_request_id, step_no, approver_user_id)`; `ck_delegated CHECK ((decision = 'delegated') = (delegated_to_user_id IS NOT NULL))`; append-only grants | *Soft delete:* no. *Retention:* permanent. *PII:* `personal` (`comment` may name people). *Audit:* yes — this is a decision of record. *Volume:* ~12,000 |
|
||
|
||
`approver_user_id NOT NULL` on `approval_decision` is the schema-level form of "approval is always
|
||
a human act". There is no system principal that can approve, and `route_snapshot` means the chain
|
||
that was actually followed is on disk even after the route is edited.
|
||
|
||
**Queries these support.** "Requisitions awaiting my approval" (the prototype's notification
|
||
`Job requisition JOB-1004 awaits your approval`, `js/data.js:250`); approval cycle time per
|
||
department; overdue approvals for the SLA sweep; the full approval trail on a requisition or offer.
|
||
|
||
---
|
||
|
||
## 11. Job postings and publishing
|
||
|
||
Two tables, **split across two phases**. BRD §8.2 specifies 8 platforms with per-platform connection
|
||
state and cost banding; the prototype models publishings with views/clicks/applications
|
||
(`js/jobboard.js:16-19`) and platform metadata (`DB.publishPlatforms`).
|
||
|
||
**`app.job_posting` is Phase 1, migration `006a`. `app.job_posting_metric` and all external-platform
|
||
state are Phase 4, migration `027`.** The split is forced, not stylistic:
|
||
`job_application.job_posting_id` (§15.1) is a **Phase 1** column — "what advert did this candidate
|
||
read" is answerable from day one for the careers portal — and a Phase 1 migration cannot reference a
|
||
table created in a Phase 4 one. Phase 1 therefore creates `job_posting` with one row per published
|
||
version for the careers portal only (`publish_platform` = the internal portal), and Phase 4 adds the
|
||
eight external platforms, their metrics and their reconciliation state on top of the same table. `08`
|
||
GAP-25 recommends exactly this split; §33.1 records the ordering violation it resolves.
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.job_posting` | A specific job **version** published to a specific platform. Pins the exact text the applicant read | `id`, `public_id`, `job_version_id` FK NOT NULL, `publish_platform_id` FK, `external_id` NULL, `external_url` NULL, `state` CHECK in (`draft`,`publishing`,`live`,`paused`,`expired`,`removed`,`failed`), `published_at`, `unpublished_at`, `expires_at`, `cost_band_id` FK vocabulary, `posted_by_user_id`, `last_sync_at`, `last_error jsonb` NULL, audit | two partial unique **indexes** per §4.8 — `CREATE UNIQUE INDEX uq_job_posting_external ON app.job_posting (publish_platform_id, external_id) WHERE external_id IS NOT NULL;` and `CREATE UNIQUE INDEX uq_job_posting_live ON app.job_posting (job_version_id, publish_platform_id) WHERE state IN ('publishing','live','paused');` — plus `ix (state, last_sync_at)` for reconciliation | *Status:* Tier 2. *Soft delete:* no. *Retention:* permanent. *PII:* `internal`. *Audit:* yes — publishing has cost implications, so publish/unpublish is HR-admin-gated and audited. *Volume:* ~6,000 |
|
||
| `app.job_posting_metric` | Daily per-posting funnel counters from the platform | `job_posting_id`, `as_of_date`, `views`, `clicks`, `applications`, `spend_amount`/`spend_currency_code`, `fetched_at` | PK `(job_posting_id, as_of_date)`; money pair CHECK; all counters `NOT NULL DEFAULT 0 CHECK (>= 0)` | *Soft delete:* no. *Retention:* 3 years. *PII:* `internal`. *Audit:* no — machine-fetched counters, high volume, no decision value in a change trail. *Volume:* ~1.1M |
|
||
|
||
**Why posting references `job_version` and not `job`.** An application must be able to answer
|
||
"what did this candidate actually read", which is the difference between a defensible and an
|
||
indefensible rejection. Pinning the version on the posting, and the posting on the application,
|
||
makes that a two-hop FK rather than a guess.
|
||
|
||
**`job_posting_metric` is a rare "no audit" table.** Stated explicitly because §4.10's default is
|
||
audit-on: these are externally-sourced daily counters with no actor and no decision content, and
|
||
auditing 1.1M machine writes would bloat the audit partition for nothing. The fetch itself is
|
||
audited once per `ingestion_run`.
|
||
|
||
*Queries.* The Job Board screen's per-platform aggregate (`js/jobboard.js:16-19`); channel
|
||
performance ranking (BRD §5); cost per application per platform; the reconciliation sweep for
|
||
postings whose state has drifted.
|
||
|
||
---
|
||
|
||
## 12. Recruitment intake and inbox — the raw layer
|
||
|
||
### ERD 3 — Intake, candidates and applications
|
||
|
||
```mermaid
|
||
erDiagram
|
||
source_channel ||--o{ intake_channel : "type of"
|
||
intake_channel ||--o{ raw_intake : "delivers"
|
||
intake_channel ||--o{ ingestion_run : "polled by"
|
||
raw_intake ||--o{ raw_intake_attachment : "files"
|
||
raw_intake ||--o{ intake_parse_attempt : "parsed by"
|
||
raw_intake ||--o| intake_resolution : "resolved by"
|
||
raw_intake ||--o{ raw_intake_read : "read state"
|
||
intake_resolution }o--o| candidate : "created or matched"
|
||
intake_resolution }o--o| job_application : "created"
|
||
raw_intake ||--o{ candidate : "origin (NOT NULL)"
|
||
raw_intake ||--o{ job_application : "origin (NOT NULL)"
|
||
candidate ||--o{ candidate_email : "addresses"
|
||
candidate ||--o{ candidate_phone : "numbers"
|
||
candidate ||--o{ candidate_skill : "skills"
|
||
candidate ||--o{ candidate_employment : "employment"
|
||
candidate ||--o{ candidate_education : "education"
|
||
candidate ||--o{ candidate_document : "CV revisions"
|
||
candidate ||--o| candidate_search_index: "search doc"
|
||
candidate ||--o{ job_application : "applies"
|
||
job ||--o{ job_application : "receives"
|
||
job_version ||--o{ job_application : "applied against"
|
||
job_application ||--o{ job_application_stage_history : "stage history"
|
||
|
||
raw_intake {
|
||
bigint id PK
|
||
uuid public_id
|
||
bigint intake_channel_id FK
|
||
text external_message_id
|
||
timestamptz received_at
|
||
jsonb payload
|
||
bytea payload_sha256
|
||
text state
|
||
bigint assigned_to_user_id FK
|
||
}
|
||
intake_parse_attempt {
|
||
bigint id PK
|
||
bigint raw_intake_id FK
|
||
text parser_name
|
||
text parser_version
|
||
text status
|
||
jsonb parsed
|
||
numeric confidence
|
||
}
|
||
intake_resolution {
|
||
bigint id PK
|
||
bigint raw_intake_id FK
|
||
text resolution_kind
|
||
text decision_mode
|
||
bigint decided_by_user_id FK
|
||
bigint candidate_id FK
|
||
bigint job_application_id FK
|
||
}
|
||
candidate {
|
||
bigint id PK
|
||
uuid public_id
|
||
text reference_code
|
||
text full_name_original
|
||
text name_normalised
|
||
bigint created_from_raw_intake_id FK
|
||
bigint merged_into_candidate_id FK
|
||
date retention_due_on
|
||
}
|
||
job_application {
|
||
bigint id PK
|
||
uuid public_id
|
||
text reference_code
|
||
bigint candidate_id FK
|
||
bigint job_id FK
|
||
bigint job_version_id FK
|
||
bigint raw_intake_id FK
|
||
int attempt_no
|
||
bigint current_stage_id FK
|
||
bigint status_id FK
|
||
text state
|
||
}
|
||
```
|
||
|
||
**The rule this section exists to make physical.** Nothing becomes a candidate before it exists as
|
||
raw arrival. The prototype's inbox is already resolved — each row carries name, email, `jobId`,
|
||
`atsScore` and recruiter directly (`js/data.js:284-300`) — so there is no representable state for
|
||
"arrived and cannot become a candidate", which is precisely the state that must exist.
|
||
|
||
### 12.1 `app.intake_channel`
|
||
|
||
*Purpose.* A *configured connection*, distinct from the channel *type*. `ref.source_channel` holds
|
||
the 11 channel types from BRD §8.1; `intake_channel` holds the concrete instances — two Outlook
|
||
mailboxes, one careers-form endpoint, one LinkedIn webhook — each with its own credential
|
||
reference, delta cursor and health.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `key` | `text` | no | UNIQUE, e.g. `outlook_careers_mailbox` |
|
||
| `source_channel_id` | `bigint` | no | FK `ref.source_channel` |
|
||
| `name` | `text` | no | |
|
||
| `transport` | `text` | no | CHECK in (`graph_mail`, `webhook`, `form_post`, `manual_ui`, `file_drop`, `api_pull`) |
|
||
| `endpoint_ref` | `text` | yes | Mailbox address, webhook path, form slug |
|
||
| `credential_ref` | `text` | yes | **A pointer into the secret store, never a secret.** CHECK it does not look like a token |
|
||
| `delta_cursor` | `text` | yes | Graph delta link / last-seen id |
|
||
| `delta_cursor_updated_at` | `timestamptz` | yes | |
|
||
| `is_enabled` | `boolean` | no | Default false — a channel is off until deliberately turned on |
|
||
| `health_state` | `text` | no | CHECK in (`unknown`, `healthy`, `degraded`, `failing`), default `unknown` |
|
||
| `last_success_at`, `last_failure_at` | `timestamptz` | yes | |
|
||
| `last_error` | `jsonb` | yes | |
|
||
| `poll_interval_seconds` | `int` | yes | |
|
||
| `auto_create_candidate` | `boolean` | no | Default **false**. Per-channel switch; see §12.4 layer 5 |
|
||
| audit | | | |
|
||
|
||
**Constraints.** `uq_intake_channel_key`. `ck_intake_channel_manual CHECK (transport <> 'manual_ui'
|
||
OR is_enabled)` — the manual channel must always exist and be enabled, because recruiter UI entry
|
||
also goes through raw intake (§12.4 layer 1). `ck_credential_not_inline CHECK (credential_ref IS
|
||
NULL OR credential_ref !~ '^(ey|sk-|Bearer )')` — a cheap, blunt guard against someone pasting a
|
||
token into a config field.
|
||
|
||
**Profile.** *Status:* `health_state`, Tier 2. *Soft delete:* no (disable instead). *Retention:*
|
||
permanent. *PII:* `internal`. *Audit:* yes — enabling a channel or changing a credential reference
|
||
is a security-relevant change. *Volume:* ~20. *Queries:* the worker's poll schedule; the
|
||
integrations health screen; per-channel intake volume.
|
||
|
||
### 12.2 `app.raw_intake`
|
||
|
||
*Purpose.* The immutable landing row. Everything that arrives, from every channel, lands here
|
||
first and is never edited afterwards.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `public_id` | `uuid` | no | UNIQUE |
|
||
| `intake_channel_id` | `bigint` | no | FK |
|
||
| `external_message_id` | `text` | yes | Provider message id / webhook delivery id. NULL for manual UI entry |
|
||
| `received_at` | `timestamptz` | no | When the source says it arrived |
|
||
| `ingested_at` | `timestamptz` | no | When we recorded it. Both, because they differ and lag matters |
|
||
| `payload` | `jsonb` | no | Full envelope: headers, body, form fields. Legitimate JSONB (P6) |
|
||
| `payload_sha256` | `bytea` | no | Over the canonicalised payload |
|
||
| `sender_address_raw` | `text` | yes | Preserved exactly. Attacker-controlled |
|
||
| `subject_raw` | `text` | yes | Preserved exactly. Attacker-controlled |
|
||
| `declared_job_reference` | `text` | yes | What the sender claimed to be applying for, before any resolution |
|
||
| `state` | `text` | no | CHECK in (`received`, `parsing`, `parsed`, `needs_review`, `resolved_new_candidate`, `resolved_existing_candidate`, `rejected_unusable`, `quarantined`) |
|
||
| `state_changed_at` | `timestamptz` | no | |
|
||
| `assigned_to_user_id` | `bigint` | yes | Triage owner |
|
||
| `triage_priority` | `smallint` | no | Default 0 |
|
||
| `resolved_at` | `timestamptz` | yes | |
|
||
| `ingestion_run_id` | `bigint` | yes | FK, which poll produced it |
|
||
| `message_thread_id` | `bigint` | yes | FK `app.message_thread` — links a candidate reply to the outbound message it answers |
|
||
| `created_at` | | no | No `updated_*` except the state columns, which have a narrow grant |
|
||
|
||
**Constraints.**
|
||
`uq_raw_intake_external UNIQUE (intake_channel_id, external_message_id)` and
|
||
`uq_raw_intake_payload UNIQUE (intake_channel_id, payload_sha256)` — the pair that makes
|
||
redelivery idempotent. A webhook retried five times produces one row.
|
||
`ck_raw_intake_terminal CHECK ((state IN ('rejected_unusable','quarantined')) <= (resolved_at IS
|
||
NOT NULL))` — the two terminal states with **no candidate** must still record when triage
|
||
finished.
|
||
Grants: `INSERT, SELECT`, plus `UPDATE (state, state_changed_at, assigned_to_user_id,
|
||
triage_priority, resolved_at)` only. The payload can never be overwritten.
|
||
|
||
**Indexes.** `ix_raw_intake_inbox (state, received_at DESC) WHERE state IN
|
||
('received','parsing','parsed','needs_review')` — the inbox list, which is the screen this table
|
||
exists for. `ix_raw_intake_assigned (assigned_to_user_id, received_at DESC) WHERE resolved_at IS
|
||
NULL`. `ix_raw_intake_channel (intake_channel_id, received_at DESC)` for per-source attribution.
|
||
GIN on `payload` is **not** created by default: indexing every raw envelope is pure write cost, and
|
||
the queries that need it are forensic. It is added only if a specific payload predicate becomes a
|
||
list filter.
|
||
|
||
**Profile.** *Status:* `state`, Tier 2 — a technical pipeline state, not a business vocabulary,
|
||
which is why it is a CHECK rather than a `ref` table. *Required:* channel, timestamps, payload,
|
||
hash, state. *Soft delete:* **no, and nobody can hard-delete a submission** — that is the point of
|
||
the layer. *Retention:* 24 months from `received_at` for unresolved and rejected rows; resolved
|
||
rows follow their candidate's retention (§31). Purge pseudonymises `payload`, `sender_address_raw`
|
||
and `subject_raw` rather than deleting the row. *PII:* `payload` is `sensitive_personal` — it
|
||
contains the CV and everything the sender wrote. *Audit:* insert and every state change.
|
||
*Volume:* ~250,000 (**assumption**: 200,000 applications plus ~25% unusable/duplicate/spam).
|
||
*Queries:* the Recruitment Inbox list with per-source attribution and unread state (BRD §5);
|
||
idempotency check on delivery; "where did this candidate come from" as a single FK hop from
|
||
`candidate.created_from_raw_intake_id`; the stale-queue alert.
|
||
|
||
### 12.3 Remaining intake tables
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.raw_intake_attachment` | One row per attached file | `raw_intake_id`, `stored_file_id` FK, `filename_raw`, `mime_type_declared`, `mime_type_sniffed`, `byte_size`, `sha256`, `attachment_index`, `is_probable_cv boolean`, `extraction_state` CHECK in (`pending`,`extracted`,`unsupported`,`failed`) | `uq (raw_intake_id, attachment_index)`; `ix (sha256)` — a duplicate-detection signal; `ix (extraction_state) WHERE extraction_state = 'pending'` | *Soft delete:* no. *Retention:* with the intake; blob deleted by purge. *PII:* `sensitive_personal`. *Audit:* insert + download. *Volume:* ~300,000 |
|
||
| `app.raw_intake_read` | Per-user read/unread state for the inbox | `raw_intake_id`, `user_id`, `read_at` | PK `(raw_intake_id, user_id)` | *Soft delete:* no. *Retention:* with the intake. *PII:* `internal`. *Audit:* no. *Volume:* ~500,000 |
|
||
| `app.ingestion_run` | One poll or webhook batch per channel | `intake_channel_id`, `started_at`, `finished_at`, `trigger_kind` CHECK in (`schedule`,`manual`,`webhook`), `items_seen`, `items_new`, `items_duplicate`, `items_failed`, `cursor_before`, `cursor_after`, `status` CHECK in (`running`,`succeeded`,`partial`,`failed`), `error jsonb` | `ix (intake_channel_id, started_at DESC)`; counters `NOT NULL DEFAULT 0` | *Soft delete:* no. *Retention:* 13 months. *PII:* `internal`. *Audit:* no (the run rows *are* the record). *Volume:* ~2M — the highest-frequency row in this area, because a 5-minute mail poll is 105,000 runs/year per channel. **Retain 13 months and prune**, and do not audit them |
|
||
| `app.ingestion_dead_letter` | Deliveries that could not even become a `raw_intake` row (malformed envelope, unknown channel, oversized payload) | `intake_channel_id` NULL, `ingestion_run_id` NULL, `external_ref`, **`stored_file_id bigint NULL REFERENCES app.stored_file(id)`** (the payload bytes, via the one registry), **`raw_body_excerpt bytea NULL`** (the diagnostic snippet only), `error jsonb`, `retry_count`, `first_seen_at`, `last_retry_at`, `resolved_at`, `resolved_by_user_id` | `ix (resolved_at) WHERE resolved_at IS NULL`; `ix_dead_letter_age (first_seen_at)` for the time sweep; `ck_dead_letter_excerpt CHECK (octet_length(raw_body_excerpt) <= 8192)` | *Soft delete:* no. *Retention:* **time-based, unconditional** — blob deleted at 90 days, excerpt pseudonymised and row retained 12 months (§31.2). *PII:* `sensitive_personal` (the payload may be a CV). *Audit:* yes. *Volume:* ~2,000. **This table is why "no document is ever silently lost" is true** — the failure mode with no dead-letter table is a `try/except: pass` in the adapter |
|
||
| `app.integration_webhook_event` | Non-application inbound webhooks: delivery receipts, posting state changes, assessment completions | `source` CHECK, `event_kind`, `external_id`, `payload jsonb`, `signature_verified boolean`, `received_at`, `processed_at`, `processing_error jsonb` | `uq (source, external_id)`; `ix (processed_at) WHERE processed_at IS NULL` | *Soft delete:* no. *Retention:* 13 months. *PII:* `personal`. *Audit:* no. *Volume:* ~800,000 |
|
||
|
||
**Why `ingestion_dead_letter` does not hold the payload inline.** An earlier draft gave it
|
||
`raw_body bytea`, which is the one place in an otherwise disciplined design where candidate document
|
||
bytes would live in a relational row rather than in `app.stored_file` plus object storage — and it
|
||
would do so for deliveries whose *stated* causes include "oversized payload". Three consequences
|
||
made that untenable, and all three are why the column is now a `stored_file_id`:
|
||
|
||
1. **Erasure could not reach it.** The row has no `candidate_id` and no link to any subject, so the
|
||
§31.2 purge — which is driven entirely by `candidate.retention_due_on` and enumerates the
|
||
columns it pseudonymises — can never find it. A CV that failed at the envelope stage would sit
|
||
in the database indefinitely and would survive a subject erasure request.
|
||
2. **`files.delete_for_subject()` and the retention sweep only know about `stored_file`.** Routing
|
||
the bytes through the registry puts them inside the one mechanism that already covers virus
|
||
scanning, blob deletion and `deleted_from_store_at`.
|
||
3. **Row size.** A 25 MB malformed attachment inline is a TOAST'd row on a table the dead-letter
|
||
list screen scans.
|
||
|
||
**The compensating control is a time sweep, and §31.2 says so explicitly.** Subject-driven erasure
|
||
structurally cannot reach unlinked intake, because there is no identified subject to key it on.
|
||
`ingestion_dead_letter` and `integration_webhook_event` therefore both carry **unconditional
|
||
time-based** policies triggered on `first_seen_at` / `received_at`, independent of any candidate.
|
||
|
||
`intake_parse_attempt` and `parse_issue` are documented in §18 with the rest of document
|
||
processing. `intake_resolution` is next, because it is the invariant boundary.
|
||
|
||
### 12.4 `app.intake_resolution` — the decision record
|
||
|
||
*Purpose.* The decision that turns (or refuses to turn) an intake into an identity. It has its own
|
||
actor, timestamp, mode and reason, which is what makes the pipeline reviewable.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `raw_intake_id` | `bigint` | no | FK, UNIQUE — one resolution per intake |
|
||
| `resolution_kind` | `text` | no | CHECK in (`create_candidate`, `attach_to_existing_candidate`, `mark_duplicate`, `reject_unusable`, `quarantine`) |
|
||
| `decision_mode` | `text` | no | CHECK in (`human`, `automatic`) |
|
||
| `decided_by_user_id` | `bigint` | yes | Required when `decision_mode = 'human'` |
|
||
| `decided_at` | `timestamptz` | no | |
|
||
| `candidate_id` | `bigint` | yes | FK — set for create/attach |
|
||
| `job_application_id` | `bigint` | yes | FK — set when an application was created too |
|
||
| `duplicate_of_candidate_id` | `bigint` | yes | FK — set for `mark_duplicate` |
|
||
| `reject_reason_id` | `bigint` | yes | FK `ref.rejection_reason` — set for `reject_unusable` |
|
||
| `reject_note` | `text` | yes | |
|
||
| `auto_create_evidence` | `jsonb` | yes | The matching evidence that justified an automatic creation |
|
||
| `matching_config_version_id` | `bigint` | yes | FK — which thresholds were in force |
|
||
| `created_at` | | no | Append-only |
|
||
|
||
**Constraints.**
|
||
|
||
```sql
|
||
CONSTRAINT uq_intake_resolution UNIQUE (raw_intake_id),
|
||
CONSTRAINT ck_res_human CHECK (decision_mode <> 'human' OR decided_by_user_id IS NOT NULL),
|
||
CONSTRAINT ck_res_candidate CHECK ((resolution_kind IN ('create_candidate','attach_to_existing_candidate'))
|
||
= (candidate_id IS NOT NULL)),
|
||
CONSTRAINT ck_res_duplicate CHECK ((resolution_kind = 'mark_duplicate') = (duplicate_of_candidate_id IS NOT NULL)),
|
||
CONSTRAINT ck_res_reject CHECK ((resolution_kind = 'reject_unusable') = (reject_reason_id IS NOT NULL)),
|
||
CONSTRAINT ck_res_auto_evidence CHECK (decision_mode = 'human'
|
||
OR resolution_kind <> 'create_candidate'
|
||
OR auto_create_evidence IS NOT NULL)
|
||
```
|
||
|
||
**Indexes.** `ix_intake_resolution_candidate (candidate_id)`;
|
||
`ix_intake_resolution_kind (resolution_kind, decided_at)` for triage throughput reporting.
|
||
|
||
**Migration placement — created in 011, not 010.** This table's FKs reach `app.job_application`
|
||
(migration 011) and `app.matching_config_version` (013). It is the resolution *of* an application,
|
||
so it belongs with `job_application` in **011**, and `matching_config_version_id` is added there as
|
||
a plain nullable `bigint` and given its `REFERENCES` clause by `ALTER TABLE` in **013**. §33 records
|
||
both steps. Creating it in 010 as an earlier draft did is a forward reference and would fail —
|
||
see §33's forward-reference rule and the CI gate that now enforces it.
|
||
|
||
**Profile.** *Status:* n/a — the resolution *is* terminal. *Soft delete:* no. *Retention:*
|
||
permanent (this is the lawful-basis and provenance record). *PII:* `personal`. *Audit:* yes.
|
||
*Volume:* ~250,000. *Queries:* triage throughput per recruiter; automatic-vs-human creation ratio,
|
||
which is the metric that tells you whether the auto-create thresholds are sane; "why was this
|
||
rejected".
|
||
|
||
### 12.5 The five layers that stop a malformed email creating a candidate
|
||
|
||
All five are in the database. This is the single most important invariant in the schema, so the
|
||
mechanism is spelled out rather than summarised.
|
||
|
||
| Layer | Mechanism | What it stops |
|
||
|---|---|---|
|
||
| **1. Ordering** | `candidate.created_from_raw_intake_id bigint NOT NULL REFERENCES app.raw_intake(id)`, **non-deferrable**. Likewise `job_application.raw_intake_id bigint NOT NULL`. Manual recruiter entry is not an exception — the UI writes a `raw_intake` row on the `manual_ui` channel first | A candidate row physically cannot be inserted before its intake row exists. No import, integration or psql session can produce an origin-less candidate |
|
||
| **2. Shape** | `candidate` has **no** email or phone column. `candidate_email.address_normalised text NOT NULL CHECK (address_normalised ~ '^[^@[:space:],;<>]+@[^@[:space:].,;<>]+([.][^@[:space:].,;<>]+)+$' AND length(address_normalised) BETWEEN 6 AND 254 AND address_normalised = lower(address_normalised))`, alongside `address_original text NOT NULL`. `candidate_phone.e164 text CHECK (e164 ~ '^[+][1-9][0-9]{6,14}$')` | This is where a malformed address dies. A truncated or garbled address fails the CHECK and the insert aborts |
|
||
| **3. Contactability** | `CONSTRAINT TRIGGER` on `candidate`, `DEFERRABLE INITIALLY DEFERRED`, raising unless at least one `candidate_email` or `candidate_phone` row exists at COMMIT | A contactless ghost candidate cannot commit even if layer 2 aborted only the contact insert. The intake stays in `needs_review` instead |
|
||
| **4. Identity uniqueness** | `CREATE UNIQUE INDEX uq_candidate_email ON app.candidate_email (address_normalised) WHERE deleted_at IS NULL AND suppressed_by_merge_id IS NULL AND is_identifying` and the matching `uq_candidate_phone` on `e164` | "One identifying contact point, one identity" is a database fact. If application-side matching misses, the insert fails and the intake is forced into duplicate review. The `is_identifying` term is what keeps agency and shared contact points storable — see the counter-cases below |
|
||
| **5. No silent auto-create** | `ck_res_auto_evidence` above, plus `intake_channel.auto_create_candidate` defaulting to false | An automatically created identity must carry the evidence that justified it. Thresholds stay in versioned config where they can be tuned; the *requirement to have evidence* is in the schema |
|
||
|
||
**Why layer 3 must be a deferrable constraint trigger.** A plain CHECK cannot span tables. A
|
||
`NOT NULL primary_email_id` on `candidate` would create a circular FK and would wrongly forbid the
|
||
legitimate phone-only referral. Firing at COMMIT is what allows `candidate` and `candidate_email`
|
||
to be inserted in either order inside one transaction.
|
||
|
||
**Validation is syntactic only, deliberately.** The regex proves shape, not deliverability.
|
||
`candidate_email.verified_at` is set by an actual send or a verification service, never inferred
|
||
from the regex passing.
|
||
|
||
**Known counter-cases, flagged now — and they apply to phone numbers exactly as much as to
|
||
addresses.** The global unique indexes on `candidate_email.address_normalised` *and*
|
||
`candidate_phone.e164` each make "one contact point, one candidate" a hard rule, and real cases
|
||
violate both:
|
||
|
||
| Counter-case | Breaks the email index | Breaks the phone index |
|
||
|---|---|---|
|
||
| Shared household contact point | family address | family landline / shared mobile |
|
||
| Agency submission | every candidate arrives on the agency mailbox | every candidate arrives on the agency switchboard number |
|
||
| Generic organisational contact on a referral or campus form | `info@` | a reception number |
|
||
|
||
Those intakes fail resolution and pile up in `needs_review` **identically**, so the mitigation must
|
||
cover both columns. The mechanism: `ref.non_identifying_contact` holds the known shared addresses,
|
||
domains and numbers; `ref.source_channel.is_identifying` marks whole channels (agency, campus) whose
|
||
contact points must never be treated as identity; and — because **an index predicate may not
|
||
subquery** — each of `candidate_email` and `candidate_phone` carries a trigger-maintained
|
||
`is_identifying boolean NOT NULL DEFAULT true`, denormalised from those two `ref` tables, so the
|
||
predicate can reference it directly:
|
||
|
||
```sql
|
||
CREATE UNIQUE INDEX uq_candidate_email ON app.candidate_email (address_normalised)
|
||
WHERE deleted_at IS NULL AND suppressed_by_merge_id IS NULL AND is_identifying;
|
||
|
||
CREATE UNIQUE INDEX uq_candidate_phone ON app.candidate_phone (e164)
|
||
WHERE deleted_at IS NULL AND suppressed_by_merge_id IS NULL AND is_identifying;
|
||
```
|
||
|
||
A non-identifying contact point is still stored, still displayed and still contactable — it simply
|
||
stops being an identity key, which is the correct semantics. **Decide the seed list before go-live,
|
||
not after the queue backs up.** Risk R4 (§32.2) is updated to name `candidate_phone.e164` alongside
|
||
the email index.
|
||
|
||
---
|
||
|
||
## 13. Candidates
|
||
|
||
**The promotion rule** (binding, from `_decisions.md`): a field is a **first-class column** only if
|
||
the system must FILTER, SORT, JOIN or ENFORCE UNIQUENESS on it, or an invariant or report depends
|
||
on it. It becomes a **child table** if a candidate can legitimately have more than one, or if each
|
||
instance needs its own provenance, verification state or date range. It stays in **JSONB** only if
|
||
it is parser-derived, shape-unstable, read as a whole for display or re-parse, and never a query
|
||
predicate, never an FK target and never referenced by any constraint. Promotion out of JSONB is
|
||
one-way, happens on write, and leaves the JSONB untouched.
|
||
|
||
Without the first clause everything becomes a column and the parser's long tail (publications,
|
||
references, language-proficiency detail, section offsets) forces monthly migrations. Without the
|
||
last clause JSONB becomes a shadow schema whose shape nothing enforces, which is where "we thought
|
||
that field was always present" incidents come from.
|
||
|
||
### 13.1 `app.candidate`
|
||
|
||
*Purpose.* A person, independent of any application. This table is the single highest-value
|
||
correction to the prototype, where one flat array carries `jobId`, `jobTitle`, `stage`, `aiScore`
|
||
and `recruiter` directly on the candidate (`js/data.js:117-127`) so one person cannot hold two
|
||
applications.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `public_id` | `uuid` | no | UNIQUE |
|
||
| `reference_code` | `text` | no | UNIQUE, `CAN-5001` shape (`js/data.js:118`) |
|
||
| `full_name_original` | `text` | no | Exactly as parsed or entered. **Attacker-controlled** |
|
||
| `display_name` | `text` | no | Recruiter-correctable; defaults to `full_name_original` |
|
||
| `name_normalised` | `text` | no | `GENERATED ALWAYS AS (lower(public.unaccent_immutable(coalesce(display_name,'')))) STORED`. Trigram-indexed. Feeds both search and duplicate detection. **`unaccent_immutable`, never bare `unaccent` — see the note below** |
|
||
| `country_code` | `char(2)` | yes | ISO 3166-1 alpha-2, CHECK `~ '^[A-Z]{2}$'` |
|
||
| `location_text` | `text` | yes | As stated by the candidate; preserved even when `location_id` resolves |
|
||
| `location_id` | `bigint` | yes | FK `ref.location` when it maps |
|
||
| `current_title` | `text` | yes | Filterable, so a column (`js/candidates.js:68`) |
|
||
| `current_employer_name` | `text` | yes | |
|
||
| `current_employer_normalised` | `text` | yes | `GENERATED ALWAYS AS (lower(public.unaccent_immutable(coalesce(current_employer_name,'')))) STORED`, trigram-indexed. A duplicate-detection signal |
|
||
| `total_experience_months` | `int` | yes | CHECK 0–840. **Months, not years** |
|
||
| `highest_education_level_id` | `bigint` | yes | FK `ref.education_level` |
|
||
| `notice_period_days` | `int` | yes | |
|
||
| `availability_kind` | `text` | yes | CHECK in (`immediate`, `notice_period`, `date`, `unknown`) — the prototype has an "Immediate Availability" saved search (`js/data.js:413`) |
|
||
| `available_from` | `date` | yes | |
|
||
| `desired_salary_amount` | `numeric(14,2)` | yes | `sensitive_personal` |
|
||
| `desired_salary_currency_code` | `char(3)` | yes | |
|
||
| `source_channel_id` | `bigint` | no | FK `ref.source_channel` — first-touch attribution |
|
||
| `created_from_raw_intake_id` | `bigint` | **no** | FK `app.raw_intake`, non-deferrable. §12.5 layer 1 |
|
||
| `status_id` | `bigint` | no | FK `ref.lifecycle_status` domain `candidate`: `active`, `passive`, `do_not_contact`, `merged`, `purged` |
|
||
| `status_domain` | `text` gen. | no | Constant `'candidate'` |
|
||
| `merged_into_candidate_id` | `bigint` | yes | FK self. Set on the losing side of a merge |
|
||
| `is_pseudonymised` | `boolean` | no | Default false. Set by the retention purge |
|
||
| `pseudonymised_at` | `timestamptz` | yes | |
|
||
| `retention_due_on` | `date` | yes | Maintained by trigger from last meaningful activity |
|
||
| `last_activity_at` | `timestamptz` | yes | Maintained by trigger from applications, interviews and messages |
|
||
| audit + soft delete | | | |
|
||
|
||
**Constraints.** `uq_candidate_public_id`, `uq_candidate_reference_code`.
|
||
`ck_candidate_merge_not_self CHECK (merged_into_candidate_id <> id)`.
|
||
`ck_candidate_merged_status` — a `DEFERRABLE` constraint trigger asserting that
|
||
`merged_into_candidate_id IS NOT NULL` exactly when the status key is `merged` (a CHECK cannot see
|
||
`ref.lifecycle_status.key`, which is why it is a trigger).
|
||
`ck_candidate_pseudonymised CHECK (is_pseudonymised = (pseudonymised_at IS NOT NULL))`.
|
||
Money pair CHECK on the desired-salary pair. Plus the deferred **contactability** trigger (§12.5
|
||
layer 3).
|
||
|
||
**Every generated normalisation column uses `public.unaccent_immutable`, and this is not
|
||
cosmetic.** The one-argument `public.unaccent(text)` shipped by the `unaccent` extension is
|
||
declared `STABLE`, not `IMMUTABLE`, because it resolves its dictionary at run time. PostgreSQL 16
|
||
therefore **rejects** the obvious spelling:
|
||
|
||
```sql
|
||
-- FAILS: ERROR: generation expression is not immutable
|
||
name_normalised text GENERATED ALWAYS AS (lower(public.unaccent(coalesce(display_name,'')))) STORED
|
||
|
||
-- WORKS: the wrapper pins the dictionary, so the expression is immutable
|
||
name_normalised text NOT NULL GENERATED ALWAYS AS (lower(public.unaccent_immutable(coalesce(display_name,'')))) STORED
|
||
```
|
||
|
||
Migration `001` creates the wrapper that pins the dictionary and is safe to mark immutable:
|
||
|
||
```sql
|
||
CREATE FUNCTION public.unaccent_immutable(text) RETURNS text
|
||
LANGUAGE sql IMMUTABLE STRICT PARALLEL SAFE AS
|
||
$$ SELECT public.unaccent('public.unaccent'::regdictionary, $1) $$;
|
||
```
|
||
|
||
and migration `001` then **asserts** it, so a botched extension install fails the migration rather
|
||
than the ninth one:
|
||
|
||
```sql
|
||
DO $$
|
||
BEGIN
|
||
IF NOT EXISTS (
|
||
SELECT 1 FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespace
|
||
WHERE n.nspname = 'public' AND p.proname = 'unaccent_immutable'
|
||
AND p.provolatile = 'i'
|
||
) THEN
|
||
RAISE EXCEPTION 'public.unaccent_immutable is missing or not IMMUTABLE';
|
||
END IF;
|
||
END $$;
|
||
```
|
||
|
||
**The tradeoff, stated honestly:** marking the wrapper `IMMUTABLE` is a promise the database cannot
|
||
verify. If someone `ALTER TEXT SEARCH DICTIONARY unaccent`s the rules, every stored generated value
|
||
and every expression index silently becomes stale. That is acceptable because the `unaccent`
|
||
dictionary is never edited in this deployment, and it is the *only* way to have accent-folded
|
||
generated columns at all. The alternative — normalising in application code and storing a plain
|
||
column — was rejected because a psql insert then bypasses normalisation and duplicate detection
|
||
silently stops matching, which is exactly the class of defect P9 exists to remove.
|
||
|
||
**Columns bound by this rule:** `candidate.name_normalised`,
|
||
`candidate.current_employer_normalised`, `candidate_employment.employer_normalised`,
|
||
`candidate_education.institution_normalised`, `candidate_link.url_normalised`,
|
||
`ref.skill.name_normalised`, `ref.skill_alias.alias_normalised`, and the `unaccent` calls inside
|
||
`app.build_candidate_search_document()` (§29.3).
|
||
|
||
**CI gate (junior workstream).** A grep over `migrations/*.sql` that fails the build on a bare
|
||
`unaccent(` appearing inside a `GENERATED ALWAYS AS` clause or an index expression:
|
||
|
||
```bash
|
||
grep -nE '(GENERATED ALWAYS AS|CREATE (UNIQUE )?INDEX)[^;]*[^_]unaccent\(' migrations/*.sql \
|
||
&& { echo "bare unaccent() in a generated column or index expression"; exit 1; }
|
||
```
|
||
|
||
This gate is the reason the rule survives; the wrapper being *documented* is not enough, because
|
||
the natural thing to type is `unaccent(`.
|
||
|
||
**Indexes.**
|
||
|
||
| Index | Purpose |
|
||
|---|---|
|
||
| `ix_candidate_name_trgm` GIN `(name_normalised gin_trgm_ops)` | Fuzzy name search **and** duplicate detection — one mechanism, tuned once |
|
||
| `ix_candidate_employer_trgm` GIN `(current_employer_normalised gin_trgm_ops)` | Employer-similarity duplicate signal |
|
||
| `ix_candidate_live` `(status_id, last_activity_at DESC) WHERE deleted_at IS NULL` | The Candidates list default ordering |
|
||
| `ix_candidate_retention` `(retention_due_on) WHERE deleted_at IS NULL AND is_pseudonymised = false` | The nightly purge is an index range scan, not a full-table computation |
|
||
| `ix_candidate_experience` `(total_experience_months) WHERE deleted_at IS NULL` | The experience-range filter |
|
||
| `ix_candidate_merged` `(merged_into_candidate_id) WHERE merged_into_candidate_id IS NOT NULL` | Redirect resolution |
|
||
| `ix_candidate_intake` `(created_from_raw_intake_id)` | Provenance lookups |
|
||
|
||
**Profile.** *Status:* Tier 1 + `candidate_status_history`. *Required:* name, source channel,
|
||
intake origin, status; everything else is §4.9 reason 1 or 2. *Soft delete:* yes. *Retention:*
|
||
policy `candidate_no_hire` — 24 months from `last_activity_at` unless a `retention_hold` exists
|
||
(**assumption**; the BRD requires honouring rights requests but names no period). *PII:*
|
||
`personal`; `desired_salary_*` is `sensitive_personal`. *Audit:* yes, data changes **and** profile
|
||
views as explicit access events. *Volume:* ~125,000. *Queries:* the Candidates list with filters
|
||
and sorts (`js/candidates.js`); fuzzy name/employer search; the duplicate detector's candidate
|
||
generation; retention sweep; merge redirect.
|
||
|
||
**Why `total_experience_months` and not the prototype's integer years** (`js/data.js:121`): months
|
||
is what CV date arithmetic actually produces, and rounding to years loses ordering between two
|
||
candidates 11 months apart. The raw experience string stays in `intake_parse_attempt.parsed`.
|
||
|
||
### 13.2 Candidate child tables
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | PII | Volume |
|
||
|---|---|---|---|---|---|
|
||
| `app.candidate_email` | Work and personal addresses with per-address provenance and verification | `candidate_id`, `address_original`, `address_normalised`, `kind` (`personal`/`work`/`unknown`, CHECK), `is_primary`, `verified_at`, `source` CHECK in (`parsed`,`form`,`recruiter`,`integration`), `is_identifying boolean NOT NULL DEFAULT true` (trigger-maintained from `ref.non_identifying_contact` and `ref.source_channel.is_identifying`), `parse_attempt_id` NULL, `suppressed_by_merge_id` NULL, soft delete | The §12.5 layer-2 CHECK; `CREATE UNIQUE INDEX uq_candidate_email ON app.candidate_email (address_normalised) WHERE deleted_at IS NULL AND suppressed_by_merge_id IS NULL AND is_identifying;`; `CREATE UNIQUE INDEX uq_candidate_email_primary ON app.candidate_email (candidate_id) WHERE is_primary AND deleted_at IS NULL;` | `personal` | ~180,000 |
|
||
| `app.candidate_phone` | Same shape for phones | `candidate_id`, `e164`, `raw_input`, `kind`, `is_primary`, `verified_at`, `source`, `is_identifying boolean NOT NULL DEFAULT true` (same trigger), `suppressed_by_merge_id`, soft delete | E.164 CHECK; `CREATE UNIQUE INDEX uq_candidate_phone ON app.candidate_phone (e164) WHERE deleted_at IS NULL AND suppressed_by_merge_id IS NULL AND is_identifying;` — btree, and also the exact-phone duplicate signal. **The `is_identifying` term is required here for the same reason as on email** (§12.5): agency switchboards and shared household numbers otherwise jam `needs_review` | `personal` | ~140,000 |
|
||
| `app.candidate_skill` | Skills with proficiency, provenance and confidence | `candidate_id`, `skill_id` NULL FK `ref.skill`, `raw_label` NULL, `proficiency` NULL CHECK 1–5, `years_months` NULL, `source` CHECK in (`parsed`,`self_declared`,`recruiter`,`assessment`,`ai_inferred`), `confidence numeric(4,3)` NULL, `is_confirmed_by_recruiter`, `parse_attempt_id` NULL, `ai_run_id` NULL | `ck_candidate_skill_ref CHECK (skill_id IS NOT NULL OR raw_label IS NOT NULL)`; `CREATE UNIQUE INDEX uq_candidate_skill ON app.candidate_skill (candidate_id, skill_id) WHERE skill_id IS NOT NULL;`; `ix (skill_id)` for "who knows PostgreSQL" | `personal` | ~700,000 |
|
||
| `app.candidate_employment` | Employment history — a duplicate signal and a requirement input, so columns not JSONB | `candidate_id`, `employer_name`, `employer_normalised text GENERATED ALWAYS AS (lower(public.unaccent_immutable(coalesce(employer_name,'')))) STORED`, `title`, `start_date`, `end_date` NULL (`NULL` = current), `is_current` generated `(end_date IS NULL)`, `location_text`, `description`, `source`, `parse_attempt_id` | `ck_employment_dates CHECK (end_date IS NULL OR end_date >= start_date)`; `ix (candidate_id, start_date DESC)`; GIN trigram on `employer_normalised`; a "one current role" partial unique index is **deliberately not created** — concurrent roles are legitimate | `personal` | ~400,000 |
|
||
| `app.candidate_education` | Degrees and institutions | `candidate_id`, `institution_name`, `institution_normalised text GENERATED ALWAYS AS (lower(public.unaccent_immutable(coalesce(institution_name,'')))) STORED`, `education_level_id` FK, `field_of_study`, `start_date`, `end_date`, `grade_text`, `source`, `parse_attempt_id` | `ck` date order; `ix (candidate_id)` | `personal` | ~200,000 |
|
||
| `app.candidate_link` | LinkedIn, GitHub, portfolio | `candidate_id`, `link_type_id` FK vocabulary, **`link_type_key text NOT NULL`** (denormalised by the same trigger idiom as `job_application.status_key`), `url_original`, `url_normalised text GENERATED ALWAYS AS (lower(public.unaccent_immutable(coalesce(url_original,'')))) STORED`, `suppressed_by_merge_id` NULL, soft delete | `CREATE UNIQUE INDEX uq_candidate_link_url ON app.candidate_link (url_normalised) WHERE link_type_key = 'linkedin' AND deleted_at IS NULL AND suppressed_by_merge_id IS NULL;` — identical normalised LinkedIn URL is a strong duplicate signal | `personal` | ~150,000 |
|
||
| `app.candidate_consent` | Lawful basis, granted and withdrawn. **Append-only** | `candidate_id`, `purpose_id` FK vocabulary, `lawful_basis` CHECK in (`consent`,`legitimate_interest`,`contract`,`legal_obligation`), `granted_at`, `withdrawn_at`, `source`, `evidence_ref`, `raw_intake_id` NULL | `ck (withdrawn_at IS NULL OR withdrawn_at >= granted_at)`; `ix (candidate_id, purpose_id)`; append-only grants | `personal` | ~250,000 |
|
||
| `app.candidate_tag` | Recruiter tags | `candidate_id`, `tag_id` FK `ref.tag`, `added_by_user_id`, soft delete | `CREATE UNIQUE INDEX uq_candidate_tag ON app.candidate_tag (candidate_id, tag_id) WHERE deleted_at IS NULL;` | `internal` | ~200,000 |
|
||
| `app.candidate_note` | Free-text recruiter notes, optionally scoped to an application or interview | `candidate_id` **NOT NULL**, `job_application_id` NULL, `interview_id` NULL, `body`, `note_kind_id` FK vocabulary, `is_private boolean`, `author_user_id`, soft delete | `ck_note_scope` — a trigger asserting that a supplied `job_application_id` belongs to this candidate (a CHECK cannot span tables); `ix (candidate_id, created_at DESC)` | `sensitive_personal` — notes contain opinions about people | ~150,000 |
|
||
| `app.candidate_field_provenance` | Per-field origin of every promoted candidate column | `candidate_id`, `column_name`, `source_kind` CHECK in (`parsed`,`form`,`recruiter`,`integration`,`ai`,`merge`), `parse_attempt_id` NULL, `ai_run_id` NULL, `merge_id` NULL, `confidence numeric(4,3)` NULL, `set_at`, `set_by_user_id` NULL, `previous_value text` NULL | `uq (candidate_id, column_name, set_at)`; `ix (candidate_id)` | `internal` | ~900,000 |
|
||
| `app.candidate_status_history` | Status over time, trigger-written | Standard history shape (§9.5) | `ex` overlap on `(candidate_id, tstzrange)` | `internal` | ~250,000 |
|
||
| `app.candidate_search_index` | The FTS document. §29 | `candidate_id` PK, `document tsvector`, `refreshed_at`, `source_version int` | GIN on `document` | derived `personal` | ~125,000 |
|
||
| `app.candidate_embedding` | **Phase 2.** pgvector embeddings, versioned per model | `candidate_id`, `model_id`, `model_version`, `source_document_id` FK, `embedding vector(1024)`, `generated_at`, `ai_run_id` | `uq (candidate_id, model_id, model_version, source_document_id)`; HNSW index on `embedding` | derived `sensitive_personal` | ~200,000 |
|
||
|
||
**`candidate_field_provenance` replaces `CandidateProfileVersion`.** `_decisions.md` Part 1 lists a
|
||
`CandidateProfileVersion` entity "from a parsed doc". A whole profile version row would duplicate
|
||
`intake_parse_attempt.parsed`, which already *is* the immutable parsed profile. What the version
|
||
was actually for is answering "where did this candidate's job title come from, and how confident
|
||
were we" — which is per-field, not per-profile. This table answers it directly, supports the
|
||
explainability requirement for AI-derived fields (BRD §7.1), and makes a recruiter correction
|
||
visible as a provenance row with `source_kind = 'recruiter'`. **Recorded in §30 and §32.**
|
||
|
||
**`candidate_link.link_type_key` exists so the index predicate needs no magic id.** The LinkedIn
|
||
uniqueness rule has to discriminate on link type, but the discriminator is `link_type_id`, an FK to
|
||
`ref.vocabulary_value`, and **an index predicate may not contain a subquery**. That leaves two
|
||
options: hardcode a seeded literal id (`WHERE link_type_id = 7`), or denormalise the key. §16 takes
|
||
the first option honestly and visibly for `assignment_role` — "the subquery in an index predicate is
|
||
not permitted, so this is implemented as `role_id = 1` with the id pinned by the seed migration and
|
||
asserted by a CI test" — and that is defensible there because `assignment_role` is a six-row table
|
||
whose primary-recruiter row will never move. Here the key is denormalised instead, because
|
||
`ref.vocabulary_value` holds twelve vocabularies and its ids are not conceptually pinned, so a magic
|
||
number in a uniqueness predicate is a re-seed away from silently unenforcing the rule. The
|
||
maintenance cost is one more trigger of a shape already used on `job_application.status_key` and
|
||
`offer.status_key`, and the payoff is that the predicate is readable and self-asserting.
|
||
|
||
**Skills are a controlled vocabulary with an escape hatch.** `ref.skill` plus `ref.skill_alias`
|
||
because the prototype already uses a fixed `skillsPool` (`js/data.js:46`), so the taxonomy is
|
||
nearly free — and a taxonomy is what makes requirement matching, faceting and scoring reproducible,
|
||
whereas free-text skills make every score depend on spelling. The nullable `skill_id` plus
|
||
`raw_label` pair is deliberate: parser output that maps to nothing must still be storable and
|
||
reviewable rather than dropped, and it becomes the queue that grows the taxonomy.
|
||
|
||
**Rejected here:** skills as `text[]` on candidate (cannot carry proficiency, provenance or
|
||
confidence, and GIN on a text array still leaves spelling variants unmatched); employment history
|
||
in JSONB (employer and date-range overlap are duplicate-detection signals and requirement inputs,
|
||
so they must be queryable columns); a generic `candidate_attribute` EAV table (unqueryable without
|
||
pivots, untypable).
|
||
|
||
### 13.3 Legitimate JSONB — the complete list
|
||
|
||
These columns, and **no others**, may be JSONB in this schema:
|
||
|
||
`raw_intake.payload`, `intake_parse_attempt.parsed`, `intake_parse_attempt.error`,
|
||
`candidate_document.layout_metadata`, `ats_result.evidence`,
|
||
`ats_result_criterion.matched_evidence`, `ai.ai_model_invocation.request`/`.response`,
|
||
`ai.ai_suggestion.payload`, `audit.audit_event.before`/`.after`, `job_version.custom_fields`,
|
||
`approval_request.route_snapshot`, `integration_webhook_event.payload`,
|
||
`ingestion_dead_letter.error`, `intake_resolution.auto_create_evidence`,
|
||
`duplicate_candidate_pair.signals`, `candidate_merge_operation.previous_value`/`.new_value`,
|
||
`scoring_config_version.hyperparameters`, `assessment_result.breakdown`,
|
||
`saved_search.filters`, `saved_report.definition`, `talent_pool.criteria`,
|
||
`ref.vocabulary_value.metadata`, `setting.value`.
|
||
|
||
**GIN indexes on JSONB are added only where a query needs them**: `intake_parse_attempt.parsed`
|
||
and `ats_result.evidence`. Indexing every raw payload is pure write cost.
|
||
|
||
---
|
||
|
||
## 14. Candidate documents
|
||
|
||
### 14.1 `app.candidate_document`
|
||
|
||
*Purpose.* CV revisions and other candidate files, attached to the person. This is a child table
|
||
rather than a column on `candidate` because CV revisions are exactly what an `ats_result` must pin:
|
||
a score computed against the July CV must not appear to have been computed against the September
|
||
one.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `public_id` | `uuid` | no | UNIQUE |
|
||
| `candidate_id` | `bigint` | no | FK |
|
||
| `stored_file_id` | `bigint` | no | FK `app.stored_file` |
|
||
| `document_type_id` | `bigint` | no | FK vocabulary: `cv`, `cover_letter`, `certificate`, `portfolio`, `id_document`, `reference_letter`, `right_to_work` |
|
||
| `revision_no` | `int` | no | Per candidate per type |
|
||
| `is_primary_cv` | `boolean` | no | Default false |
|
||
| `source_raw_intake_id` | `bigint` | yes | FK — NULL only for a recruiter-attached file whose intake row is the manual-UI one, which is still set in practice |
|
||
| `raw_intake_attachment_id` | `bigint` | yes | FK — the specific attachment it came from |
|
||
| `sha256` | `bytea` | no | Denormalised from `stored_file` for local uniqueness checks |
|
||
| `filename_display` | `text` | no | Sanitised for display; the raw name stays on `stored_file` |
|
||
| `page_count` | `int` | yes | |
|
||
| `language_code` | `char(2)` | yes | Detected |
|
||
| `extracted_text` | `text` | yes | Full text. Weight `D` in the search document (§29) |
|
||
| `extracted_text_chars` | `int` | yes | Cheap emptiness check without reading the text |
|
||
| `layout_metadata` | `jsonb` | yes | Section offsets, block coordinates. Legitimate JSONB |
|
||
| `parse_state` | `text` | no | CHECK in (`pending`, `parsing`, `parsed`, `partial`, `failed`, `unsupported`) — the four BRD §9.2 document states plus two the BRD's set cannot express |
|
||
| `latest_parse_attempt_id` | `bigint` | yes | FK, denormalised pointer |
|
||
| `uploaded_by_user_id` | `bigint` | yes | |
|
||
| `expires_on` | `date` | yes | For certificates and right-to-work documents |
|
||
| audit + soft delete | | | |
|
||
|
||
**Constraints.** `uq_candidate_document_revision UNIQUE (candidate_id, document_type_id,
|
||
revision_no)`. Two partial unique **indexes** per §4.8:
|
||
`CREATE UNIQUE INDEX uq_candidate_document_primary_cv ON app.candidate_document (candidate_id) WHERE is_primary_cv AND deleted_at IS NULL;`
|
||
— exactly one current CV — and
|
||
`CREATE UNIQUE INDEX uq_candidate_document_sha ON app.candidate_document (candidate_id, sha256) WHERE deleted_at IS NULL;`
|
||
— the same file re-sent is not a new revision.
|
||
`ck_candidate_document_text CHECK ((parse_state IN ('parsed','partial')) <= (extracted_text IS NOT
|
||
NULL))`.
|
||
|
||
**Indexes.** `ix_candidate_document_candidate (candidate_id, document_type_id, revision_no DESC)
|
||
WHERE deleted_at IS NULL`; `ix_candidate_document_sha (sha256)` — identical-document duplicate
|
||
signal across candidates; `ix_candidate_document_parse (parse_state) WHERE parse_state IN
|
||
('pending','parsing')`; `ix_candidate_document_expiry (expires_on) WHERE expires_on IS NOT NULL`.
|
||
|
||
**Profile.** *Status:* `parse_state`, Tier 2. *Required:* candidate, file, type, revision, hash,
|
||
state. *Soft delete:* yes. *Retention:* with the candidate; the purge deletes the blob and nulls
|
||
`extracted_text`, keeping the row so `ats_result.candidate_document_id` stays valid. *PII:*
|
||
`sensitive_personal` (`extracted_text` is the full CV). *Audit:* yes, including **download as an
|
||
explicit access event** — for candidate PII, the read is the event that matters. *Volume:*
|
||
~200,000. *Queries:* the document list on a candidate profile; the CV the score was computed
|
||
against; the parse queue; FTS document assembly; the retention purge's blob list; identical-file
|
||
duplicate detection.
|
||
|
||
**Why `extracted_text` lives here and not on the parse attempt.** The parse attempt owns the
|
||
*attempt* (parser, version, confidence, errors, structured output); the document owns the
|
||
*current best text*, because that is what search and scoring read on every request and it must be
|
||
one indexed lookup, not a join to the latest successful attempt. The attempt's `parsed` JSONB
|
||
retains the full structured extraction. The redundancy is deliberate and one-directional: a
|
||
successful parse writes `extracted_text` on the document; nothing writes back.
|
||
|
||
---
|
||
|
||
## 15. Applications
|
||
|
||
### 15.1 `app.job_application`
|
||
|
||
*Purpose.* The candidate ↔ job join. It owns stage and status, and it is the **only** thing an ATS
|
||
score attaches to. In the prototype there is no such entity — `jobId`, `stage` and `aiScore` hang
|
||
off the candidate (`js/data.js:117-127`) — which is why a candidate there cannot hold two
|
||
applications or two scores.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `public_id` | `uuid` | no | UNIQUE. Appears in candidate status emails, which is why §3 exists |
|
||
| `reference_code` | `text` | no | UNIQUE, `APP-30001` shape (`js/data.js:298`) |
|
||
| `candidate_id` | `bigint` | no | FK |
|
||
| `job_id` | `bigint` | no | FK. Uniqueness is per **job**, not per version or posting |
|
||
| `job_version_id` | `bigint` | no | FK — the version in force when the application was created |
|
||
| `job_posting_id` | `bigint` | yes | FK — the exact posting the candidate read, when known |
|
||
| `raw_intake_id` | `bigint` | **no** | FK, non-deferrable. §12.5 layer 1 |
|
||
| `source_channel_id` | `bigint` | no | FK `ref.source_channel` |
|
||
| `pipeline_config_version_id` | `bigint` | no | FK — pinned at creation, so a later config edit cannot retro-change which transitions were legal |
|
||
| `attempt_no` | `int` | no | Default 1 |
|
||
| `current_stage_id` | `bigint` | no | FK `ref.pipeline_stage`. Denormalised current state |
|
||
| `status_id` | `bigint` | no | FK `ref.lifecycle_status` domain `job_application` |
|
||
| `status_domain` | `text` gen. | no | Constant `'job_application'` |
|
||
| `status_key` | `text` | no | Denormalised from `ref.lifecycle_status.key`, maintained by trigger. Read by the UI and by reporting group-bys |
|
||
| `status_is_terminal` | `boolean` | no | Denormalised from `ref.lifecycle_status.is_terminal` by the **same** trigger. Exists **only** so `state` can be generated without a hardcoded key list (§4.6) |
|
||
| `status_is_negative` | `boolean` | no | Denormalised from `ref.lifecycle_status.is_negative` by the same trigger. Read by funnel reporting and by the terminal-negative transition guard |
|
||
| `state` | `text` gen. | no | `GENERATED ALWAYS AS (CASE WHEN status_is_terminal THEN 'terminal' ELSE 'active' END) STORED` |
|
||
| `applied_at` | `timestamptz` | no | |
|
||
| `stage_entered_at` | `timestamptz` | no | Maintained by trigger. Makes "days in current stage" a subtraction |
|
||
| `terminal_at` | `timestamptz` | yes | |
|
||
| `rejection_reason_id` | `bigint` | yes | FK `ref.rejection_reason` |
|
||
| `withdrawal_reason` | `text` | yes | |
|
||
| `superseded_by_application_id` | `bigint` | yes | FK self. Set by merge step 3 (§19.4) |
|
||
| `cooling_off_override_by_user_id` | `bigint` | yes | |
|
||
| `cooling_off_override_reason` | `text` | yes | |
|
||
| `current_ats_result_id` | `bigint` | yes | FK `app.ats_result`, `DEFERRABLE`. The pointer to the current score row, for provenance and for the explanation drill-down |
|
||
| `current_overall_score` | `numeric(6,3)` | yes | **The score *value*, denormalised.** Maintained by the same trigger that flips `ats_result.is_current`. This is what the shortlist index sorts on |
|
||
| `current_band` | `text` | yes | The band label of the current score, denormalised alongside it so a band filter needs no join |
|
||
| `current_primary_recruiter_id` | `bigint` | yes | FK `app.app_user`. Denormalised from assignment |
|
||
| `sla_due_at` | `timestamptz` | yes | Derived from the pipeline config's stage SLA |
|
||
| audit + soft delete | | | |
|
||
|
||
**Constraints — the reapplication rule.**
|
||
|
||
```sql
|
||
-- at most one live application per (candidate, job)
|
||
CREATE UNIQUE INDEX uq_application_live ON app.job_application (candidate_id, job_id)
|
||
WHERE state = 'active' AND deleted_at IS NULL AND superseded_by_application_id IS NULL;
|
||
|
||
-- attempts are explicit and orderable
|
||
CONSTRAINT uq_application_attempt UNIQUE (candidate_id, job_id, attempt_no),
|
||
|
||
-- terminality and its timestamp cannot disagree
|
||
CONSTRAINT ck_application_terminal CHECK ((state = 'terminal') = (terminal_at IS NOT NULL)),
|
||
CONSTRAINT ck_application_reject CHECK (rejection_reason_id IS NULL OR state = 'terminal'),
|
||
CONSTRAINT ck_application_override CHECK ((cooling_off_override_by_user_id IS NULL)
|
||
= (cooling_off_override_reason IS NULL)),
|
||
CONSTRAINT ck_application_superseded CHECK (superseded_by_application_id <> id)
|
||
```
|
||
|
||
plus a plain `BEFORE INSERT` **row** trigger enforcing the **cooling-off period**:
|
||
|
||
```sql
|
||
CREATE TRIGGER tg_application_cooling_off
|
||
BEFORE INSERT ON app.job_application
|
||
FOR EACH ROW EXECUTE FUNCTION app.tg_check_cooling_off();
|
||
```
|
||
|
||
A new attempt is permitted only when the previous attempt is terminal and
|
||
`now() >= previous.terminal_at + cooling_off`, where `cooling_off` comes from versioned config
|
||
(default 90 days; 0 for `withdrawn_by_candidate`; shorter when the rejection reason is
|
||
`role_filled`, from `ref.rejection_reason.cooling_off_days`). The override columns are the audited
|
||
escape hatch.
|
||
|
||
**Why a plain row trigger and not a constraint trigger.** Two reasons, one syntactic and one
|
||
semantic. Syntactically, `CREATE CONSTRAINT TRIGGER … BEFORE INSERT` is **not valid PostgreSQL** —
|
||
constraint triggers may only be `AFTER ROW`, and the `BEFORE` form fails with
|
||
`ERROR: syntax error at or near "BEFORE"` (§4.8). Semantically, the rule needs no deferral at all:
|
||
it reads only *already-committed prior attempts*, so nothing later in the transaction can change the
|
||
answer, and failing immediately gives the recruiter the error at the point of the offending insert
|
||
rather than at COMMIT. If a future requirement ever demands end-of-transaction evaluation — for
|
||
instance a bulk import that legitimately inserts two attempts in one transaction — the correct form
|
||
is `CREATE CONSTRAINT TRIGGER tg_application_cooling_off AFTER INSERT ON app.job_application
|
||
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW …`. **Audited for the same mistake:** §9.1
|
||
`ck_job_current_version`, §12.5 layer 3 contactability, §13.1 `ck_candidate_merged_status` and both
|
||
weight-sum triggers are genuine COMMIT-time checks and are all correctly `AFTER`; this was the only
|
||
`BEFORE` constraint trigger in the document.
|
||
|
||
**The stage must belong to the application's own pinned pipeline.** `pipeline_config_version_id` is
|
||
`NOT NULL` and pinned at creation, and `current_stage_id` is a global FK to `ref.pipeline_stage`
|
||
with — on its own — nothing tying the two together. Without a constraint, an application can sit in
|
||
a stage its own pipeline does not contain, which happens for real after a department pipeline change,
|
||
a psql fix, a bulk import, or a merge that carried an application between jobs. The consequence is
|
||
the worst kind: §17's `stages_for(job_version)` renders columns that do not include that stage, so
|
||
**the application silently vanishes from the kanban while still counting as `state = 'active'` in
|
||
every funnel number**. Two deferred constraint triggers close it:
|
||
|
||
```sql
|
||
CREATE CONSTRAINT TRIGGER tg_application_stage_in_pipeline
|
||
AFTER INSERT OR UPDATE OF current_stage_id, pipeline_config_version_id
|
||
ON app.job_application
|
||
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW
|
||
EXECUTE FUNCTION app.tg_assert_stage_in_pipeline();
|
||
-- asserts: EXISTS (SELECT 1 FROM app.pipeline_config_stage pcs
|
||
-- WHERE pcs.pipeline_config_version_id = NEW.pipeline_config_version_id
|
||
-- AND pcs.pipeline_stage_id = NEW.current_stage_id)
|
||
|
||
CREATE CONSTRAINT TRIGGER tg_stage_history_in_pipeline
|
||
AFTER INSERT ON app.job_application_stage_history
|
||
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW
|
||
EXECUTE FUNCTION app.tg_assert_history_stages_in_pipeline();
|
||
-- asserts the same for BOTH stage_id and from_stage_id, against the parent
|
||
-- application's pinned pipeline_config_version_id
|
||
```
|
||
|
||
Deferred rather than immediate because a stage move and a pipeline re-pin can legitimately happen in
|
||
one transaction (the merge routine does exactly that). **Two safety nets accompany it**, because the
|
||
trigger cannot retroactively fix rows written before it existed: a nightly reconciliation query
|
||
listing applications whose `current_stage_id` is not in their pinned config, surfaced on the same
|
||
operational dashboard as the `actor_unknown` counts (§9.5); and "application in a stage outside its
|
||
pinned pipeline" as a named case in the constraint test suite (§2, P9).
|
||
|
||
Four details in the reapplication block are load-bearing and are worth stating as decisions:
|
||
|
||
- **Uniqueness is per `job`, not per `job_version` or `job_posting`.** The classic recruiter
|
||
complaint is the same person arriving twice for one requisition through LinkedIn and the careers
|
||
page — both real sources in the prototype's inbox (`js/data.js:284-300`). Per-posting uniqueness
|
||
would permit exactly that; per-version uniqueness would let a requirement edit silently allow a
|
||
second live application.
|
||
- **`state` is generated from `status_is_terminal`, never maintained in parallel and never from a
|
||
key list in DDL.** Generating it removes the bug class where a status transition forgets to update
|
||
the flag the partial index depends on. Reading the *denormalised `ref` boolean* rather than
|
||
`status_key IN ('hired','rejected','withdrawn','expired')` removes a second problem: with the key
|
||
list in DDL, adding "On Hold" as a terminal status — precisely what §4.6 Tier 1 exists to allow
|
||
without a deployment — would require `ALTER TABLE … ALTER COLUMN state` and a full rewrite of
|
||
~200,000 rows. It also fixes a concrete defect: `superseded_by_merge` (§19.4 mechanic 3) is a
|
||
terminal status that a hardcoded four-key list would omit, leaving merge-superseded applications
|
||
`state = 'active'` — still on the kanban, still in "my candidates", still in the SLA sweep and
|
||
still in funnel counts, with `ck_application_terminal` forcing `terminal_at` to stay NULL.
|
||
`superseded_by_merge` is seeded with `is_terminal = true, is_negative = false` (it is not a
|
||
rejection — the candidate is still live under the surviving identity), and merge mechanic 3 sets
|
||
`terminal_at = merge.performed_at` in the same statement so `ck_application_terminal` holds.
|
||
- **`superseded_by_application_id` is excluded from the index predicate on purpose.** It is the
|
||
exact escape hatch merge needs when both merged identities applied to the same job; without it,
|
||
merge would be blocked by this constraint.
|
||
- **The cooling-off override exists deliberately.** A hard block would be circumvented by recruiters
|
||
creating a duplicate candidate record to get around it, which is strictly worse than an audited
|
||
override. The constraint should shape behaviour, not invite evasion.
|
||
|
||
**Indexes.**
|
||
|
||
| Index | Purpose |
|
||
|---|---|
|
||
| `uq_application_live` (above) | The invariant, and the "does this person already have a live application" lookup |
|
||
| `ix_application_pipeline (job_id, current_stage_id) WHERE state = 'active' AND deleted_at IS NULL` | The Pipeline kanban, which loads one job's active applications grouped by stage |
|
||
| `ix_application_recruiter (current_primary_recruiter_id, stage_entered_at) WHERE state = 'active' AND deleted_at IS NULL` | "My candidates", the most-loaded list screen |
|
||
| `ix_application_candidate (candidate_id, applied_at DESC)` | The candidate profile's application history |
|
||
| `ix_application_sla (sla_due_at) WHERE state = 'active' AND sla_due_at IS NOT NULL` | The SLA breach sweep |
|
||
| `ix_application_status_time (status_id, applied_at)` | Funnel and time-to-hire reporting |
|
||
| `ix_application_shortlist (job_id, current_overall_score DESC) WHERE state = 'active' AND deleted_at IS NULL` | **The ranked shortlist for a requisition** — §29.6 Q7, the primary shortlisting surface. One index scan, correctly ordered, no join |
|
||
| `ix_application_band (job_id, current_band) WHERE state = 'active' AND deleted_at IS NULL` | The score-band facet on the candidate/application list |
|
||
|
||
**Why the score *value* is denormalised and not just the pointer.** An earlier draft justified
|
||
`current_ats_result_id` as "denormalised so the list screen sorts by score without a join" and
|
||
indexed `(job_id, current_ats_result_id)`. That does not work, and the reason is worth stating so
|
||
nobody reintroduces it: the column stores an **id**, so ordering by it orders by score-row
|
||
*insertion sequence*, not by score. A rescore inserts a new `ats_result` with a higher id and a
|
||
*lower* score, and the "ranked" list ranks it first. `ix_application_score` therefore served nothing
|
||
— which the old index note half-admitted, conceding "the score value itself is read from
|
||
`ats_result`", i.e. the join it was supposed to avoid was still happening. Denormalising
|
||
`current_overall_score` and `current_band` fixes it at no extra maintenance cost, because the trigger
|
||
that flips `ats_result.is_current` already runs at exactly the right moment and already writes
|
||
`current_ats_result_id`; it now writes three columns instead of one. The pointer is retained — it is
|
||
what the explanation drill-down and the provenance query need.
|
||
|
||
**Profile.** *Status:* Tier 1 (`status_id`) **and** stage (`current_stage_id`), with two separate
|
||
history tables. Status and stage are genuinely different dimensions — an application can be
|
||
`active` in stage `Interview`, or `rejected` from stage `Interview` — and the prototype conflates
|
||
them (`status: stage` at `js/data.js:122`), which is why its funnel numbers cannot distinguish
|
||
"currently interviewing" from "rejected after interview". *Required:* candidate, job, version,
|
||
intake, channel, pipeline version, stage, status, `applied_at`. *Soft delete:* yes. *Retention:*
|
||
with the candidate; the skeleton row survives pseudonymisation so funnel metrics stay intact.
|
||
*PII:* `internal` in itself — its PII is by reference. *Audit:* yes. *Volume:* ~200,000.
|
||
*Queries:* the pipeline board; "my candidates"; the candidate's application history; funnel
|
||
conversion by stage; time-to-hire; source performance; SLA breaches; reapplication check.
|
||
|
||
### 15.2 Application history and source attribution
|
||
|
||
| Table | Purpose | Key columns | Constraints | Volume |
|
||
|---|---|---|---|---|
|
||
| `app.job_application_stage_history` | Every stage transition as a **transition row**, not a diff | `job_application_id`, `stage_id`, `from_stage_id` NULL (NULL = entry), `valid_from`, `valid_to`, `actor_user_id`, `actor_kind` CHECK, `actor_unknown`, `change_reason`, `request_id`, `ai_suggestion_id` NULL | `ex` overlap on `(job_application_id, tstzrange(valid_from, valid_to))`; append-only; trigger-written | ~900,000 |
|
||
| `app.job_application_status_history` | Status over time, same shape | `job_application_id`, `status_id`, `valid_from`, `valid_to`, actor columns, `rejection_reason_id` NULL | `ex` overlap; append-only; trigger-written | ~500,000 |
|
||
|
||
**Why transition rows and not generic row shadowing.** `django-simple-history`-style shadow tables
|
||
mirror columns instead of modelling transitions, so "who moved this candidate from Screening to
|
||
Interview and why" becomes a diff-inference problem instead of a `SELECT`. Storing
|
||
`valid_from`/`valid_to` rather than only `changed_at` is a deliberate denormalisation: time-in-stage
|
||
is the most-queried recruiting metric and becomes a subtraction rather than a window function over
|
||
the whole history.
|
||
|
||
**`ai_suggestion_id` on the stage history is the "AI never auto-rejects" audit trail.** When a
|
||
recruiter accepts an AI shortlisting suggestion, the transition row records which suggestion it
|
||
came from — and `actor_kind` is still `user`, because a human performed it. The service-layer guard
|
||
refuses any transition into a terminal-negative stage where `actor_kind <> 'user'`, and
|
||
`ats_result.review_outcome` carries the same rule as a CHECK (§20.4).
|
||
|
||
**Source attribution is not a table.** First touch is `raw_intake` (which channel delivered the
|
||
application), last touch is `job_posting_id` (which advert they read), and the reporting dimension
|
||
is `source_channel_id`. A multi-touch attribution table is rejected for Phase 1: it needs
|
||
candidate-side tracking (UTM capture, cookie identity) that this platform does not have and BRD
|
||
§8.1 does not ask for. **Recorded in §30.**
|
||
|
||
---
|
||
|
||
## 16. Recruiter assignments
|
||
|
||
Two concrete tables with the same shape, not one polymorphic table.
|
||
|
||
| Table | Purpose | Key columns |
|
||
|---|---|---|
|
||
| `app.job_assignment` | Who owns this requisition, in what role, over what period | `job_id`, `user_id`, `role_id` FK `ref.assignment_role`, `valid_from`, `valid_to` NULL, `assigned_by_user_id`, `reason`, `allocation_pct` NULL |
|
||
| `app.job_application_assignment` | Per-application ownership, for the cases where it differs from the requisition's | `job_application_id`, `user_id`, `role_id`, `valid_from`, `valid_to` NULL, `assigned_by_user_id`, `reason` |
|
||
|
||
**Constraints (both).**
|
||
|
||
```sql
|
||
CONSTRAINT ck_assignment_interval CHECK (valid_to IS NULL OR valid_to > valid_from),
|
||
CONSTRAINT ex_job_assignment_overlap EXCLUDE USING gist (
|
||
job_id WITH =, user_id WITH =, role_id WITH =, tstzrange(valid_from, valid_to) WITH &&
|
||
);
|
||
|
||
-- exactly one primary recruiter per requisition AT EVERY INSTANT, not just one open-ended row
|
||
ALTER TABLE app.job_assignment
|
||
ADD CONSTRAINT ex_job_primary_recruiter EXCLUDE USING gist (
|
||
job_id WITH =, tstzrange(valid_from, valid_to) WITH &&
|
||
) WHERE (role_id = 1); -- primary_recruiter, id pinned by the seed migration
|
||
```
|
||
|
||
**Why an `EXCLUDE` and not `CREATE UNIQUE INDEX … WHERE valid_to IS NULL`.** A partial unique index
|
||
on the open-ended row enforces "at most one *open-ended* primary recruiter", which is a different and
|
||
weaker rule than "one primary recruiter at a time". It permits a scheduled handover to be entered as
|
||
two bounded rows that overlap, and it permits a future-dated primary assignment to coexist with the
|
||
current one with nothing objecting. The `EXCLUDE … WHERE (role_id = 1)` form enforces the rule that
|
||
was actually intended, at every instant, and it composes correctly with the time predicate below —
|
||
so a handover entered as `[Mon, Fri)` and `[Fri, ∞)` is accepted while `[Mon, Fri)` and
|
||
`[Wed, ∞)` is refused. The narrower partial unique index would have been acceptable *only* alongside
|
||
a CHECK forbidding future-dated primary assignments, which is a worse product.
|
||
|
||
The `role_id = 1` literal is the same honest, documented compromise §16 has always made — **a
|
||
subquery in an index or `EXCLUDE` predicate is not permitted**, so the id is pinned by the seed
|
||
migration and asserted by a CI test rather than hidden. (Contrast `candidate_link.link_type_key`
|
||
in §13.2, where the key is denormalised instead; that is the right call there because
|
||
`ref.vocabulary_value` holds twelve vocabularies whose ids are not conceptually pinned, whereas
|
||
`ref.assignment_role` is a six-row table whose `primary_recruiter` row will never move.) The
|
||
alternative here, a denormalised `is_exclusive_primary boolean` maintained by trigger, was rejected
|
||
as one more thing to keep in sync.
|
||
|
||
**Indexes — "current" is the §7.4 time predicate, here too.**
|
||
|
||
| Index | Purpose |
|
||
|---|---|
|
||
| `ix_job_assignment_user_period (user_id, valid_from, valid_to)` | The workload query, and the one branch-2 read that must see bounded grants. A scheduled handover — Ahmed covers requisition 412 Monday to Friday — has `valid_to IS NOT NULL` from creation, and a `valid_to IS NULL` predicate would make him invisible to `v_user_effective_scope` for the whole cover period |
|
||
| `ix_job_assignment_job_period (job_id, valid_from, valid_to)` | "Who owns this requisition now / on date D" |
|
||
| `ix_job_assignment_open (user_id) WHERE valid_to IS NULL` | Fast path only, for the open-ended majority. Never the sole source of "current" |
|
||
|
||
**Denormalised pointers maintained by trigger:** `job.current_primary_recruiter_id` and
|
||
`job_application.current_primary_recruiter_id`. Every list screen in the 23-route IA filters or
|
||
displays by recruiter (`js/app.js:7-16`), and a temporal join on every row of every list is the
|
||
wrong default.
|
||
|
||
**Profile (both).** *Status:* interval form, no status column. *Required:* all but `reason`,
|
||
`allocation_pct` and `valid_to`. *Soft delete:* no — unassignment is `valid_to`, and the row is the
|
||
history. *Retention:* permanent. *PII:* `internal`. *Audit:* yes. *Volume:* ~4,000 /
|
||
~30,000. *Queries:* recruiter workload and SLA dashboards (BRD §9.1 `Recruiter: workload,
|
||
efficiency, SLA`); "who owned this in March"; reassignment on offboarding; the access-scope view
|
||
(§7.5) branch 2.
|
||
|
||
**Why two tables rather than one with a `subject_type`.** A polymorphic FK cannot be enforced by the
|
||
database at all, and unenforceable references to jobs and applications are exactly what this model
|
||
exists to eliminate. The cost is one duplicated table shape — cheap and obvious. The alternative
|
||
cost is a `subject_id` that can point at a deleted row with nothing to stop it.
|
||
|
||
**Why an interval table rather than a scalar recruiter column.** The prototype has
|
||
`recruiter`/`recruiterId` as a single scalar on both job and candidate (`js/data.js:96`,
|
||
`js/data.js:123`), which cannot answer "who owned this in March" and has no primary/supporting
|
||
distinction. The `EXCLUDE` constraint preserves the one good property the scalar accidentally had —
|
||
a single unambiguous owner — while adding the flexibility and history it lacked. Without it,
|
||
"flexible assignment" degrades into two people each believing they own the requisition.
|
||
|
||
---
|
||
|
||
## 17. Pipeline configuration
|
||
|
||
### ERD 4 — Pipeline, interviews and assessments
|
||
|
||
```mermaid
|
||
erDiagram
|
||
pipeline_config ||--|{ pipeline_config_version : "versions"
|
||
pipeline_config_version ||--o{ pipeline_config_stage : "ordered stages"
|
||
pipeline_config_version ||--o{ pipeline_transition_rule : "allowed moves"
|
||
pipeline_stage ||--o{ pipeline_config_stage : "used as"
|
||
pipeline_config_version ||--o{ job_application : "pinned by"
|
||
job_application ||--o{ interview : "evaluated in"
|
||
interview ||--o{ interview_participant : "panel"
|
||
interview ||--o{ interview_slot : "reschedules"
|
||
interview ||--o{ interview_status_history : "status over time"
|
||
interview ||--o{ scorecard : "scored by"
|
||
scorecard_template ||--|{ scorecard_template_version : "versions"
|
||
scorecard_template_version||--o{ scorecard_template_criterion : "criteria"
|
||
scorecard_template_version||--o{ scorecard : "pinned by"
|
||
scorecard ||--o{ scorecard_criterion_score : "ratings"
|
||
scorecard_template_criterion ||--o{ scorecard_criterion_score : "criterion"
|
||
job_application ||--o{ assessment_assignment : "assessed by"
|
||
assessment_template ||--|{ assessment_template_version: "versions"
|
||
assessment_template_version ||--o{ assessment_assignment : "pinned by"
|
||
assessment_assignment ||--o| assessment_result : "result"
|
||
app_user ||--o{ interview_participant : "attends"
|
||
app_user ||--o{ user_availability_rule : "available"
|
||
|
||
pipeline_config_version {
|
||
bigint id PK
|
||
bigint pipeline_config_id FK
|
||
int version_no
|
||
timestamptz published_at
|
||
}
|
||
pipeline_config_stage {
|
||
bigint id PK
|
||
bigint pipeline_config_version_id FK
|
||
bigint pipeline_stage_id FK
|
||
int order_index
|
||
int sla_days
|
||
boolean requires_scorecard
|
||
}
|
||
pipeline_transition_rule {
|
||
bigint id PK
|
||
bigint pipeline_config_version_id FK
|
||
bigint from_stage_id FK
|
||
bigint to_stage_id FK
|
||
text required_permission_key
|
||
boolean requires_reason
|
||
boolean is_terminal_negative
|
||
}
|
||
interview {
|
||
bigint id PK
|
||
uuid public_id
|
||
bigint job_application_id FK
|
||
timestamptz starts_at
|
||
timestamptz ends_at
|
||
text scheduling_timezone
|
||
timestamp local_start_wall
|
||
tstzrange slot
|
||
bigint status_id FK
|
||
}
|
||
interview_participant {
|
||
bigint id PK
|
||
bigint interview_id FK
|
||
bigint user_id FK
|
||
bigint candidate_id FK
|
||
text response_status
|
||
tstzrange slot
|
||
}
|
||
scorecard {
|
||
bigint id PK
|
||
bigint interview_id FK
|
||
bigint interviewer_user_id FK
|
||
bigint template_version_id FK
|
||
text recommendation
|
||
timestamptz submitted_at
|
||
timestamptz locked_at
|
||
}
|
||
assessment_assignment {
|
||
bigint id PK
|
||
uuid public_id
|
||
bigint job_application_id FK
|
||
bigint template_version_id FK
|
||
bigint status_id FK
|
||
timestamptz due_at
|
||
}
|
||
```
|
||
|
||
`pipeline` owns the **rules**; `job_application` owns the **state**. They are deliberately separate
|
||
even though both concern stages: stage configuration is versioned reference data with a slow change
|
||
cadence and a `configure` permission, while stage state changes constantly under an `edit`
|
||
permission. Merging them would put a configuration screen behind a recruiter permission.
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.pipeline_config` | Identity of a named pipeline. Global default, or scoped to a department | `key` UNIQUE, `name`, `department_id` NULL FK, `is_default`, `current_version_id` FK, audit, soft delete | `uq_pipeline_default UNIQUE ((true)) WHERE is_default AND deleted_at IS NULL` — exactly one global default; deferred check that `current_version_id` is set at COMMIT | *Soft delete:* yes. *Retention:* permanent. *PII:* `internal`. *Audit:* yes. *Volume:* ~15 |
|
||
| `app.pipeline_config_version` | Immutable published configuration | `pipeline_config_id`, `version_no`, `published_at`, `published_by_user_id`, `change_reason`, `config_hash bytea` | `uq (pipeline_config_id, version_no)`; INSERT/SELECT-only grants + immutability trigger | *Soft delete:* no. *Volume:* ~60 |
|
||
| `app.pipeline_config_stage` | Which stages, in what order, with SLA and gates | `pipeline_config_version_id`, `pipeline_stage_id` FK `ref.pipeline_stage`, `order_index`, `sla_days` NULL, `is_optional`, `requires_scorecard`, `requires_assessment`, `requires_approval`, `auto_advance_on_all_scorecards` | `uq (pipeline_config_version_id, pipeline_stage_id)`; `uq (pipeline_config_version_id, order_index)`; immutable | *Volume:* ~450 |
|
||
| `app.pipeline_transition_rule` | The allowed moves. **An absent row means the transition is forbidden** | `pipeline_config_version_id`, `from_stage_id` NULL (NULL = entry point), `to_stage_id` NOT NULL, `required_permission_key` NULL, `requires_reason`, `requires_rejection_reason`, `requires_approval`, `is_terminal_negative`, `allowed_actor_kinds text[] NOT NULL DEFAULT ARRAY['user']` with `ck_allowed_actor_kinds CHECK (cardinality(allowed_actor_kinds) >= 1 AND allowed_actor_kinds <@ ARRAY['user','system','integration','ai_agent'])` | `uq (pipeline_config_version_id, from_stage_id, to_stage_id)` — with `from_stage_id` nullable this needs two partial unique indexes, one `WHERE from_stage_id IS NULL`; `ck_no_self_transition CHECK (from_stage_id IS DISTINCT FROM to_stage_id)`; immutable | *Volume:* ~900 |
|
||
|
||
**Allow-list, not deny-list.** `is_allowed(from, to)` is a row lookup; a missing row is a refusal.
|
||
The alternative — all transitions legal unless a rule forbids them — means every new stage silently
|
||
opens paths nobody designed. The prototype has no transition concept at all: stages are strings on
|
||
the candidate (`js/data.js:27`, `js/data.js:122`) and any value can be assigned.
|
||
|
||
**`allowed_actor_kinds` is where "AI must never auto-reject" is enforced at the rules layer.** It is
|
||
a `text[]` whose members are drawn from the one repository-wide actor vocabulary —
|
||
`('user','system','integration','ai_agent')`, per `_decisions.md` RULING-01 — so the rules layer and
|
||
the guard layer compare against literally the same strings. It deliberately has **no** private
|
||
vocabulary of its own: an earlier draft used a scalar `allowed_actor_kind CHECK in
|
||
('user','user_or_system')`, which invented a fifth value (`user_or_system`) that no `actor_kind`
|
||
column can ever hold, meaning a rule row and a runtime actor could never be compared without a
|
||
translation step nobody would remember to write. An array of real enum members removes the
|
||
translation entirely: the check is `SET LOCAL app.actor_kind = ANY(rule.allowed_actor_kinds)`.
|
||
|
||
No rule in the seeded configuration includes anything other than `'user'` in
|
||
`allowed_actor_kinds` on a transition whose `is_terminal_negative` is true, and a CHECK enforces that
|
||
combination is impossible:
|
||
`ck_terminal_negative_user_only CHECK (NOT (is_terminal_negative AND allowed_actor_kinds <> ARRAY['user']))`.
|
||
The service guard (`actor_kind = 'user'`, the exact literal), this constraint, and `ats_result`'s
|
||
`review_outcome` CHECK are three independent expressions of the same
|
||
rule — deliberately redundant, because it is the requirement most likely to be eroded by a
|
||
convenience feature.
|
||
|
||
**Pipeline stages are a `ref` table, not an enum,** because the prototype hardcodes seven stage
|
||
strings (`js/data.js:27`) and the business will add and reorder them. An enum makes reordering and
|
||
per-department variation painful and cannot carry `order_index` or `is_terminal`.
|
||
|
||
*Queries.* `stages_for(job_version)` to render the kanban columns; `is_allowed(from, to)` on every
|
||
drag-and-drop; the stage-gate check before a transition commits; funnel definitions for analytics.
|
||
|
||
---
|
||
|
||
## 18. CV parsing runs
|
||
|
||
### ERD 5 — ATS scoring and document processing
|
||
|
||
```mermaid
|
||
erDiagram
|
||
raw_intake_attachment |o--o{ intake_parse_attempt : "parsed from"
|
||
candidate_document |o--o{ intake_parse_attempt : "attempts for"
|
||
intake_parse_attempt ||--o{ parse_issue : "issues"
|
||
stored_file ||--o{ candidate_document : "bytes"
|
||
stored_file ||--o{ raw_intake_attachment : "bytes"
|
||
scoring_config ||--|{ scoring_config_version : "versions"
|
||
scoring_config_version ||--|{ scoring_config_criterion : "weighted criteria"
|
||
scoring_config_version ||--o{ ats_result : "pinned by"
|
||
job_version ||--o{ ats_result : "pinned by"
|
||
candidate_document |o--o{ ats_result : "pinned by"
|
||
intake_parse_attempt |o--o{ ats_result : "pinned by"
|
||
job_application ||--o{ ats_result : "scored"
|
||
ats_result ||--o{ ats_result_criterion : "contributions"
|
||
ats_result ||--o{ ats_result_skill : "matched or missing"
|
||
job_requirement |o--o{ ats_result_criterion : "against"
|
||
ats_result ||--o| ats_result : "superseded_by"
|
||
candidate ||--o{ candidate_job_match : "pool match"
|
||
job_version ||--o{ candidate_job_match : "against"
|
||
relevance_config_version |o--o{ candidate_job_match : "ranked with"
|
||
scoring_config_version ||--o{ evaluation_run : "gated by"
|
||
evaluation_run ||--o{ evaluation_metric : "metrics"
|
||
|
||
intake_parse_attempt {
|
||
bigint id PK
|
||
bigint raw_intake_id FK
|
||
bigint raw_intake_attachment_id FK
|
||
bigint candidate_document_id FK
|
||
text parser_name
|
||
text parser_version
|
||
text ai_model_version
|
||
text status
|
||
jsonb parsed
|
||
numeric confidence
|
||
}
|
||
parse_issue {
|
||
bigint id PK
|
||
bigint parse_attempt_id FK
|
||
text severity
|
||
text field_path
|
||
text code
|
||
}
|
||
ats_result {
|
||
bigint id PK
|
||
uuid public_id
|
||
bigint job_application_id FK
|
||
bigint job_id FK
|
||
bigint job_version_id FK
|
||
bigint scoring_config_version_id FK
|
||
bigint candidate_document_id FK
|
||
bigint parse_attempt_id FK
|
||
text algorithm_code_version
|
||
numeric overall_score
|
||
text band
|
||
bytea input_fingerprint
|
||
boolean is_current
|
||
bigint superseded_by_id FK
|
||
text review_outcome
|
||
bigint reviewed_by_user_id FK
|
||
}
|
||
ats_result_criterion {
|
||
bigint id PK
|
||
bigint ats_result_id FK
|
||
text criterion_key
|
||
bigint job_requirement_id FK
|
||
numeric normalised_score
|
||
numeric weight_applied
|
||
numeric contribution
|
||
}
|
||
candidate_job_match {
|
||
bigint id PK
|
||
bigint candidate_id FK
|
||
bigint job_version_id FK
|
||
bigint relevance_config_version_id FK
|
||
numeric score
|
||
boolean is_current
|
||
bigint superseded_by_id FK
|
||
}
|
||
```
|
||
|
||
### 18.1 `app.intake_parse_attempt`
|
||
|
||
*Purpose.* One append-only row per attempt to turn a document into structured fields. Many attempts
|
||
per document: a retry after a transient failure, a re-parse with a better parser, a re-parse with a
|
||
different model.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `raw_intake_id` | `bigint` | no | FK — every parse traces to an arrival |
|
||
| `raw_intake_attachment_id` | `bigint` | yes | FK — NULL when parsing the email body rather than an attachment |
|
||
| `candidate_document_id` | `bigint` | yes | FK — set once the document row exists, which is what lets `ats_result` pin both. **Write-once:** settable while NULL, never re-pointable |
|
||
| `attempt_no` | `int` | no | Per `raw_intake_id` |
|
||
| `parser_name` | `text` | no | e.g. `pdfplumber`, `pymupdf`, `python-docx`, `ocrmypdf`, `unstructured` |
|
||
| `parser_version` | `text` | no | |
|
||
| `ocr_used` | `boolean` | no | Default false |
|
||
| `ai_model_id` | `bigint` | yes | FK `ai.ai_model_config_version` when a model did field extraction |
|
||
| `ai_run_id` | `bigint` | yes | FK `ai.ai_model_invocation` |
|
||
| `status` | `text` | no | CHECK in (`running`, `succeeded`, `partial`, `failed`, `unsupported`) |
|
||
| `parsed` | `jsonb` | yes | The structured extraction with per-field confidence. Legitimate JSONB |
|
||
| `error` | `jsonb` | yes | Structured error, not a string |
|
||
| `confidence` | `numeric(4,3)` | yes | CHECK 0–1. Overall confidence |
|
||
| `field_count` | `int` | yes | |
|
||
| `text_char_count` | `int` | yes | |
|
||
| `injection_signal` | `boolean` | no | Default false. **Prompt-injection detected in this document's extracted text** — the queryable fact `05` §6.5 layer 6 depends on |
|
||
| `injection_signal_codes` | `text[]` | no | Default `'{}'`. Which detector patterns fired, as stable machine codes. Never candidate text — see below |
|
||
| `started_at`, `finished_at` | `timestamptz` | no / yes | |
|
||
| `duration_ms` | `int` | yes | |
|
||
| `created_at` | | no | Append-only |
|
||
|
||
**Constraints.** `uq_parse_attempt UNIQUE (raw_intake_id, attempt_no)`.
|
||
`ck_parse_terminal CHECK ((status IN ('succeeded','partial','failed','unsupported')) = (finished_at
|
||
IS NOT NULL))`. `ck_parse_output CHECK (status NOT IN ('succeeded','partial') OR parsed IS NOT
|
||
NULL)`. `ck_parse_error CHECK (status <> 'failed' OR error IS NOT NULL)` — a failure with no
|
||
recorded reason is the exact thing BRD §6.3 forbids ("parse failures are visible rather than
|
||
silent"). `ck_parse_injection CHECK (injection_signal = (cardinality(injection_signal_codes) > 0))` —
|
||
a biconditional in the §20.4 style, because both halves of the broken state are real: a signal with no
|
||
codes cannot be triaged, and codes with no signal is a detector that ran, found something, and failed
|
||
to raise it.
|
||
|
||
**`injection_signal` is a persisted column and not a log line, and that is the whole point.** `05`
|
||
§6.5 layer 6 makes prompt-injection detection one of eight defences against adversarial text inside a
|
||
CV — the highest-likelihood AI threat in the system (T-4), where the candidate has direct motive,
|
||
direct control of the input, and a public technique. A positive signal has three consequences, all of
|
||
which need the fact to be *on a row*: it routes the intake to `needs_review` (`raw_intake.state`), it
|
||
shows a visible banner on the candidate profile and the score panel, and it alerts `hr_admin`. None of
|
||
those is servable from application logs, and the second is servable months later only if the fact was
|
||
stored at parse time. It lives here rather than on `candidate_document` because detection is a
|
||
property of **one parse of one document**: a re-parse with a better extractor legitimately finds
|
||
invisible-layer text the first parser never saw, and that difference is itself the signal.
|
||
|
||
**Codes, not spans — and specifically not in `parse_issue.message`.** `injection_signal_codes` holds
|
||
stable machine codes (`injection_instruction_override`, `injection_role_play`,
|
||
`injection_delimiter_escape`, `injection_encoded_block`, `injection_homoglyph`,
|
||
`injection_zero_width`, `injection_bidi_control`, `injection_invisible_pdf_text`,
|
||
`injection_render_mismatch` — the nine detector classes `05` §6.5 layer 6 enumerates). It stores no
|
||
offsets and no quoted text. Two reasons, and the second is a constraint this document already
|
||
imposes: the offending text is already on disk in `candidate_document.extracted_text`, so an offset
|
||
list would be a second copy of hostile candidate content with its own erasure obligation; and
|
||
`parse_issue` — the obvious place for per-pattern detail — is classified `internal` precisely because
|
||
its `message` "must not quote candidate content" (§18.2), so an injection payload is the one thing it
|
||
must never carry. The reviewer reads the document; the schema records that there is a reason to.
|
||
**The document is never rejected and the text is never silently stripped** (`05` §6.5): a candidate
|
||
must not be penalised by an automated system for something no human has looked at, and quiet
|
||
stripping destroys the evidence. `text[]` rather than JSONB keeps this off §13.3's closed list —
|
||
there is nothing nested here, and `cardinality()` in the CHECK above is immutable where a JSONB
|
||
predicate would be clumsier for no gain.
|
||
|
||
**Grants: insert, one closing update, then frozen (§4.2).** This row is opened with
|
||
`status = 'running'` *before* the parse runs — `04` §4 lines 821 and 848 show exactly that sequence —
|
||
so pure INSERT/SELECT grants would make it impossible for the parse pipeline to record any result at
|
||
all, and `ck_parse_terminal` plus `ck_parse_output` make the terminal write **mandatory**. The
|
||
resolution is the same narrow, column-scoped grant §12.2 uses for `raw_intake.state` and §20.4 uses
|
||
for `ats_result.is_current`:
|
||
|
||
```sql
|
||
REVOKE UPDATE, DELETE ON app.intake_parse_attempt FROM talentflow_app;
|
||
GRANT SELECT, INSERT ON app.intake_parse_attempt TO talentflow_app;
|
||
GRANT UPDATE (status, parsed, error, confidence, field_count, text_char_count,
|
||
injection_signal, injection_signal_codes,
|
||
finished_at, duration_ms, candidate_document_id)
|
||
ON app.intake_parse_attempt TO talentflow_app;
|
||
|
||
CREATE TRIGGER tg_parse_attempt_immutable
|
||
BEFORE UPDATE OR DELETE ON app.intake_parse_attempt
|
||
FOR EACH ROW EXECUTE FUNCTION app.tg_parse_attempt_freeze();
|
||
```
|
||
|
||
`app.tg_parse_attempt_freeze()` enforces three rules, which together are what "append-only" means for
|
||
this table:
|
||
|
||
| Rule | Raises when |
|
||
|---|---|
|
||
| A closed attempt is immutable | `TG_OP = 'UPDATE' AND OLD.finished_at IS NOT NULL` and any column other than `candidate_document_id` differs — a parse result may be written **once** |
|
||
| `candidate_document_id` is write-once | `OLD.candidate_document_id IS NOT NULL AND NEW.candidate_document_id IS DISTINCT FROM OLD.candidate_document_id` — the document link is set at resolution time, after the parse, and can never be re-pointed |
|
||
| Nothing is ever deleted | `TG_OP = 'DELETE'`, unconditionally |
|
||
|
||
**Why `candidate_document_id` is exempt from the closed-row freeze.** The ordering is real, not an
|
||
oversight: the parse finishes (writing `finished_at`), and only *then* does `intake_resolution` decide
|
||
whether a candidate and therefore a `candidate_document` exists. If the column were frozen at
|
||
`finished_at`, the link would never be written and `ats_result` could never pin both the document and
|
||
the attempt — the pin §20.4 depends on. **The alternative was considered and rejected:** drop the
|
||
column entirely and derive the link through `candidate_document.raw_intake_attachment_id`, which
|
||
removes the one mutable column from an otherwise insert-then-close table. It was rejected because the
|
||
derivation is not total — an attempt against the email *body* has
|
||
`raw_intake_attachment_id IS NULL` and no path to a document at all — so `ats_result` would lose the
|
||
pin in exactly the case (body-parsed applications) where provenance is hardest to reconstruct. Paying
|
||
for that with one write-once column, guarded by the rule above, is the cheaper side.
|
||
|
||
**Indexes.** `ix_parse_attempt_intake (raw_intake_id, attempt_no DESC)`;
|
||
`ix_parse_attempt_document (candidate_document_id) WHERE candidate_document_id IS NOT NULL`;
|
||
`ix_parse_attempt_failed (finished_at DESC) WHERE status IN ('failed','partial')` — the parse-failure
|
||
queue, which is a screen, not a log grep;
|
||
`ix_parse_attempt_injection (finished_at DESC) WHERE injection_signal` — the injection triage queue
|
||
and the `hr_admin` alert query, partial so it costs nothing on the overwhelming majority of rows;
|
||
**GIN on `parsed`** (one of only two JSONB GIN indexes in
|
||
the schema) because the field-level review UI queries into it.
|
||
|
||
**Profile.** *Status:* `status`, Tier 2. *Required:* intake, parser name and version, status,
|
||
`started_at`, `injection_signal`, `injection_signal_codes`. *Soft delete:* no. *Retention:* `parsed`
|
||
is pseudonymised with the candidate; the row and its metrics survive — **including
|
||
`injection_signal` and its codes**, which carry no candidate content and are the record that a
|
||
document was flagged. *PII:* `parsed` is `sensitive_personal`; `injection_signal` and
|
||
`injection_signal_codes` are `internal`. *Audit:* insert only, plus an explicit
|
||
`intake.injection_signal_raised` event when the signal is set (`05` §7.3). *Volume:*
|
||
~400,000 (**assumption**: 1.3 attempts per document), of which the injection-flagged subset is
|
||
expected in the low hundreds. *Queries:* the parse-failure queue; the injection triage queue and the
|
||
candidate-profile banner; injection-pattern frequency by detector code, which is how the detector's
|
||
false-positive rate is measured rather than assumed; parser
|
||
success rate by version, which is how a parser regression is detected; the attempt an `ats_result`
|
||
pinned; replaying a parser version over historical intake.
|
||
|
||
**Why attempts are separated from the intake row.** A failed parse must be retryable and
|
||
re-auditable without mutating the arrival record, and a parser version bump must be replayable over
|
||
historical intake. Storing the parse result *on* the intake row — the prototype's shape, where
|
||
`resumeStatus` is a single string on the inbox item (`js/data.js:300`) — loses the raw payload the
|
||
moment a parser writes over it and cannot represent two attempts.
|
||
|
||
### 18.2 `app.parse_issue`
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `parse_attempt_id` | `bigint` | no | FK |
|
||
| `severity` | `text` | no | CHECK in (`info`, `warning`, `error`) |
|
||
| `code` | `text` | no | Stable machine code, e.g. `missing_email`, `date_range_unparseable`, `ocr_low_confidence`, `encrypted_pdf` |
|
||
| `field_path` | `text` | yes | JSON pointer into `parsed` |
|
||
| `message` | `text` | no | Human-readable |
|
||
| `page_no` | `int` | yes | |
|
||
| `created_at` | | no | |
|
||
|
||
**Indexes.** `ix_parse_issue_attempt (parse_attempt_id)`; `ix_parse_issue_code (code, created_at)` —
|
||
the aggregate that tells you which parser weakness to fix next.
|
||
|
||
**Profile.** *Soft delete:* no. *Retention:* with the attempt. *PII:* `internal` (`message` must not
|
||
quote candidate content; a CI-checkable rule for the parser authors). *Audit:* no. *Volume:*
|
||
~600,000. *Queries:* the issue list on a document; top parse failure codes by parser version.
|
||
|
||
**Not a table: a parser registry.** `parser_name` and `parser_version` stay as text columns. A
|
||
`parser_component`/`parser_component_version` pair would be a foreign key to nowhere useful — no
|
||
query joins to it, nothing constrains it, and parser versions are produced by `pip freeze` rather
|
||
than administered. **Recorded in §30.**
|
||
|
||
---
|
||
|
||
## 19. Duplicate detection and merge
|
||
|
||
### 19.1 Matching configuration
|
||
|
||
| Table | Purpose | Key columns | Constraints |
|
||
|---|---|---|---|
|
||
| `app.matching_config` | Identity of the duplicate-matching configuration | `key` UNIQUE, `name`, `current_version_id`, audit | deferred current-version check |
|
||
| `app.matching_config_version` | Immutable thresholds and signal weights | `matching_config_id`, `version_no`, `auto_link_threshold numeric(5,4)`, `review_threshold numeric(5,4)`, `auto_create_max_score numeric(5,4)`, `signal_weights jsonb`, `detector_name`, `detector_version`, `published_at`, `published_by_user_id`, `config_hash` | `uq (matching_config_id, version_no)`; `ck (review_threshold <= auto_link_threshold)`; immutable |
|
||
|
||
Thresholds live in versioned config rather than in the schema because they will be tuned, and
|
||
storing `matching_config_version_id` on both `duplicate_candidate_pair` and `intake_resolution`
|
||
means a threshold change shows up as a change in *what was flagged* rather than as an unexplained
|
||
shift in queue volume.
|
||
|
||
### 19.2 `app.duplicate_candidate_pair`
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `candidate_a_id` | `bigint` | no | FK |
|
||
| `candidate_b_id` | `bigint` | no | FK |
|
||
| `match_score` | `numeric(5,4)` | no | CHECK 0–1 |
|
||
| `signals` | `jsonb` | no | **Per-signal values**, not just a total |
|
||
| `detector_name` | `text` | no | |
|
||
| `detector_version` | `text` | no | |
|
||
| `matching_config_version_id` | `bigint` | no | FK |
|
||
| `detected_at` | `timestamptz` | no | |
|
||
| `last_detected_at` | `timestamptz` | no | Bumped when a later run re-observes the pair |
|
||
| `state` | `text` | no | CHECK in (`open`, `confirmed_duplicate`, `confirmed_distinct`, `merged`) |
|
||
| `reviewed_by_user_id` | `bigint` | yes | |
|
||
| `reviewed_at` | `timestamptz` | yes | |
|
||
| `note` | `text` | yes | |
|
||
| `merge_id` | `bigint` | yes | FK `app.candidate_merge` |
|
||
|
||
**Constraints.**
|
||
`ck_pair_canonical CHECK (candidate_a_id < candidate_b_id)` and
|
||
`uq_pair UNIQUE (candidate_a_id, candidate_b_id)` — together, the thing that stops the same pair
|
||
being re-flagged in the opposite order on every detector run, which is the most common source of
|
||
reviewer fatigue in duplicate queues.
|
||
`ck_pair_reviewed CHECK ((state IN ('confirmed_duplicate','confirmed_distinct')) <=
|
||
(reviewed_by_user_id IS NOT NULL))`.
|
||
`ck_pair_merged CHECK ((state = 'merged') = (merge_id IS NOT NULL))`.
|
||
|
||
**Indexes.** `ix_pair_open (match_score DESC) WHERE state = 'open'` — the review queue, highest
|
||
score first; `ix_pair_a (candidate_a_id)`, `ix_pair_b (candidate_b_id)` for "is this candidate in
|
||
any open pair"; `ix_pair_distinct (candidate_a_id, candidate_b_id) WHERE state =
|
||
'confirmed_distinct'` — **the detector must consult this and skip these pairs.**
|
||
|
||
**The six signals, each stored with its own value:**
|
||
|
||
| Signal | Source | Index that serves it |
|
||
|---|---|---|
|
||
| Exact normalised email | `candidate_email.address_normalised` | the unique index itself |
|
||
| Exact E.164 phone | `candidate_phone.e164` | btree unique |
|
||
| Trigram similarity on name | `candidate.name_normalised` | GIN trigram |
|
||
| Trigram similarity on employer + title + employment date overlap | `candidate.current_employer_normalised`, `candidate_employment` | GIN trigram + `(candidate_id, start_date)` |
|
||
| Identical document hash | `candidate_document.sha256` | btree |
|
||
| Identical normalised LinkedIn URL | `candidate_link.url_normalised` | partial unique index |
|
||
|
||
**Profile.** *Status:* `state`, Tier 2. *Soft delete:* no. *Retention:* permanent — the
|
||
`confirmed_distinct` memory must outlive the candidates' activity, or two different people with the
|
||
same common name resurface in the queue forever. *PII:* `internal` (the linkage is; `signals` holds
|
||
similarity numbers, not values). *Audit:* yes. *Volume:* ~40,000. *Queries:* the duplicate review
|
||
queue ordered by score; the detector's skip list; per-signal precision analysis after a threshold
|
||
change.
|
||
|
||
**Rejected:** auto-merge above a similarity threshold (the failure mode is silent and
|
||
cross-contaminates two people's application history and compensation data); storing only a composite
|
||
score (the reviewer cannot see *why*, so review quality collapses); `fuzzystrmatch`/Levenshtein
|
||
alone (no index support at scale, unlike trigram GIN).
|
||
|
||
### 19.3 `app.candidate_merge`
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `public_id` | `uuid` | no | UNIQUE |
|
||
| `surviving_candidate_id` | `bigint` | no | FK |
|
||
| `merged_candidate_id` | `bigint` | no | FK |
|
||
| `duplicate_pair_id` | `bigint` | yes | FK — NULL for a merge initiated directly from a profile |
|
||
| `performed_by_user_id` | `bigint` | **no** | Human-only merge, expressed as a constraint |
|
||
| `performed_at` | `timestamptz` | no | |
|
||
| `reason` | `text` | **no** | |
|
||
| `operation_count` | `int` | no | Written at completion; a cheap integrity check against `candidate_merge_operation` |
|
||
| `reversed_at` | `timestamptz` | yes | |
|
||
| `reversed_by_user_id` | `bigint` | yes | |
|
||
| `reversal_reason` | `text` | yes | |
|
||
| `reversal_blocked_reason` | `text` | yes | CHECK in (`retention_purge`, `later_merge_conflict`, `manual_hold`) |
|
||
|
||
**Constraints.** `ck_merge_distinct CHECK (surviving_candidate_id <> merged_candidate_id)`.
|
||
`ck_merge_reversal CHECK ((reversed_at IS NULL) = (reversed_by_user_id IS NULL))`.
|
||
`ck_merge_reversal_reason CHECK (reversed_at IS NULL OR reversal_reason IS NOT NULL)`. A candidate
|
||
cannot be the loser of two live merges, expressed as a partial unique **index** per §4.8:
|
||
`CREATE UNIQUE INDEX uq_merge_loser ON app.candidate_merge (merged_candidate_id) WHERE reversed_at IS NULL;`
|
||
|
||
**Stack-discipline trigger.** A `BEFORE UPDATE` trigger on `candidate_merge`, firing when
|
||
`reversed_at` transitions from NULL, queries `candidate_merge_operation` for later unreversed
|
||
operations on the same `(target_table, target_row_pk)`. If any exist, reversal is **refused** with
|
||
an error naming the blocking merge.
|
||
|
||
**Profile.** *Status:* reversal columns, not a status. *Soft delete:* no. *Retention:* permanent.
|
||
*PII:* `internal`. *Audit:* yes, high value. *Volume:* ~6,000. *Queries:* the merge history on a
|
||
candidate; the unmerge confirmation screen; "which merges are blocked and why".
|
||
|
||
### 19.4 `app.candidate_merge_operation` and the merge mechanics
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `merge_id` | `bigint` | no | FK |
|
||
| `seq` | `int` | no | UNIQUE with `merge_id`. **Application order** — the order the merge performed the operations in |
|
||
| `reversal_rank` | `int` | no | **Reversal order within a target row.** Lower ranks are undone first. Assigned per `op_kind` from a fixed table (below), not derived from `seq` |
|
||
| `op_kind` | `text` | no | CHECK in (`reparent_row`, `set_field`, `suppress_row`, `renumber_attempt`, `supersede_application`) |
|
||
| `target_table` | `text` | no | CHECK against a whitelist asserted in migration |
|
||
| `target_row_pk` | `bigint` | no | |
|
||
| `column_name` | `text` | yes | Required for `set_field` and `reparent_row` |
|
||
| `previous_value` | `jsonb` | no | **The reversal payload** |
|
||
| `new_value` | `jsonb` | yes | |
|
||
| `created_at` | | no | Append-only |
|
||
|
||
**Constraints.** `uq_merge_op UNIQUE (merge_id, seq)`. `ck_merge_op_column CHECK (op_kind NOT IN
|
||
('set_field','reparent_row') OR column_name IS NOT NULL)`. `ck_merge_op_rank CHECK (reversal_rank
|
||
BETWEEN 1 AND 9)`. Append-only grants.
|
||
`ix_merge_op_target (target_table, target_row_pk)` — the index the stack-discipline
|
||
trigger uses, and the reason that check is affordable.
|
||
`ix_merge_op_replay (merge_id, target_table, target_row_pk, reversal_rank)` — the index the reversal
|
||
replay reads in order (§19.5).
|
||
|
||
**The six mechanics, as implemented:**
|
||
|
||
1. **The losing candidate row is retained** with status `merged` and `merged_into_candidate_id` set.
|
||
Its `public_id` and `reference_code` stay resolvable, and
|
||
`GET /candidates/{loser_public_id}` returns `301` to the survivor. Not sentimentality: the loser's
|
||
`public_id` is already inside sent candidate emails and recruiter bookmarks, and its
|
||
`reference_code` may be written on a paper interview note.
|
||
2. **Child rows are re-parented** by `UPDATE ... SET candidate_id = survivor`, one operation row per
|
||
affected row with `previous_value = {"candidate_id": <loser>}`.
|
||
3. **Colliding applications to the same job:** the earlier-created application stays live; the other
|
||
gets `superseded_by_application_id` set, status `superseded_by_merge`, and — in the **same
|
||
statement** — `terminal_at = merge.performed_at`, so `ck_application_terminal` holds against the
|
||
now-terminal `state` (`superseded_by_merge` is seeded `is_terminal = true, is_negative = false`;
|
||
see §15.1). `attempt_no` collisions are renumbered. All recorded as operations.
|
||
4. **Scalar survivor fields** are overwritten only where the survivor is NULL, or per-field by
|
||
explicit recruiter choice in the merge UI. Each overwrite is an `op_kind = 'set_field'` with
|
||
`previous_value`.
|
||
5. **Duplicate email/phone/link rows** that would violate the global unique index get
|
||
`suppressed_by_merge_id` set — never deleted. **The unique indexes are partial on
|
||
`WHERE suppressed_by_merge_id IS NULL AND deleted_at IS NULL` specifically to allow this.** This
|
||
is the detail that breaks naive merges: two duplicates by definition may share an email address,
|
||
so re-parenting both violates a naive `UNIQUE (address)`, and the tempting fix — deleting one —
|
||
is exactly the data loss the requirement forbids.
|
||
6. **The merge also writes `audit_event` rows**, but the undo log is a separate,
|
||
application-readable table.
|
||
|
||
**Why re-pointing and not copying.** Copied application, interview and score rows would appear twice
|
||
in every funnel report and would leave the originals orphaned under a dead identity. **Why the undo
|
||
log is not `audit_event`:** audit must stay append-only and hash-chained for forensics, while the
|
||
undo log is operational data the merge feature reads and writes; conflating them would make the
|
||
audit table mutable. Audit rows are also partitioned, may be archived off-box, and may have PII
|
||
redacted under retention — an unreliable base for an operational undo.
|
||
|
||
### 19.5 Reversal semantics
|
||
|
||
Reversal replays `candidate_merge_operation` for the `merge_id` **in dependency order per target
|
||
row** — `ORDER BY target_table, target_row_pk, reversal_rank` — restoring `previous_value` for each
|
||
operation, then sets the reversal columns and clears the loser's `merged_into_candidate_id` and
|
||
status.
|
||
|
||
**Why not descending `seq`, which is what an earlier draft specified.** Descending `seq` looks
|
||
obviously right (undo in reverse) and is wrong for the one case merge exists to handle. Trace it.
|
||
Mechanic 2 emits `reparent_row` on the losing candidate's application at seq *k*; mechanic 3 then
|
||
emits `supersede_application` on that same row at seq *k+1*. Reversing descending undoes *k+1*
|
||
first, which clears `superseded_by_application_id` and restores the prior status — leaving **two
|
||
rows** with the same `candidate_id` (still the survivor, because *k* has not been undone yet), the
|
||
same `job_id`, `state = 'active'`, `deleted_at IS NULL` and `superseded_by_application_id IS NULL`.
|
||
That is precisely the tuple `uq_application_live` forbids (§15.1), so the statement raises. And
|
||
because `uq_application_live` is a **partial** unique index, it **cannot be `DEFERRABLE`** (§4.8) —
|
||
there is no way to postpone the check to COMMIT — so the violation aborts the whole reversal
|
||
transaction. **Every merge that collided on a job would be permanently unreversible, which is the
|
||
exact case §19.4 mechanic 3 was written for.**
|
||
|
||
The fix is to stop treating `seq` as the reversal order. `seq` is *application* order; reversal order
|
||
is a property of the operation kind, because what must be undone first is whatever the database's
|
||
uniqueness rules depend on. `reversal_rank` encodes it, assigned from a fixed table:
|
||
|
||
| `op_kind` | `reversal_rank` | Undone at this point because |
|
||
|---|---|---|
|
||
| `reparent_row` | 1 | The row must be back under the **losing** candidate before any predicate that keys on `(candidate_id, job_id)` is re-evaluated |
|
||
| `renumber_attempt` | 2 | `attempt_no` must be restored while the row is already back under the loser, so `uq_application_attempt` sees the original pair |
|
||
| `supersede_application` | 3 | Only now is clearing `superseded_by_application_id` safe: the two rows no longer share a `candidate_id`, so `uq_application_live` is satisfied |
|
||
| `suppress_row` | 4 | Un-suppressing an email/phone/link re-arms `uq_candidate_email` / `uq_candidate_phone`, which requires the rows to already be back under their original candidate |
|
||
| `set_field` | 5 | Scalar restores on the survivor depend on nothing and are cheapest last |
|
||
|
||
Ordering by `(target_table, target_row_pk, reversal_rank)` rather than globally by rank keeps
|
||
per-row dependency correct while leaving unrelated rows free to be processed in any order.
|
||
|
||
**The alternative considered.** Emit a single compound `unmerge_application_collision` operation
|
||
whose `previous_value` carries both `candidate_id` and the supersession state, so one `UPDATE`
|
||
restores both and the ordering problem disappears. Rejected — but narrowly. It is genuinely simpler
|
||
at the SQL level; it was rejected because it makes the undo log's grain inconsistent (most operations
|
||
are one column on one row, this one is several), and because the completeness test in §19.4's
|
||
mitigation — "enumerate every table carrying `candidate_id` and assert an operation exists for each"
|
||
— is much easier to write against a uniform per-column grain. `reversal_rank` keeps the grain
|
||
uniform and puts the ordering knowledge in one five-row table.
|
||
|
||
**Required fixture, and it is the only thing that would have caught this.** A reversal test that
|
||
merges two candidates **who both applied to the same job**, then asserts a clean round-trip: two live
|
||
applications restored, one under each candidate, both `state = 'active'`, both
|
||
`superseded_by_application_id IS NULL`, `terminal_at` back to NULL, original `attempt_no` values, and
|
||
`uq_application_live` satisfied. A reversal test using two candidates with disjoint applications
|
||
passes under either ordering and proves nothing.
|
||
|
||
Three hard rules:
|
||
|
||
| Rule | Behaviour | Why |
|
||
|---|---|---|
|
||
| **Stack discipline** | A merge may be reversed only if no later unreversed merge touched any row this merge touched. Enforced by the trigger in §19.3. Out-of-order unmerge is **refused**, never attempted | If a second merge re-parented a row the first merge had already moved, the first merge's `previous_value` is stale and restoring it would move the row to a candidate it never belonged to — silent corruption nobody would notice for months. Refusing is strictly better, and the fix path (reverse the later merge first) is obvious once the error names the blocking merge |
|
||
| **Order within a merge is `reversal_rank`, not reverse `seq`** | Replay is `ORDER BY target_table, target_row_pk, reversal_rank`, per the table above | `uq_application_live` is a partial unique index and therefore cannot be deferred, so a wrong intra-merge order aborts the transaction rather than resolving at COMMIT |
|
||
| **Post-merge rows stay with the survivor** | Rows created after `performed_at` remain with the survivor. Mechanically checkable via `created_at > performed_at`, and the unmerge confirmation screen **must list exactly which rows will stay** before the recruiter confirms | A note written while the identities were merged has no defensible pre-merge owner, and guessing one would fabricate provenance. Showing the list turns an invisible surprise into an informed decision |
|
||
| **Retention interlock** | If a purge has pseudonymised or blob-deleted either candidate, it sets `reversal_blocked_reason = 'retention_purge'` and the trigger refuses | Reversal without the loser's PII produces a shell identity that looks like data loss. Blocking with a stated reason is more honest than half-reversing |
|
||
|
||
There is no time limit on reversal otherwise — a fixed window would be arbitrary, and duplicate
|
||
errors are often discovered when the candidate reapplies a year later.
|
||
|
||
**Highest-risk logic in the schema, and its mitigation.** Reversal correctness rests entirely on the
|
||
completeness of `candidate_merge_operation`: any table carrying `candidate_id` that merge re-parents
|
||
but does not record silently becomes unreversible, and the failure surfaces only the first time
|
||
someone unmerges months later. Mitigation, and it is a required deliverable: **enumerate every table
|
||
carrying `candidate_id` in one place — a `SELECT` over `information_schema.columns` in a test — and
|
||
assert that the merge routine records an operation for each of them.** That query is the guard; a
|
||
code review is not.
|
||
|
||
---
|
||
|
||
## 20. ATS scoring snapshots
|
||
|
||
There is nothing to port. The prototype's score is `aiScore: int(52,98)` (`js/data.js:123`) with a
|
||
separate client-side relevance blend (`js/candidates.js:18`) — no components, no evidence, no model,
|
||
no version. BRD §6.1 requires the opposite: *"the score must be reproducible: the same candidate and
|
||
role must yield the same score absent a model or data change"*, with matched and missing skills
|
||
exposed.
|
||
|
||
### 20.1 `app.scoring_config` and `app.scoring_config_version`
|
||
|
||
| Table | Purpose | Key columns | Constraints |
|
||
|---|---|---|---|
|
||
| `app.scoring_config` | Identity of a scoring configuration | `key` UNIQUE, `name`, `owner_user_id`, `description`, `current_version_id`, audit, soft delete | deferred current-version check |
|
||
| `app.scoring_config_version` | Immutable configuration | `scoring_config_id`, `version_no`, `algorithm_key`, `algorithm_code_version`, `aggregation_method` CHECK in (`weighted_sum`,`weighted_mean`,`gated_weighted_sum`), `band_thresholds jsonb`, `hyperparameters jsonb`, `excluded_attributes text[]`, `activation_evaluation_run_id` NULL FK, `published_at`, `activated_at`, `deactivated_at`, `published_by_user_id`, `config_hash bytea` | `uq (scoring_config_id, version_no)`; `ck_activation_gate CHECK (activated_at IS NULL OR activation_evaluation_run_id IS NOT NULL)` — a **not-null guard only**, see the gate discussion below; INSERT/SELECT-only grants plus `UPDATE (activated_at, deactivated_at, activation_evaluation_run_id)` and the `to_jsonb`-form immutability trigger (§9.2) |
|
||
|
||
**The fairness gate is a trigger, not a CHECK — and this document must not overclaim it.**
|
||
`ck_activation_gate` says only "if `activated_at` is set then `activation_evaluation_run_id` is not
|
||
null". A single-table CHECK **cannot see** `evaluation_run.status`, so on its own it permits
|
||
activation pointing at a run that `failed`, that `error`ed, that is still `running`, that was
|
||
performed against a *different* config version, or that ran against an unfrozen
|
||
`evaluation_dataset`. Describing that as "no config version can be activated without a passing
|
||
fairness evaluation" — which §21 also repeated — is the one place this document claimed a guarantee
|
||
it did not have, and it matters because legal and the business are told this control gates AI
|
||
ranking. The real gate:
|
||
|
||
```sql
|
||
CREATE CONSTRAINT TRIGGER tg_scoring_activation_gate
|
||
AFTER INSERT OR UPDATE OF activated_at, activation_evaluation_run_id
|
||
ON app.scoring_config_version
|
||
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW
|
||
EXECUTE FUNCTION app.tg_assert_activation_gate();
|
||
```
|
||
|
||
`app.tg_assert_activation_gate()` fires when `activated_at` transitions from NULL and asserts:
|
||
|
||
```sql
|
||
EXISTS (SELECT 1
|
||
FROM app.evaluation_run r
|
||
JOIN app.evaluation_dataset d ON d.id = r.evaluation_dataset_id
|
||
WHERE r.id = NEW.activation_evaluation_run_id
|
||
AND r.status = 'passed'
|
||
AND r.scoring_config_version_id = NEW.id -- the run must be OF this version
|
||
AND r.finished_at IS NOT NULL
|
||
AND d.frozen_at IS NOT NULL) -- against a frozen dataset
|
||
```
|
||
|
||
and a **second** trigger on `app.evaluation_run` refuses any status change away from `passed` while
|
||
an active `scoring_config_version` references it — otherwise the gate could be satisfied and then
|
||
retroactively invalidated. The CHECK is retained as the cheap always-on half; the trigger is the
|
||
gate. §21 is worded to match. The document is otherwise scrupulous about the CHECK-versus-trigger
|
||
distinction (§13.1 `ck_candidate_merged_status`, §23.1) and this section is now consistent with it.
|
||
|
||
`excluded_attributes text[]` is the machine-readable statement of which candidate attributes the
|
||
scorer must not consume — the input to the fairness evaluation and to a test that asserts the scorer
|
||
never reads them. `band_thresholds` is JSONB because bands are a small ordered map read as a whole
|
||
(`{"strong": 85, "good": 70, "fair": 55}`) and nothing constrains it; `hyperparameters` is JSONB
|
||
because it is algorithm-specific and not enumerable across algorithms.
|
||
|
||
**`algorithm_code_version` on the config version** records which scorer implementation the config was
|
||
authored against, which is how a config/code mismatch becomes detectable rather than mysterious.
|
||
|
||
### 20.2 `app.scoring_config_criterion`
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `scoring_config_version_id` | `bigint` | no | FK |
|
||
| `criterion_key` | `text` | no | e.g. `skill_match`, `experience_fit`, `education_fit`, `location_fit`, `tenure_stability` |
|
||
| `label` | `text` | no | Shown in the explainability UI |
|
||
| `weight` | `numeric(6,4)` | no | CHECK 0–1 |
|
||
| `scale_min`, `scale_max` | `numeric(8,3)` | no | |
|
||
| `transform` | `text` | no | CHECK in (`linear`, `step`, `log`, `sigmoid`) |
|
||
| `is_mandatory_gate` | `boolean` | no | Default false. A failed gate caps the overall score |
|
||
| `display_order` | `int` | no | |
|
||
|
||
**Constraints.** `uq (scoring_config_version_id, criterion_key)`;
|
||
`ck (scale_max > scale_min)`; immutable per §9.2's `to_jsonb` form.
|
||
|
||
**Weight-sum invariant on the parent *and* the child, exactly as §9.2.** Two deferred constraint
|
||
triggers calling one function that asserts `count(*) >= 1 AND abs(sum(weight) - 1.0) <= 0.0001` for
|
||
the version:
|
||
|
||
```sql
|
||
CREATE CONSTRAINT TRIGGER tg_scoring_criterion_weights
|
||
AFTER INSERT OR UPDATE OR DELETE ON app.scoring_config_criterion
|
||
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW
|
||
EXECUTE FUNCTION app.tg_assert_criterion_weights();
|
||
|
||
CREATE CONSTRAINT TRIGGER tg_scoring_config_version_weights
|
||
AFTER INSERT OR UPDATE ON app.scoring_config_version
|
||
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW
|
||
EXECUTE FUNCTION app.tg_assert_criterion_weights();
|
||
```
|
||
|
||
The child trigger alone has the identical gap as `job_requirement`'s did: a
|
||
`scoring_config_version` published with **zero** criteria never fires it, weights sum to 0, and the
|
||
scorer aggregates nothing. ERD 5 states this as
|
||
`scoring_config_version ||--|{ scoring_config_criterion`, which the parent trigger makes true.
|
||
|
||
**Criteria are rows, not JSON,** because they are enumerable, joined to `job_requirement`, displayed
|
||
in the explainability UI, and aggregated in reports. A JSONB weights blob is unqueryable for the
|
||
explainability screen and unconstrainable by a sum-to-one check.
|
||
|
||
*Volume:* config versions ~50, criteria ~350.
|
||
|
||
### 20.3 Why config binding is orthogonal to job versioning
|
||
|
||
Covered in §9.4. The short form: putting `scoring_config_version_id` on `job_version` would force a
|
||
new job version on every scoring tweak, so "what changed in this requisition" becomes unanswerable.
|
||
`job_scoring_assignment` keeps them independent, and this is safe *only* because `ats_result` pins
|
||
both versions itself.
|
||
|
||
### 20.4 `app.ats_result`
|
||
|
||
*Purpose.* An immutable snapshot of one score for one application, pinned to every input that could
|
||
change the number.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `public_id` | `uuid` | no | UNIQUE |
|
||
| `job_application_id` | `bigint` | **no** | FK. **There is no `candidate_id` column** — the score is reachable only through the application |
|
||
| `job_id` | `bigint` | **no** | FK `app.job`, denormalised and immutable. A trigger asserts it equals the application's `job_id`. Exists because applications pin *different* `job_version_id`s, so a per-requisition query cannot go through `job_version_id` — see the index note |
|
||
| `job_version_id` | `bigint` | no | FK — the requirements used |
|
||
| `scoring_config_version_id` | `bigint` | no | FK — the weights used |
|
||
| `candidate_document_id` | `bigint` | yes | FK — the CV read |
|
||
| `parse_attempt_id` | `bigint` | yes | FK — the parse of that CV |
|
||
| `algorithm_code_version` | `text` | **no** | The scorer build |
|
||
| `ai_model_id` | `bigint` | yes | FK `ai.ai_model_config_version` |
|
||
| `ai_model_version` | `text` | yes | Denormalised string, so the pin survives even if the registry row is reorganised |
|
||
| `prompt_template_version` | `text` | yes | |
|
||
| `ai_run_id` | `bigint` | yes | FK `ai.ai_model_invocation` |
|
||
| `overall_score` | `numeric(6,3)` | no | CHECK 0–100 |
|
||
| `band` | `text` | no | The recommendation band the UI renders |
|
||
| `mandatory_gate_failed` | `boolean` | no | Default false |
|
||
| `input_fingerprint` | `bytea` | **no** | sha256 over **every** pinned input — see the fingerprint note below. Not three inputs; all eight |
|
||
| `computed_at` | `timestamptz` | no | |
|
||
| `compute_duration_ms` | `int` | yes | |
|
||
| `is_current` | `boolean` | no | |
|
||
| `superseded_by_id` | `bigint` | yes | FK self |
|
||
| `reviewed_by_user_id` | `bigint` | yes | |
|
||
| `reviewed_at` | `timestamptz` | yes | |
|
||
| `review_outcome` | `text` | yes | CHECK in (`agree`, `disagree_too_high`, `disagree_too_low`, `not_applicable`) |
|
||
| `review_note` | `text` | yes | |
|
||
| `created_at` | | no | Append-only |
|
||
|
||
**Constraints.**
|
||
|
||
```sql
|
||
CONSTRAINT ck_ats_score CHECK (overall_score BETWEEN 0 AND 100),
|
||
-- biconditional, NOT (is_current = false OR superseded_by_id IS NULL)
|
||
CONSTRAINT ck_ats_superseded CHECK (is_current = (superseded_by_id IS NULL)),
|
||
CONSTRAINT ck_ats_review_human CHECK (review_outcome IS NULL OR reviewed_by_user_id IS NOT NULL),
|
||
CONSTRAINT ck_ats_reviewed CHECK ((reviewed_at IS NULL) = (reviewed_by_user_id IS NULL)),
|
||
CONSTRAINT ck_ats_review_actor CHECK (reviewed_by_actor_kind = 'user');
|
||
|
||
CREATE UNIQUE INDEX uq_ats_result_current ON app.ats_result (job_application_id) WHERE is_current;
|
||
|
||
-- at COMMIT, an application with any score has EXACTLY one current score
|
||
CREATE CONSTRAINT TRIGGER tg_ats_exactly_one_current
|
||
AFTER INSERT OR UPDATE OF is_current, superseded_by_id ON app.ats_result
|
||
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW
|
||
EXECUTE FUNCTION app.tg_assert_one_current_score();
|
||
```
|
||
|
||
**Why `ck_ats_superseded` is a biconditional and not a disjunction.** The disjunctive form
|
||
`(is_current = false) OR (superseded_by_id IS NULL)` permits `is_current = false AND
|
||
superseded_by_id IS NULL` — which is exactly the broken state: a score that has dropped out of
|
||
currency with **no successor**, severing the history chain. `uq_ats_result_current` guarantees *at
|
||
most* one current row and never *at least* one, so under the old CHECK an application could end up
|
||
with scores but no current score, and `job_application.current_ats_result_id` would then point at a
|
||
superseded row or nothing at all. The biconditional makes the state unrepresentable: the newest row
|
||
is `(true, NULL)`, every superseded row is `(false, NOT NULL)`, and there is no third combination.
|
||
The deferred trigger supplies the "at least one" half that no unique index can express. It has to be
|
||
deferred because a rescore legitimately has both rows non-current for the duration of one statement.
|
||
|
||
`reviewed_by_actor_kind text NOT NULL DEFAULT 'user'` with the CHECK above is the schema-level
|
||
actor assertion — see the review-columns discussion below for what it does and does not buy.
|
||
`ats_result` has no `updated_by_user_id` (§4.2 append-only), so without that column nothing on the
|
||
row records the *kind* of principal that flipped `is_current`.
|
||
|
||
**The identical change applies to `app.candidate_job_match`** (§20.6), which carries the same
|
||
`is_current` / `superseded_by_id` pair and currently has no constraint on it at all.
|
||
|
||
**Append-only with a narrow exception, implemented as privilege rather than as trigger logic:**
|
||
|
||
```sql
|
||
GRANT SELECT, INSERT ON app.ats_result TO talentflow_app;
|
||
GRANT UPDATE (is_current, superseded_by_id, reviewed_by_user_id, reviewed_at,
|
||
review_outcome, review_note) ON app.ats_result TO talentflow_app;
|
||
```
|
||
|
||
Column-level `UPDATE` grants rather than a trigger with a column allowlist, because Postgres
|
||
enforces them at the privilege layer, before any trigger logic can be wrong.
|
||
|
||
**Rescoring** inserts a new row, flips the prior row's `is_current` to false, and sets
|
||
`superseded_by_id`. Mutating `overall_score` in place would destroy the historical score, which is
|
||
the exact requirement.
|
||
|
||
**What `ck_ats_review_human` actually guarantees, stated precisely.** It guarantees that **a review
|
||
verdict cannot exist without a named human reviewer** — no more than that. It is a not-null pairing
|
||
on a plain `bigint` FK, so any writer holding the column grant satisfies it by supplying any user id;
|
||
"a scoring job has no authenticated user" is a property of the application's request pipeline, not of
|
||
the schema. And `review_outcome`'s domain is (`agree`, `disagree_too_high`, `disagree_too_low`,
|
||
`not_applicable`) — **none of which is a rejection**. Rejection happens through
|
||
`application.transition()`.
|
||
|
||
So "AI must never auto-reject" rests on the two mechanisms that genuinely hold it:
|
||
|
||
| Mechanism | Kind of guarantee |
|
||
|---|---|
|
||
| `pipeline_transition_rule`'s `ck_terminal_negative_user_only CHECK (NOT (is_terminal_negative AND allowed_actor_kinds <> ARRAY['user']))` (§17) | A real single-table database invariant. No rule row can ever permit a non-human terminal-negative transition |
|
||
| The service guard in `application.transition()`, plus `_decisions.md` Part 1 rule 3 (the `intelligence` tier physically cannot import a domain module's writer) | Process and dependency-graph enforcement, verified by `import-linter` in CI |
|
||
|
||
Presenting three "independent expressions" when one of them was a process claim dressed as a
|
||
constraint was worse than presenting two, because it invited someone to remove the one that works.
|
||
|
||
**Closing the residual hole on the review columns properly.** Two additions, neither of which
|
||
pretends to be the auto-reject guarantee:
|
||
|
||
1. `reviewed_by_actor_kind text NOT NULL DEFAULT 'user' CHECK (reviewed_by_actor_kind = 'user')` — a
|
||
genuine schema-level assertion that whatever wrote a review claimed to be a human principal.
|
||
2. A separate **`talentflow_worker`** role for the scoring job, holding
|
||
`UPDATE (is_current, superseded_by_id)` but **not** the review columns, so the worker
|
||
*cannot* write a verdict at the privilege layer. The accompanying permission test asserts exactly
|
||
that: connect as `talentflow_worker`, attempt `UPDATE app.ats_result SET review_outcome = 'agree'`,
|
||
assert `ERROR: permission denied for column review_outcome`.
|
||
|
||
**`input_fingerprint` must cover everything that changes the score, because it is what gates
|
||
recomputation.** §29.6 Q18 uses the fingerprint to *skip* unchanged inputs on a rescore, so anything
|
||
omitted from it is an input whose change is silently ignored — the displayed score stays the one the
|
||
previous code computed, which is the precise drift this section exists to prevent. Three inputs
|
||
(feature vector, `content_hash`, `config_hash`) is not enough: a scorer release
|
||
(`algorithm_code_version`), a model or prompt version change, or a **re-parse of the same CV** all
|
||
produce an identical three-input fingerprint. The definition is therefore all eight pinned inputs:
|
||
|
||
```
|
||
input_fingerprint = sha256(
|
||
canonical_feature_vector -- see canonicalisation below
|
||
|| job_version.content_hash -- bytea, 32 bytes
|
||
|| scoring_config_version.config_hash -- bytea, 32 bytes
|
||
|| algorithm_code_version -- text
|
||
|| coalesce(ai_model_config_version_id::text, '')
|
||
|| coalesce(prompt_template_version, '')
|
||
|| coalesce(candidate_document.sha256::text,'')
|
||
|| coalesce(parse_attempt_id::text, '')
|
||
)
|
||
```
|
||
|
||
**Canonicalisation, so the value is reproducible rather than incidentally stable.** Fields are
|
||
concatenated in the literal order above, separated by a single `0x1F` unit-separator byte (so
|
||
`'ab' || ''` and `'a' || 'b'` cannot collide); the feature vector is serialised as JSON with keys
|
||
sorted ascending by byte value, no insignificant whitespace, numerics rendered at fixed scale
|
||
matching their column, `null` for absent, and UTF-8 NFC normalisation on every string. A CI test
|
||
recomputes a golden fingerprint from a fixed fixture and fails on any drift, because a
|
||
canonicalisation change is a silent full-rescore event.
|
||
|
||
**Indexes.**
|
||
|
||
| Index | Purpose |
|
||
|---|---|
|
||
| `uq_ats_result_current (job_application_id) WHERE is_current` | The working index for every score read |
|
||
| `ix_ats_result_fingerprint (job_application_id, input_fingerprint) WHERE is_current` | "Has **this application's** fingerprint changed since its current score" — the actual rescore-skip question. Indexing `input_fingerprint` alone answered "does any application anywhere have this fingerprint", which is meaningless: a fingerprint match on a *different* application implies nothing, since the feature vector and document hash differ |
|
||
| `ix_ats_result_job_score (job_id, overall_score DESC) WHERE is_current` | Score history and calibration across a requisition. **On `job_id`, not `job_version_id`** |
|
||
| `ix_ats_result_config (scoring_config_version_id, computed_at)` | "Which scores came from config v4" — the query a rollback needs |
|
||
|
||
**Why the shortlist index is on `job_id` and why the shortlist itself no longer reads this table.**
|
||
Applications pin the `job_version_id` that was in force when they applied (§15.1), so a requisition
|
||
with four versions has its applications spread across four distinct `job_version_id` values. An index
|
||
on `(job_version_id, overall_score DESC)` therefore turns "ranked shortlist for requisition JOB-1042"
|
||
into N index scans plus a merge — or a join through `job_application` to recover `job_id` — which is
|
||
why the denormalised immutable `job_id` column exists here. Even with it, the **live shortlist screen**
|
||
reads `ix_application_shortlist (job_id, current_overall_score DESC)` on `job_application` (§15.1),
|
||
because that is one index scan over the rows the screen already needs; `ats_result`'s `job_id` index
|
||
serves score *history* and calibration, where every row matters and `is_current` alone is not the
|
||
filter. §29.6 Q1 and Q7 name these indexes.
|
||
|
||
**Required test:** bump only `algorithm_code_version` — same CV, same parse, same requirements, same
|
||
weights — and assert a **new** `ats_result` row is written rather than the rescore being skipped.
|
||
|
||
**Profile.** *Status:* `is_current` + `review_outcome`, both Tier 2. *Required:* application, job
|
||
version, config version, algorithm version, score, band, fingerprint, `computed_at`. *Soft delete:*
|
||
no. *Retention:* the score survives candidate pseudonymisation — it is a statistical record with no
|
||
personal content once `evidence` is purged. *PII:* `sensitive_personal` (`evidence` quotes CV text);
|
||
the score and band alone are `internal`. *Audit:* insert and every review write. *Volume:* ~350,000
|
||
(**assumption**: 1.75 scores per application, from rescores on requisition-version publish and
|
||
config activation). *Queries:* the current score for an application; the ranked shortlist for a
|
||
requisition (BRD §6.1's "AI Relevance" sort, `js/candidates.js:18`); the score history for one
|
||
application; scores by config version for a rollback; score-vs-outcome calibration.
|
||
|
||
**Five things can change the number on screen** — the requirements, the weights, the scorer code, the
|
||
CV, and the parse of that CV — so all five are pinned **and all five are inside
|
||
`input_fingerprint`**, along with the model and prompt versions when a model participated. Pinning
|
||
without fingerprinting would have been half the job: the pins make a historical score explainable,
|
||
the fingerprint makes a *stale* score detectable. The FK to `job_application` with no `candidate_id`
|
||
is the structural fix for the prototype, where `aiScore` hangs off the candidate and a candidate
|
||
therefore cannot hold two different scores for two jobs.
|
||
|
||
### 20.5 `app.ats_result_criterion` and `app.ats_result_skill`
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Volume |
|
||
|---|---|---|---|---|
|
||
| `app.ats_result_criterion` | Per-criterion arithmetic, **stored not recomputed** | `ats_result_id`, `criterion_key`, `job_requirement_id` NULL FK, `raw_value numeric(12,3)` NULL, `normalised_score numeric(8,4)`, `weight_applied numeric(6,4)` NOT NULL, `contribution numeric(8,4)` NOT NULL, **`match_state text` NOT NULL**, `gate_passed boolean` NULL, `matched_evidence jsonb` NULL | `uq (ats_result_id, criterion_key)`; `ix (job_requirement_id)`; **`ix_ats_criterion_match (match_state, job_requirement_id)`**; append-only; `ck (weight_applied BETWEEN 0 AND 1)`; `ck_ats_criterion_match_state`, `ck_ats_criterion_unassessed` (below) | ~2.4M |
|
||
| `app.ats_result_skill` | Matched and missing skills — what the UI actually renders (BRD §9.1 `matchedSkills[]`, `missingSkills[]`) | `ats_result_id`, `skill_id` NULL FK, `raw_label` NULL, `match_kind` CHECK in (`matched`,`missing`,`adjacent`,`extra`), `job_requirement_id` NULL FK, `candidate_skill_id` NULL FK, `is_mandatory`, `confidence numeric(4,3)` NULL, `evidence_snippet text` NULL | `ck (skill_id IS NOT NULL OR raw_label IS NOT NULL)`; `CREATE UNIQUE INDEX uq_ats_result_skill ON app.ats_result_skill (ats_result_id, skill_id, match_kind) WHERE skill_id IS NOT NULL;` (index form, §4.8); `ix (skill_id, match_kind)` — the pipeline-wide skill-gap query (BRD AI-12) | ~3.5M |
|
||
|
||
**`weight_applied` and `contribution` are stored, not recomputed on read.** The arithmetic that
|
||
produced the displayed total is on disk, so a historical score is reproducible even if a config row
|
||
were somehow altered. This is the difference between "we can probably reproduce it" and "here is what
|
||
we computed". Recomputing contributions on read from the config version is correct in theory, but any
|
||
change to the aggregation code silently rewrites historical reports.
|
||
|
||
**`match_state` — the column that makes "missing requirements" a fact instead of an inference.**
|
||
|
||
```sql
|
||
match_state text NOT NULL, -- no DEFAULT: see below
|
||
CONSTRAINT ck_ats_criterion_match_state
|
||
CHECK (match_state IN ('matched', 'partial', 'missing', 'not_assessed')),
|
||
CONSTRAINT ck_ats_criterion_unassessed
|
||
CHECK (job_requirement_id IS NOT NULL OR match_state = 'not_assessed')
|
||
```
|
||
|
||
`05` §5.2 makes this row of the explainability record load-bearing: matched requirements are
|
||
`job_requirement_id` + `match_state = 'matched'`, missing ones are `match_state IN ('missing',
|
||
'partial')` with `job_requirement.is_mandatory` distinguishing a gate failure from a soft gap, and
|
||
the document's claim is that **"missing requirements are a queryable fact, not an inference from a
|
||
null"**. Without the column that claim is false, and falsifiably so: a requirement the candidate does
|
||
not meet produces *no* `ats_result_criterion` row, which is indistinguishable from a requirement the
|
||
scorer never evaluated — a parser failure, a criterion the config does not carry, a requirement added
|
||
after the score was computed. Those three have opposite meanings for a recruiter and the same shape on
|
||
disk. `08` records the consequence as **GAP-27**: REQ-SCR-03 and REQ-AIC-04 both named
|
||
`ats_result_criterion.match_state` as their database entity while the column existed nowhere, so both
|
||
traced to nothing, and REQ-AIC-04's pipeline-wide skill-gap aggregate over `missing` / `partial` was
|
||
not expressible at all.
|
||
|
||
Three details, each of which is a decision rather than a formality:
|
||
|
||
- **`NOT NULL` with no `DEFAULT`.** A default would let a scorer omit the field and silently produce
|
||
`not_assessed`, which is the exact inference-from-absence this column exists to eliminate — the
|
||
failure would move from "no row" to "a row that says nothing", which is worse because it looks
|
||
populated. `ats_result_criterion` is created in migration `012` and no data precedes it, so
|
||
`NOT NULL` costs no backfill (§33). The scorer's output schema is JSON-schema-constrained with
|
||
`match_state` enumerated and required (`05` §6.5 layer 3), so a model response that omits it voids
|
||
the run before it reaches the database.
|
||
- **Tier 2, not a `ref` table** (§4.6). Four values, engineer-owned, on the hottest append path in the
|
||
schema at ~2.4M rows; the set is fixed by the explainability contract in `05` §5.2 and is not
|
||
something a recruiting lead edits. A `ref` FK here would be ceremony plus a join per criterion row.
|
||
- **`ck_ats_criterion_unassessed` ties the state to the requirement.** `match_state` describes *the
|
||
requirement this criterion evaluated*, so a criterion with no requirement — `location_fit`,
|
||
`tenure_stability` — has nothing to match and must say `not_assessed`. Without the CHECK,
|
||
`criterion_key = 'tenure_stability', match_state = 'missing'` is storable and would be counted as a
|
||
missing requirement by every aggregate that reads this column.
|
||
|
||
**What `match_state` cannot enforce, stated so nobody over-reads it.** "A mandatory miss is a gate,
|
||
not a number" (`05` §5.2) is a **cross-table** relationship — `match_state = 'missing'` here,
|
||
`is_mandatory` on `job_requirement`, `gate_passed` here, and `mandatory_gate_failed` on the parent
|
||
`ats_result` — and no single-table CHECK can span it. It is asserted by `05` §6.5 layer 5's
|
||
consistency tripwire (a mandatory-gate failure coexisting with a high band forces `needs_review`) and
|
||
by a constraint test, exactly as §20.1 distinguishes `ck_activation_gate` from
|
||
`tg_scoring_activation_gate`. This document is scrupulous about that distinction and this column does
|
||
not weaken it.
|
||
|
||
**Profile (both).** *Soft delete:* no. *Retention:* `matched_evidence` and `evidence_snippet` are
|
||
pseudonymised with the candidate; the numbers survive — and so does `match_state`, which is a
|
||
statistical fact with no personal content and is what makes a purged candidate still count in
|
||
"which requirement most often fails". *PII:* `sensitive_personal` for the evidence
|
||
columns, `internal` for the rest, `match_state` included. *Audit:* insert only. *Queries:* the score
|
||
explanation panel, which renders matched and missing requirements as two lists rather than one list
|
||
and a set of absences; skill-gap analysis across a pipeline (REQ-AIC-04, a `GROUP BY match_state` over
|
||
the pipeline's current scores); "which requirement most often fails" across a requisition, which is
|
||
`ix_ats_criterion_match` filtered to `('missing','partial')`.
|
||
|
||
### 20.6 `app.candidate_job_match` — matching without an application
|
||
|
||
*Purpose.* A candidate-to-requisition fit score where **no application exists**: talent-pool
|
||
rematching and cross-brand matching (BRD AI-2, "match a candidate to the best-fit open roles across
|
||
all Utopia brands"). This cannot be an `ats_result`, because `ats_result.job_application_id` is
|
||
`NOT NULL` and no application exists to point at — and inventing a placeholder application would
|
||
corrupt every funnel metric.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `candidate_id` | `bigint` | no | FK |
|
||
| `job_version_id` | `bigint` | no | FK |
|
||
| `scoring_config_version_id` | `bigint` | no | FK |
|
||
| `relevance_config_version_id` | `bigint` | yes | FK `app.relevance_config_version` (§29.4). Pinned whenever the row was produced by a **ranked** search or rematch, so "why did this candidate rank third in March" is answerable. NULL only for `origin = 'recruiter_request'` single-candidate scoring, where no blend was applied |
|
||
| `algorithm_code_version` | `text` | no | |
|
||
| `score` | `numeric(6,3)` | no | CHECK 0–100 |
|
||
| `band` | `text` | no | |
|
||
| `evidence` | `jsonb` | yes | |
|
||
| `ai_run_id` | `bigint` | yes | FK |
|
||
| `input_fingerprint` | `bytea` | no | Same eight-input definition and canonicalisation as §20.4, with `relevance_config_version.config_hash` substituted for the parse pins that do not apply |
|
||
| `computed_at` | `timestamptz` | no | |
|
||
| `is_current` | `boolean` | no | |
|
||
| `superseded_by_id` | `bigint` | yes | FK self |
|
||
| `origin` | `text` | no | CHECK in (`pool_rematch`, `cross_brand_scan`, `recruiter_request`) |
|
||
| `surfaced_at` | `timestamptz` | yes | When it was shown to a recruiter |
|
||
| `dismissed_at`, `dismissed_by_user_id` | | yes | A dismissed suggestion must not resurface |
|
||
|
||
**Constraints.**
|
||
|
||
```sql
|
||
CREATE UNIQUE INDEX uq_candidate_job_match_current
|
||
ON app.candidate_job_match (candidate_id, job_version_id) WHERE is_current;
|
||
|
||
-- the same biconditional §20.4 uses; this table previously had no constraint on the pair at all
|
||
CONSTRAINT ck_cjm_superseded CHECK (is_current = (superseded_by_id IS NULL)),
|
||
CONSTRAINT ck_cjm_dismissed CHECK ((dismissed_at IS NULL) = (dismissed_by_user_id IS NULL)),
|
||
CONSTRAINT ck_cjm_relevance CHECK (origin = 'recruiter_request'
|
||
OR relevance_config_version_id IS NOT NULL)
|
||
```
|
||
|
||
plus the deferred "exactly one current per `(candidate_id, job_version_id)` that has any row" trigger,
|
||
mirroring `tg_ats_exactly_one_current`. Append-only grants plus
|
||
`UPDATE (is_current, superseded_by_id, surfaced_at, dismissed_at, dismissed_by_user_id)`.
|
||
|
||
**Indexes.** `ix_cjm_job (job_version_id, score DESC) WHERE is_current AND dismissed_at IS NULL` —
|
||
"who should we talk to for this role"; `ix_cjm_candidate (candidate_id, score DESC) WHERE is_current`
|
||
— "what else could this person do".
|
||
|
||
**Profile.** *Status:* `is_current`. *Soft delete:* no. *Retention:* purged with the candidate;
|
||
non-current rows pruned after 12 months (this table can grow without bound if a nightly job scores
|
||
every candidate against every open requisition). **Bound it explicitly: only pool members and
|
||
candidates active in the last 12 months are rescored, and only against open requisitions.** *PII:*
|
||
`internal`; `evidence` is `sensitive_personal`. *Audit:* insert; surfacing is an access event.
|
||
*Volume:* ~500,000 with the bound above; **millions without it**, which is why the bound is part of
|
||
the design and not an optimisation. *Queries:* the "you should talk to…" suggestion list (BRD AI-9);
|
||
pool rematch results; cross-brand candidate matching.
|
||
|
||
### 20.7 `app.ats_result_override` — a human disagreeing, without erasing the disagreement
|
||
|
||
*Purpose.* One append-only row recording that a named human replaced the AI's recommendation band on
|
||
one `ats_result` with their own, and why. `ats_result` already supports **review** —
|
||
`reviewed_by_user_id`, `reviewed_at`, `review_outcome`, `review_note` — but review is an *opinion
|
||
about* a score whose domain (`agree`, `disagree_too_high`, `disagree_too_low`, `not_applicable`)
|
||
contains no action. Override is the action, and `05` §9.1 I-6 records that it had no home in the
|
||
schema at all.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | PK |
|
||
| `public_id` | `uuid` | no | UUIDv7, unique. Cited in the score panel and in audit |
|
||
| `ats_result_id` | `bigint` | no | FK `app.ats_result`. **The specific result row overridden**, not the application — see the rescore note |
|
||
| `overridden_band` | `text` | no | The band the human asserts. Same domain as `ats_result.band`, which is config-driven (`scoring_config_version.band_thresholds`), so no CHECK — a CHECK here would put band names in DDL and re-create the §4.6 Tier 1 problem |
|
||
| `override_reason_id` | `bigint` | no | FK `ref.vocabulary_value` (`vocabulary_key = 'ats_override_reason'`) via the composite-FK idiom of §4.7 |
|
||
| `override_reason_vocabulary_key` | `text` | no | `GENERATED ALWAYS AS ('ats_override_reason') STORED` — the constant column that makes the composite FK type-safe (§4.7) |
|
||
| `note` | `text` | **no** | Mandatory free text **in addition to** the vocabulary. `05` §5.2 requires the reason; the vocabulary makes it analysable and the note makes it useful |
|
||
| `overridden_by_user_id` | `bigint` | no | FK `app.app_user` |
|
||
| `overridden_by_actor_kind` | `text` | no | Default `'user'`, `CHECK (overridden_by_actor_kind = 'user')` — the §20.4 idiom, and here it is definitional |
|
||
| `overridden_at` | `timestamptz` | no | Default `now()` |
|
||
| `is_current` | `boolean` | no | |
|
||
| `superseded_by_id` | `bigint` | yes | FK self |
|
||
| `created_at` | | no | Append-only, no `updated_at` (§4.2) |
|
||
|
||
**Constraints.**
|
||
|
||
```sql
|
||
CONSTRAINT fk_ats_override_reason
|
||
FOREIGN KEY (override_reason_id, override_reason_vocabulary_key)
|
||
REFERENCES ref.vocabulary_value (id, vocabulary_key),
|
||
CONSTRAINT ck_ats_override_note CHECK (btrim(note) <> ''),
|
||
CONSTRAINT ck_ats_override_actor CHECK (overridden_by_actor_kind = 'user'),
|
||
-- the same biconditional as §20.4, for the same reason
|
||
CONSTRAINT ck_ats_override_superseded CHECK (is_current = (superseded_by_id IS NULL));
|
||
|
||
CREATE UNIQUE INDEX uq_ats_result_override_current
|
||
ON app.ats_result_override (ats_result_id) WHERE is_current;
|
||
|
||
-- an override may only be placed on the score that is currently in force
|
||
CREATE CONSTRAINT TRIGGER tg_ats_override_target_current
|
||
AFTER INSERT ON app.ats_result_override
|
||
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW
|
||
EXECUTE FUNCTION app.tg_assert_override_target_current();
|
||
```
|
||
|
||
`app.tg_assert_override_target_current()` asserts
|
||
`EXISTS (SELECT 1 FROM app.ats_result r WHERE r.id = NEW.ats_result_id AND r.is_current)`. It is a
|
||
**constraint trigger and `AFTER` per §4.8** — a `BEFORE INSERT` constraint trigger is a syntax error,
|
||
and the check must be deferred because a rescore-then-override in one transaction legitimately has the
|
||
target row non-current for the duration of a statement. Overriding a superseded score is otherwise
|
||
storable and meaningless: the panel renders the current score, so an override attached to an older row
|
||
would be invisible while appearing in the override-rate aggregate.
|
||
|
||
**`ck_ats_override_actor` is definitional, not defensive.** `ats_result`'s equivalent CHECK (§20.4)
|
||
exists to assert that *whatever* wrote a review claimed to be a human principal. Here the claim is the
|
||
table's reason for existing: an override is by construction a human disagreeing with a model, so an
|
||
`actor_kind` of `system`, `integration` or `ai_agent` on this table is not a lesser case, it is a
|
||
contradiction. The literal is `'user'` per **RULING-01** — the value set is exactly
|
||
(`user`, `system`, `integration`, `ai_agent`), there is no `human` member, and a guard spelled
|
||
`actor_kind <> 'human'` would compare against a value the CHECK cannot hold.
|
||
|
||
**Append-only with the §20.4 grant, and no "exactly one current" trigger.**
|
||
|
||
```sql
|
||
REVOKE UPDATE, DELETE ON app.ats_result_override FROM talentflow_app;
|
||
GRANT SELECT, INSERT ON app.ats_result_override TO talentflow_app;
|
||
GRANT UPDATE (is_current, superseded_by_id) ON app.ats_result_override TO talentflow_app;
|
||
|
||
CREATE TRIGGER tg_ats_result_override_immutable
|
||
BEFORE UPDATE ON app.ats_result_override
|
||
FOR EACH ROW
|
||
WHEN ((to_jsonb(OLD) - 'is_current' - 'superseded_by_id')
|
||
IS DISTINCT FROM (to_jsonb(NEW) - 'is_current' - 'superseded_by_id'))
|
||
EXECUTE FUNCTION app.tg_raise_immutable();
|
||
|
||
CREATE TRIGGER tg_ats_result_override_no_delete
|
||
BEFORE DELETE ON app.ats_result_override
|
||
FOR EACH ROW EXECUTE FUNCTION app.tg_raise_immutable();
|
||
```
|
||
|
||
The `UPDATE`/`DELETE` split is §9.2's shape and is forced by §4.8's fourth syntax fact — the
|
||
allow-list compares against `NEW`, which a combined `BEFORE UPDATE OR DELETE` `WHEN` clause may not
|
||
reference — and the `DELETE` half needs no `WHEN` clause, because an override row is evidence of a
|
||
human disagreeing with a model and is never removed.
|
||
|
||
`talentflow_worker` — the scoring job's role (§20.4) — holds **no privilege on this table at all**,
|
||
which is the privilege-layer statement of "AI never overrides a human, and never overrides itself".
|
||
The accompanying test connects as `talentflow_worker` and asserts
|
||
`ERROR: permission denied for table ats_result_override`.
|
||
|
||
Unlike `ats_result`, this table has **no** deferred "exactly one current" constraint trigger, and the
|
||
asymmetry is deliberate. §20.4 needs one because an application with any score must have *exactly*
|
||
one current score — `uq_ats_result_current` supplies "at most one" and no unique index can supply "at
|
||
least one". Here the correct cardinality is **zero or one**: the overwhelming majority of results are
|
||
never overridden, and that is the healthy state. `uq_ats_result_override_current` is therefore the
|
||
complete constraint, and adding the parent trigger would forbid the normal case.
|
||
|
||
**A second override supersedes the first; it does not replace it.** A recruiter overrides, a hiring
|
||
manager corrects them: two rows, the second carrying `is_current`, the first carrying
|
||
`superseded_by_id`. The chain is the record of a disagreement between two humans, which is precisely
|
||
the thing a hiring decision may later have to be defended against.
|
||
|
||
**Rescoring does not carry an override forward, and this is a decision with a visible consequence.**
|
||
An override binds to the `ats_result` row it overrode. When a requisition version is published or a
|
||
config activated, `ats_result` inserts a new row (§20.4) and the override does **not** follow it:
|
||
the new score was computed from different inputs, so re-applying a human's band judgement to it would
|
||
be attributing to that human an opinion they never formed about a score they never saw. The override
|
||
row is retained on the superseded result as evidence, and the score panel must say so —
|
||
"this candidate's previous score was overridden to *Strong* by A. Mujtaba on 12 March; the score has
|
||
since been recomputed" — rather than silently dropping the override or silently keeping it. That
|
||
sentence is a UI deliverable in Ahmed's score-explanation panel (`05` §4, "Human review"), and it is
|
||
the one place where this design's honesty depends on rendering rather than on a constraint.
|
||
|
||
**Why the reason is a vocabulary *and* a note.** `05` §5.2 rejects a free-text-only reason outright:
|
||
"reasons must be enumerable to be analysable". The override rate — and specifically the override rate
|
||
*by reason*, by role, and by requisition — is the single most useful fairness signal available without
|
||
protected-attribute data (`05` §5.5 Track A), and free text cannot be grouped. The mandatory note is
|
||
kept because the enumerated reason alone loses the case detail an audit needs. `ref.vocabulary_value`
|
||
rather than a dedicated `ref` table per §4.7 consolidation 2: this is a pure-label vocabulary with no
|
||
structural columns, seeded with values such as `evidence_misread`, `requirement_misinterpreted`,
|
||
`cv_out_of_date`, `context_not_in_cv`, `parse_error`, `model_disagreement`, `other` — and adding one is
|
||
an `INSERT`, not a migration.
|
||
|
||
**Indexes.**
|
||
|
||
| Index | Purpose |
|
||
|---|---|
|
||
| `uq_ats_result_override_current (ats_result_id) WHERE is_current` | The score panel's read: "is this score overridden" |
|
||
| `ix_ats_override_reason (override_reason_id, overridden_at DESC) WHERE is_current` | The override-rate-by-reason aggregate — the Track A fairness signal |
|
||
| `ix_ats_override_user (overridden_by_user_id, overridden_at DESC)` | Override rate by reviewer, which is how a single person systematically overriding one direction becomes visible |
|
||
|
||
**Profile.** *Status:* `is_current`, Tier 2. *Required:* everything except `superseded_by_id`
|
||
(§4.9 reason 1). *Soft delete:* no — forbidden, per §31.3's append-only list. *Retention:* survives
|
||
candidate pseudonymisation. `note` is the only column that could quote candidate content and is
|
||
pseudonymised with the candidate; the band, reason, actor and timestamps are retained permanently
|
||
because they are the governance record, exactly as `ai_run` metrics are (§31.2). *PII:* `internal`
|
||
for band, reason and ids; `note` is `personal` — not `sensitive_personal`, because a reason written by
|
||
a recruiter about their own judgement is not CV content, and the classification drives whether audit
|
||
payloads may carry it (§28.1). *Audit:* insert, and it is on `05` §7.3's AI event list as
|
||
"**score overridden (with reason)**" — one of the events `05` §4 alerts on. *Volume:* ~15,000 over 5
|
||
years (**assumption**: ~4% of the ~350,000 `ats_result` rows are overridden, plus a small
|
||
supersession tail). If the real rate is materially higher, that is a finding about the scoring config
|
||
rather than a capacity problem. *Queries:* the score explanation panel; override rate by reason, by
|
||
role and by requisition (the fairness signal); "every AI-influenced decision a human reversed", which
|
||
is the query an external challenge to a hiring decision starts from.
|
||
|
||
**Why not columns on `ats_result`.** Three reasons. (1) `ats_result` is an immutable snapshot of a
|
||
*computation*; an override is a later human act, and putting it on the same row means the row is no
|
||
longer a snapshot of anything. (2) The narrow `UPDATE` grant on `ats_result` would have to widen to
|
||
carry override columns, and it is the grant that makes "the AI's number stays on disk" true at the
|
||
privilege layer — widening it to admit a band write is exactly the hole the design is built to avoid.
|
||
(3) An override can itself be overridden, which a column set on a one-row-per-score table cannot
|
||
represent without losing the first one. `05` §5.2's rule is one sentence and this is the schema shape
|
||
that satisfies it: **an override is a new row, never a mutation.**
|
||
|
||
---
|
||
|
||
## 21. Fairness evaluation (Phase 3)
|
||
|
||
Separate from `scoring` because it must act as a **gate a non-engineer can verify**, and because its
|
||
readers are legal and the business (BRD §7.2).
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.evaluation_dataset` | A frozen, named set of applications used for evaluation | `key` UNIQUE, `name`, `definition_sql text`, `row_count`, `frozen_at`, `snapshot_hash bytea`, `created_by_user_id` | immutable after `frozen_at` | *Retention:* permanent. *PII:* `internal` (it holds ids, not attributes). *Volume:* ~20 |
|
||
| `app.evaluation_run` | One evaluation of one scoring config version | `scoring_config_version_id` FK, `evaluation_dataset_id` FK, `started_at`, `finished_at`, `status` CHECK in (`running`,`passed`,`failed`,`error`), `verdict_note`, `run_by_user_id`, `methodology_version` | `ix (scoring_config_version_id, finished_at DESC)`; `uq (scoring_config_version_id, evaluation_dataset_id, started_at)`; append-only | *Retention:* permanent. *Audit:* yes. *Volume:* ~200 |
|
||
| `app.evaluation_metric` | One metric value per group | `evaluation_run_id`, `metric_key` (e.g. `selection_rate`, `impact_ratio`, `score_mean`), `group_dimension`, `group_value`, `value numeric(12,6)`, `sample_size`, `threshold numeric(12,6)` NULL, `passed boolean` NULL | `uq (evaluation_run_id, metric_key, group_dimension, group_value)`; `ck (sample_size >= 0)` | *PII:* `internal` — **aggregates only, minimum cell size enforced by the job, never per-person rows**. *Volume:* ~10,000 |
|
||
|
||
**No `special_category` data is stored to make this work.** Group dimensions available in Phase 1–3
|
||
are the ones the system already holds for other reasons: source channel, location, education level,
|
||
grade applied for, and years of experience band. Protected-attribute evaluation would require
|
||
`special_category` data, which §4.10 excludes from Phase 1 by decision. **Stated plainly so nobody
|
||
believes this table delivers protected-attribute fairness testing without that separate decision.**
|
||
|
||
**The gate is a deferred constraint trigger, not a CHECK.** `app.tg_assert_activation_gate()` on
|
||
`app.scoring_config_version` fires when `activated_at` transitions from NULL and refuses unless the
|
||
referenced `evaluation_run` has `status = 'passed'`, `finished_at IS NOT NULL`, a
|
||
`scoring_config_version_id` equal to the version being activated, and an `evaluation_dataset` with
|
||
`frozen_at IS NOT NULL`. A companion trigger on `app.evaluation_run` refuses any status change away
|
||
from `passed` while an active config version references it. The single-table CHECK
|
||
`ck_activation_gate` remains as a not-null guard only — it cannot see another table's `status` and
|
||
must not be described as the gate. Full statement and reasoning in §20.1.
|
||
|
||
---
|
||
|
||
## 22. Communications
|
||
|
||
### ERD 6 — Communications and chatbot
|
||
|
||
```mermaid
|
||
erDiagram
|
||
message_template ||--|{ message_template_version : "versions"
|
||
message_template_version||--o{ outbound_message : "rendered from"
|
||
message_thread ||--o{ outbound_message : "in thread"
|
||
message_thread ||--o{ raw_intake : "replies land as"
|
||
outbound_message ||--o{ outbound_message_event : "delivery events"
|
||
candidate ||--o{ outbound_message : "recipient"
|
||
job_application |o--o{ outbound_message : "about"
|
||
candidate_email |o--o{ communication_suppression: "suppressed"
|
||
app_user ||--o{ notification : "receives"
|
||
app_user ||--o{ notification_preference : "configures"
|
||
|
||
ai_capability ||--o{ prompt_template : "for"
|
||
prompt_template ||--|{ prompt_template_version : "versions"
|
||
ai_model_config ||--|{ ai_model_config_version : "versions"
|
||
ai_capability ||--o{ ai_model_invocation : "invoked"
|
||
prompt_template_version ||--o{ ai_model_invocation : "pinned"
|
||
ai_model_config_version ||--o{ ai_model_invocation : "pinned"
|
||
app_user |o--o{ ai_model_invocation : "on behalf of"
|
||
ai_model_invocation ||--o{ ai_suggestion : "produces"
|
||
ai_model_invocation ||--o{ ai_review : "reviewed"
|
||
ai_suggestion ||--o{ ai_feedback : "rated"
|
||
conversation ||--o{ conversation_message : "messages"
|
||
conversation_message ||--o{ conversation_tool_invocation : "tool calls"
|
||
query_intent ||--o{ conversation_tool_invocation : "whitelisted as"
|
||
app_user ||--o{ conversation : "owns"
|
||
|
||
outbound_message {
|
||
bigint id PK
|
||
uuid public_id
|
||
text channel
|
||
bigint template_version_id FK
|
||
text to_address_snapshot
|
||
text subject_snapshot
|
||
text body_snapshot
|
||
bigint sent_by_user_id FK
|
||
bigint ai_run_id FK
|
||
text status
|
||
timestamptz sent_at
|
||
}
|
||
ai_model_invocation {
|
||
bigint id PK
|
||
uuid public_id
|
||
bigint ai_capability_id FK
|
||
bigint prompt_template_version_id FK
|
||
bigint model_config_version_id FK
|
||
bigint actor_user_id FK
|
||
text status
|
||
jsonb request
|
||
jsonb response
|
||
int total_tokens
|
||
numeric cost_amount
|
||
}
|
||
ai_suggestion {
|
||
bigint id PK
|
||
bigint ai_run_id FK
|
||
text subject_kind
|
||
bigint job_application_id FK
|
||
bigint candidate_id FK
|
||
text suggestion_kind
|
||
jsonb payload
|
||
text status
|
||
bigint decided_by_user_id FK
|
||
}
|
||
conversation_tool_invocation {
|
||
bigint id PK
|
||
bigint conversation_message_id FK
|
||
bigint query_intent_id FK
|
||
jsonb arguments
|
||
bigint acting_user_id FK
|
||
boolean authorization_passed
|
||
}
|
||
```
|
||
|
||
**Phasing — 1 for the minimal send path, 2 for the pipeline.** This area was previously flagged
|
||
Phase 2 as a whole, which contradicted `04` §9.1 row 5 and left Phase 1 with no way to answer a
|
||
CV it could not parse. Ruled (`00` §4 DEF-07 note, `07` §17 divergence 6, `08` §7 finding 10):
|
||
|
||
| | Phase 1 — migration `011a` | Phase 2 — migration `018` |
|
||
|---|---|---|
|
||
| Tables | `message_template`, `message_template_version` (one seeded transactional template, not an authoring UI), `message_thread`, `outbound_message`, `outbound_message_event`, `notification` | `notification_preference` |
|
||
| Behaviour | One row written **before** the provider call; `Mail.Send` through the `MailProvider` port; the `outbound:{public_id}` + `sent_at IS NULL` idempotency guard; NDR matched to its row by thread token; in-app notification rows | Template authoring and versioning UI, retry with backoff, complaint handling, digest sends, per-user preferences, the notification centre |
|
||
|
||
Seven of the eight tables are therefore Phase 1 objects. The area count in §5 reads
|
||
"1 minimal send / 2 full pipeline" for that reason, and §33.1 explains why the Phase 1 file is
|
||
numbered `011a` rather than carrying a split phase label on `018` — a phase label cannot reorder an
|
||
apply sequence, and `017b` sits in between.
|
||
|
||
**Templates are versioned; sent messages are immutable snapshots.** These are two different
|
||
obligations. A template must be editable (BRD §5.22 lists Email Templates as a settings screen;
|
||
`js/settings.js:108-115` shows six templates as inert UI). A sent message must never change, because
|
||
what the candidate received is a fact — and a rejection email whose text can be edited after the
|
||
fact is a legal problem, not a data-modelling preference.
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.message_template` | Identity of a template | `key` UNIQUE, `name`, `category` CHECK in (`application_received`,`interview_invitation`,`assessment_assignment`,`offer_letter`,`rejection`,`reference_request`,`general`) — the six from `js/settings.js:109` plus one, `current_version_id`, `is_transactional`, audit, soft delete | deferred current-version check | *Soft delete:* yes. *Retention:* permanent. *PII:* `internal`. *Audit:* yes. *Volume:* ~40 |
|
||
| `app.message_template_version` | Immutable template content | `message_template_id`, `version_no`, `channel` CHECK in (`email`,`sms`,`in_app`), `locale`, `subject_template`, `body_template`, `body_format` CHECK in (`markdown`,`html`,`text`), `declared_variables text[]`, `published_at`, `published_by_user_id`, `content_hash`, `ai_suggestion_id` NULL | `uq (message_template_id, version_no)`; `ck (channel <> 'email' OR subject_template IS NOT NULL)`; immutable | *Soft delete:* no. *Volume:* ~200 |
|
||
| `app.message_thread` | Groups an outbound message with the candidate replies that land back through intake | `id`, `candidate_id` NULL FK, `job_application_id` NULL FK, `subject_normalised`, `external_thread_ref` NULL, `last_message_at`, `message_count` | `ix (candidate_id, last_message_at DESC)`; `CREATE UNIQUE INDEX uq_message_thread_ref ON app.message_thread (external_thread_ref) WHERE external_thread_ref IS NOT NULL;` | *PII:* `personal`. *Retention:* with the candidate. *Volume:* ~180,000 |
|
||
| `app.outbound_message` | **The immutable sent record** | see below | see below | see below |
|
||
| `app.outbound_message_event` | Provider delivery events | `outbound_message_id`, `event_kind` CHECK in (`queued`,`sent`,`delivered`,`deferred`,`bounced`,`complained`,`opened`,`clicked`,`failed`), `occurred_at`, `provider_payload jsonb`, `provider_event_id` | `CREATE UNIQUE INDEX uq_outbound_event_provider ON app.outbound_message_event (outbound_message_id, provider_event_id) WHERE provider_event_id IS NOT NULL;`; `ix (outbound_message_id, occurred_at)`; append-only | *PII:* `personal`. *Retention:* 24 months. *Audit:* no. *Volume:* ~900,000 |
|
||
| `app.notification` | In-app notifications (`js/data.js:244-252`) | `user_id`, `kind_id` FK vocabulary, `title`, `body`, `candidate_id` NULL, `job_application_id` NULL, `job_id` NULL, `interview_id` NULL, `offer_id` NULL, `approval_request_id` NULL, `created_at`, `read_at`, `dismissed_at` | `ck num_nonnulls(subjects) <= 1`; `ix (user_id, created_at DESC) WHERE read_at IS NULL` | *PII:* `personal` (bodies name candidates). *Retention:* 12 months, then hard-deleted — a read notification has no historical value; the underlying event is in `audit_event`. *Audit:* no. *Volume:* ~1.5M |
|
||
| `app.notification_preference` | Per-user, per-kind, per-channel opt-in | `user_id`, `kind_id`, `channel` CHECK, `is_enabled`, `digest_frequency` CHECK in (`immediate`,`hourly`,`daily`,`off`) | PK `(user_id, kind_id, channel)` | *PII:* `internal`. *Volume:* ~3,000 |
|
||
| `app.communication_suppression` | **The do-not-send list.** Backs `GET/POST/DELETE /api/v1/suppression-list` (`06` §2.18) and the bounce/unsubscribe handling `04` §2.11 depends on | `id`, `address_normalised text NOT NULL` (the suppression is on the *address*, not the identity, because a bounce is a property of the mailbox), `candidate_email_id bigint NULL FK` (populated when the address resolves to a known candidate email, for the profile-side badge), `reason` CHECK in (`unsubscribed`,`hard_bounce`,`complaint`,`pseudonymised`,`manual`), `source` CHECK in (`provider_webhook`,`candidate_link`,`recruiter`,`retention_purge`), `added_at`, `added_by_user_id` NULL, `released_at`, `released_by_user_id` NULL, `release_reason` | `CREATE UNIQUE INDEX uq_suppression_live ON app.communication_suppression (address_normalised) WHERE released_at IS NULL;` — one live suppression per address, releasable and re-addable; `ck_suppression_release CHECK ((released_at IS NULL) = (released_by_user_id IS NULL))`; `ck_suppression_hard CHECK (released_at IS NULL OR reason <> 'complaint')` — a spam complaint is **never** releasable, which is a deliverability requirement not a policy preference; `ix (candidate_email_id)` | *Soft delete:* no — release is `released_at` and the row is the evidence. *Retention:* permanent; the address is already pseudonymised by the candidate purge, and the `reason = 'pseudonymised'` row is what stops a purged candidate being mailed. *PII:* `personal`. *Audit:* yes — adding and releasing are both decisions of record. *Volume:* ~15,000 |
|
||
|
||
**Why the suppression list is a table and not a flag on `candidate_email`.** Three reasons.
|
||
(1) A suppression frequently arrives for an address the system has never seen as a
|
||
`candidate_email` — a forwarded invitation, a typo'd address, a provider complaint on an alias — so
|
||
the row must be able to exist without a candidate. (2) `candidate_email` is soft-deletable and
|
||
merge-suppressible; a do-not-send obligation must survive both, and a flag on a row that a merge can
|
||
suppress is a flag that can be lost. (3) The send path's check must be a single indexed lookup on the
|
||
normalised address before any join, which is what makes it cheap enough to run on every send.
|
||
**The send path consults this table unconditionally** and writes `outbound_message.status =
|
||
'suppressed'` rather than skipping silently — a suppressed send is a visible outcome, not an absence.
|
||
|
||
### 22.1 `app.outbound_message` in detail
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `public_id` | `uuid` | no | UNIQUE. Used in unsubscribe/track links |
|
||
| `channel` | `text` | no | CHECK in (`email`, `sms`, `in_app`) |
|
||
| `message_template_version_id` | `bigint` | yes | NULL for a freehand message |
|
||
| `message_thread_id` | `bigint` | yes | FK |
|
||
| `candidate_id` | `bigint` | yes | FK — recipient when the recipient is a candidate |
|
||
| `recipient_user_id` | `bigint` | yes | FK — recipient when internal |
|
||
| `job_application_id` | `bigint` | yes | FK — what it is about |
|
||
| `interview_id`, `offer_id`, `assessment_assignment_id` | `bigint` | yes | FK, context |
|
||
| `to_address_snapshot` | `text` | no | The address as sent. **Not a join to `candidate_email`** |
|
||
| `from_address` | `text` | no | |
|
||
| `reply_to_address` | `text` | yes | |
|
||
| `subject_snapshot` | `text` | yes | |
|
||
| `body_snapshot` | `text` | no | The exact body sent |
|
||
| `rendered_variables` | `jsonb` | yes | The substitutions used |
|
||
| `sent_by_user_id` | `bigint` | yes | NULL for system-triggered transactional mail |
|
||
| `ai_run_id` | `bigint` | yes | FK — set when the body was AI-drafted (BRD AI-6) |
|
||
| `ai_reviewed_by_user_id` | `bigint` | yes | **An AI-drafted candidate-facing message requires a named human sender or reviewer** |
|
||
| `provider` | `text` | yes | |
|
||
| `provider_message_id` | `text` | yes | |
|
||
| `status` | `text` | no | CHECK in (`queued`, `sent`, `delivered`, `bounced`, `failed`, `suppressed`) |
|
||
| `queued_at`, `sent_at`, `delivered_at`, `bounced_at`, `failed_at` | `timestamptz` | no / yes | |
|
||
| `failure_reason` | `text` | yes | |
|
||
| `created_at` | | no | Append-only |
|
||
|
||
**Constraints.**
|
||
`ck_outbound_recipient CHECK (num_nonnulls(candidate_id, recipient_user_id) = 1)`.
|
||
`ck_outbound_sent CHECK ((status <> 'queued') = (sent_at IS NOT NULL))`.
|
||
`ck_outbound_ai_human CHECK (ai_run_id IS NULL OR sent_by_user_id IS NOT NULL OR
|
||
ai_reviewed_by_user_id IS NOT NULL)` — **no AI-drafted message leaves the system without a named
|
||
human**. `CREATE UNIQUE INDEX uq_outbound_provider ON app.outbound_message (provider, provider_message_id) WHERE provider_message_id IS NOT NULL;` (index form, §4.8).
|
||
|
||
Grants: `INSERT, SELECT` plus `UPDATE (status, sent_at, delivered_at, bounced_at, failed_at,
|
||
failure_reason, provider_message_id)` only. The body, subject and recipient snapshot can never be
|
||
rewritten.
|
||
|
||
**Indexes.** `ix_outbound_candidate (candidate_id, sent_at DESC)` — the communication log on a
|
||
candidate profile; `ix_outbound_application (job_application_id, sent_at DESC)`;
|
||
`ix_outbound_status (status, queued_at) WHERE status IN ('queued','failed')` — the send/retry queue;
|
||
`ix_outbound_thread (message_thread_id, sent_at)`.
|
||
|
||
**Profile.** *Status:* Tier 2. *Required:* channel, recipient, addresses, body, status,
|
||
`queued_at`. *Soft delete:* no. *Retention:* 24 months after send, then the body and address are
|
||
pseudonymised while the metadata (that a rejection email was sent, when, by whom) is retained —
|
||
because *that* is the defensibility record. *PII:* `sensitive_personal` (`body_snapshot` may contain
|
||
offer terms and rejection reasoning). *Audit:* insert; delivery events are not audited (they are the
|
||
`outbound_message_event` rows). *Volume:* ~600,000. *Queries:* the communication log on a candidate
|
||
or application; the send queue and retry sweep; bounce handling, which must mark
|
||
`candidate_email.verified_at` back to NULL; "did we ever tell this candidate they were rejected".
|
||
|
||
**Why the snapshot columns and not joins.** `to_address_snapshot` rather than a join to
|
||
`candidate_email` because the candidate's address changes and merges, and the record must say where
|
||
the message actually went. `body_snapshot` rather than re-rendering from the template version
|
||
because variable values change: re-rendering the interview invitation next year would produce a
|
||
different, wrong body. The redundancy is the point.
|
||
|
||
---
|
||
|
||
## 23. Interviews, feedback and scorecards
|
||
|
||
### 23.1 `app.interview`
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `public_id` | `uuid` | no | UNIQUE — appears in candidate confirmation links |
|
||
| `job_application_id` | `bigint` | no | FK. Interviews attach to the **application**, not the candidate |
|
||
| `interview_type_id` | `bigint` | no | FK vocabulary — the 7 types (BRD §9.2, `js/data.js:131`) |
|
||
| `interview_mode_id` | `bigint` | no | FK vocabulary — Video Call / On-site / Phone (`js/data.js:132`) |
|
||
| `round_no` | `int` | no | Default 1 |
|
||
| `stage_id` | `bigint` | yes | FK `ref.pipeline_stage` — which stage this interview belongs to |
|
||
| `starts_at` | `timestamptz` | no | The resolved instant |
|
||
| `ends_at` | `timestamptz` | no | |
|
||
| `scheduling_timezone` | `text` | no | IANA, validated against `pg_timezone_names` |
|
||
| `local_start_wall` | `timestamp` | no | **The only naked timestamp in the schema** — the organiser's chosen wall-clock time |
|
||
| `slot` | `tstzrange` | no | `GENERATED ALWAYS AS (tstzrange(starts_at, ends_at, '[)')) STORED` |
|
||
| `location_text` | `text` | yes | For on-site |
|
||
| `meeting_url` | `text` | yes | For video |
|
||
| `meeting_provider` | `text` | yes | |
|
||
| `status_id` | `bigint` | no | FK `ref.lifecycle_status` domain `interview`: `scheduled`, `confirmed`, `completed`, `cancelled`, `no_show`, `rescheduled` |
|
||
| `status_domain` | `text` gen. | no | Constant `'interview'` |
|
||
| `organiser_user_id` | `bigint` | no | FK |
|
||
| `scorecard_template_version_id` | `bigint` | yes | FK — pinned at scheduling |
|
||
| `reschedule_count` | `int` | no | Default 0 |
|
||
| `cancelled_reason` | `text` | yes | |
|
||
| `calendar_external_id` | `text` | yes | |
|
||
| audit + soft delete | | | |
|
||
|
||
**Constraints.** `ck_interview_times CHECK (ends_at > starts_at)`.
|
||
`ck_interview_tz` — a trigger validating `scheduling_timezone` against `pg_timezone_names` (a CHECK
|
||
cannot call a set-returning function reliably in all versions, so this is a trigger and is
|
||
documented as such). `ck_interview_mode_location CHECK (meeting_url IS NOT NULL OR location_text IS
|
||
NOT NULL OR interview_mode is phone)` — implemented as a trigger for the same reason (it needs the
|
||
vocabulary key).
|
||
|
||
**Indexes.** `ix_interview_application (job_application_id, starts_at)`;
|
||
`ix_interview_upcoming (starts_at) WHERE deleted_at IS NULL` filtered by scheduled/confirmed status
|
||
in the query — the Interviews screen's default view; GiST `ix_interview_slot USING gist (slot)`.
|
||
|
||
**Profile.** *Status:* Tier 1 + `interview_status_history`. *Required:* application, type, mode,
|
||
both instants, timezone, wall time, organiser, status. *Soft delete:* yes (cancellation is a status;
|
||
deletion is a mistake being hidden). *Retention:* with the candidate. *PII:* `personal`. *Audit:*
|
||
yes. *Volume:* ~120,000. *Queries:* upcoming interviews for a user; the interview panel view; a
|
||
candidate's interview history; no-show sweep; reminder sends; the interviewer's assigned-interview
|
||
scope (§7.5 branch 3).
|
||
|
||
**Both time columns exist for a stated reason**, repeated here because it is the rule most likely to
|
||
be "simplified" later: for a single one-off interview `starts_at` alone would suffice; the moment
|
||
there are panel slots, availability windows or a reschedule across a DST boundary, intent and
|
||
instant diverge, and recomputing intent from UTC after a tzdata release can give a different
|
||
wall-clock answer.
|
||
|
||
**The divergence risk, and its mitigation.** Storing both permits divergence if any write path
|
||
updates one without the other, and **a CHECK cannot verify the relationship because it requires
|
||
timezone resolution** (`local_start_wall AT TIME ZONE scheduling_timezone = starts_at` is not
|
||
immutable and cannot appear in a CHECK). Mitigation, required: a single scheduling service function
|
||
is the only writer, plus a nightly reconciliation job reporting rows where the relationship does not
|
||
hold.
|
||
|
||
### 23.2 Remaining interview tables
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.interview_participant` | The panel, plus the candidate | `interview_id`, `user_id` NULL FK, `candidate_id` NULL FK, `external_name` NULL, `external_email` NULL, `participant_role_id` FK vocabulary (`interviewer`,`shadow`,`coordinator`,`candidate`,`observer`), `is_required`, `response_status` CHECK in (`invited`,`accepted`,`declined`,`tentative`,`no_response`), `attended boolean` NULL, `slot tstzrange` **and `interview_is_blocking boolean NOT NULL`** — both mirrored from the interview by trigger | `ck num_nonnulls(user_id, candidate_id, external_email) = 1`; the two `EXCLUDE` constraints below | *PII:* `personal`. *Retention:* with the interview. *Audit:* yes. *Volume:* ~360,000 |
|
||
| `app.interview_slot` | Reschedule history — every proposed and superseded time | `interview_id`, `seq`, `starts_at`, `ends_at`, `scheduling_timezone`, `local_start_wall`, `state` CHECK in (`proposed`,`active`,`superseded`,`declined`), `changed_by_user_id`, `change_reason`, `created_at` | `uq (interview_id, seq)`; `CREATE UNIQUE INDEX uq_interview_slot_active ON app.interview_slot (interview_id) WHERE state = 'active';` (index form, §4.8); append-only | *PII:* `internal`. *Volume:* ~180,000 |
|
||
| `app.interview_status_history` | Status over time, trigger-written | standard shape (§9.5) | `ex` overlap | *Volume:* ~300,000 |
|
||
| `app.user_availability_rule` / `_exception` | §7.6 | | | |
|
||
|
||
**Two double-booking `EXCLUDE` constraints, and both need `interview_is_blocking`:**
|
||
|
||
```sql
|
||
-- a panel member cannot be in two overlapping LIVE interviews
|
||
ALTER TABLE app.interview_participant
|
||
ADD CONSTRAINT ex_participant_double_book EXCLUDE USING gist (
|
||
user_id WITH =, slot WITH &&
|
||
) WHERE (user_id IS NOT NULL
|
||
AND interview_is_blocking
|
||
AND response_status IN ('invited','accepted','tentative'));
|
||
|
||
-- and neither can the CANDIDATE
|
||
ALTER TABLE app.interview_participant
|
||
ADD CONSTRAINT ex_candidate_double_book EXCLUDE USING gist (
|
||
candidate_id WITH =, slot WITH &&
|
||
) WHERE (candidate_id IS NOT NULL AND interview_is_blocking);
|
||
```
|
||
|
||
**Why `interview_is_blocking` is not optional.** `response_status` describes the *participant's*
|
||
answer, not the *interview's* existence. Without a mirrored interview-level term, a **cancelled**
|
||
interview keeps blocking the panel member's calendar: `interview.status_id` (`cancelled`, `no_show`,
|
||
`rescheduled`) and `interview.deleted_at` appear nowhere in the predicate, so rescheduling into the
|
||
freed slot fails with an exclusion violation the recruiter cannot resolve from the UI — a dead end,
|
||
because the UI has no control that edits `response_status` on a cancelled interview. The mirrored
|
||
boolean is `true` only when `interview.deleted_at IS NULL` and the interview's status key is
|
||
`scheduled` or `confirmed`, so cancelling genuinely releases the time.
|
||
|
||
**Why the second constraint exists.** The first covers `user_id` only, so the candidate
|
||
(`participant_role = 'candidate'`, `candidate_id` populated) could be invited to two overlapping
|
||
panels with nothing objecting — a real and embarrassing recruiting error that no application-level
|
||
check reliably prevents under concurrent scheduling by two recruiters. It has no `response_status`
|
||
term because a candidate's non-response does not free their calendar.
|
||
|
||
**`slot` and `interview_is_blocking` are mirrored onto `interview_participant` by trigger** rather
|
||
than joined, because an `EXCLUDE` constraint can only see columns in its own table. This is a
|
||
genuine, unavoidable denormalisation. **The mirror trigger fires on `interview` INSERT and on UPDATE
|
||
OF `starts_at`, `ends_at`, `status_id` and `deleted_at`** — not on time changes alone, which was the
|
||
gap that let cancellations keep blocking. It is one of the named cases in the constraint test suite:
|
||
schedule, cancel, and assert the freed slot is immediately re-bookable by the same panel member.
|
||
|
||
### 23.3 Scorecards
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.scorecard_template` | Identity of an evaluation template. The prototype has four (`js/data.js:418-423`) | `key` UNIQUE, `name`, `department_id` NULL FK, `interview_type_id` NULL FK, `current_version_id`, audit, soft delete | deferred current-version check | *Volume:* ~20 |
|
||
| `app.scorecard_template_version` | Immutable | `scorecard_template_id`, `version_no`, `rating_scale_min`, `rating_scale_max`, `requires_overall_comment`, `recommendation_options text[]`, `published_at`, `published_by_user_id` | `uq (template_id, version_no)`; `ck (rating_scale_max > rating_scale_min)`; immutable | *Volume:* ~60 |
|
||
| `app.scorecard_template_criterion` | The criteria — 5 per template in the prototype | `scorecard_template_version_id`, `criterion_key`, `label`, `description`, `weight numeric(6,4)` NULL, `is_required`, `display_order`, `allows_na boolean` | `uq (version_id, criterion_key)`; deferred weight-sum trigger **only when any weight is non-null** (unweighted scorecards are legitimate); immutable | *Volume:* ~300 |
|
||
| `app.scorecard` | One interviewer's submission for one interview | `interview_id`, `interviewer_user_id`, `scorecard_template_version_id`, `recommendation` CHECK in (`strong_yes`,`yes`,`no`,`strong_no`,`no_decision`), `overall_rating numeric(5,2)` NULL, `overall_comment`, `is_draft`, `submitted_at`, `locked_at`, `unlock_requested_by_user_id` NULL, `unlocked_by_user_id` NULL, `unlock_reason` NULL, `ai_summary_run_id` NULL | `uq_scorecard UNIQUE (interview_id, interviewer_user_id)`; `ck_scorecard_submit CHECK (is_draft = (submitted_at IS NULL))`; `ck_scorecard_lock CHECK (locked_at IS NULL OR submitted_at IS NOT NULL)`; grants allow `UPDATE` while `is_draft`, and a trigger refuses any content update once `locked_at IS NOT NULL` unless an `unlocked_by_user_id` is being set in the same statement | *Status:* draft/submitted/locked as columns, not a status FK — three states with strict ordering are better expressed as timestamps. *PII:* `sensitive_personal` — evaluative opinions about a person. *Retention:* with the candidate; the ratings survive pseudonymisation, the free-text comments do not. *Audit:* yes, including the unlock path. *Volume:* ~340,000 |
|
||
| `app.scorecard_criterion_score` | Per-criterion rating | `scorecard_id`, `scorecard_template_criterion_id`, `rating numeric(5,2)` NULL, `is_na boolean`, `comment` | `uq (scorecard_id, criterion_id)`; `ck (is_na = (rating IS NULL))`; `ck rating within the template version's scale` — a trigger, since the scale is on the grandparent | *PII:* `sensitive_personal`. *Volume:* ~1.7M |
|
||
|
||
**Scorecards lock on submit**, and the unlock path is audited with a named actor and a reason. Without
|
||
the lock, an interviewer who hears the hiring decision can retroactively align their feedback, which
|
||
destroys the value of structured evaluation. Without an unlock path, a genuine mis-click becomes a
|
||
support ticket with no resolution. Both are modelled.
|
||
|
||
**Feedback is scorecards; there is no separate feedback table.** Ad-hoc commentary from a hiring
|
||
manager who did not interview goes in `candidate_note` scoped to the application (§13.2). A third
|
||
"feedback" entity would sit between the two with no distinct queries. **Recorded in §30.**
|
||
|
||
---
|
||
|
||
## 24. Assessments (Phase 3)
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.assessment_template` | Identity | `key` UNIQUE, `name`, `assessment_type_id` FK vocabulary — the 6 types (BRD §9.2), `current_version_id`, audit, soft delete | deferred current-version check | *Volume:* ~25 |
|
||
| `app.assessment_template_version` | Immutable definition | `assessment_template_id`, `version_no`, `duration_minutes`, `max_score numeric(8,2)`, `passing_score numeric(8,2)` NULL, `provider` CHECK in (`internal`,`external`), `provider_key` NULL, `instructions`, `is_timed`, `allows_retake`, `published_at`, `published_by_user_id` | `uq (template_id, version_no)`; **`ck_template_max_score CHECK (max_score > 0)`**; `ck (passing_score IS NULL OR passing_score <= max_score)`; immutable per §9.2's `to_jsonb` form | *Volume:* ~80 |
|
||
| `app.assessment_assignment` | One assessment given to one application | `id`, `public_id`, `job_application_id` FK, `assessment_template_version_id` FK, `assigned_by_user_id`, `assigned_at`, `due_at`, `expires_at`, `status_id` FK `ref.lifecycle_status` domain `assessment_assignment` (`pending`,`in_progress`,`completed`,`expired`,`cancelled` — the prototype's five, `js/data.js:158`), `status_domain` gen., `started_at`, `attempt_no`, `candidate_access_token_id` NULL FK, `provider_ref` NULL, `invite_message_id` NULL FK | `uq (job_application_id, assessment_template_version_id, attempt_no)`; `ck (due_at > assigned_at)`; `ix (status_id, due_at)` for the expiry sweep; `ix (job_application_id)` | *Status:* Tier 1. *PII:* `internal`. *Retention:* with the candidate. *Audit:* yes. *Volume:* ~90,000 |
|
||
| `app.assessment_result` | The outcome. **Genuinely insert-only** — the row is written when the result is *known* | `assessment_assignment_id` **UNIQUE**, `score numeric(8,2)` NULL, `max_score numeric(8,2)`, `percentage numeric(5,2)` generated (null-safe, below), `passed boolean` NULL, `submitted_at`, `graded_at`, `graded_by_user_id` NULL, `time_taken_minutes`, `breakdown jsonb`, `report_file_id` NULL FK `stored_file`, `provider_payload jsonb` NULL | `uq (assessment_assignment_id)` — one result per attempt; **`ck_result_max_score CHECK (max_score > 0)`**; `ck (score IS NULL OR score <= max_score)`; `ck_result_graded CHECK ((graded_by_user_id IS NULL) = (graded_at IS NULL))`; **`INSERT, SELECT` grants only — no `UPDATE` grant at all** | *PII:* `sensitive_personal`. *Retention:* with the candidate; the score survives, the `breakdown` and report blob do not. *Audit:* yes. *Volume:* ~70,000 |
|
||
|
||
**Two defects closed here, both worth the space.**
|
||
|
||
**(a) The pending state lives on the assignment, not on a half-written result.** An earlier draft gave
|
||
this table "append-only grants plus `UPDATE (graded_by_user_id, graded_at, passed)` for human
|
||
grading" — which does not work, because `score` was not grantable and **writing the mark is the
|
||
entire act of human grading**. A grader could record who graded and whether the candidate passed, but
|
||
not the number. Rather than widen the grant to `score` and `time_taken_minutes` (the obvious fix, and
|
||
the one that quietly turns an append-only table into a mutable one), the row is now **inserted only
|
||
when the result is known**: at submission for an auto-graded external assessment, at grading for an
|
||
`provider = 'internal'` human-graded one. The pending state is already representable —
|
||
`assessment_assignment.status_id` carries `in_progress` — so nothing is lost, and the table keeps
|
||
INSERT/SELECT grants with no exception. Consequence to implement: the grading UI reads the submission
|
||
from `assessment_assignment` plus the provider payload, and its save is an `INSERT`, not an `UPDATE`.
|
||
|
||
**(b) `percentage` must be null-safe, or the insert aborts on division by zero.**
|
||
`percentage numeric(5,2) GENERATED ALWAYS AS (round(score * 100 / max_score, 2)) STORED` raises
|
||
`ERROR: division by zero` when `max_score = 0`, and nothing forbade zero —
|
||
`ck (passing_score IS NULL OR passing_score <= max_score)` is satisfied by `0`. The `max_score > 0`
|
||
CHECK on both this table and the template version prevents it at source; the generated column is
|
||
made null-safe anyway, because a defence that depends on one CHECK never being dropped is not a
|
||
defence:
|
||
|
||
```sql
|
||
percentage numeric(5,2) GENERATED ALWAYS AS (
|
||
CASE WHEN score IS NULL OR max_score IS NULL OR max_score = 0
|
||
THEN NULL
|
||
ELSE round(score * 100 / max_score, 2)
|
||
END) STORED
|
||
```
|
||
|
||
`breakdown jsonb` holds per-section detail for display. It is legitimate JSONB under P6 — nothing
|
||
constrains it, nothing filters on it, and section structures differ per provider. If a section score
|
||
ever becomes a report dimension, it gets promoted to an `assessment_result_section` table; that is a
|
||
migration, not a JSONB query.
|
||
|
||
**Retakes are `attempt_no` on the assignment, not a nullable second result.** `allows_retake` on the
|
||
template version governs whether a second assignment may be created, and each attempt keeps its own
|
||
score, which is what a fairness review of retake policy needs.
|
||
|
||
*Queries.* The Assessments screen filtered by status and due date; the expiry sweep; a candidate's
|
||
assessment history; average score by template version (which is how you detect that a new version is
|
||
harder, not that candidates got worse); score-vs-hire-outcome correlation.
|
||
|
||
---
|
||
|
||
## 25. Offers
|
||
|
||
### ERD 7 — Offers and talent pool
|
||
|
||
```mermaid
|
||
erDiagram
|
||
job_application ||--o{ offer : "offered"
|
||
offer ||--|{ offer_version : "revisions"
|
||
offer ||--o{ offer_status_history : "status over time"
|
||
offer_version ||--o{ offer_response : "responded to"
|
||
offer_version |o--o{ approval_request : "approved via"
|
||
offer_version |o--o| stored_file : "letter"
|
||
fx_rate |o--o{ offer_version : "converted with"
|
||
currency ||--o{ offer_version : "denominated in"
|
||
talent_pool ||--o{ talent_pool_member : "members"
|
||
candidate ||--o{ talent_pool_member : "belongs to"
|
||
job_application |o--o{ talent_pool_member : "sourced from"
|
||
candidate ||--o{ candidate_job_match : "rematched"
|
||
app_user ||--o{ saved_search : "owns"
|
||
app_user ||--o{ talent_pool : "owns"
|
||
|
||
offer {
|
||
bigint id PK
|
||
uuid public_id
|
||
bigint job_application_id FK
|
||
bigint current_version_id FK
|
||
bigint status_id FK
|
||
timestamptz issued_at
|
||
}
|
||
offer_version {
|
||
bigint id PK
|
||
uuid public_id
|
||
bigint offer_id FK
|
||
int version_no
|
||
numeric base_salary_amount
|
||
char base_salary_currency_code
|
||
numeric base_salary_reporting_amount
|
||
bigint fx_rate_id FK
|
||
date start_date
|
||
date expiry_date
|
||
bigint letter_file_id FK
|
||
bytea content_hash
|
||
}
|
||
offer_response {
|
||
bigint id PK
|
||
bigint offer_version_id FK
|
||
text response
|
||
timestamptz responded_at
|
||
bigint recorded_by_user_id FK
|
||
}
|
||
talent_pool {
|
||
bigint id PK
|
||
uuid public_id
|
||
text name
|
||
bigint owner_user_id FK
|
||
boolean is_dynamic
|
||
jsonb criteria
|
||
}
|
||
talent_pool_member {
|
||
bigint id PK
|
||
bigint talent_pool_id FK
|
||
bigint candidate_id FK
|
||
bigint source_application_id FK
|
||
timestamptz added_at
|
||
timestamptz removed_at
|
||
}
|
||
fx_rate {
|
||
bigint id PK
|
||
char base_currency_code FK
|
||
char quote_currency_code FK
|
||
numeric rate
|
||
date as_of_date
|
||
}
|
||
```
|
||
|
||
**Offer amounts get immutable versions rather than a history table**, because a revised offer is a
|
||
distinct document with its own approval and its own letter — not a field edit. The status of the offer
|
||
*as a whole* gets a history table, because "sent → negotiating → accepted" is a lifecycle.
|
||
|
||
### 25.1 `app.offer`
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | |
|
||
| `public_id` | `uuid` | no | UNIQUE — appears in the candidate's acceptance link |
|
||
| `job_application_id` | `bigint` | no | FK |
|
||
| `current_version_id` | `bigint` | yes | FK, `DEFERRABLE`; deferred check that it is set at COMMIT |
|
||
| `status_id` | `bigint` | no | FK `ref.lifecycle_status` domain `offer`: Draft, Sent, Negotiating, Accepted, Declined, Expired (BRD §9.2, `js/data.js:169`) |
|
||
| `status_domain` | `text` gen. | no | Constant `'offer'` |
|
||
| `status_key` | `text` | no | Trigger-maintained, read by the UI and by reporting |
|
||
| `status_is_terminal` | `boolean` | no | Trigger-maintained from `ref.lifecycle_status.is_terminal` — the **same pattern as `job_application`** (§4.6, §15.1), so the live-offer index predicate is not a hardcoded exclusion list |
|
||
| `status_is_negative` | `boolean` | no | Trigger-maintained from `ref.lifecycle_status.is_negative` |
|
||
| `issued_at` | `timestamptz` | yes | Set when a human issues. **Never set by a job** |
|
||
| `issued_by_user_id` | `bigint` | yes | |
|
||
| `responded_at` | `timestamptz` | yes | |
|
||
| `closed_at` | `timestamptz` | yes | |
|
||
| audit + soft delete | | | |
|
||
|
||
**Constraints.** One live offer per application, while permitting a fresh offer after a decline —
|
||
written as a partial unique **index** (§4.8) whose predicate reads the denormalised `ref` boolean, not
|
||
a hardcoded key list:
|
||
|
||
```sql
|
||
CREATE UNIQUE INDEX uq_offer_live ON app.offer (job_application_id)
|
||
WHERE NOT status_is_terminal AND deleted_at IS NULL;
|
||
```
|
||
|
||
`ck_offer_issued CHECK ((issued_at IS NULL) = (issued_by_user_id IS NULL))` — **issuing an offer is
|
||
always a human act; there is no system principal that can issue.**
|
||
|
||
**Why the predicate is `NOT status_is_terminal` and not `status_key NOT IN ('declined','expired')`.**
|
||
Identical reasoning to §15.1's `state`, and the same concrete failure: the hardcoded exclusion list
|
||
puts the terminal set in DDL, so seeding a new terminal offer status — `rescinded` is the obvious one,
|
||
and `superseded_by_revision` is arguable — silently leaves those offers counting as *live* and blocks
|
||
a fresh offer on the application forever. `ref.lifecycle_status` already carries `is_terminal` for the
|
||
`offer` domain (`accepted`, `declined`, `expired` true; `draft`, `sent`, `negotiating` false), so the
|
||
boolean is free and the rule becomes a data change plus a bounded backfill rather than an index
|
||
rebuild. Note that `accepted` is terminal here, which is correct: an accepted offer must also block a
|
||
second live offer on the same application.
|
||
|
||
**Indexes.** `ix_offer_status (status_id, closed_at)`; `ix_offer_application
|
||
(job_application_id)`.
|
||
|
||
**Profile.** *Status:* Tier 1 + `offer_status_history`. *Soft delete:* yes (a draft created in error).
|
||
*Retention:* permanent for accepted offers (they are employment records); 24 months for declined and
|
||
expired, then pseudonymised with the candidate. *PII:* `internal` in itself. *Audit:* yes, high value.
|
||
*Volume:* ~12,000. *Queries:* the Offers screen by status; offer acceptance rate by department and
|
||
grade; expiring-offer sweep; time-from-offer-to-acceptance.
|
||
|
||
### 25.2 `app.offer_version`
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id`, `public_id` | | no | |
|
||
| `offer_id` | `bigint` | no | FK |
|
||
| `version_no` | `int` | no | UNIQUE with `offer_id` |
|
||
| `base_salary_amount` | `numeric(14,2)` | no | |
|
||
| `base_salary_currency_code` | `char(3)` | no | FK `ref.currency` |
|
||
| `base_salary_period` | `text` | no | CHECK in (`annual`, `monthly`, `hourly`, `daily`) — a bare number is ambiguous, and the prototype's `base` (`js/data.js:175`) is implicitly annual USD with nothing saying so |
|
||
| `signing_bonus_amount` / `_currency_code` | | yes | |
|
||
| `annual_bonus_pct` | `numeric(5,2)` | yes | CHECK 0–100. The prototype stores `bonus: int(5,25) + '%'` as a string (`js/data.js:177`) |
|
||
| `equity_units` | `numeric(14,2)` | yes | |
|
||
| `equity_instrument` | `text` | yes | CHECK in (`rsu`, `option`, `phantom`, `none`) |
|
||
| `equity_vesting_months` | `int` | yes | |
|
||
| `allowances` | `jsonb` | yes | Named allowances. Display-only under P6 |
|
||
| `benefits_summary` | `text` | yes | |
|
||
| `employment_type_id`, `grade_id`, `location_id`, `department_id` | `bigint` | no / yes | FKs — the offer's own terms, which may differ from the requisition's |
|
||
| `reports_to_user_id` | `bigint` | yes | |
|
||
| `start_date` | `date` | no | |
|
||
| `expiry_date` | `date` | no | |
|
||
| `notice_period_days` | `int` | yes | |
|
||
| `probation_months` | `int` | yes | |
|
||
| `base_salary_reporting_amount` / `_currency_code` | | yes | Converted for reporting. **Currency-normalised but NOT period-normalised** — never rank or band on this |
|
||
| `annualisation_hours_per_year` | `int` | yes | Required when `base_salary_period = 'hourly'`, else NULL. Default 2080. **The assumption is pinned on the row, not implied by the code that read it** |
|
||
| `annualisation_days_per_year` | `int` | yes | Required when `base_salary_period = 'daily'`, else NULL. Default 260 |
|
||
| `annualisation_months_per_year` | `int` | yes | Required when `base_salary_period = 'monthly'`, else NULL. Default 12 |
|
||
| `base_salary_annualised_reporting_amount` | `numeric(14,2)` | yes | **The column every banding, ranking and variance query reads.** Computed at write time from `base_salary_amount`, `base_salary_period`, the annualisation basis above and `fx_rate_id`. Stored, not derived at read, for the same reason as every other pinned figure: a re-run report must show the same number |
|
||
| `fx_rate_id` | `bigint` | yes | FK `app.fx_rate` — the pinned rate |
|
||
| `letter_file_id` | `bigint` | yes | FK `app.stored_file` — the generated letter |
|
||
| `letter_ai_run_id` | `bigint` | yes | FK — set when the letter was AI-generated (BRD AI-7) |
|
||
| `approval_request_id` | `bigint` | yes | FK |
|
||
| `content_hash` | `bytea` | no | |
|
||
| `change_reason` | `text` | yes | Required for `version_no > 1` |
|
||
| `created_at`, `created_by_user_id` | | no | `created_by_user_id` **NOT NULL** — §4.2 |
|
||
|
||
**Constraints.** `uq_offer_version UNIQUE (offer_id, version_no)`. Money pair CHECKs on all four
|
||
pairs (§4.4).
|
||
|
||
**The date CHECK, and the generated column it depends on.** `expiry_date` and `start_date` must not
|
||
precede the offer's own creation day. `created_at::date` is **not immutable** and cannot appear in a
|
||
CHECK, so the comparison is against a stored generated column — but the obvious spelling of *that*
|
||
fails for the identical reason, because the `timestamptz → date` cast depends on the session
|
||
`TimeZone`:
|
||
|
||
```sql
|
||
-- FAILS: ERROR: generation expression is not immutable
|
||
created_on date GENERATED ALWAYS AS ((created_at)::date) STORED
|
||
|
||
-- WORKS: the zone is a literal, so the expression is immutable
|
||
created_on date NOT NULL GENERATED ALWAYS AS (((created_at AT TIME ZONE 'UTC'))::date) STORED,
|
||
CONSTRAINT ck_offer_dates CHECK (expiry_date >= created_on AND start_date >= created_on)
|
||
```
|
||
|
||
**Offer dates are therefore evaluated in UTC, and that is a product decision, not an accident**
|
||
(§4.3). The boundary case worth stating to the business: an offer created at 05:30 Asia/Karachi is
|
||
00:30 UTC on the same date, so a same-day expiry behaves as the recruiter expects; one created at
|
||
04:00 Karachi on the 3rd is 23:00 UTC on the **2nd**, so `expiry_date = 3rd` is accepted and
|
||
`expiry_date = 2nd` is also accepted. The alternative — storing the creating user's zone on the row
|
||
and comparing in it — was rejected because the zone would then be an input to an immutability
|
||
constraint on an immutable table, and because offer expiry is a legal date read off a letter, which is
|
||
better anchored to one zone than to whoever happened to type it.
|
||
|
||
`ck_offer_fx CHECK ((fx_rate_id IS NULL) = (base_salary_reporting_amount IS NULL))`.
|
||
`ck_offer_annualised CHECK ((base_salary_reporting_amount IS NULL) = (base_salary_annualised_reporting_amount IS NULL))`.
|
||
`ck_offer_annualisation_basis CHECK (
|
||
(base_salary_period = 'hourly') = (annualisation_hours_per_year IS NOT NULL) AND
|
||
(base_salary_period = 'daily') = (annualisation_days_per_year IS NOT NULL) AND
|
||
(base_salary_period = 'monthly') = (annualisation_months_per_year IS NOT NULL))`.
|
||
`ck_offer_equity CHECK ((equity_units IS NULL) = (equity_instrument IS NULL OR equity_instrument =
|
||
'none'))`. INSERT/SELECT-only grants plus the §9.2 `to_jsonb`-form immutability trigger.
|
||
|
||
**Indexes.** `ix_offer_version_offer (offer_id, version_no DESC)`;
|
||
`ix_offer_version_start (start_date)` for start-date planning;
|
||
`ix_offer_version_grade (grade_id, base_salary_annualised_reporting_amount)` for compensation banding.
|
||
|
||
**Why the banding index is on the annualised column, not `base_salary_reporting_amount`.** Converting
|
||
currency does **not** normalise period. An index on `(grade_id, base_salary_reporting_amount)` sorts
|
||
an hourly rate of USD 65 next to an annual salary of USD 135,000 as if the first were the lower-paid
|
||
offer, so the banding report — and the approval-escalation variance query below — are simply wrong,
|
||
silently, in whichever direction the mix of period values happens to fall. Two consequences of fixing
|
||
it properly:
|
||
|
||
- **`job_version` needs `salary_period` too** (§9.2), for exactly the reason this section already gave
|
||
for the offer: a bare number is ambiguous. Without it, "offer-vs-requisition-range variance, which
|
||
is how you find approvals that should have been escalated" compares a periodised offer against a
|
||
period-less range, and the `approval_route.amount_threshold_amount` control (§10) that depends on
|
||
that variance inherits the same error.
|
||
- **The approval threshold is compared against the annualised reporting amount**, stated in §10.
|
||
|
||
**Both sides of the variance query read annualised reporting amounts**, so like is compared with
|
||
like:
|
||
|
||
```sql
|
||
-- offers that landed above their requisition's range: the escalation check
|
||
SELECT o.public_id, ov.base_salary_annualised_reporting_amount, jv.salary_max_annualised_amount
|
||
FROM app.offer_version ov
|
||
JOIN app.offer o ON o.id = ov.offer_id AND o.current_version_id = ov.id
|
||
JOIN app.job_application ja ON ja.id = o.job_application_id
|
||
JOIN app.job_version jv ON jv.id = ja.job_version_id
|
||
WHERE jv.salary_max_annualised_amount IS NOT NULL
|
||
AND ov.base_salary_annualised_reporting_amount > jv.salary_max_annualised_amount;
|
||
```
|
||
|
||
**Profile.** *Status:* none — the `offer` carries it. *Required:* base salary pair and period, both
|
||
dates, org FKs, hash, creator. *Soft delete:* no. *Retention:* permanent for the accepted version.
|
||
*PII:* **`sensitive_personal`** — this is compensation data, the most sensitive non-special-category
|
||
data in the system, and the classification drives column-level access in §31. *Audit:* insert only,
|
||
but see the audit-PII rule: for `sensitive_personal` columns the audit payload stores a hash and the
|
||
fact of change, not the value. *Volume:* ~18,000. *Queries:* the current offer terms; the revision
|
||
diff; compensation banding by grade and department; offer-vs-requisition-range variance, which is how
|
||
you find approvals that should have been escalated.
|
||
|
||
### 25.3 `app.offer_status_history` and `app.offer_response`
|
||
|
||
| Table | Purpose | Key columns | Constraints | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.offer_status_history` | Status over time, trigger-written | standard shape (§9.5) | `ex` overlap on `(offer_id, tstzrange)` | *Volume:* ~50,000 |
|
||
| `app.offer_response` | The candidate's act, recorded once per version | `offer_version_id`, `response` CHECK in (`accepted`,`declined`,`counter_proposed`,`no_response`), `responded_at`, `channel` CHECK in (`portal`,`email`,`verbal`,`letter`), `counter_notes`, `decline_reason_id` NULL FK, `recorded_by_user_id` NULL, `candidate_access_token_id` NULL FK, `raw_intake_id` NULL FK | `uq (offer_version_id)`; `ck_response_provenance CHECK (recorded_by_user_id IS NOT NULL OR candidate_access_token_id IS NOT NULL)` — a response with neither a recording recruiter nor an authenticated candidate token has no provenance; append-only | *PII:* `personal`; `counter_notes` is `sensitive_personal`. *Audit:* yes. *Volume:* ~12,000 |
|
||
|
||
**`ck_response_provenance` is worth the awkwardness.** The realistic paths are: the candidate clicks
|
||
Accept on a tokenised page (token id present), or a recruiter records a verbal acceptance (user id
|
||
present). A row with neither is a data-entry artefact, and offer acceptance is not something to be
|
||
vague about.
|
||
|
||
---
|
||
|
||
## 26. Talent pool, saved searches, currency and FX
|
||
|
||
### 26.1 Talent pool
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.talent_pool` | A named collection of candidates to re-surface (BRD §5, "retain and re-surface previously sourced candidates not hired for their original role") | `id`, `public_id`, `key` UNIQUE, `name`, `description`, `owner_user_id`, `is_dynamic boolean`, `criteria jsonb` NULL, `department_id` NULL, `is_shared`, `last_rematch_at`, audit, soft delete | `ck_pool_dynamic CHECK (is_dynamic = (criteria IS NOT NULL))`; `ix (owner_user_id) WHERE deleted_at IS NULL` | *Soft delete:* yes. *Retention:* permanent (membership is purged with the candidate). *PII:* `internal`. *Audit:* yes. *Volume:* ~200 |
|
||
| `app.talent_pool_member` | Membership with provenance and a removal record | `talent_pool_id`, `candidate_id`, `source_application_id` NULL FK, `added_by_user_id`, `added_at`, `reason_id` NULL FK `ref.rejection_reason`, `note`, `removed_at`, `removed_by_user_id`, `last_contacted_at` | `CREATE UNIQUE INDEX uq_pool_member ON app.talent_pool_member (talent_pool_id, candidate_id) WHERE removed_at IS NULL;` (index form, §4.8); `ix (candidate_id) WHERE removed_at IS NULL` | *Soft delete:* `removed_at` serves the purpose; no separate `deleted_at`. *Retention:* purged with the candidate. *PII:* `internal`. *Audit:* yes — adding someone to a pool is a processing decision that needs a lawful basis. *Volume:* ~80,000 |
|
||
| `app.saved_search` | Saved filter sets, per user or shared. The prototype has four (`js/data.js:410-415`) | `owner_user_id`, `name`, `entity_kind` CHECK in (`candidate`,`job_application`,`job`,`interview`,`offer`), `filters jsonb`, `is_shared`, `last_run_at`, `last_result_count`, soft delete | `CREATE UNIQUE INDEX uq_saved_search_name ON app.saved_search (owner_user_id, name) WHERE deleted_at IS NULL;` (index form, §4.8); `ix (owner_user_id)` | *Soft delete:* yes. *Retention:* permanent. *PII:* `internal` — **but a filter can encode a discriminatory query, so saved searches are readable by compliance.** *Audit:* creation and execution. *Volume:* ~1,000 |
|
||
|
||
**Dynamic pools are a stored query, not stored membership**, and `criteria jsonb` is display/execution
|
||
data only — no constraint or index references it (P6). A dynamic pool's members are computed at read
|
||
time through the same filter builder the Candidates screen uses, so there is exactly one filter
|
||
implementation and one authorization path. Static pools use `talent_pool_member` rows.
|
||
|
||
**Rematch results are `candidate_job_match` (§20.6)**, not a pool-specific table, because the
|
||
question "how well does this pool member fit this open requisition" is the same question cross-brand
|
||
matching asks and must produce the same, pinned, explainable answer. **Recorded in §30.**
|
||
|
||
### 26.2 `app.fx_rate` and `ref.currency`
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `ref.currency` | ISO 4217 | **PK `code char(3)`**, `name`, `minor_unit smallint` CHECK 0–4, `symbol`, `is_active`, `is_reporting_currency` | exactly one reporting currency, as a partial unique **index** on a constant expression (§4.8): `CREATE UNIQUE INDEX uq_currency_reporting ON ref.currency ((true)) WHERE is_reporting_currency;`; `ck (code ~ '^[A-Z]{3}$')` | *Volume:* ~20 |
|
||
| `app.fx_rate` | Point-in-time rates, pinned by every conversion | `base_currency_code` FK, `quote_currency_code` FK, `rate numeric(18,10)` CHECK > 0, `as_of_date`, `source` CHECK in (`ecb`,`manual`,`provider`), `fetched_at`, `created_by_user_id` NULL | `uq_fx_rate UNIQUE (base_currency_code, quote_currency_code, as_of_date, source)`; `ck (base <> quote)`; `ix (quote_currency_code, as_of_date DESC)` for "the rate in force on this date"; append-only | *Soft delete:* no. *Retention:* permanent — a report must be reproducible, which requires the rate that was used. *PII:* `internal`. *Audit:* insert only. *Volume:* ~40,000 (**assumption**: 8 currency pairs, daily, 5 years plus manual corrections) |
|
||
|
||
**A wrong rate is corrected by inserting a new rate and recomputing the reporting columns**, never by
|
||
updating the rate row — because the original conversion is already on a board report. The original
|
||
amount and currency on the domain row are never touched.
|
||
|
||
**`is_reporting_currency` on `ref.currency`** is how the reporting-column conversions know their
|
||
target. **Assumption:** one group reporting currency. If Utopia Brands ever reports in two, this
|
||
becomes a small `reporting_currency` table and the offer's single reporting pair becomes a child
|
||
table — a contained change, flagged here so it is not a surprise.
|
||
|
||
---
|
||
|
||
## 27. AI orchestration and chatbot (`ai` schema)
|
||
|
||
All model access is funnelled through this schema. Its shape is what makes three governance
|
||
requirements structural rather than documentary: every AI output is versioned and addressable, AI
|
||
never writes domain state, and the chatbot cannot exceed the asking user's permissions.
|
||
|
||
### 27.1 Registry and versioning
|
||
|
||
| Table | Purpose | Key columns | Constraints | Profile |
|
||
|---|---|---|---|---|
|
||
| `ai.ai_capability` | The 15 named capabilities (BRD §7, `js/data.js:448-464`), with **true per-capability availability** | `key` UNIQUE, `name`, `description`, `availability` CHECK in (`unavailable`,`internal_alpha`,`beta`,`general`), `phase smallint`, `is_enabled`, `requires_human_review boolean`, **`allows_system_actor boolean NOT NULL DEFAULT false`**, `max_cost_per_call_amount`/`_currency_code`, `rate_limit_per_user_hour` | money pair CHECK. **`allows_system_actor` defaults to `false`, so a newly seeded capability is human-attributed until someone deliberately says otherwise**, and it must be `false` for every capability that answers a question or returns data to a person — see §27.2 | *Volume:* 15. **`availability` exists because the prototype's AI Studio shows all 15 as Beta or Coming Soon (`js/data.js:448-464`), which creates a stakeholder expectation that they are nearly done. Only three are Phase 1** |
|
||
| `ai.prompt_template` | Identity of a prompt | `ai_capability_id` FK, `key` UNIQUE, `name`, `current_version_id` | | *Volume:* ~30 |
|
||
| `ai.prompt_template_version` | Immutable prompt text | `prompt_template_id`, `version_no`, `system_prompt`, `user_prompt_template`, `output_schema jsonb` NULL, `declared_variables text[]`, `published_at`, `published_by_user_id`, `content_hash` | `uq (prompt_template_id, version_no)`; immutable | *Volume:* ~150 |
|
||
| `ai.ai_model_config` | Identity of a model configuration | `key` UNIQUE, `name`, `provider` CHECK in (`api_provider`,`private_endpoint`,`self_hosted`), `current_version_id` | | *Volume:* ~10 |
|
||
| `ai.ai_model_config_version` | Immutable model settings | `ai_model_config_id`, `version_no`, `model_id text`, `model_version text`, `temperature numeric(3,2)`, `max_output_tokens`, `parameters jsonb`, `input_cost_per_1k_amount`/`_currency_code`, `output_cost_per_1k_amount`/`_currency_code`, `data_processing_agreement_ref`, `retains_data boolean`, `published_at` | `uq (config_id, version_no)`; money pair CHECKs; immutable | *Volume:* ~40 |
|
||
|
||
**`retains_data` and `data_processing_agreement_ref` are columns, not documentation.** BRD §7.4
|
||
requires that candidate data not leave controlled infrastructure and flags model hosting as the
|
||
decision with the widest downstream impact (BRD "DECISION REQUIRED"). Recording per-model-version
|
||
whether the provider retains data makes "which candidate data went to a retaining provider" a query
|
||
rather than an investigation. **Model hosting itself is an open question, labelled as such: Phase 1
|
||
assumes a contracted API provider under a data-processing agreement.**
|
||
|
||
### 27.2 `ai.ai_model_invocation` — the run ledger
|
||
|
||
*Purpose.* One row per model call, written **before its result is usable**, so every AI output is
|
||
addressable and versioned.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id`, `public_id` | | no | |
|
||
| `ai_capability_id` | `bigint` | no | FK |
|
||
| `prompt_template_version_id` | `bigint` | yes | NULL for a raw call with no template |
|
||
| `ai_model_config_version_id` | `bigint` | no | FK |
|
||
| `actor_user_id` | `bigint` | yes | **The human on whose behalf the call is made.** `NOT NULL` for `actor_kind IN ('user','ai_agent')`, and NULL **only** for `actor_kind = 'system'` — see the actor discussion below |
|
||
| `actor_kind` | `text` | no | CHECK in (`user`, `ai_agent`, `system`). `ai_agent` means the chatbot acting *for* `actor_user_id`, never instead of them. `system` means unattended pipeline work with no human present, and is tightly constrained |
|
||
| `trigger_kind` | `text` | no | CHECK in (`interactive`, `batch`, `webhook`, `scheduled`) |
|
||
| `subject_kind` | `text` | yes | CHECK — what it was about |
|
||
| `job_application_id`, `candidate_id`, `job_version_id`, `candidate_document_id`, `conversation_message_id` | `bigint` | yes | Typed nullable subject, `num_nonnulls <= 1` |
|
||
| `request` | `jsonb` | no | The rendered request. Legitimate JSONB |
|
||
| `response` | `jsonb` | yes | Raw response |
|
||
| `status` | `text` | no | CHECK in (`running`, `succeeded`, `failed`, `refused`, `timeout`, `rate_limited`, `circuit_open`) |
|
||
| `error` | `jsonb` | yes | |
|
||
| `input_tokens`, `output_tokens`, `total_tokens` | `int` | yes | |
|
||
| `cost_amount` | `numeric(18,8)` | yes | **The one documented exception to §4.4's `numeric(14,2)` and to the minor-unit rounding trigger** (§4.4 rule 6). A single invocation costs fractions of a cent; at `numeric(14,2)` every row rounds to `0.00` and `ix_run_cost` returns zeros |
|
||
| `cost_currency_code` | `char(3)` | yes | FK `ref.currency`. Retained — the exception is to the scale and the rounding rule, not to the currency rule |
|
||
| `latency_ms` | `int` | yes | |
|
||
| `started_at`, `finished_at` | | no / yes | |
|
||
| `request_id` | `uuid` | yes | Correlates to `audit.audit_event` |
|
||
| `created_at` | | no | Append-only |
|
||
|
||
**Constraints.**
|
||
|
||
```sql
|
||
CONSTRAINT ck_run_terminal CHECK ((status <> 'running') = (finished_at IS NOT NULL)),
|
||
CONSTRAINT ck_run_error CHECK (status NOT IN ('failed','timeout','refused') OR error IS NOT NULL),
|
||
CONSTRAINT ck_run_money CHECK ((cost_amount IS NULL) = (cost_currency_code IS NULL)),
|
||
|
||
-- a human-attributed run has a human; a system run must not fake one
|
||
CONSTRAINT ck_run_actor_pairing CHECK
|
||
((actor_kind IN ('user','ai_agent')) = (actor_user_id IS NOT NULL)),
|
||
|
||
-- a system run is unattended by definition, so it can never be interactive
|
||
CONSTRAINT ck_run_system_trigger CHECK
|
||
(actor_kind <> 'system' OR (actor_user_id IS NULL
|
||
AND trigger_kind IN ('batch','scheduled','webhook')))
|
||
```
|
||
|
||
plus a deferred constraint trigger asserting the third rule, which a CHECK cannot express because it
|
||
reads another table: **a `system` run is permitted only for a capability flagged
|
||
`ai_capability.allows_system_actor = true`.**
|
||
|
||
**Grants: insert, one closing update, then genuinely immutable.** An earlier draft said "append-only
|
||
grants — nothing may update a run, ever", which cannot be reconciled with the rest of this very
|
||
section: the row is written **before** its result is usable with `status = 'running'`, and
|
||
`ck_run_terminal` requires `finished_at` to be set the moment `status` leaves `'running'`. The
|
||
documented lifecycle (`02` §5: "open AiRun row BEFORE the call, status=running" … "close AiRun, tokens,
|
||
cost, latency, status=succeeded") *requires* writing `status`, `response`, `finished_at`, `latency_ms`,
|
||
the three token columns, `cost_amount`, `cost_currency_code` and `error`. With INSERT/SELECT-only
|
||
grants **no run could ever leave `running`**, and the ledger the whole governance story rests on could
|
||
not function. The narrow closing grant, in exactly the form §12.2 uses for `raw_intake.state` and
|
||
§20.4 for `ats_result.is_current`:
|
||
|
||
```sql
|
||
REVOKE UPDATE, DELETE ON ai.ai_model_invocation FROM talentflow_app;
|
||
GRANT SELECT, INSERT ON ai.ai_model_invocation TO talentflow_app;
|
||
GRANT UPDATE (status, response, error, input_tokens, output_tokens, total_tokens,
|
||
cost_amount, cost_currency_code, latency_ms, finished_at)
|
||
ON ai.ai_model_invocation TO talentflow_app;
|
||
|
||
-- a closed run can never be reopened or rewritten
|
||
CREATE TRIGGER tg_run_no_reopen
|
||
BEFORE UPDATE ON ai.ai_model_invocation
|
||
FOR EACH ROW
|
||
WHEN (OLD.status <> 'running')
|
||
EXECUTE FUNCTION app.tg_raise_immutable();
|
||
|
||
-- and no run is ever deleted, closed or not
|
||
CREATE TRIGGER tg_run_no_delete
|
||
BEFORE DELETE ON ai.ai_model_invocation
|
||
FOR EACH ROW EXECUTE FUNCTION app.tg_raise_immutable();
|
||
```
|
||
|
||
Two triggers rather than one `BEFORE UPDATE OR DELETE`, per §4.8's fourth syntax fact: a `WHEN` clause
|
||
cannot read `TG_OP`, so the `DELETE` arm cannot be expressed as a disjunct and becomes its own
|
||
unconditional trigger. `OLD.status <> 'running'` is legal in the `UPDATE` half because `OLD` is
|
||
available there.
|
||
|
||
**The correct wording, and it is a meaningful difference:** *a run may be closed once and never
|
||
reopened.* Not "nothing may update a run, ever". The guarantee that matters for forensics is that a
|
||
**terminal** run is frozen — which the trigger delivers at the row level, independently of the grant —
|
||
and that guarantee is now real instead of being contradicted by the lifecycle two paragraphs away.
|
||
§4.2's append-only list and §31.3's forbidden-write list both name this exception explicitly.
|
||
|
||
**`actor_kind = 'system'` exists because the Phase 1 intake pipeline has no human present, and
|
||
pretending otherwise would have meant inserting a fake user.** `04` §4 line 831 reads
|
||
`WK->>AIO: invoke document_classification (actor = system, capability-scoped)` and line 838 does the
|
||
same for `cv_field_extraction`; the nightly cross-brand scan (§20.6 `origin = 'cross_brand_scan'`) and
|
||
every `trigger_kind IN ('batch','scheduled')` run have no `app_user` either. With
|
||
`actor_user_id NOT NULL` and `actor_kind` restricted to (`user`,`ai_agent`), **no AI-assisted parse
|
||
could write a ledger row at all**, so document classification and field extraction — the two
|
||
capabilities the Phase 1 vertical slice depends on — could not run.
|
||
|
||
The chatbot guarantee survives intact, and this is the part to read carefully:
|
||
|
||
| Rule | Where |
|
||
|---|---|
|
||
| A `system` run has **no** `actor_user_id` — it cannot borrow or impersonate one | `ck_run_actor_pairing` |
|
||
| A `system` run can never be `interactive` | `ck_run_system_trigger` |
|
||
| A `system` run is permitted **only** on a capability with `allows_system_actor = true`, and that flag is `false` for **every** assistant, answering, ranking-for-display and summarisation capability | deferred trigger against `ai.ai_capability` |
|
||
| Where a human *did* cause the work — a manual CV upload, a publish-triggered rescore, a recruiter-requested match — the pipeline **propagates that user** and must not fall back to `system` | service rule, asserted by a test that uploads a CV as a named recruiter and checks `actor_user_id` on the resulting run |
|
||
|
||
`allows_system_actor` is thus the whole boundary: it is `true` for `document_classification`,
|
||
`cv_field_extraction` and the batch matching capabilities (none of which answer a question or return
|
||
data to a person), and `false` for everything the chatbot can reach. A chatbot answer therefore still
|
||
cannot be produced by a principal with no permissions, because a chatbot capability can never carry
|
||
`actor_kind = 'system'`. **`04` line 831 and this section now agree, and the AI-boundary decision in
|
||
`_decisions.md` — which states the absolute form, "there is no service account and no system
|
||
principal" — needs the same amendment: there is no system principal *for any capability that returns
|
||
data to a user*, which is the property the constraint was protecting.**
|
||
|
||
**Indexes.** `ix_run_capability_time (ai_capability_id, started_at DESC)`;
|
||
`ix_run_actor (actor_user_id, started_at DESC)`; `ix_run_subject_application
|
||
(job_application_id) WHERE job_application_id IS NOT NULL`;
|
||
`ix_run_cost (started_at) WHERE cost_amount IS NOT NULL` for the cost rollup;
|
||
`ix_run_failed (started_at DESC) WHERE status <> 'succeeded'`.
|
||
|
||
**Profile.** *Status:* Tier 2. *Required:* capability, model version, actor, request, status,
|
||
`started_at`. *Soft delete:* no. *Retention:* 24 months, then `request` and `response` are
|
||
pseudonymised while the metadata (capability, model version, tokens, cost, latency, outcome) is
|
||
retained permanently — the metrics are the model-governance record and contain no PII. *PII:*
|
||
`request`/`response` are `sensitive_personal` (they carry CV text). *Audit:* the run row *is* the
|
||
record; additionally every AI-influenced *decision* writes an `audit_event` carrying `ai_run_id`.
|
||
*Volume:* ~1.2M. *Queries:* "every AI-influenced decision on this candidate" (join through
|
||
`ai_suggestion` and `ats_result.ai_run_id`); cost per capability per month; failure rate per model
|
||
version, which is the signal that a provider changed something; the AI Studio availability screen.
|
||
|
||
**The `actor_user_id` / `allows_system_actor` pair is the single most important thing in this schema
|
||
for the chatbot constraint.** `invoke()` takes the human actor and calls `iam.can()` with that actor.
|
||
For every capability the chatbot can reach, `allows_system_actor = false`, so `actor_user_id` is
|
||
`NOT NULL` by constraint and there is no service account to borrow — a chatbot answer cannot contain
|
||
data the asking user could not already see. The standard mistake — a service account with post-hoc
|
||
filtering — fails the first time the filter has a bug, and is a direct violation of the access-control
|
||
constraint. `actor_kind = 'system'` does not reopen that door, because a `system` run is confined by
|
||
constraint to unattended, non-answering capabilities that return nothing to a person.
|
||
|
||
**`ref.currency` FK and the cost exception, restated in one line** so nobody "tidies" the column back
|
||
to `numeric(14,2)`: cost is a *metered internal consumption* figure, not a payable amount, and the
|
||
minor-unit rounding trigger deliberately does not apply to it (§4.4 rule 6). The monthly rollup rounds
|
||
to minor units at presentation. `job_posting_metric.spend_amount` is the opposite case — a real invoice
|
||
from an external platform — and stays `numeric(14,2)` under the ordinary rules.
|
||
|
||
### 27.3 Suggestions, review and feedback
|
||
|
||
| Table | Purpose | Key columns | Constraints | Profile |
|
||
|---|---|---|---|---|
|
||
| `ai.ai_suggestion` | **The structural boundary: AI output is a suggestion record, never a domain write** | `ai_run_id` FK, `suggestion_kind` CHECK in (`shortlist_rank`,`stage_advance`,`skill_extraction`,`jd_draft`,`email_draft`,`interview_questions`,`candidate_match`,`summary`,`next_action`), `subject_kind` CHECK, typed nullable subject FKs (`job_application_id`, `candidate_id`, `job_version_id`, `talent_pool_id`), `payload jsonb`, `confidence numeric(4,3)` NULL, `status` CHECK in (`pending`,`accepted`,`rejected`,`superseded`,`expired`), `decided_by_user_id` NULL, `decided_at` NULL, `decision_note`, `expires_at` | `ck num_nonnulls(subjects) = 1`; `ck_suggestion_decided CHECK ((status IN ('accepted','rejected')) = (decided_by_user_id IS NOT NULL))`; `ix (status, subject) WHERE status = 'pending'` | *Soft delete:* no. *Retention:* 12 months for undecided; decided suggestions retained with the candidate. *PII:* `payload` is `sensitive_personal`. *Audit:* yes on the decision. *Volume:* ~800,000 |
|
||
| `ai.ai_review` | An explicit human review of a run's quality | `ai_run_id`, `reviewer_user_id`, `verdict` CHECK in (`correct`,`partially_correct`,`incorrect`,`harmful`), `note`, `reviewed_at` | `uq (ai_run_id, reviewer_user_id)` | *Audit:* yes. *Volume:* ~20,000 |
|
||
| `ai.ai_feedback` | Lightweight thumbs-up/down from the surface | `ai_suggestion_id` NULL, `ai_run_id` NULL, `user_id`, `rating smallint` CHECK in (-1, 1), `note`, `created_at` | `ck num_nonnulls(ai_suggestion_id, ai_run_id) = 1` | *Volume:* ~60,000 |
|
||
|
||
**`ai_suggestion` is what makes "AI must never auto-reject" true by construction.** No AI code path
|
||
can write to `job_application`; it can only insert a suggestion. A domain service's
|
||
`accept_suggestion()` applies it, with the human as the actor. The enforcement points, stated at their
|
||
real strength (§20.4 corrects an earlier overclaim here):
|
||
|
||
| Point | Strength |
|
||
|---|---|
|
||
| `ai_suggestion.ck_suggestion_decided` — an accepted or rejected suggestion must name `decided_by_user_id` | **Database invariant.** A suggestion cannot be applied by nobody |
|
||
| `pipeline_transition_rule.ck_terminal_negative_user_only` — no rule row may permit a non-`'user'` actor on a terminal-negative transition | **Database invariant**, and the load-bearing one |
|
||
| `application.transition()`'s actor guard, plus the `intelligence → domain` import ban enforced by `import-linter` | **Process and dependency graph.** Verified in CI, not by the database |
|
||
| `ats_result.ck_ats_review_human` + `ck_ats_review_actor` | Guarantees a review verdict names a human reviewer. It does **not** by itself prevent auto-rejection — `review_outcome` has no rejecting value, and rejection happens through `transition()` |
|
||
|
||
### 27.4 Chatbot
|
||
|
||
| Table | Purpose | Key columns | Constraints | Profile |
|
||
|---|---|---|---|---|
|
||
| `ai.query_intent` | **The whitelist.** Every question the chatbot can answer maps to a named, parameterised, typed intent implemented as a service call | `key` UNIQUE, `name`, `description`, `parameter_schema jsonb`, `required_permission_key` NOT NULL, `service_function text`, `is_enabled`, `phase smallint`, `max_rows int` | `ck (required_permission_key <> '')` | *Volume:* ~40. **The entire Phase 1 chatbot capability surface is these rows** |
|
||
| `ai.conversation` | A chat session | `id`, `public_id`, `user_id` FK, `title`, `screen_context jsonb` NULL, `started_at`, `last_message_at`, `message_count`, `is_archived` | `ix (user_id, last_message_at DESC)` | *PII:* `personal` — a user's questions are personal data about the user. *Retention:* 12 months. *Volume:* ~150,000 |
|
||
| `ai.conversation_message` | One turn | `conversation_id`, `seq`, `role` CHECK in (`user`,`assistant`,`system`), `content text`, `ai_run_id` NULL FK, `token_count`, `created_at` | `uq (conversation_id, seq)`; append-only | *PII:* `sensitive_personal` (an answer may quote candidate data). *Retention:* 12 months. *Volume:* ~900,000 |
|
||
| `ai.conversation_tool_invocation` | Every tool call the assistant made, with the authorization outcome | `conversation_message_id`, `query_intent_id` FK, `arguments jsonb`, `acting_user_id` **NOT NULL**, `authorization_passed boolean` **NOT NULL**, `denial_reason` NULL, `row_count` NULL, `result_ref` NULL, `duration_ms`, `created_at` | `ck (authorization_passed OR denial_reason IS NOT NULL)`; append-only; `ix (acting_user_id, created_at DESC)` | *Audit:* yes — and **every chatbot answer additionally writes an `audit.audit_event` with `actor_kind = 'ai_agent'` and `on_behalf_of_user_id` set to the asking user**, because a read is the event that matters. *Volume:* ~600,000 |
|
||
|
||
**Phase 1: the chatbot has no SQL access.** It calls the same authorization-checked service layer as
|
||
the UI, through these whitelisted intents. **Text-to-SQL against any application role is
|
||
prohibited** — it is an unbounded read capability that no prompt-level guard reliably constrains.
|
||
|
||
**Phase 2, only if ad-hoc querying is genuinely required:** a dedicated PostgreSQL role for the AI
|
||
path with row-level security policies keyed to `current_setting('app.actor_user_id')` plus
|
||
column-level privileges excluding `sensitive_personal` columns, so the access boundary is enforced by
|
||
the database rather than by prompt engineering. RLS is deferred rather than dismissed because it
|
||
demands disciplined `SET LOCAL` usage on every pooled connection and is easy to get subtly wrong;
|
||
introducing it in Phase 1 alongside a brand-new authorization layer doubles the risk.
|
||
|
||
**Accepted consequence, stated:** deferring RLS means Phase 1 candidate PII protection rests entirely
|
||
on a new application authorization layer, in a codebase that today has none at all
|
||
(`_repo-findings.md` §D). There is no database-level backstop. The mitigation is concrete: one
|
||
centralised authorization module, no direct repository access from controllers, and a test asserting
|
||
that every candidate-reading endpoint passes through it.
|
||
|
||
---
|
||
|
||
## 28. Audit, governance, activity history and operational surfaces
|
||
|
||
### 28.1 `audit.audit_event`
|
||
|
||
*Purpose.* One append-only record of every state change and every AI-influenced or access decision,
|
||
across all modules.
|
||
|
||
**Partitioned.** `PARTITION BY RANGE (occurred_at)`, monthly partitions,
|
||
`PRIMARY KEY (id, occurred_at)` with `id` from a single shared sequence.
|
||
|
||
> **Encode this once and remember it:** a partitioned table's primary key must include the partition
|
||
> key. Any ORM or tooling assuming a single-column integer PK on `audit_event` will misbehave, and a
|
||
> developer will hit it the first time they try to reference an audit row.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` | no | From `audit.audit_event_id_seq`, shared across partitions |
|
||
| `occurred_at` | `timestamptz` | no | Partition key |
|
||
| `actor_user_id` | `bigint` | yes | NULL for genuinely system-originated events |
|
||
| `actor_kind` | `text` | no | CHECK in (`user`, `system`, `integration`, `ai_agent`) |
|
||
| `on_behalf_of_user_id` | `bigint` | yes | Set when `actor_kind = 'ai_agent'` |
|
||
| `request_id` | `uuid` | yes | |
|
||
| `session_id` | `bigint` | yes | |
|
||
| `ip` | `inet` | yes | |
|
||
| `user_agent` | `text` | yes | |
|
||
| `source_service` | `text` | no | Which code path wrote the row. CHECK in (`web`, `worker`, `integration_adapter`, `migration`, `admin`, `assistant`, `unknown`), default `'unknown'` — see below |
|
||
| `action` | `text` | no | e.g. `candidate.viewed`, `application.stage_changed`, `offer.approved`, `export.run` |
|
||
| `entity_table` | `text` | no | |
|
||
| `entity_pk` | `bigint` | yes | |
|
||
| `entity_public_id` | `uuid` | yes | Denormalised so an archived partition is still readable without the source table |
|
||
| `before` | `jsonb` | yes | |
|
||
| `after` | `jsonb` | yes | |
|
||
| `changed_columns` | `text[]` | yes | |
|
||
| `outcome` | `text` | no | CHECK in (`success`, `denied`, `error`) |
|
||
| `denial_reason` | `text` | yes | |
|
||
| `ai_run_id` | `bigint` | yes | Every AI-influenced decision carries it |
|
||
| `row_hash` | `bytea` | no | `sha256(canonical row)` — **the row's own hash, computed on the insert path.** No chain input |
|
||
| `prev_hash` | `bytea` | yes | The predecessor's `row_hash`. **Written by the sealing job, not by the writer.** NULL means "inside the current sealing window" |
|
||
| `chain_seq` | `int` | yes | Position within the partition's chain. Written by the sealing job alongside `prev_hash` |
|
||
| `sealed_at` | `timestamptz` | yes | When the sealing job chained this row |
|
||
| `redacted_at` | `timestamptz` | yes | Set by the authorised redaction path (§28.2) |
|
||
| `pre_redaction_row_hash` | `bytea` | yes | The `row_hash` as it was before redaction. **Without this column the tamper-evidence claim and the erasure claim are mutually exclusive** |
|
||
|
||
**Constraints and indexes.** `ck_audit_denied CHECK ((outcome = 'denied') = (denial_reason IS NOT
|
||
NULL))`. `ck_audit_ai CHECK (actor_kind <> 'ai_agent' OR on_behalf_of_user_id IS NOT NULL)`.
|
||
`ck_audit_source CHECK (source_service IN ('web','worker','integration_adapter','migration','admin','assistant','unknown'))`.
|
||
`ck_audit_sealed CHECK ((prev_hash IS NULL) = (chain_seq IS NULL) AND (chain_seq IS NULL) = (sealed_at IS NULL))` —
|
||
sealing writes all three or none. `ck_audit_redaction CHECK ((redacted_at IS NULL) = (pre_redaction_row_hash IS NULL))`.
|
||
Per-partition indexes: `(entity_table, entity_pk, occurred_at DESC)` — the entity's trail;
|
||
`(actor_user_id, occurred_at DESC)` — "everything actor X did in this window", the forensic query
|
||
that matters; `(action, occurred_at DESC)`; `(ai_run_id) WHERE ai_run_id IS NOT NULL`;
|
||
`ix_audit_unsealed (occurred_at, id) WHERE prev_hash IS NULL` — the sealing job's working set, and the
|
||
only index it needs.
|
||
|
||
**`source_service` answers "which code path did this", and without it that question has no answer.**
|
||
`05` §9.1 I-5 records the gap precisely: the column list covers actor, action, entity, before/after,
|
||
correlation ids, address, outcome and hashes, but nothing identifying the writer, while the audit
|
||
requirement names a source service and the deployment runs **two processes from one image** (`02` §10)
|
||
plus per-channel integration adapters. "Did the web tier or the worker change this application's
|
||
stage" is a first-question-in-every-investigation, and `request_id` does not answer it: a single
|
||
recruiter action propagates one `request_id` through the API layer, into a queue job, and into an
|
||
`ai_model_invocation`, which is exactly why the correlation is useful and exactly why it cannot
|
||
discriminate the writer.
|
||
|
||
Two mechanics worth pinning:
|
||
|
||
- **The value arrives the same way the actor does**, through `SET LOCAL app.source_service` alongside
|
||
`app.actor_user_id` / `app.actor_kind` / `app.change_reason` / `app.request_id` (§31.4), read by the
|
||
generic trigger with `current_setting('app.source_service', true)`.
|
||
- **`NOT NULL DEFAULT 'unknown'` rather than a nullable column, and `unknown` is a seventh value on
|
||
purpose.** `05` §7.1 enumerates six; the seventh follows this document's existing rule for the same
|
||
failure. When the setting is unset the row is written with `actor_kind = 'system'` and
|
||
`actor_unknown = true` so the gap is **visible rather than silently misattributed** (§31.4, R3), and
|
||
`source_service` must behave identically — defaulting a missing value to `'admin'` or `'migration'`
|
||
would fabricate provenance in the one table whose value is that it does not. `source_service =
|
||
'unknown'` joins `actor_unknown` in the required data-quality alarm, and a rising count means a call
|
||
site is not setting the GUC, not that a mystery service exists.
|
||
|
||
**Two classes of writer, because one is not enough.**
|
||
|
||
1. **A generic trigger on every classified table** for data-change events. Cannot be bypassed.
|
||
2. **Explicit application writes for access events** — profile viewed, document downloaded, export
|
||
run, chatbot query answered. **No trigger can observe a read**, and for candidate PII the read is
|
||
the compliance event that matters most.
|
||
|
||
**Tamper resistance, in ascending strength and stated honestly.**
|
||
|
||
| Layer | Mechanism | What it actually gives |
|
||
|---|---|---|
|
||
| 1 | `talentflow_app` holds only `INSERT` and `SELECT`. `UPDATE`, `DELETE`, `TRUNCATE` revoked. DDL lives in a separate migration role. Two narrow exceptions, each with its own role: `talentflow_sealer` (layer 3) and `talentflow_redactor` (§28.2) | Prevention against the application, including a compromised application |
|
||
| 2 | `BEFORE UPDATE OR DELETE` trigger raising an exception, **with two named holes and no others** — see the trigger definition in §28.2 | Prevention that travels with the schema, in case a grant is misconfigured during environment setup |
|
||
| 3 | `row_hash` per row on the insert path, plus a **single-writer sealing job** that chains rows per partition, plus a nightly verifier that recomputes and alerts | **Tamper evidence, not prevention.** It detects modification by anyone who bypasses layers 1–2, including a DBA. It cannot stop them. Claiming otherwise would be dishonest |
|
||
| 4 | WAL archiving/PITR plus a daily export of the closed partition to write-once object storage (S3 Object Lock or immutable blob), with that partition's final head hash recorded | The only genuine independent check |
|
||
|
||
**Layer 3: hashing is decoupled from insertion, and it has to be.** An earlier draft specified
|
||
`row_hash = sha256(prev_hash || canonical row)` computed **by the generic audit trigger on every
|
||
classified write**. That is not safely computable under concurrency, and the reason is worth spelling
|
||
out because the naive form looks right:
|
||
|
||
- Two concurrent transactions both read the same `prev_hash` (whatever the current head is). Both
|
||
insert. The chain **forks**, and the nightly verifier fails permanently from that point on — with no
|
||
tampering having occurred.
|
||
- The only way to prevent the fork on the insert path is to serialise every audit insert on a
|
||
per-partition lock. But **every domain write fires the audit trigger**, on a table projected at ~40M
|
||
rows, so that lock becomes a global write bottleneck on the hottest path in the database. The audit
|
||
mechanism would throttle the product.
|
||
- `id` comes from a **shared sequence**, so id order is not commit order. A verifier replaying by
|
||
`(occurred_at, id)` cannot reproduce whatever order the concurrent writers happened to observe, even
|
||
if no fork occurred.
|
||
|
||
The design instead separates the two concerns:
|
||
|
||
| Step | Who | What |
|
||
|---|---|---|
|
||
| Insert | the audit trigger / the application, as `talentflow_app` | `row_hash = sha256(canonical row)`. `prev_hash`, `chain_seq` and `sealed_at` are left NULL. No lock, no contention, no ordering assumption |
|
||
| Seal | `talentflow_sealer`, hourly and again at partition close | `SELECT pg_advisory_xact_lock(hashtext(partition_name))` — one writer per partition, ever — then walk `WHERE prev_hash IS NULL AND occurred_at < now() - interval '5 minutes'` in `(occurred_at, id)` order, setting `prev_hash`, `chain_seq` and `sealed_at`, and record the partition's running head hash in `audit.partition_seal` |
|
||
| Verify | the nightly verifier | Replay `(occurred_at, id)` order over sealed rows only. Deterministic, because the sealing job established that order and nothing else writes these columns |
|
||
|
||
The five-minute lag is what makes the ordering safe: any transaction still in flight will have
|
||
committed before the sealer considers its window, so `(occurred_at, id)` is a stable total order by the
|
||
time it is read. Layer 4 already assumed a per-partition final hash exported to write-once storage, so
|
||
this fits the existing design rather than adding to it.
|
||
|
||
**Stated explicitly, because it is the honest limit of the claim:** events inside the current sealing
|
||
window are **hash-covered individually but not yet chained**. An attacker who modifies a row *and*
|
||
recomputes its `row_hash` within that window, before sealing, leaves no chain evidence. Reducing the
|
||
window increases sealing frequency and contention; five minutes is the chosen balance, and layer 4 —
|
||
not layer 3 — is what covers the residual.
|
||
|
||
**The PII rule that resolves append-only versus erasure.** For columns classified
|
||
`sensitive_personal` or above, `before`/`after` store **a hash and the fact of change, not the
|
||
value**. Raw values for those columns live only in the entity and its history tables, where the
|
||
retention purge can reach them. For merely `personal` columns raw values are stored, and a narrow,
|
||
authorised, logged redaction path exists — spelled out mechanically in §28.2, because "a redaction path
|
||
exists" against a table with revoked `UPDATE` and an unconditional immutability trigger is not a
|
||
design, it is a contradiction.
|
||
|
||
**Accepted weakening, stated:** keeping `sensitive_personal` values out of audit payloads weakens
|
||
forensic reconstruction. An investigation can prove a compensation field changed and when, but not to
|
||
what value, unless `offer_version` history is intact. This is a deliberate trade against erasure
|
||
obligations and **must be documented so nobody later assumes audit alone is sufficient evidence.**
|
||
|
||
**Profile.** *Status:* `outcome`. *Soft delete:* no, and no delete path exists at all. *Retention:*
|
||
partitions older than 13 months are detached, compressed and archived; audit retention is set
|
||
**independently of candidate retention**. *PII:* mixed by design — see the rule above. *Volume:*
|
||
**~40M rows over 5 years** (**assumption**: data-change plus access auditing over candidates,
|
||
applications, scores, interviews and offers). This is the largest table in the database by an order of
|
||
magnitude, which is the entire reason for partitioning: archival is a `DETACH` rather than a
|
||
multi-hour `DELETE` that bloats the table and defeats autovacuum. *Queries:* the entity trail on any
|
||
record; the actor trail; "every AI-influenced decision on this candidate"; the access-review export;
|
||
the nightly hash verification.
|
||
|
||
**Rejected:** a separate audit database or SIEM stream in Phase 1 (new infrastructure for no Phase 1
|
||
requirement); blockchain or external notarisation (cost and complexity far beyond an internal HR
|
||
system); per-entity audit tables (fragments the forensic query that matters); one unpartitioned table
|
||
(archival becomes a mass `DELETE`); trigger-only auditing (cannot capture reads, which is the
|
||
compliance requirement that matters most for candidate PII).
|
||
|
||
### 28.2 Governance tables
|
||
|
||
| Table | Purpose | Key columns | Constraints | Volume |
|
||
|---|---|---|---|---|
|
||
| `audit.audit_event_redaction` | The logged, **authorised** exception path for erasing a `personal` value from an audit payload | `audit_event_id`, `audit_event_occurred_at` (both, to match the composite PK), `columns_redacted text[]`, `reason`, `legal_basis`, `performed_by_user_id` NOT NULL, `performed_at`, `approved_by_user_id` NOT NULL | append-only; both approver and performer required — a single person cannot redact audit; `ck_redaction_two_person CHECK (approved_by_user_id <> performed_by_user_id)` | ~200 |
|
||
| `audit.partition_seal` | One row per audit partition per sealing run: the chain head, so the verifier and the layer-4 export have something to compare against | `partition_name` , `sealed_through_occurred_at`, `sealed_through_id`, `head_row_hash bytea`, `row_count_sealed`, `sealed_at`, `sealer_version` | `uq (partition_name, sealed_through_id)`; append-only; readable by `talentflow_readonly` | ~800 |
|
||
| `audit.pii_classification` | The machine-readable registry three jobs must read | PK `(table_name, column_name)`, `class` CHECK in (`internal`,`personal`,`sensitive_personal`,`special_category`), **`data_subject_kind` CHECK in (`candidate`,`staff`,`both`,`none`)** (the `staff` subject flag — see below), `lawful_basis`, `retention_policy_id` NULL FK, `masking_strategy` CHECK in (`none`,`hash`,`truncate`,`tokenise`,`null_out`,`pseudonymise`), `notes` | `ck_no_special_category CHECK (class <> 'special_category')` **in Phase 1** — a constraint, so adding such data is a deliberate migration and not an accident; `ck_pii_subject_kind CHECK (class = 'internal' OR data_subject_kind <> 'none')` | ~900 |
|
||
| `audit.retention_policy` | Named policies | `key` UNIQUE, **`subject_id bigint NOT NULL REFERENCES ref.retention_subject(id)`** (replaces the hardcoded `subject` CHECK — see below), `basis`, `retain_for interval` NOT NULL, `trigger_event` CHECK in (`last_activity`,`created_at`,`terminal_at`,`consent_withdrawn`,`first_seen_at`,`sent_at`,`finished_at`), `purge_action` CHECK in (`pseudonymise`,`delete_blob`,`hard_delete`), `is_active` | `ck (retain_for > interval '0')`; `ix (subject_id)` | ~17 |
|
||
| `audit.retention_hold` | Legal holds the purge must skip | `subject_type` CHECK, `candidate_id` NULL, `job_application_id` NULL, `raw_intake_id` NULL, `reason` NOT NULL, `placed_by_user_id` NOT NULL, `placed_at`, `expires_on` NULL, `released_at`, `released_by_user_id` | `ck num_nonnulls(subjects) = 1`; `ix (candidate_id) WHERE released_at IS NULL` | ~500 |
|
||
| `audit.retention_action` | What each purge run actually did | `subject_type`, `candidate_id` NULL, `retention_policy_id`, `executed_at`, `columns_affected text[]`, `rows_affected int`, `blob_keys_deleted text[]`, `run_id uuid`, `dry_run boolean` | append-only; `ix (candidate_id)`, `ix (executed_at)` | ~150,000 |
|
||
|
||
**Why a table and not column comments.** Three separate jobs must *read* the classification — the
|
||
purge job, the subject-access export, and the non-production anonymisation script — and a comment is
|
||
not queryable. **The CI completeness check** (every column on a candidate-touching table has a row) is
|
||
a well-scoped, independently demonstrable junior task and is assigned as such.
|
||
|
||
#### `data_subject_kind` — the `staff` flag, and why `personal` was not enough on its own
|
||
|
||
`05` §9.1 I-7 identifies a real under-classification: `recruiter_performance` is person-attributable
|
||
data about **employees**, not candidates, and the four classes treat `internal` as the low class, so
|
||
staff performance data classified `internal` reads as "unprotected". The fix `05` §9.2 asks for is to
|
||
classify recruiter-attributable metrics as `personal` **with a `staff` subject flag**, and the flag has
|
||
to be a column because the three jobs that read this registry branch on it differently:
|
||
|
||
| Reader | What it needs `data_subject_kind` for |
|
||
|---|---|
|
||
| The retention purge (§31.2) | A candidate purge must touch `candidate` and `both` columns and must **not** touch `staff` ones. `app_user.full_name` is `personal`, and pseudonymising it on a candidate's erasure request would erase a recruiter's identity from every requisition they own |
|
||
| The subject-access export (`05` §4) | A candidate SAR assembles `candidate` + `both`; a staff SAR — a different obligation with a different lawful basis — assembles `staff` + `both`. One registry, two column sets, selected by this column rather than by a hardcoded table list in the export code |
|
||
| The non-production anonymiser (`05` §4) | Anonymises both subject kinds, but a production restore into staging is prohibited precisely because a table list maintained by hand would miss the staff columns |
|
||
|
||
`ck_pii_subject_kind CHECK (class = 'internal' OR data_subject_kind <> 'none')` is the invariant that
|
||
makes the flag trustworthy: any column classified `personal` or above must name whose data it is.
|
||
Without it, `none` becomes the value people reach for when they are unsure, and a `personal` column
|
||
belonging to nobody is invisible to every job in the table above. `internal` columns — ids, statuses,
|
||
counters — are legitimately `none`, which is why the CHECK exempts exactly that class and no other.
|
||
|
||
**The classification lands on base columns, not on the analytics views.** `recruiter_performance` is a
|
||
read-only SQL view (§28.4), and this registry's PK is `(table_name, column_name)` over tables. The
|
||
staff-attributable columns the view reads — `app_user.full_name`, `app_user.email_normalised`,
|
||
`job_assignment.user_id`, `interview_participant.user_id`, `scorecard.created_by_user_id`,
|
||
`ats_result.reviewed_by_user_id`, `ats_result_override.overridden_by_user_id` (§20.7),
|
||
`access_grant.granted_by_user_id` (§7.7) — carry `data_subject_kind = 'staff'`, and the view inherits
|
||
its obligations from them. Extending the CI completeness check to assert that every column on a
|
||
staff-attributable table has a row is the same shape of check as the candidate one, and lands in the
|
||
same junior workstream.
|
||
|
||
#### How the audit redaction path actually executes
|
||
|
||
§28.1 layer 1 revokes `UPDATE` on `audit.audit_event` and layer 2 raises on any update. A redaction
|
||
path that "erases a `personal` value from an audit payload" necessarily **UPDATEs** `before`/`after` on
|
||
an existing audit row. Naming the table without naming the mechanism left two mutually exclusive
|
||
readings — either the erasure path is impossible (leaving `personal` values in audit payloads
|
||
permanently, contradicting §31.4's promise) or the immutability trigger has an unspecified hole. It is
|
||
the second, and the hole is now specified, narrow, and two-person:
|
||
|
||
```sql
|
||
-- a role that can do exactly one thing
|
||
CREATE ROLE talentflow_redactor NOLOGIN;
|
||
GRANT USAGE ON SCHEMA audit TO talentflow_redactor;
|
||
GRANT SELECT ON audit.audit_event TO talentflow_redactor;
|
||
GRANT UPDATE (before, after, redacted_at, pre_redaction_row_hash)
|
||
ON audit.audit_event TO talentflow_redactor;
|
||
GRANT INSERT, SELECT ON audit.audit_event_redaction TO talentflow_redactor;
|
||
|
||
-- layer 2, with its two holes named and nothing else permitted
|
||
CREATE FUNCTION audit.tg_audit_event_guard() RETURNS trigger LANGUAGE plpgsql AS $$
|
||
BEGIN
|
||
IF TG_OP = 'DELETE' THEN
|
||
RAISE EXCEPTION 'audit.audit_event rows are never deleted';
|
||
END IF;
|
||
|
||
-- hole 1: the sealing job, writing chain columns exactly once, while they are NULL
|
||
IF current_user = 'talentflow_sealer'
|
||
AND OLD.prev_hash IS NULL
|
||
AND (to_jsonb(OLD) - 'prev_hash' - 'chain_seq' - 'sealed_at')
|
||
= (to_jsonb(NEW) - 'prev_hash' - 'chain_seq' - 'sealed_at') THEN
|
||
RETURN NEW;
|
||
END IF;
|
||
|
||
-- hole 2: an authorised, approved, logged redaction
|
||
IF current_user = 'talentflow_redactor'
|
||
AND OLD.redacted_at IS NULL
|
||
AND (to_jsonb(OLD) - 'before' - 'after' - 'redacted_at' - 'pre_redaction_row_hash')
|
||
= (to_jsonb(NEW) - 'before' - 'after' - 'redacted_at' - 'pre_redaction_row_hash')
|
||
AND NEW.redacted_at IS NOT NULL
|
||
AND NEW.pre_redaction_row_hash = OLD.row_hash THEN
|
||
RETURN NEW; -- the matching redaction row is asserted at COMMIT, below
|
||
END IF;
|
||
|
||
RAISE EXCEPTION 'audit.audit_event is immutable (attempted by %)', current_user;
|
||
END $$;
|
||
|
||
CREATE TRIGGER tg_audit_event_guard
|
||
BEFORE UPDATE OR DELETE ON audit.audit_event
|
||
FOR EACH ROW EXECUTE FUNCTION audit.tg_audit_event_guard();
|
||
|
||
-- the paperwork must exist in the SAME transaction, checked at COMMIT
|
||
CREATE CONSTRAINT TRIGGER tg_audit_redaction_authorised
|
||
AFTER UPDATE OF redacted_at ON audit.audit_event
|
||
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW
|
||
EXECUTE FUNCTION audit.tg_assert_redaction_logged();
|
||
-- asserts EXISTS (SELECT 1 FROM audit.audit_event_redaction r
|
||
-- WHERE r.audit_event_id = NEW.id
|
||
-- AND r.audit_event_occurred_at = NEW.occurred_at
|
||
-- AND r.performed_by_user_id IS NOT NULL
|
||
-- AND r.approved_by_user_id IS NOT NULL)
|
||
```
|
||
|
||
**The verifier's semantics, which is the half that makes the two claims compatible.** A redaction
|
||
necessarily invalidates that row's `row_hash` — the payload changed — so without a rule the nightly
|
||
verifier would alert forever after the first erasure request, and the practical response to a verifier
|
||
that always alerts is to stop reading it. The rule:
|
||
|
||
| Row state | Verifier compares `row_hash` against | Chain verdict |
|
||
|---|---|---|
|
||
| `redacted_at IS NULL` | recomputed `sha256(canonical row)` | intact if equal |
|
||
| `redacted_at IS NOT NULL` | **`pre_redaction_row_hash`** | intact **provided** a matching `audit_event_redaction` row exists with both a performer and an approver |
|
||
| `redacted_at IS NOT NULL`, no matching redaction row | — | **TAMPER.** This is precisely the signal the chain exists to raise |
|
||
|
||
The chain itself is unaffected, because `prev_hash` links `row_hash` values and `pre_redaction_row_hash`
|
||
preserves the original. **Without those two columns the tamper-evidence claim and the erasure claim are
|
||
mutually exclusive**, which is why they are columns and not process notes.
|
||
|
||
#### Why `retention_policy.subject` is an FK, not a CHECK
|
||
|
||
The hardcoded `subject CHECK IN ('candidate','raw_intake','audit','session','outbound_message','ai_run','staff_record')`
|
||
— seven values — could not hold the policies it exists to drive. §31.2 tabulates **fifteen**, whose
|
||
subjects include `conversation`, `notification`, `report_run_output`, `ingestion_run` and
|
||
`job_posting_metric`; migration 004 seeds `retention_policy`, so five seeded rows would violate the
|
||
CHECK and **migration 004 would abort**. Separately, `ingestion_dead_letter` and
|
||
`integration_webhook_event` carry retention periods in §12.3 but appeared in neither the CHECK nor
|
||
§31.2, so nothing drove their purge at all.
|
||
|
||
Extending the CHECK to fifteen values would fix the immediate break, but §4.6's own Tier 1 reasoning
|
||
applies exactly: **"a new thing that needs a retention period" is a value the business adds without a
|
||
deploy** — a new integration, a new AI artefact, a new report output. So `subject` becomes
|
||
`subject_id bigint NOT NULL REFERENCES ref.retention_subject(id)`, seeded with all fifteen plus the two
|
||
missing ones, and `ref.retention_subject.is_subject_driven` records whether a policy for it can be keyed
|
||
on a data subject at all (§12.3: unlinked intake cannot).
|
||
|
||
**Two CI tests, both cheap and both required:**
|
||
|
||
1. Every distinct `subject_id` in the `retention_policy` seed resolves to a live
|
||
`ref.retention_subject` row — trivially true with an FK, which is the point of using one.
|
||
2. **Every table classified `personal` or above in `audit.pii_classification` is covered by exactly
|
||
one active `retention_policy`.** Not "at least one" — two policies with different periods on one
|
||
table is an ambiguity the purge job would resolve arbitrarily. This test is what would have caught
|
||
`ingestion_dead_letter` and `integration_webhook_event` having periods in prose and no policy row.
|
||
|
||
### 28.3 Activity history is a view, not a table
|
||
|
||
The prototype has an activity feed built from six templates (`js/data.js:184-196`). The temptation is
|
||
an `activity_event` table. **Rejected:** it would duplicate `audit.audit_event` and drift from it, and
|
||
the drift would be invisible until someone noticed the feed and the audit trail disagreed.
|
||
|
||
```sql
|
||
CREATE VIEW app.v_activity_feed AS
|
||
SELECT ae.occurred_at, ae.actor_user_id, ae.action, ae.entity_table,
|
||
ae.entity_pk, ae.entity_public_id
|
||
FROM audit.audit_event ae
|
||
WHERE ae.outcome = 'success'
|
||
AND ae.action IN ('application.created','application.stage_changed','interview.scheduled',
|
||
'assessment.completed','offer.sent','offer.accepted','application.rejected',
|
||
'candidate.merged')
|
||
AND ae.occurred_at > now() - interval '30 days';
|
||
```
|
||
|
||
Bounded to 30 days, which keeps it inside the two most recent partitions and makes it fast. The
|
||
whitelist is what makes it a feed rather than a firehose. Rendering joins out to the entity for names,
|
||
under the reader's own authorization scope — **the feed is filtered by scope like any other query; an
|
||
activity feed that leaks candidate names across departments is a real and easy mistake.**
|
||
**Recorded in §30.**
|
||
|
||
### 28.4 Operational surfaces
|
||
|
||
Five tables that no §10 area names but that the 23-route IA and the API contract require
|
||
(`js/app.js:7-16`; `06` §1.9, §2.18).
|
||
|
||
| Table | Purpose | Key columns | Constraints / indexes | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.task` | Recruiter tasks and SLA items, including AI next-best-action suggestions (`js/data.js:400-407`) | `id`, `public_id`, `title`, `description`, `assignee_user_id` FK, `created_by_user_id`, `due_at`, `completed_at`, `completed_by_user_id`, `status` CHECK in (`open`,`in_progress`,`done`,`cancelled`), `priority smallint`, `origin` CHECK in (`manual`,`rule`,`ai_suggestion`,`sla`), `task_kind_id` FK vocabulary, `ai_suggestion_id` NULL FK, typed nullable subjects (`candidate_id`, `job_application_id`, `job_id`, `interview_id`, `offer_id`, `raw_intake_id`), soft delete | `ck num_nonnulls(subjects) <= 1`; `ck_task_done CHECK ((status = 'done') = (completed_at IS NOT NULL))`; `ck_task_ai CHECK ((origin = 'ai_suggestion') = (ai_suggestion_id IS NOT NULL))`; `ix (assignee_user_id, due_at) WHERE status IN ('open','in_progress') AND deleted_at IS NULL` | *Soft delete:* yes. *Retention:* 24 months after completion. *PII:* `personal` (titles name candidates). *Audit:* yes. *Volume:* ~400,000 |
|
||
| `app.setting` | Workspace settings, replacing the inert Settings screen (`js/settings.js`). **Created in Phase 1** — see the note below | `key` UNIQUE, `scope` CHECK in (`workspace`,`department`,`user`), `department_id` NULL, `user_id` NULL, `value jsonb`, `value_type` CHECK, `updated_by_user_id`, `updated_at` | `uq (key, scope, coalesce(department_id,0), coalesce(user_id,0))`; `ck` scope matches the populated column | *PII:* `internal`. *Audit:* yes — a security setting change matters. *Volume:* ~500 |
|
||
| `app.api_idempotency_record` | The server side of the mandatory idempotency key on every creating `POST` (`06` §1.9). **Phase 1** — the guarantee is load-bearing from the first `POST /applications` a flaky mobile connection retries | `id`, `idempotency_key text NOT NULL`, `actor_user_id bigint NOT NULL FK`, `method text NOT NULL`, `path text NOT NULL`, `request_body_sha256 bytea NOT NULL`, `response_status smallint` NULL, `response_body_sha256 bytea` NULL, `created_resource_table text` NULL, `created_resource_pk bigint` NULL, `state text NOT NULL` CHECK in (`in_flight`,`completed`,`failed`), `first_seen_at`, `completed_at` NULL, `expires_at timestamptz NOT NULL` | `uq_idempotency UNIQUE (actor_user_id, method, path, idempotency_key)` — the key is scoped **per actor** so one user's key cannot collide with or replay another's; `ck_idem_complete CHECK ((state = 'completed') = (completed_at IS NOT NULL))`; `ix_idem_expiry (expires_at)` for the sweep | *Soft delete:* no. *Retention:* **24 hours**, hard-deleted by a `maintenance`-queue sweep — this is operational scaffolding with no historical value, and the second table after `user_session` where hard delete is correct. *PII:* `internal` (hashes, not bodies). *Audit:* no. *Volume:* ~50,000 live at any time |
|
||
| `app.saved_report` | Report definitions | `key`, `name`, `owner_user_id`, `definition jsonb`, `is_shared`, `schedule_cron` NULL, `last_run_at`, soft delete | `CREATE UNIQUE INDEX uq_saved_report_name ON app.saved_report (owner_user_id, name) WHERE deleted_at IS NULL;` (index form, §4.8) | *PII:* `internal`. *Volume:* ~300 |
|
||
| `app.report_run` | Executions, including exports — because an export of candidate data is an access event | `saved_report_id` NULL, `run_by_user_id` NOT NULL, `parameters jsonb`, `started_at`, `finished_at`, `row_count`, `output_file_id` NULL FK `stored_file`, `status` CHECK, `export_format` NULL | append-only; `ix (run_by_user_id, started_at DESC)` | *Audit:* yes, as an **access** event with the row count — "who exported 4,000 candidate records last Tuesday" must be answerable. *Volume:* ~60,000 |
|
||
|
||
**`app.setting` is a Phase 1 table, not a Phase 2 one, and the reason is a specific requirement.**
|
||
REQ-SEC-05 requires **match-score visibility to be a configuration setting in Phase 1**, and `05`
|
||
§2.10 rule 5 commits to it concretely: "ATS score visibility is a configurable role setting, defaulting
|
||
to `hr_admin` + `recruiter` + `hiring_manager` in Phase 1 and off for `interviewer` permanently".
|
||
With no settings storage in Phase 1 that visibility rule becomes exactly the hardcoded constant the
|
||
requirement forbids — and it is not a cosmetic setting: **OBD-05 is an open business decision, so the
|
||
answer WILL change after Phase 1 ships**, and changing a constant is a code deploy while changing a
|
||
row is not.
|
||
|
||
The table is trivial (key, scope, value, updated_by, updated_at), `05`'s field-policy layer already
|
||
needs a settings read, and `00` PROP-06 already makes the Django admin the Phase 1 administration
|
||
surface — so registering `app.setting` there adds **no UI work at all**. It therefore moves to
|
||
migration **012** (Phase 1), seeded with the score-visibility keys; the rest of the workspace-settings
|
||
surface and the Settings screen itself stay in Phase 4 as planned. Migration 020 keeps `task`,
|
||
`saved_search`, `saved_report` and `report_run`.
|
||
|
||
**Phase 1 acceptance criterion, which GAP-05 lacked:** flipping the `ats.score_visible_to_roles`
|
||
setting changes what a `hiring_manager` sees **on the next request, with no deploy** — asserted in the
|
||
A-22 field-policy tests, and asserted negatively for `interviewer`, whose exclusion is not
|
||
configurable.
|
||
|
||
**Analytics owns no tables.** Reporting reads **read-only SQL views declared in migrations** — the one
|
||
sanctioned cross-module read (`_decisions.md` Part 1 rule 2). Declaring them in migrations keeps the
|
||
exception auditable and reviewable in a diff. Phase 2 adds materialised views for the funnel and
|
||
time-to-hire rollups, refreshed nightly; they are derived data with no retention or PII obligation of
|
||
their own beyond inheriting their sources'.
|
||
|
||
### 28.5 `app.candidate_erasure_request` (Phase 3)
|
||
|
||
*Purpose.* The candidate deletion / erasure workflow as a first-class record: a request arrives, an
|
||
identity is verified, a legal-hold check runs, a decision is made and recorded with its basis, and the
|
||
execution is tied to the `audit.retention_action` row that actually did the work. `05` §4 commits to
|
||
this as "**a first-class workflow, not a database operation**", and `05` §10 places it in Phase 3
|
||
alongside subject-access requests. Nothing in §31.2's retention mechanism covers it: the nightly purge
|
||
is a time-driven sweep over `candidate.retention_due_on`, and an erasure request is a *demand* arriving
|
||
at an arbitrary moment with a statutory clock, an identity to verify, and a refusal path.
|
||
|
||
| Column | Type | Null | Notes |
|
||
|---|---|---|---|
|
||
| `id` | `bigint` identity | no | PK |
|
||
| `public_id` | `uuid` | no | UUIDv7, unique. The citable identifier, quoted in the confirmation sent to the candidate. **No `reference_code`** — §3.1 allocates exactly three reference-code sequences (`candidate`, `job`, `job_application`) and this table deliberately does not add a fourth: that idiom exists for high-volume, recruiter-spoken entities, and ~300 rows over five years do not justify a sequence, a trigger and a fourth prefix nobody will memorise |
|
||
| `candidate_id` | `bigint` | yes | FK `app.candidate`. **Nullable, and the nullability is the point** — see below |
|
||
| `claimed_identifier` | `text` | no | The email address or phone number the request arrived for, normalised. Present even when `candidate_id` is set, because it is what was *claimed*, not what was matched |
|
||
| `received_at` | `timestamptz` | no | Default `now()` |
|
||
| `received_on` | `date` | no | `GENERATED ALWAYS AS (((received_at AT TIME ZONE 'UTC'))::date) STORED` — the statutory clock starts on a calendar day, and §4.3's zone-literal idiom is mandatory here |
|
||
| `received_via` | `text` | no | CHECK in (`email`, `portal`, `post`, `phone`, `in_person`, `regulator`, `agency`) |
|
||
| `requested_scope` | `text` | no | CHECK in (`full_erasure`, `documents_only`, `restrict_processing`, `withdraw_consent`). Not every request is a full erasure and treating them alike over-executes |
|
||
| `request_text` | `text` | yes | What the candidate actually asked for, verbatim. `sensitive_personal` |
|
||
| `logged_by_user_id` | `bigint` | yes | FK `app.app_user`. NULL when the request self-served through the portal (§4.9 reason 3) |
|
||
| `status` | `text` | no | CHECK in (`received`, `awaiting_verification`, `verified`, `on_hold`, `granted`, `partially_granted`, `refused`, `executed`, `withdrawn`). Tier 2 (§4.6) — a legal workflow, not a business-editable lifecycle |
|
||
| `due_on` | `date` | no | The statutory response deadline. Starts at `received_on + 30` and is extendable |
|
||
| `extension_reason` | `text` | yes | Required when `due_on > received_on + 30` |
|
||
| `identity_verified_at` | `timestamptz` | yes | |
|
||
| `verification_method` | `text` | yes | CHECK in (`sso_session`, `token_link`, `document_check`, `known_contact_reply`, `manual_attestation`) |
|
||
| `verified_by_user_id` | `bigint` | yes | FK `app.app_user` |
|
||
| `retention_hold_id` | `bigint` | yes | FK `audit.retention_hold`. Set when a legal hold blocks execution — the reason a request sits at `on_hold` |
|
||
| `decision` | `text` | yes | CHECK in (`granted`, `partially_granted`, `refused`) |
|
||
| `decision_note` | `text` | yes | |
|
||
| `refusal_basis` | `text` | yes | Mandatory when `decision = 'refused'`. A refusal without a stated lawful basis is not a refusal, it is a non-answer |
|
||
| `decided_by_user_id` | `bigint` | yes | FK `app.app_user` |
|
||
| `decided_at` | `timestamptz` | yes | |
|
||
| `executed_at` | `timestamptz` | yes | |
|
||
| `retention_action_id` | `bigint` | yes | FK `audit.retention_action` — **the link that makes the claim checkable**, see below |
|
||
| `confirmation_sent_at` | `timestamptz` | yes | |
|
||
| audit + soft delete | | | §4.2, §4.5 — soft delete is **not** applied here; see the profile |
|
||
|
||
**Constraints.**
|
||
|
||
```sql
|
||
CONSTRAINT uq_candidate_erasure_public_id UNIQUE (public_id),
|
||
CONSTRAINT ck_erasure_verified CHECK (num_nonnulls(identity_verified_at, verification_method) IN (0, 2)),
|
||
CONSTRAINT ck_erasure_decided CHECK (num_nonnulls(decision, decided_by_user_id, decided_at) IN (0, 3)),
|
||
CONSTRAINT ck_erasure_refusal CHECK (decision IS DISTINCT FROM 'refused' OR refusal_basis IS NOT NULL),
|
||
CONSTRAINT ck_erasure_extension CHECK ((due_on > received_on + 30) = (extension_reason IS NOT NULL)),
|
||
CONSTRAINT ck_erasure_due_order CHECK (due_on >= received_on),
|
||
-- an erasure may not execute unverified, undecided, or without the action that performed it
|
||
CONSTRAINT ck_erasure_execution CHECK (
|
||
executed_at IS NULL
|
||
OR (identity_verified_at IS NOT NULL
|
||
AND decision IN ('granted','partially_granted')
|
||
AND retention_action_id IS NOT NULL)),
|
||
CONSTRAINT ck_erasure_status_terminal CHECK (
|
||
(status = 'executed') = (executed_at IS NOT NULL)),
|
||
CONSTRAINT ck_erasure_hold CHECK (status <> 'on_hold' OR retention_hold_id IS NOT NULL)
|
||
```
|
||
|
||
**`ck_erasure_execution` is the most important constraint on this table**, and it encodes the three
|
||
things `05` §4 requires before an irreversible act: identity verification *before* action, a decision
|
||
that actually granted something, and a pointer to the `retention_action` row that performed the
|
||
pseudonymisation. All three are single-table facts, so all three are a CHECK rather than a process
|
||
promise. What the CHECK cannot verify is that the `retention_action` row covers *this* candidate — that
|
||
is a cross-table assertion and belongs in a deferred constraint trigger of the §20.1 shape; it is
|
||
called out here rather than claimed, because this document's convention is to be explicit about what a
|
||
CHECK does and does not buy.
|
||
|
||
**`candidate_id` is nullable, and that is a requirement rather than a concession.** Three cases produce
|
||
a request with no candidate: an erasure demand for a person the system holds no record of (which must
|
||
still be logged, verified, answered and retained as evidence that it was answered); a request arriving
|
||
before matching, since a claimed email may resolve to several candidates or none; and a request whose
|
||
subject is unlinked intake — `app.ingestion_dead_letter` and `app.integration_webhook_event` have no
|
||
`candidate_id` by construction (§31.2), so a subject-keyed request cannot reach them and the
|
||
compensating control is the unconditional time sweep. §4.9 reason 2 applies: legitimately absent.
|
||
|
||
**Why `app` and not `audit`.** Every table in the `audit` schema is append-only — `UPDATE`, `DELETE`
|
||
and `TRUNCATE` are revoked for the application role (§1.1) and §31.3 forbids soft delete and mutation
|
||
across all of `audit.*`. This table is a **workflow with a mutable status**: it moves through
|
||
`received` → `awaiting_verification` → `verified` → `granted` → `executed`, with a UI Ahmed owns
|
||
(`05` §10, "erasure request UI"). Putting a mutable workflow in `audit` would either break that
|
||
schema's guarantee or require a grant exception on the one schema whose value is not having any. Its
|
||
*immutable* consequences are already in `audit`: `audit.retention_action` records what the purge did,
|
||
and `audit.audit_event` records "erasure request received/verified/decided/executed/refused"
|
||
(`05` §7.3) — five events, each append-only, each hash-chained. The workflow row is the thing being
|
||
worked on; the audit trail is the record of having worked on it.
|
||
|
||
**Two known interactions, both of which must be on the confirmation screen.** (1) Execution runs the
|
||
**same pseudonymisation path** as retention (§31.2), not a row delete — so the candidate's skeleton
|
||
row, application structure, stage history and score numbers survive, and the confirmation must say
|
||
what was erased rather than implying the record is gone. (2) Erasure **permanently blocks merge
|
||
reversal** for the affected candidate: §31.2 step 6 sets
|
||
`candidate_merge.reversal_blocked_reason = 'retention_purge'`, and that is irreversible. `05` §4 names
|
||
this interaction explicitly and requires it stated at the point of decision, not discovered afterwards.
|
||
|
||
**Soft delete is not erasure, and this table is where that rule is enforceable.** §31.3's last row
|
||
makes it a policy; here it is a mechanism. A recruiter clicking Delete writes `candidate.deleted_at`
|
||
and hides a record. A rights request writes a row in this table, and only a row in this table can lead
|
||
to `retention_action`. There is no path from `deleted_at` to pseudonymisation, deliberately.
|
||
|
||
**Indexes.** `uq_candidate_erasure_public_id (public_id)`;
|
||
`ix_erasure_open (due_on) WHERE executed_at IS NULL AND status NOT IN ('refused','withdrawn')` — the
|
||
statutory-deadline queue, which is the screen this table exists to drive;
|
||
`ix_erasure_candidate (candidate_id) WHERE candidate_id IS NOT NULL` — "has this candidate ever
|
||
requested erasure", asked before any merge and before any re-engagement campaign;
|
||
`ix_erasure_identifier (claimed_identifier)` — matching an inbound request to a candidate, and
|
||
detecting a repeat request.
|
||
|
||
**Profile.** *Status:* `status`, Tier 2. *Required:* `public_id`, claimed identifier, receipt, channel,
|
||
scope, status, `due_on`; everything downstream of receipt is nullable under §4.9 reason 1 and gated by
|
||
the all-or-nothing CHECKs above. *Soft delete:* **no, and this is one of the few places the absence is
|
||
a control** — a deleted erasure request would be a deleted record of a legal obligation. *Retention:*
|
||
permanent. The request and its execution are the evidence that the obligation was met, and they are
|
||
**not themselves subject to erasure**: `claimed_identifier` and `request_text` are pseudonymised when
|
||
the candidate is (they are the candidate's own words), while the ids, dates, decision, basis and
|
||
actors are retained indefinitely — the same split §31.2 applies to `outbound_message`, and the same
|
||
reasoning as `05` §7.5's "audit retention is set independently". *PII:* `claimed_identifier` is
|
||
`personal`, `request_text` is `sensitive_personal` (it may recite employment history and grievances),
|
||
everything else `internal`; `data_subject_kind = 'candidate'` on the first two (§28.2). *Audit:* yes —
|
||
all five lifecycle events, and it is on `05` §4's real-time alert list by virtue of being a data
|
||
lifecycle operation. *Volume:* ~300 over 5 years (**assumption**: a low rate against 25,000 new
|
||
candidate identities a year; the figure that would change this is a public campaign or a regulator
|
||
referral, neither of which is predictable). *Queries:* the statutory-deadline queue; "has this
|
||
candidate requested erasure" before a merge or a campaign; the annual rights-request report by channel,
|
||
scope and outcome; the refusal register with bases, which is the first thing a regulator asks for.
|
||
|
||
### 28.6 The three read-only database roles
|
||
|
||
Not tables — **roles**, with `GRANT`s. `05` §2.7 and adr/0009 both specify three narrowly-defined
|
||
database roles as Phase 2+ defence in depth, "none of them the application role", and `08` GAP-27
|
||
records that none of them appeared anywhere in this document. They are the mechanism behind the honest
|
||
gap adr/0009 accepts: application-layer enforcement is bypassable by anything that reaches the
|
||
database directly, and these three roles are what make each of those paths survivable rather than
|
||
unguarded.
|
||
|
||
| Role | Purpose | Posture | Created | Policied |
|
||
|---|---|---|---|---|
|
||
| `ats_support_readonly` | Production `psql` for debugging — the developer at a production prompt, which is the path adr/0009 names as its largest Phase 1 exposure | `SELECT` only, no column privilege on any contact column, `candidate_document.extracted_text`, or any money column; RLS on the tables it can read; every connection logged | `001` | `021a`, extended by `023` |
|
||
| `ats_report_reader` | BI / spreadsheet access, **if** the business ever demands direct connectivity | `SELECT` on `analytics` views only, never a base table; RLS on the views; no contact, document or money columns | `001` | `021a` |
|
||
| `ats_ai_reader` | The only role an ad-hoc AI query path may ever use, **if** Phase 2 concludes fixed tool intents are insufficient (ADR 0010) | `SELECT` only; RLS keyed to `current_setting('app.actor_user_id')` via helpers mirroring `scopes_for`; column privileges **exclude every `sensitive_personal` column** per `audit.pii_classification` | `001` | `027a` — **conditional**, see below |
|
||
|
||
**All three are created with zero privileges in migration `001`, and that split is deliberate.**
|
||
`CREATE ROLE … NOLOGIN` with no `GRANT` at all is inert: the role cannot connect, cannot read, and
|
||
cannot be used by accident. Doing it in `001` alongside
|
||
`talentflow_migrate`/`_app`/`_readonly`/`_sealer`/`_redactor`/`_worker` means provisioning happens
|
||
once, every later migration has a role name to
|
||
`GRANT` to, and the Phase 2/4 work is **policy** rather than provisioning — which matters because
|
||
adr/0009's V-triggers (V2: a BI tool connects; V4: more than two `psql` PII fixes in a quarter) can
|
||
fire at any time and the response must be "write the policies", not "get a role created in
|
||
production".
|
||
|
||
**RLS and the application role — the lockout trap, named because it is the exact failure adr/0009
|
||
rejects RLS-as-primary over.** `talentflow_migrate` owns the tables, so `talentflow_app` is **not** the
|
||
owner and is therefore *subject* to any RLS enabled on them. Enabling RLS for the benefit of a
|
||
read-only role would silently reduce the application to zero rows on every policied table. Every
|
||
`ALTER TABLE … ENABLE ROW LEVEL SECURITY` in `021a` and `027a` is therefore paired, in the same
|
||
statement block, with a permissive pass-through for the runtime role:
|
||
|
||
```sql
|
||
ALTER TABLE app.candidate ENABLE ROW LEVEL SECURITY;
|
||
|
||
-- the pass-through, without which the application reads zero rows
|
||
CREATE POLICY p_candidate_app ON app.candidate
|
||
FOR ALL TO talentflow_app USING (true) WITH CHECK (true);
|
||
|
||
CREATE POLICY p_candidate_support ON app.candidate
|
||
FOR SELECT TO ats_support_readonly USING (deleted_at IS NULL AND is_pseudonymised = false);
|
||
```
|
||
|
||
`ALTER ROLE talentflow_app BYPASSRLS` would be shorter and is **rejected**: it requires superuser,
|
||
which managed Postgres does not reliably grant, and it is a role attribute that no migration diff makes
|
||
visible — the `pg_dump`-plus-catalogue gate (ADR 0017) sees policies, so the guarantee should live in
|
||
policies. A constraint test connects as `talentflow_app` after each RLS migration and asserts a known
|
||
row is still visible; that test is the regression guard for the whole mechanism, and its absence is
|
||
what would let a lockout ship.
|
||
|
||
**"Restricting a column to zero rows" is a column privilege, not a policy — `05` §2.7's phrasing
|
||
describes the intent and this is the mechanism.** RLS filters *rows*; it cannot hide a column, and a
|
||
policy that returned zero rows whenever a query touched `candidate_document.extracted_text` is not
|
||
expressible. The enforceable form is `GRANT SELECT (…)` naming the readable columns, which is what
|
||
`021a` writes:
|
||
|
||
```sql
|
||
GRANT SELECT (id, public_id, candidate_id, document_type_id, stored_file_id,
|
||
page_count, created_at) -- extracted_text deliberately absent
|
||
ON app.candidate_document TO ats_support_readonly;
|
||
```
|
||
|
||
Column privileges are also what the ADR's `ats_ai_reader` requirement actually needs: "column
|
||
privileges exclude every `sensitive_personal` column" is generated from `audit.pii_classification`
|
||
(§28.2), so the exclusion list is derived from the registry rather than maintained twice — and the CI
|
||
completeness check on that registry becomes the thing that keeps the grant honest.
|
||
|
||
**`ats_ai_reader` is conditional and its migration is written only if the condition fires.** ADR 0010
|
||
prohibits text-to-SQL and adr/0009's V3 makes this role's policies contingent on business sign-off for
|
||
ad-hoc assistant querying. `05` §10 places it in Phase 4 for the same reason. `027a` is therefore
|
||
listed in §33 as a **conditional** migration: the role exists from `001`, and the grants, the
|
||
`scopes_for`-mirroring RLS helper functions and their own test suite are authored **before the first
|
||
query** and not before. Writing them speculatively would mean maintaining a second authorization
|
||
implementation — the exact cost adr/0009 rejects Option B to avoid.
|
||
|
||
---
|
||
|
||
## 29. Search and query design
|
||
|
||
### 29.1 Evaluating the options
|
||
|
||
| Mechanism | What it is genuinely good at | Where it fails for us | Phase 1 verdict |
|
||
|---|---|---|---|
|
||
| **btree / composite btree** | Equality and range on stage, job, recruiter, department, location, experience range, score band, dates. **This is the large majority of what the prototype's screens actually filter on** (`js/candidates.js`) | Substring and typo matching; ranking | **Adopt.** The primary mechanism |
|
||
| **Partial btree** | The hot list screens: active applications, undeleted candidates, unread inbox, pending parses. Smaller index, and the predicate is already in every query | Nothing — this is free precision | **Adopt** for every listed hot path |
|
||
| **`pg_trgm` GIN** | Typo-tolerant name and employer matching via `similarity()` and `%`. **The same indexes duplicate detection needs**, so one mechanism is tuned, understood and tested rather than two | Multi-word relevance ranking; phrase queries; long documents (a GIN trigram index over CV text would be enormous and slow to build) | **Adopt** for `name_normalised`, `current_employer_normalised`, `ref.skill.name_normalised` |
|
||
| **Full-text search (`tsvector` + GIN, `ts_rank_cd`)** | Multi-field weighted relevance across name, title, employer, skills, education and CV text; stemming; phrase and boolean queries | Typos (`Micheal` will not match `Michael`); cross-row documents need a materialised table | **Adopt** via `candidate_search_index` |
|
||
| **JSONB GIN** | Ad-hoc predicates into parser output and score evidence | Cannot be constrained (P6 forbids invariants on it); indexing every payload is pure write cost | **Adopt narrowly** — two columns only: `intake_parse_attempt.parsed`, `ats_result.evidence` |
|
||
| **`pgvector` + HNSW** | Semantic "find me people like this" and hybrid retrieval where the query is part concept, part keyword | Needs an embedding pipeline, per-model versioning, and re-embedding on model change; no Phase 1 requirement demands it | **Phase 2**, in the same database. This is what makes "no separate AI service in Phase 1" achievable rather than aspirational |
|
||
| **OpenSearch / Elasticsearch** | BM25 tuning experiments, learning-to-rank, sub-second facets across many dimensions, cross-entity autocomplete | A second datastore, a permanent dual-write and reindex-drift cost, a second authorization implementation — which directly threatens the chatbot access-control constraint | **Rejected for Phase 1.** Forbidden by constraint, and unjustifiable at 125,000 candidates. §29.5 gives the numeric trigger |
|
||
|
||
**The Phase 1 answer, in one sentence:** structured filters on partial and composite btree indexes,
|
||
free-text relevance on one trigger-maintained `tsvector` table, fuzzy matching on two trigram GIN
|
||
indexes shared with duplicate detection, facets by `GROUP BY`. Nothing else.
|
||
|
||
### 29.2 Why a search-index table and not a generated column
|
||
|
||
The usual advice is a `GENERATED ... AS (to_tsvector(...)) STORED` column on `candidate`. **It cannot
|
||
work here:** a generated column may only reference columns in its own row, and skills, education,
|
||
employment and CV text are child tables — which they must be (§13). One row per candidate in a
|
||
dedicated table, maintained by triggers on the contributing tables, keeps the GIN index small and
|
||
makes a reindex a targeted `UPDATE`.
|
||
|
||
### 29.3 Exact DDL
|
||
|
||
```sql
|
||
CREATE TABLE app.candidate_search_index (
|
||
candidate_id bigint PRIMARY KEY REFERENCES app.candidate(id) ON DELETE CASCADE,
|
||
document tsvector NOT NULL,
|
||
refreshed_at timestamptz NOT NULL DEFAULT now(),
|
||
source_version int NOT NULL DEFAULT 1 -- bumped when the builder changes, to drive a rebuild sweep
|
||
);
|
||
|
||
CREATE INDEX ix_candidate_search_document
|
||
ON app.candidate_search_index USING gin (document);
|
||
|
||
-- The document builder. Weighting: A name, B title/employer/skills, C education/location, D CV text.
|
||
CREATE OR REPLACE FUNCTION app.build_candidate_search_document(p_candidate_id bigint)
|
||
RETURNS tsvector LANGUAGE sql STABLE AS $$
|
||
SELECT
|
||
setweight(to_tsvector('simple', public.unaccent_immutable(coalesce(c.display_name, ''))), 'A')
|
||
|| setweight(to_tsvector('english', public.unaccent_immutable(coalesce(c.current_title, ''))), 'B')
|
||
|| setweight(to_tsvector('simple', public.unaccent_immutable(coalesce(c.current_employer_name, ''))), 'B')
|
||
|| setweight(to_tsvector('simple', public.unaccent_immutable(coalesce(
|
||
(SELECT string_agg(coalesce(s.name, cs.raw_label), ' ')
|
||
FROM app.candidate_skill cs
|
||
LEFT JOIN ref.skill s ON s.id = cs.skill_id
|
||
WHERE cs.candidate_id = c.id), ''))), 'B')
|
||
|| setweight(to_tsvector('simple', public.unaccent_immutable(coalesce(
|
||
(SELECT string_agg(ce.employer_name || ' ' || coalesce(ce.title,''), ' ')
|
||
FROM app.candidate_employment ce WHERE ce.candidate_id = c.id), ''))), 'B')
|
||
|| setweight(to_tsvector('english', public.unaccent_immutable(coalesce(
|
||
(SELECT string_agg(ed.institution_name || ' ' || coalesce(ed.field_of_study,''), ' ')
|
||
FROM app.candidate_education ed WHERE ed.candidate_id = c.id), ''))), 'C')
|
||
|| setweight(to_tsvector('simple', public.unaccent_immutable(coalesce(c.location_text, ''))), 'C')
|
||
|| setweight(to_tsvector('english', public.unaccent_immutable(left(coalesce(
|
||
(SELECT cd.extracted_text FROM app.candidate_document cd
|
||
WHERE cd.candidate_id = c.id AND cd.is_primary_cv AND cd.deleted_at IS NULL
|
||
LIMIT 1), ''), 200000))), 'D')
|
||
FROM app.candidate c
|
||
WHERE c.id = p_candidate_id;
|
||
$$;
|
||
```
|
||
|
||
`left(..., 200000)` bounds the CV contribution: `tsvector` has a hard 1 MB limit and a 100-page
|
||
scanned CV will otherwise abort the insert — a failure mode that shows up in production, not in
|
||
testing.
|
||
|
||
Maintenance triggers on `candidate`, `candidate_skill`, `candidate_employment`,
|
||
`candidate_education` and `candidate_document` enqueue a refresh rather than rebuilding inline: an
|
||
inline rebuild would put six subqueries on the critical path of every skill insert during a bulk
|
||
parse. **The queue is the same Postgres-backed queue used everywhere else, so the enqueue is
|
||
transactional with the write** — the row and its reindex job commit or roll back together.
|
||
|
||
**Trigram indexes** (created once, used by both search and duplicate detection):
|
||
|
||
```sql
|
||
CREATE EXTENSION IF NOT EXISTS pg_trgm;
|
||
CREATE EXTENSION IF NOT EXISTS unaccent;
|
||
|
||
CREATE INDEX ix_candidate_name_trgm
|
||
ON app.candidate USING gin (name_normalised gin_trgm_ops);
|
||
|
||
CREATE INDEX ix_candidate_employer_trgm
|
||
ON app.candidate USING gin (current_employer_normalised gin_trgm_ops);
|
||
|
||
CREATE INDEX ix_skill_name_trgm
|
||
ON ref.skill USING gin (name_normalised gin_trgm_ops);
|
||
|
||
CREATE INDEX ix_candidate_employment_employer_trgm
|
||
ON app.candidate_employment USING gin (employer_normalised gin_trgm_ops);
|
||
|
||
-- tune once, in a migration, and record the value:
|
||
SET pg_trgm.similarity_threshold = 0.35; -- per-session; set on the app role via ALTER ROLE
|
||
```
|
||
|
||
`name_normalised` is
|
||
`GENERATED ALWAYS AS (lower(public.unaccent_immutable(coalesce(display_name,'')))) STORED`.
|
||
|
||
**Use the wrapper everywhere, never bare `unaccent`.** The one-argument `public.unaccent(text)` is
|
||
declared `STABLE`, so PostgreSQL 16 rejects it inside a generated column or an index expression with
|
||
`ERROR: generation expression is not immutable`. The wrapper is created **and asserted** in migration
|
||
`001`, and a CI grep gate rejects bare `unaccent(` inside a `GENERATED ALWAYS AS` clause or an index
|
||
expression — the full statement, the immutability assertion, the volatility tradeoff and the complete
|
||
list of bound columns are in §13.1.
|
||
|
||
```sql
|
||
CREATE OR REPLACE FUNCTION public.unaccent_immutable(text)
|
||
RETURNS text LANGUAGE sql IMMUTABLE STRICT PARALLEL SAFE AS
|
||
$$ SELECT public.unaccent('public.unaccent'::regdictionary, $1) $$;
|
||
```
|
||
|
||
`app.build_candidate_search_document()` above calls the wrapper for the same reason it matters in a
|
||
generated column: its output feeds a GIN-indexed `tsvector`, and a folding change Postgres cannot
|
||
detect would silently desynchronise the index from the data.
|
||
|
||
### 29.4 Ranking, and the versioned config it reads
|
||
|
||
```
|
||
final_rank = w_text * ts_rank_cd(csi.document, query, 32)
|
||
+ w_fuzzy * similarity(c.name_normalised, :term)
|
||
+ w_recency * exp(-age_days / :half_life_days)
|
||
+ w_score * (ja.current_overall_score / 100)
|
||
```
|
||
|
||
The four weights and `half_life_days` live in a **versioned config row**, not in code — the same rule
|
||
as scoring configs, so a relevance change is attributable rather than a mystery. This replaces the
|
||
prototype's ad-hoc client-side relevance blend (`js/candidates.js:18`), which mixes a random
|
||
`aiScore` with string matching in the browser.
|
||
|
||
`ts_rank_cd` with normalisation flag 32 (`rank/(rank+1)`) keeps the text component in `[0,1)` so the
|
||
weights are comparable.
|
||
|
||
**Two tables, and they are named here because "a versioned config row" without a table is not a
|
||
design.** REQ-SRC-04 explicitly requires the blend to be attributable to a versioned configuration and
|
||
REQ-SCR-09 depends on the same thing; without these tables the weights end up as constants in the
|
||
search service — precisely the failure mode the versioning principle (§5.3, P3) exists to prevent, and
|
||
the one that makes *"why did this candidate rank third in March"* unanswerable. Both are created in
|
||
**migration 014**, alongside `candidate_search_index`.
|
||
|
||
| Table | Purpose | Key columns | Constraints | Profile |
|
||
|---|---|---|---|---|
|
||
| `app.relevance_config` | Identity of a relevance configuration | `id`, `key` UNIQUE, `name`, `owner_user_id` FK, `description`, `current_version_id` NULL FK `DEFERRABLE`, audit, soft delete | deferred current-version check, as every other `*_config` | *Soft delete:* yes. *Retention:* permanent. *PII:* `internal`. *Audit:* yes. *Volume:* ~3 (one per searchable entity) |
|
||
| `app.relevance_config_version` | **Immutable** weights | `id`, `relevance_config_id` FK, `version_no int`, **`w_text numeric(6,4)`, `w_fuzzy numeric(6,4)`, `w_recency numeric(6,4)`, `w_score numeric(6,4)`** — four typed columns, **not JSON** — `half_life_days int NOT NULL`, `published_at`, `created_by_user_id` NOT NULL, `config_hash bytea`, `superseded_at` NULL | `uq (relevance_config_id, version_no)`; `ck` each weight `BETWEEN 0 AND 1`; `ck (half_life_days > 0)`; a **`DEFERRABLE` sum-to-1.0 constraint trigger** (`abs(w_text + w_fuzzy + w_recency + w_score - 1.0) <= 0.0001`); INSERT/SELECT-only grants plus the §9.2 `to_jsonb`-form immutability trigger allowing only `superseded_at` | *Soft delete:* no. *Retention:* permanent. *PII:* `internal`. *Audit:* insert only. *Volume:* ~30 |
|
||
|
||
**Weights are four `numeric(6,4)` columns rather than a `weights jsonb` blob** for exactly the reason
|
||
§20.2 gives for scoring criteria: a JSON blob cannot carry a sum-to-1.0 check, cannot be range-checked
|
||
per weight, and cannot be diffed readably in a version-comparison screen. P6 applies — nothing
|
||
enforceable depends on JSONB.
|
||
|
||
**The sum-to-1.0 check is deliberate and states an assumption:** the blend is treated as normalised, so
|
||
the four weights are shares of one ranking score and `final_rank` stays in a comparable range across
|
||
config versions. If a future tuning round wants unnormalised weights, that is a new decision and the
|
||
trigger is dropped in a migration — not silently violated.
|
||
|
||
**Anything that stores a ranked result pins the version.** `app.candidate_job_match` gains
|
||
`relevance_config_version_id` (§20.6), pinned exactly as it already pins `scoring_config_version_id`,
|
||
so a stored ranking is reproducible. Live search results are not stored, so for them the attributability
|
||
requirement is met by the version being resolvable at query time and recorded on the
|
||
`report_run`/`saved_search` execution row.
|
||
|
||
**Phase 1 acceptance criterion, which closes GAP-04:** changing any weight **mints a new
|
||
`relevance_config_version` row and bumps `current_version_id`**; it does not mutate the existing row.
|
||
Asserted by a test that attempts an `UPDATE` on a published version and expects the immutability
|
||
exception.
|
||
|
||
### 29.5 When a separate search service becomes justified
|
||
|
||
Revisit **only** when any of these actually fires:
|
||
|
||
| Trigger | Threshold |
|
||
|---|---|
|
||
| (a) Corpus size | Candidate rows exceed ~2,000,000, **or** indexed searchable text exceeds ~50 GB |
|
||
| (b) Latency | p95 search latency exceeds 500 ms on the tuned FTS + trigram path, **after** index tuning and **after** moving search to a read replica |
|
||
| (c) Throughput | Sustained search throughput exceeds ~50 queries/second **and** search measurably degrades transactional write latency |
|
||
| (d) Capability | A requirement appears that Postgres genuinely cannot serve: live per-field BM25 relevance experimentation, learning-to-rank, sub-second facets across more than ~10 dimensions, or cross-entity typo-tolerant autocomplete under 50 ms |
|
||
|
||
**Exhaust these first, in order:** index and query tuning → a read replica dedicated to search → a
|
||
materialised search table → a BM25 extension (`pg_search`/ParadeDB) if ranking *quality alone* is the
|
||
gap.
|
||
|
||
**Assumption, labelled:** an internal Utopia Brands recruiting platform will hold on the order of
|
||
10⁴–10⁵ candidates accumulated over several years — the projection in §5 is 125,000 — which is three
|
||
to four orders of magnitude below trigger (a). On that basis the honest conclusion is that **Postgres
|
||
FTS plus trigram will very likely never be outgrown for this system**, and the read replica in step
|
||
(b) is the realistic ceiling of what will ever be needed. Numeric triggers rather than "when we
|
||
outgrow it", because otherwise the decision gets made by enthusiasm rather than evidence, and in
|
||
practice most "we need Elasticsearch" moments are an unindexed query or an untuned ranking function.
|
||
|
||
**Re-derive, do not inherit, if the platform is ever pointed at bulk job-board feeds or high-volume
|
||
seasonal hiring** — candidate and `raw_intake` growth could be one to two orders of magnitude higher
|
||
than assumed, and both search and audit sizing change.
|
||
|
||
### 29.6 Expected query patterns
|
||
|
||
Every query below is scope-filtered by `iam.can()` (§7.5) before it runs. That filter is not shown.
|
||
|
||
| # | Query | Screen / consumer | Frequency | Index it relies on |
|
||
|---|---|---|---|---|
|
||
| Q1 | **Application-centric** candidate list: filter by stage, department, score band; sort by score, date or stage age; paginate | Candidates — default view (`js/candidates.js`) | Very high | `ix_application_shortlist (job_id, current_overall_score DESC)`, `ix_application_band`, `ix_application_pipeline`, `ix_application_recruiter`, `ix_job_open (department_id, …)` for the department filter |
|
||
| Q1b | **Person-centric** candidate list: identity search and lifecycle filters only — no stage, no department | Candidates — "search people" | High | `ix_candidate_live (status_id, last_activity_at DESC)`, `ix_candidate_experience`, `ix_candidate_name_trgm` |
|
||
| Q2 | Free-text candidate search with ranking | Candidates search box | High | `ix_candidate_search_document` + `ix_candidate_name_trgm` |
|
||
| Q3 | Pipeline board: one job's active applications grouped by stage, with score and days-in-stage | Pipeline | Very high | `ix_application_pipeline`; `stage_entered_at` makes days-in-stage a subtraction |
|
||
| Q4 | Inbox: unresolved intake ordered by received, with source and per-user unread | Recruitment Inbox | Very high | `ix_raw_intake_inbox`, `raw_intake_read` |
|
||
| Q5 | Candidate profile: identity + all child rows + applications + documents + scores + interviews + messages | Candidate profile | High | the per-child `(candidate_id, …)` indexes; ~10 queries, deliberately not one join |
|
||
| Q6 | Score explanation: one `ats_result` with criteria, skills and pinned versions | Score panel | Medium | `ix_ats_result_current`, `uq (ats_result_id, criterion_key)` |
|
||
| Q7 | Ranked shortlist for a requisition | Shortlisting (BRD §6.1) | Medium | `ix_application_shortlist (job_id, current_overall_score DESC) WHERE state = 'active' AND deleted_at IS NULL` — **one index scan on `job_application`**, not `ats_result`. See §20.4 for why `ix_ats_result_job_score (job_version_id, …)` could not serve this |
|
||
| Q8 | Funnel conversion by stage for a period | Dashboard, Analytics | Medium | `job_application_stage_history (valid_from)`; a materialised view from Phase 2 |
|
||
| Q9 | Time-to-hire and time-in-stage distributions | Analytics | Medium | history intervals; materialised nightly |
|
||
| Q10 | Recruiter workload and SLA | Recruiter Hub | Medium | `ix_job_assignment_user_period` (the §7.4 time predicate, so bounded cover assignments are counted), `ix_application_recruiter` |
|
||
| Q11 | Source and channel performance | Analytics, Job Board | Low | `ix_raw_intake_channel`, `job_posting_metric` |
|
||
| Q12 | Duplicate review queue by descending score, skipping `confirmed_distinct` | Duplicate review | Low | `ix_pair_open`, `ix_pair_distinct` |
|
||
| Q13 | Upcoming interviews for a user; panel availability conflict check | Interviews | High | `ix_interview_upcoming`, `ix_interview_slot`, the participant `EXCLUDE` |
|
||
| Q14 | Offers by status; expiring soon | Offers | Low | `ix_offer_status` |
|
||
| Q15 | Approvals awaiting me | Notifications, Requisitions | Medium | `ix (status_id, sla_due_at)` on `approval_request` |
|
||
| Q16 | Entity audit trail; actor audit trail | Audit screen, compliance | Low | partition-local `(entity_table, entity_pk, occurred_at)` and `(actor_user_id, occurred_at)` |
|
||
| Q17 | Retention purge working set | Nightly job | Daily | `ix_candidate_retention` — an index range scan, not a full-table computation |
|
||
| Q18 | Rescore candidate set on requisition-version publish or config activation | Worker | Bursty | `ix_ats_result_fingerprint (job_application_id, input_fingerprint) WHERE is_current` — compares **this application's** current fingerprint against the recomputed eight-input value (§20.4) |
|
||
| Q19 | Cross-brand match suggestions for a candidate / for a requisition | AI suggestions | Low | `ix_cjm_candidate`, `ix_cjm_job` |
|
||
| Q20 | "Every AI-influenced decision on this candidate" | Compliance, AI governance | Low | `ix_run_subject_application`, `audit (ai_run_id)` |
|
||
|
||
**Q1 splits in two, and that is a product decision this document has to make rather than dodge.** The
|
||
old Q1 read "Candidate list: filter by stage, department, score band … Very high" — the most-loaded
|
||
screen in the product — and attributed it to `ix_candidate_live`, `ix_candidate_experience` and
|
||
`ix_ats_result_current`. **None of those serves it.** `ix_candidate_live` is
|
||
`(status_id, last_activity_at DESC)`, i.e. *candidate lifecycle* status (`active`, `passive`,
|
||
`do_not_contact`, `merged`), which is not a pipeline stage. And `app.candidate` has **no department and
|
||
no stage column at all** — correctly, because P1 keeps participation off the person — so filtering
|
||
people by stage or department means candidate → `job_application` → `job`, a direction no composite
|
||
index supported.
|
||
|
||
The resolution is to decide **which entity the screen lists**:
|
||
|
||
| View | Lists | Filters | Indexes |
|
||
|---|---|---|---|
|
||
| **Candidates (default)** | `job_application` rows, rendered candidate-first | stage, department (via `job`), score band, recruiter, source | Already exist: `ix_application_shortlist`, `ix_application_band`, `ix_application_pipeline`, `ix_application_recruiter` |
|
||
| **Search people** | `candidate` rows | name/skill text, lifecycle status, experience range, location, tag | Already exist: `ix_candidate_name_trgm`, `ix_candidate_search_document`, `ix_candidate_live`, `ix_candidate_experience` |
|
||
|
||
This costs nothing to build — every index already exists — and it is also the more honest UI: "stage"
|
||
is a property of an application, and a person with three live applications has three stages, which a
|
||
person-row list has to either hide or fabricate.
|
||
|
||
**If the business insists on a genuinely person-centric list that filters on stage and department**,
|
||
the answer is a projection table, not a join:
|
||
`app.candidate_pipeline_summary (candidate_id PK, active_application_count int, best_current_stage_id bigint, department_ids bigint[], max_current_score numeric(6,3), last_activity_at timestamptz)`
|
||
with `(best_current_stage_id, max_current_score DESC)` and a GIN on `department_ids`, refreshed through
|
||
the **same queue** that maintains `candidate_search_index` (§29.3) so the enqueue stays transactional
|
||
with the write. It is deliberately **not** in the 152-table count: it is a documented escape hatch with
|
||
a known shape, to be built only if the requirement is confirmed, because a denormalised projection is a
|
||
permanent staleness surface and should not be paid for speculatively.
|
||
|
||
**Either way, Q1 needs the latency criterion GAP-09 asks for:** p95 under 300 ms for the first page at
|
||
125,000 candidates / 200,000 applications on the seeded dataset, asserted in CI against real Postgres.
|
||
An index attribution that nobody measured is how this defect survived in the first place.
|
||
|
||
**Two patterns deliberately avoided.** (1) `SELECT *` with a deep join tree for the candidate profile
|
||
— ten targeted queries under one authorization check are faster and far easier to reason about than
|
||
one twelve-table join whose row multiplication has to be de-duplicated in the application. (2)
|
||
`OFFSET`-based pagination on the high-volume lists; keyset pagination on `(sort_key, id)` instead,
|
||
because `OFFSET 10000` reads and discards 10,000 rows.
|
||
|
||
---
|
||
|
||
## 30. Deviations from the named entity list
|
||
|
||
Tables were not created merely because the prompt named them. Every consolidation, split, rename and
|
||
omission is listed here with its reasoning.
|
||
|
||
### 30.1 Consolidations
|
||
|
||
| Named separately | Realised as | Reasoning |
|
||
|---|---|---|
|
||
| `job_status`, `candidate_status`, `application_status`, `offer_status`, `interview_status`, `assessment_status` | `ref.lifecycle_status` with a `domain` column and composite-FK domain safety (§4.7) | Six tables of identical shape and identical metadata. Domain type safety is preserved by `UNIQUE (id, domain)` plus a generated constant column, so nothing is lost but five tables and five back-office screens |
|
||
| 12 pure-label vocabularies (interview type, interview mode, assessment type, document type, note kind, task kind, consent purpose, cost band, channel type, link type, participant role, parse issue severity) | `ref.vocabulary_value` with a `vocabulary_key` and the same composite-FK idiom (§4.7) | No structural columns, no relationships beyond the label. Adding a vocabulary becomes an INSERT rather than a migration |
|
||
| "Job requisitions" and "jobs" as two areas | One aggregate: `job` (identity) + `job_version` (content) + `job_requirement` | A requisition **is** the job record of record (BRD §9.1). Two identities for one thing would force every application to choose which to reference |
|
||
| Requisition approval workflow and offer approval workflow | One engine: `approval_route`, `approval_route_step`, `approval_request`, `approval_decision` (§10) | Identical shape, identical UI, identical query. Typed nullable subject columns keep the FKs real. This is normalisation, not a generic workflow engine — see the tension note in §10 |
|
||
| A `feedback` entity separate from scorecards | `scorecard` + `scorecard_criterion_score`; ad-hoc commentary in `candidate_note` scoped to the application | Structured feedback is a scorecard; unstructured feedback is a note. A third entity between them has no distinct query |
|
||
| `candidate_note` and `job_application_note` | One `candidate_note` with nullable `job_application_id` and `interview_id` | All notes are about a person, so the FK to `candidate` stays mandatory and access control has one rule. A trigger asserts the application belongs to the candidate |
|
||
| A pool-rematch result table | `candidate_job_match` (§20.6), shared with cross-brand matching | The question is identical, and the answer must be pinned and explainable the same way. Two tables would mean two explainability implementations |
|
||
| A separate `search` module or index store | `candidate_search_index` inside the same database | Constraint, plus §29.5's numeric thresholds |
|
||
| A separate `embeddings` service | `candidate_embedding`, a pgvector column set in Phase 2 | Same reason |
|
||
| Parser registry (`parser_component` + version) | `parser_name` / `parser_version` text columns on the parse attempt | Nothing joins to it, nothing constrains it, and versions come from `pip freeze` rather than being administered |
|
||
|
||
### 30.2 Splits
|
||
|
||
| Named as one thing | Split into | Reasoning |
|
||
|---|---|---|
|
||
| "Recruitment intake and inbox" | `intake_channel` (configured connection) + `ref.source_channel` (channel type) | One channel type can have several connections — two Outlook mailboxes — and reporting groups by type while polling targets a connection |
|
||
| "Intake" as one table | `raw_intake` + `raw_intake_attachment` + `intake_parse_attempt` + `intake_resolution` (+ read state, runs, dead letters) | Arrival, files, parse attempts and the resolution decision have different mutability, different actors and different cardinality. A single table with a status column — the prototype's shape (`js/data.js:284-300`) — cannot hold multiple parse attempts, cannot distinguish "parse failed" from "human rejected", and loses the raw payload the moment a parser writes over it |
|
||
| "Candidates" as one table | `candidate` + 13 child tables | The promotion rule (§13). A wide table cannot hold two emails, two employers, or a skill's provenance |
|
||
| "Pipeline" as configuration | `pipeline_config` + `pipeline_config_version` + `pipeline_config_stage` + `pipeline_transition_rule` | Transitions are a different relation from stage membership, and both must be immutable per version so a config edit cannot retro-change which moves were legal |
|
||
| "Communications" | Templates (versioned, editable) + `outbound_message` (immutable snapshot) + delivery events + notifications | A template must change; a sent message must not. Conflating them means a rejection email whose text can be edited after the fact |
|
||
| "Offers" | `offer` + `offer_version` + `offer_status_history` + `offer_response` | A revised offer is a new document with its own approval and letter, not a field edit; the candidate's response is a separate act with its own provenance |
|
||
| "Files" | `stored_file` registry + owning-table domain rows | Retention, scanning and dedupe attach to bytes, not to domain rows (§8.1) |
|
||
| Interview status vs. reschedule times | `interview_status_history` + `interview_slot` | Two different histories: what state it was in, and what times were proposed |
|
||
| Scoring config binding | `job_scoring_assignment` and `job_pipeline_assignment`, not columns on `job_version` | Otherwise every config tweak fabricates a fake requisition revision |
|
||
|
||
### 30.3 Named but deliberately not created
|
||
|
||
| Not created | Why | What answers the need instead |
|
||
|---|---|---|
|
||
| `activity_event` / activity feed table | Would duplicate and drift from `audit.audit_event` | `app.v_activity_feed`, a 30-day whitelisted view (§28.3) |
|
||
| `CandidateProfileVersion` | Would duplicate `intake_parse_attempt.parsed`, which already is the immutable parsed profile. The real question is per-field, not per-profile | `candidate_field_provenance` (§13.2) |
|
||
| Multi-touch source attribution table | Requires candidate-side tracking this platform does not have; BRD §8.1 asks only for per-source attribution | `raw_intake` (first touch), `job_posting_id` (advert read), `source_channel_id` (reporting dimension) |
|
||
| `assessment_result_section` | Section structures differ per provider and no report dimension needs them yet | `assessment_result.breakdown jsonb`, with an explicit promotion path |
|
||
| Generic `history` / `event` table | No FKs, no typed columns, no per-dimension `EXCLUDE`, and it becomes the largest table with the worst selectivity | Six typed per-entity history tables |
|
||
| Polymorphic `assignment (subject_type, subject_id)` | Unenforceable FK | `job_assignment` + `job_application_assignment` |
|
||
| `analytics` domain tables | Analytics owns no state | Read-only SQL views declared in migrations (§28.4) |
|
||
| Per-region or per-brand databases | Forbidden by constraint; brand is `ref.business_unit`, a column | One database |
|
||
| Queue tables | Owned by `procrastinate` in schema `queue` | Library migrations |
|
||
|
||
---
|
||
|
||
## 31. Governance policies
|
||
|
||
### 31.1 PII classification summary
|
||
|
||
Full per-column detail lives in `audit.pii_classification` (~900 rows) and is asserted complete by
|
||
CI. This is the summary by class.
|
||
|
||
| Class | What is in it | Tables and columns (representative) | Access rule | Masking strategy on purge |
|
||
|---|---|---|---|---|
|
||
| `internal` | Ids, foreign keys, statuses, timestamps, configuration, counters, similarity scores, org dimensions | all `id`/`public_id`/`*_id`, `ref.*`, `job`, `job_version`, `job_application` structure, `duplicate_candidate_pair.signals`, `ats_result.overall_score`/`band`, all `*_history` structural columns | Any authenticated user within scope | `none` |
|
||
| `personal` | Name, email addresses, phones, location, links, employment and education history, session IP/user-agent, notification bodies, approval comments, chatbot questions | `candidate.full_name_original`/`display_name`/`location_text`/`current_title`/`current_employer_name`, `candidate_email.*`, `candidate_phone.*`, `candidate_employment.*`, `candidate_education.*`, `candidate_link.*`, `app_user.full_name`/`email_*`, `user_session.ip`/`user_agent`, `notification.title`/`body`, `approval_decision.comment`, `ai.conversation_message.content` | Recruiter/HR within scope; interviewers only for candidates on their own interviews | `pseudonymise` (deterministic token) or `null_out` |
|
||
| `sensitive_personal` | Compensation, CV files and extracted text, interview scorecards and comments, AI evidence and raw model payloads, recruiter notes, assessment detail, raw intake payloads | `offer_version.*_amount`, `candidate.desired_salary_amount`, `job_version.salary_*`, `stored_file` content, `candidate_document.extracted_text`, `raw_intake.payload`, `intake_parse_attempt.parsed`, `ats_result.evidence`, `ats_result_criterion.matched_evidence`, `ats_result_skill.evidence_snippet`, `scorecard.overall_comment`, `scorecard_criterion_score.*`, `candidate_note.body`, `assessment_result.breakdown`, `ai.ai_model_invocation.request`/`response`, `ai.ai_suggestion.payload`, `outbound_message.body_snapshot` | Explicit permission per module; **never in an audit `before`/`after` payload — hash and fact-of-change only** (§28.1); excluded from the Phase 2 chatbot database role by column-level privilege | `pseudonymise`, `null_out`, or `delete_blob` |
|
||
| `special_category` | **NONE IN PHASE 1.** No diversity, health or accommodation data is stored | — | — | — |
|
||
|
||
`audit.pii_classification.ck_no_special_category` makes the last row a constraint, not a promise:
|
||
introducing such data requires a deliberate migration that removes the CHECK, which forces the
|
||
separate access-control, aggregate-only-read and lawful-basis decisions to be made explicitly.
|
||
|
||
**Class is only half of a classification; the other half is whose data it is.** Every row also carries
|
||
`data_subject_kind` in (`candidate`, `staff`, `both`, `none`) with
|
||
`ck_pii_subject_kind CHECK (class = 'internal' OR data_subject_kind <> 'none')` (§28.2). The class
|
||
answers "how protected"; the subject kind answers "protected for whom", and the purge, the
|
||
subject-access export and the non-production anonymiser all branch on the second. Staff-attributable
|
||
columns — `app_user.full_name`/`email_*`, the recruiter ids on `job_assignment`,
|
||
`interview_participant`, `scorecard`, `ats_result.reviewed_by_user_id`,
|
||
`ats_result_override.overridden_by_user_id` (§20.7) and `access_grant.granted_by_user_id` (§7.7) — are
|
||
`personal` with `data_subject_kind = 'staff'`, which is what keeps a candidate's erasure request from
|
||
pseudonymising the recruiter who owns their requisition, and what stops recruiter-performance data
|
||
being read as "internal, therefore unprotected" (`05` §9.1 I-7).
|
||
|
||
**Two consequences worth naming.** (1) Compensation is `sensitive_personal`, not ordinary personal
|
||
data — that is why `offer_version` audit payloads store hashes and why compensation reporting goes
|
||
through aggregate views. (2) `ai.conversation_message.content` is `sensitive_personal` because an
|
||
assistant answer may quote candidate data, even though the user's *question* is merely `personal`.
|
||
|
||
### 31.2 Retention policy
|
||
|
||
| Policy key | Subject | Trigger event | Retain for | Purge action | Notes |
|
||
|---|---|---|---|---|---|
|
||
| `candidate_no_hire` | Candidate not hired | `last_activity_at` | **24 months** | `pseudonymise` + `delete_blob` | **Assumption** — the BRD requires honouring rights requests but names no period. Confirm with legal before go-live |
|
||
| `candidate_hired` | Candidate hired | `terminal_at` of the hired application | Duration of employment + **7 years** | `pseudonymise` on the recruiting record only | **Assumption.** The employment record itself is out of scope for this system |
|
||
| `candidate_consent_withdrawn` | Any candidate | `consent_withdrawn` | **30 days** | `pseudonymise` + `delete_blob` | Overrides the two above |
|
||
| `raw_intake_unresolved` | Intake with no candidate | `received_at` | **24 months** | `pseudonymise` payload + `delete_blob` | Row retained; the arrival record is not erased |
|
||
| `raw_intake_rejected` | Intake rejected as unusable | `resolved_at` | **12 months** | same | |
|
||
| `outbound_message` | Sent messages | `sent_at` | **24 months** | `pseudonymise` body and address | Metadata (that a rejection was sent, when, by whom) is retained permanently — that is the defensibility record |
|
||
| `ai_run` | Model invocations | `started_at` | **24 months** | `pseudonymise` request/response | Token, cost, latency and outcome metrics retained permanently; they contain no PII and are the governance record |
|
||
| `conversation` | Chatbot history | `last_message_at` | **12 months** | `hard_delete` | The only PII-bearing hard delete: a chat log has no downstream FK and no reporting value |
|
||
| `session` | Sessions and tokens | `expires_at` | **90 days** | `hard_delete` | A dead session has no historical value |
|
||
| `notification` | In-app notifications | `created_at` | **12 months** | `hard_delete` | The underlying event is in `audit_event` |
|
||
| `report_run_output` | Export files (report downloads and subject-access exports) | `finished_at` | **7 days** | `delete_blob` + stamp `stored_file.deleted_from_store_at` | Matches the `exports` container lifecycle in `adr/0003-object-storage-strategy.md` §1 ("delete after 7 days, no exceptions") — a longer policy row would be unenforceable, because the container deletes the bytes at day 7 whatever this table says. An export is a **derived copy of candidate data outside the access-controlled UI**, so it gets the shortest window in this table; re-running the report is cheap and is itself audited. The `report_run` row, its parameters and its row count are retained permanently — an export is an access event |
|
||
| `audit` | `audit.audit_event` | `occurred_at` | **13 months hot**, then archived to immutable storage | `detach` + archive | **Set independently of candidate retention** |
|
||
| `ingestion_run` | Poll runs | `started_at` | **13 months** | `hard_delete` | High-frequency operational rows with no decision content |
|
||
| `ingestion_dead_letter_blob` | Dead-lettered payload bytes | `first_seen_at` | **90 days** | `delete_blob` + stamp `stored_file.deleted_from_store_at` | **Time-based and unconditional** — see the note below. The bytes are the CV-shaped part |
|
||
| `ingestion_dead_letter_row` | Dead-letter rows | `first_seen_at` | **12 months** | `pseudonymise` `raw_body_excerpt` | Row and `error` retained so "no document is ever silently lost" stays auditable |
|
||
| `integration_webhook_event` | Non-application inbound webhooks | `received_at` | **13 months** | `pseudonymise` `payload` | Payloads carry candidate names and addresses in delivery receipts. **Time-based** — a webhook event has no reachable subject either |
|
||
| `job_posting_metric` | Platform counters | `as_of_date` | **3 years** | `hard_delete` | |
|
||
| `staff_record` | `app_user` | offboarding | Employment + **7 years** | none | Not subject to candidate erasure |
|
||
|
||
Seventeen policies. Every `subject_id` resolves to a `ref.retention_subject` row (§28.2) — the
|
||
hardcoded seven-value CHECK this table outgrew would have aborted migration 004 on five of them.
|
||
|
||
**Mechanism.** `candidate.retention_due_on date` is maintained by trigger from `last_activity_at` and
|
||
the applicable policy, so the nightly purge is an index range scan on `ix_candidate_retention` rather
|
||
than a full-table computation. `audit.retention_hold` rows are skipped. Every run writes
|
||
`audit.retention_action` recording the subject, policy, columns affected, blob keys deleted and
|
||
whether it was a dry run.
|
||
|
||
**The time sweep is not the only trigger for a purge, and the other one is a workflow.** A candidate
|
||
exercising a right to erasure does not wait for `retention_due_on`. That path is
|
||
`app.candidate_erasure_request` (§28.5, Phase 3): it carries the statutory clock, the identity
|
||
verification `05` §4 requires before any action, the `audit.retention_hold` check, the decision and its
|
||
lawful basis or refusal basis, and a `retention_action_id` FK to the row that performed the work — so
|
||
"this erasure was executed" is a checkable join rather than a claim. Execution runs **this section's
|
||
pseudonymisation path**, unchanged; the workflow decides *whether and when*, and §31.2 decides *what*.
|
||
|
||
**Subject-driven erasure cannot reach unlinked intake, and a time sweep is the compensating control.**
|
||
The candidate-keyed purge above is driven entirely by `candidate.retention_due_on`, so it can only find
|
||
rows reachable from a candidate. `app.ingestion_dead_letter` and `app.integration_webhook_event` have
|
||
**no `candidate_id` and no link to any subject** — by construction, because a dead letter is a delivery
|
||
that could not be resolved to a person at all. There is therefore nothing to key a subject erasure on,
|
||
and a CV that failed at the envelope stage would otherwise sit in the database indefinitely and survive
|
||
an erasure request. The three policy rows above are **unconditional and time-based**: they run whether or
|
||
not any subject ever asked, keyed on `first_seen_at` / `received_at`, and they are the reason
|
||
`ingestion_dead_letter` now stores its bytes in `app.stored_file` (§12.3) rather than inline — the
|
||
`delete_blob` action needs a `stored_file` row to act on. `ref.retention_subject.is_subject_driven =
|
||
false` marks exactly this class.
|
||
|
||
**Erasure and content-addressed blob sharing.** One `stored_file` row can be referenced by two
|
||
candidates' documents (§8.1, byte-identical CVs). When candidate A is purged and candidate B still
|
||
references the same bytes, the blob is **not** deleted and `deleted_from_store_at` stays NULL; A's
|
||
extracted text and A's personal columns are erased as normal. Deleting the bytes would erase B's data on
|
||
A's request. The purge's blob step is therefore conditional on `NOT EXISTS` any live referencing row —
|
||
the exact predicate is in §8.1 — and this is the one place a purge leaves a blob standing.
|
||
|
||
**The purge action is pseudonymisation, not row deletion.** This is the central reconciliation in the
|
||
whole design: the brief requires that history exist permanently and data-protection law requires
|
||
erasure. Deleting rows would tear holes in funnel metrics, break FKs from audit and history, and make
|
||
the merge undo log unreplayable. Pseudonymisation satisfies erasure of identifying data and leaves the
|
||
statistical shape intact — a purged candidate still counts in "applications per source in Q3 2027",
|
||
which is exactly right.
|
||
|
||
**What a purge actually does to one candidate:**
|
||
|
||
1. Replace `full_name_original`, `display_name`, `location_text`, `current_title`,
|
||
`current_employer_name` with deterministic tokens derived from `public_id`
|
||
(`'REDACTED-' || left(encode(digest(public_id::text, 'sha256'),'hex'), 12)`), so the same person
|
||
pseudonymises identically everywhere and joins still work.
|
||
2. Pseudonymise `candidate_email.address_*`, `candidate_phone.*`, `candidate_link.url_*`, and null
|
||
`candidate_employment.employer_name`/`description`, `candidate_education.institution_name`.
|
||
3. Null `candidate_document.extracted_text`, delete the blobs, stamp
|
||
`stored_file.deleted_from_store_at`.
|
||
4. Null `raw_intake.payload`, `intake_parse_attempt.parsed`, `ats_result.evidence`,
|
||
`ats_result_criterion.matched_evidence`, `scorecard.overall_comment`,
|
||
`scorecard_criterion_score.comment`, `candidate_note.body`, `outbound_message.body_snapshot`.
|
||
5. Delete `candidate_search_index` and `candidate_embedding` rows — **derived indexes are in scope
|
||
for erasure** (BRD §7.4 says so explicitly: "including within any derived embeddings or indexes").
|
||
6. Set `is_pseudonymised`, `pseudonymised_at`; write `retention_action`; set
|
||
`candidate_merge.reversal_blocked_reason = 'retention_purge'` on any merge involving this candidate.
|
||
|
||
**Retained through a purge:** all ids, `reference_code`, `public_id`, every timestamp, every status
|
||
and stage history row, `job_application` structure, `ats_result.overall_score`/`band`/pinned versions,
|
||
`ats_result_criterion` numbers, assessment scores, scorecard ratings, offer amounts on **accepted**
|
||
offers (an employment record), and every `audit_event`.
|
||
|
||
**Named risk:** step 6 means an aggressive purge cadence silently erodes merge reversibility over
|
||
time. Mitigation, and a decision to make explicitly: either exclude candidates involved in an
|
||
unreversed merge from purge for a defined window, or require explicit acknowledgement at purge time
|
||
that reversibility is being given up.
|
||
|
||
### 31.3 Soft-delete policy
|
||
|
||
| Rule | Detail |
|
||
|---|---|
|
||
| Shape | `deleted_at timestamptz` + `deleted_by_user_id bigint`, with `CHECK ((deleted_at IS NULL) = (deleted_by_user_id IS NULL))`. Never `is_deleted boolean` — it loses *when*, and therefore cannot drive retention |
|
||
| Where it applies | Recruiter-removable entities only: `candidate`, `job`, `job_application`, `candidate_note`, `candidate_document`, `candidate_tag`, `candidate_link`, `candidate_email`, `candidate_phone`, `interview`, `offer`, `task`, `saved_search`, `saved_report`, `talent_pool`, `app_user`, `message_template`, `scorecard_template`, `assessment_template`, `pipeline_config`, `scoring_config`, `matching_config` |
|
||
| Where it is forbidden | Every append-only table: `audit.audit_event`, `ats_result`, `ats_result_criterion`, `ats_result_skill`, `ats_result_override`, `candidate_job_match`, `access_grant`, all `*_history`, all `*_version`, `raw_intake`, `raw_intake_attachment`, `intake_parse_attempt`, `parse_issue`, `intake_resolution`, `candidate_merge`, `candidate_merge_operation`, `candidate_consent`, `outbound_message`, `outbound_message_event`, `approval_decision`, `interview_slot`, `offer_response`, `assessment_result`, `fx_rate`, all `ai.*` ledger tables, all `audit.*` — **and `candidate_erasure_request`**, where the absence is itself a control: a deleted erasure request is a deleted record of a legal obligation (§28.5) |
|
||
| Forbidden `UPDATE`, **and the four named exceptions** | No append-only table above may be updated, with exactly four column-scoped exceptions, each with its own freezing trigger and each documented at its table: `ats_result` (`is_current`, `superseded_by_id`, the three review columns — §20.4); `ai.ai_model_invocation` (the ten closing columns; a run may be **closed once and never reopened** — §27.2); `app.intake_parse_attempt` (the nine closing columns; frozen once `finished_at IS NOT NULL`, and `candidate_document_id` write-once — §18.1); `audit.audit_event` (`prev_hash`/`chain_seq`/`sealed_at` for `talentflow_sealer` while NULL, and `before`/`after`/`redacted_at`/`pre_redaction_row_hash` for `talentflow_redactor` with an approved `audit_event_redaction` row in the same transaction — §28.1, §28.2). **Anything not in that list is a bug, and the constraint test suite asserts the exception for each.** `candidate_job_match`, `outbound_message`, `access_grant` (the three revocation columns — §7.7) and `ats_result_override` (`is_current`/`superseded_by_id` — §20.7) carry narrow status-closing grants of the same shape (§20.6, §22.1) |
|
||
| Uniqueness | Every rule that must tolerate re-creation is a **partial** unique index with `WHERE deleted_at IS NULL`, and on the three merge-affected tables additionally `AND suppressed_by_merge_id IS NULL` |
|
||
| Reads | Default application reads go through `v_<entity>_live` views. Seeing deleted rows requires deliberately querying the base table |
|
||
| Cascades | **No `ON DELETE CASCADE` on any FK from a soft-deletable parent.** Soft-deleting a candidate does not touch children; the live views filter by joining to the parent's `deleted_at` |
|
||
| Restore | Clearing `deleted_at` restores the row. **A restore is audited as its own event**, because an undelete is as interesting as a delete |
|
||
| Hard delete | Permitted only on `user_session`, `notification`, `ai.conversation`, `ai.conversation_message`, `ingestion_run`, `job_posting_metric`, `raw_intake_read` — all of which have no downstream FK and no reporting or defensibility value |
|
||
| The rule that matters most | **Soft delete is not erasure and must never be presented as such.** A recruiter clicking Delete hides a record; a rights request triggers §31.2. Conflating them in the UI would be a compliance failure with a technically correct database underneath |
|
||
|
||
### 31.4 Audit policy
|
||
|
||
| Question | Answer |
|
||
|---|---|
|
||
| **What is audited** | Every data change on any table with a `personal` or higher classification, plus every **access** event: candidate profile viewed, document downloaded, export run, chatbot answer produced, search executed with a result count |
|
||
| **What is not audited** | Machine-sourced high-volume counters and operational rows with no decision content: `job_posting_metric`, `ingestion_run`, `outbound_message_event`, `notification`, `raw_intake_read`, `parse_issue`. Each is an explicit exception, listed in the table's profile |
|
||
| **Who writes it** | Two writers. A generic trigger for data changes (cannot be bypassed) and explicit application calls for access events (no trigger can observe a read) |
|
||
| **Actor propagation** | Middleware issues `SET LOCAL app.actor_user_id / app.actor_kind / app.change_reason / app.request_id / app.source_service` at transaction start; triggers read them with `current_setting(..., true)`. When unset, `actor_kind = 'system'`, `actor_unknown = true` and `source_service = 'unknown'` (§28.1) |
|
||
| **Required alarm** | A dashboard query on `actor_unknown` counts **and on `audit_event.source_service = 'unknown'`**, treated as one data-quality alarm. Without it the attribution gap is invisible, in both dimensions: who acted and which code path acted. **This is a deliverable** |
|
||
| **Audit vs. domain history** | Both exist and serve different readers, and this is not duplication. Domain history (`job_application_stage_history` etc.) is typed, FK-enforced, interval-shaped and queried by recruiters and analytics — "how long in Screening", "who owned this in March". `audit.audit_event` is untyped, cross-module, append-only and hash-chained, and is queried by compliance — "everything actor X did", "every AI-influenced decision on this candidate". Per-module logs cannot answer the second class of question; a generic log cannot efficiently answer the first |
|
||
| **AI decisions** | Every AI-influenced decision writes an `audit_event` carrying `ai_run_id`, model version, actor and timestamp. `actor_kind = 'ai_agent'` always carries `on_behalf_of_user_id`, enforced by CHECK |
|
||
| **Immutability** | Four layers (§28.1), with layer 3 stated as tamper *evidence* and only layer 4 providing a genuine independent check |
|
||
| **PII in payloads** | `sensitive_personal` columns are recorded as a hash plus fact-of-change, never a value. `personal` columns store values, with a two-person logged redaction path for erasure requests: `talentflow_redactor` holds `UPDATE (before, after, redacted_at, pre_redaction_row_hash)` and nothing else, the immutability trigger admits that role only when a matching `audit_event_redaction` row with both a performer and an approver is inserted in the same transaction, and a redacted row verifies against `pre_redaction_row_hash` so the chain stays intact (§28.2). Without those two columns the tamper-evidence claim and the erasure claim would be mutually exclusive |
|
||
| **Retention** | 13 months hot, then detach and archive to immutable storage, set independently of candidate retention |
|
||
| **Testing** | Every immutability trigger and every append-only grant gets a test that attempts the forbidden write and asserts the exception. Assigned to the junior as an owned workstream |
|
||
|
||
---
|
||
|
||
## 32. Reconciliation with `_decisions.md`, and risks
|
||
|
||
### 32.1 Points needing explicit reconciliation
|
||
|
||
Five, each flagged where it arose.
|
||
|
||
> **The binding resolution for all five, and for the two smaller alignments below, is in
|
||
> `_open-items.md`.** This table is the evidence and what this document does; the ruling is there.
|
||
> 1 → RULING-02 (plain SQL is the schema authority, Django is the runner, and the CI gate is a
|
||
> schema-drift check rather than `makemigrations --check`), 2 → RULING-05 (§8.1's additive reading
|
||
> wins; `stored_file.scan_status` is the **only** scan status), 3 → C-01, 4 → RULING-09 (one
|
||
> approval engine; known at `006`, built at `017`), 5 → C-10 (additive, not contradictory) with the
|
||
> `region` half of it still gated on **OPEN-05**. The PostgreSQL-version alignment → RULING-07
|
||
> (pinned at 16 for Phase 1). **The CamelCase↔snake_case mapping drafted at the end of this
|
||
> section is now published as `_glossary.md`** per RULING-03 — keep it here as the derivation, but
|
||
> `_glossary.md` is the artefact a reviewer checks a name against, and it is where new rows go.
|
||
|
||
| # | Tension | Part 1 says | Part 2 says | This document does | Who must resolve it |
|
||
|---|---|---|---|---|---|
|
||
| 1 | **Migration tooling vs. ORM** | Django 5 + DRF, chosen partly because "26 modules with a junior need built-in migrations (none exist, §B)" | "Ordered, up-only plain-SQL migration files… The ORM, if any, maps to the schema; it never generates it" | Follows Part 2: plain SQL is authoritative. Django's `migrations` app is used as the **runner** (`RunSQL` only, `--fake-initial` for the initial state) so there is one migration ledger, and Django models are written by hand with `managed = False` on nothing — the models mirror the schema and `makemigrations` output is never committed | **Settled by ADR 0017** (`adr/0017-plain-sql-migrations-as-schema-authority.md`), which supersedes the six separately-worded recommendations this contradiction had accumulated across `00` §10, `02` §15 I1, this row, `04` §9.1, `05` §9.1 I-1 and `07` §17. Canonical text quoted verbatim in `02-system-architecture.md` §12.4 — do not paraphrase it here. It confirms this document's position (plain SQL authoritative, Django as runner only, `managed = False` on no table) and adds the two-gate CI mechanism: `makemigrations --check` as the model-vs-state gate, plus a `pg_dump`-plus-catalogue diff for the objects Django cannot model — the ones §4.8 enumerates. Signed off in **T-04, Phase 0 week 1**; merge blocker on migration `001` |
|
||
| 2 | **`stored_file` registry vs. inline object keys** | `files` module owns `StoredFile(sha256, mime, size, storage_key, scan_status, retention_class)` with `delete_for_subject()` | Object-store key and sha256 inline on `raw_intake_attachment` and `candidate_document` | Adopts Part 1's registry **and** keeps Part 2's columns as denormalised domain data (§8.1). Additive, not contradictory: erasure, virus scanning and content dedupe each need a single place | No decision needed; recorded so nobody "simplifies" the registry away |
|
||
| 3 | **`CandidateProfileVersion`** | Listed as a `candidate` module entity | Not present; parsed output lives in `intake_parse_attempt.parsed` | Replaced by `candidate_field_provenance` (§13.2), which answers the actual question — per-field origin and confidence — without duplicating the parse record | Confirm the Part 1 module entity list is updated, or the API document will describe an entity that does not exist |
|
||
| 4 | **One approval engine** | "A generic `workflow_engine` module driving all state machines — rejected as speculative; each aggregate's transitions live in its own service until at least three modules demonstrably need the same engine" | Silent on approvals | Shares `approval_request`/`approval_decision` across requisition versions and offer versions (§10), on the grounds that route-driven sign-off is not a state machine. **Stated as a judgement call, not as compliance** | If the reviewer disagrees, the split is cheap: duplicate the two tables as `job_version_approval*` and `offer_approval*`. Doing it later is a data migration; doing it now is a copy-paste. Decide at migration `006` |
|
||
| 5 | **The nine additive security and explainability objects `05` §9.2 declares, and the Accepted ADR that depends on two of them** | `05` §9.2 declares nine objects as security and explainability controls; `adr/0009-permission-enforcement-strategy.md` — status **Accepted** — resolves scope from `access_grant` and from `job__current_version__location__region_id`, and its Consequences table states plainly that "`access_grant` and `ref.region` are additive… These must land in the Phase 1 schema, not be discovered in Phase 2". `05` §5.2 further claims that "missing requirements are a queryable fact, not an inference from a null" | Nothing. **None of the nine existed in this document** — no table, no §5 map row, no §32.1 point, and none of the 29 migrations in §33. `08` GAP-27 (its only High finding) verified it by grep: `access_grant`, `ref.region`, `match_state`, `source_service`, `candidate_erasure_request`, `injection_signal` and `ats_result_override` each appeared **0 times** | **Adopts all nine, in the migration that owns each.** Four tables — `ref.region` (§6), `app.access_grant` (§7.7), `app.ats_result_override` (§20.7), `app.candidate_erasure_request` (§28.5); four columns — `app.ats_result_criterion.match_state` (§20.5), `audit.audit_event.source_service` (§28.1), `app.intake_parse_attempt.injection_signal` + `_codes` (§18.1), `audit.pii_classification.data_subject_kind` (§28.2); plus `app.access_scope.region_id` and `ref.location.region_id` (§6, §7.3) and the three read-only roles as **roles with `GRANT`s, not tables** (§28.6). §5's map moves from 155 to 159 and §33 names every migration. Three deliberate departures from the letter of the source documents, each argued at its table: `access_grant`'s subject is P7 typed-nullable FKs, **not** `05` §2.3's `(subject_table, subject_id)`; the 30-day ceiling is `expires_at - granted_at <= interval '30 days'`, because the additive form is stable rather than immutable (§4.3); and `05` §2.7's "restricting a column to zero rows" is implemented as a **column privilege**, because RLS filters rows and cannot hide a column | **No arbitration needed for eight of the nine** — C-10 already rules the list additive rather than contradictory, and adopting it is a schema task. **One part remains open and is not decided here:** whether `region` becomes a *grantable* `scope_type` is **OPEN-05** (owner: Talent Lead + Talha). §7.3 provisions the column, the exclusive-arc branch and the `scope_key` `coalesce` entry so that adoption is a **one-line** CHECK change instead of the three coordinated edits whose third-edit omission `05` §2.3, `06` §1.15 and `08` GAP-27 all flag as a silent authorization defect. Also: `05` §9.2's own withdrawal of the `role_assignment` scope columns is honoured — `08` GAP-27's remediation text still lists them and is stale on that point |
|
||
|
||
Two smaller alignments, noted without needing a decision: Part 1 says PostgreSQL 16 and Part 2 says
|
||
16+ targeting 17 — this document follows Part 2 and adds that **the major version must be pinned at
|
||
provisioning**, because UUIDv7 generation differs (native from PG18). And Part 1's module entity names
|
||
are CamelCase Django-style (`InboundSubmission`, `ProcessingAttempt`, `Application`) while Part 2's
|
||
table names are snake_case (`raw_intake`, `intake_parse_attempt`, `job_application`); the table names
|
||
here are authoritative, and the mapping is:
|
||
`InboundSubmission→raw_intake`, `SubmissionAttachment→raw_intake_attachment`,
|
||
`ProcessingAttempt→intake_parse_attempt`, `Requisition→job`, `RequisitionVersion→job_version`,
|
||
`RequisitionRequirement→job_requirement`, `Application→job_application`,
|
||
`Assignment→job_assignment`/`job_application_assignment`, `AiRun→ai.ai_model_invocation`,
|
||
`ApplicationScore→ats_result`, `ScoreComponent→ats_result_criterion`,
|
||
`SkillMatch→ats_result_skill`, `DuplicateCandidateLink→duplicate_candidate_pair`,
|
||
`MergeOperation→candidate_merge_operation`, `Pool→talent_pool`,
|
||
`TalentPoolMembership→talent_pool_member`, `OutboundMessage→outbound_message`,
|
||
`Task→task`, `RefValue→ref.vocabulary_value`/`ref.lifecycle_status`.
|
||
|
||
### 32.2 Risks carried by this design
|
||
|
||
Every one of these is a real cost, not a caveat.
|
||
|
||
| # | Risk | Severity | Mitigation (all are deliverables, not intentions) |
|
||
|---|---|---|---|
|
||
| R1 | **PL/pgSQL volume.** History triggers, deferrable contactability and weight-sum triggers, immutability triggers, audit hash chaining, search index maintenance, the stack-discipline trigger — this is a lot of database code for a two-person team with a junior | High | Talha owns all trigger and constraint code; Ahmed Mujtaba owns migrations, reference data, the `pii_classification` CI check, search tuning and the constraint test suite. **Every trigger gets a test that attempts the forbidden write and asserts the exception** |
|
||
| R2 | **Merge reversal completeness.** Any table carrying `candidate_id` that merge re-parents but does not record in `candidate_merge_operation` becomes silently unreversible, and the failure surfaces months later | High | A test that enumerates `candidate_id` columns from `information_schema.columns` and asserts the merge routine records an operation for each. The enumeration is generated, not maintained by hand |
|
||
| R3 | **`SET LOCAL` dependency.** Trigger-written history depends on middleware always setting the actor. Background jobs, integrations, imports and psql fixes will miss it | Medium | `actor_unknown = true` makes the gap visible; a dashboard alarm on its count. Without the alarm, the attribution gap is invisible rather than absent |
|
||
| R4 | **One-contact-point-one-identity is too strict for reality, on `candidate_email.address_normalised` *and* `candidate_phone.e164` equally.** Shared household addresses and numbers, agency mailboxes and switchboards, and generic `info@` / reception contacts will fail resolution and back up `needs_review` | Medium | `ref.non_identifying_contact` plus `ref.source_channel.is_identifying`, denormalised onto **both** `candidate_email.is_identifying` and `candidate_phone.is_identifying` by trigger so the index predicates can reference them (an index predicate cannot subquery). **Decide the seed list before go-live, not after the queue backs up.** §12.5 layer 4 |
|
||
| R5 | **`interview.starts_at` vs. `local_start_wall` divergence.** A CHECK cannot verify the relationship because it needs timezone resolution | Medium | One scheduling service function is the only writer; a nightly reconciliation job reports violations |
|
||
| R6 | **Retention purge blocks merge reversal permanently.** Aggressive purge cadence silently erodes reversibility | Medium | Exclude candidates in an unreversed merge from purge for a defined window, or require explicit acknowledgement |
|
||
| R7 | **`sensitive_personal` excluded from audit payloads weakens forensics.** An investigation can prove a compensation field changed but not to what value unless `offer_version` history is intact | Medium | Documented in §31.4 so nobody later assumes audit alone is sufficient evidence |
|
||
| R8 | **No RLS in Phase 1.** Candidate PII protection rests entirely on a brand-new authorization layer in a codebase with none today (`_repo-findings.md` §D) | High | One centralised authorization module, no repository access from controllers, and a test asserting every candidate-reading endpoint passes through it |
|
||
| R9 | **`candidate_job_match` unbounded growth.** A nightly job scoring every candidate against every open requisition is millions of rows | Medium | The bound is part of the design (§20.6): pool members and candidates active in the last 12 months only, against open requisitions only |
|
||
| R10 | **Composite audit PK.** `(id, occurred_at)` will surprise any tooling assuming a single-column integer PK | Low | Documented at the top of §28.1. Cheap if known, confusing if discovered at runtime |
|
||
| R11 | **Volume assumptions.** All sizing rests on 40,000 applications/year. Bulk job-board feeds or seasonal hiring could be 10–100× | Medium | Re-derive rather than inherit. The two figures that change behaviour are `audit_event` (partition sizing) and `candidate` (search trigger (a)) |
|
||
| R12 | **XSS interaction.** `full_name_original`, `address_original`, `raw_intake.payload`, `intake_parse_attempt.parsed` and `stored_file.original_filename` are deliberately preserved unsanitised, and the prototype has no escaping anywhere with 34 `innerHTML` sites (`_repo-findings.md` §E) | High | Storage-side preservation **must** be paired with output-side escaping. Sanitising on write would violate the preserve-the-original rule and is the wrong fix. This is a rendering-layer deliverable, not a schema one, but the schema guarantees the exposure exists |
|
||
| R13 | **Two composite-FK idioms** (`ref.lifecycle_status`, `ref.vocabulary_value`) are non-obvious | Low | Written once in migration `002` with a comment, and a CI test asserting every status column has its generated domain column and composite FK |
|
||
| R14 | **Assumed retention periods.** 24 months for non-hired candidates and 7 years for staff records are assumptions, not confirmed policy | Medium | `audit.retention_policy` rows are data, so a change is an UPDATE plus a re-run — but the *first* purge is irreversible. **Confirm with legal before the purge job is enabled**, and run it in `dry_run` mode for one full cycle first |
|
||
|
||
---
|
||
|
||
## 33. Migration sequence
|
||
|
||
Ordered, up-only, plain SQL under `db/migrations/`. No down-migrations; recovery is forward-fix plus
|
||
PITR. Every migration is reviewed by Talha. Dependencies are strict — **a migration may only reference
|
||
objects created in a lower-numbered file**, and that rule is now **mechanically enforced** rather than
|
||
asserted (see §33.1, "The forward-reference CI gate"). An earlier draft of this sequence broke its own
|
||
rule in four places on the Phase 1 critical path; the gate exists because a stated ordering rule that
|
||
nothing checks is an ordering rule that will be broken again.
|
||
|
||
| # | File | Creates | Depends on | Phase | Owner |
|
||
|---|---|---|---|---|---|
|
||
| 001 | `001_extensions_roles_schemas.sql` | Extensions (`pg_trgm`, `unaccent`, `btree_gist`, `pgcrypto`); schemas `ref`, `app`, `ai`, `audit`, `staging` — **plus `CREATE SCHEMA IF NOT EXISTS queue` if procrastinate's own migrations have not already run** (see §33.1); roles `talentflow_migrate`/`_app`/`_readonly`/`_sealer`/`_redactor`/`_worker` with default privileges; **the three read-only roles `ats_ai_reader`/`ats_report_reader`/`ats_support_readonly` as `NOLOGIN` with zero privileges** — provisioning once, policy later in `021a`/`023`/`027a` (§28.6); `public.unaccent_immutable()` **and the `provolatile = 'i'` assertion block** (§13.1); `app.tg_set_updated_at()`; `app.tg_raise_immutable()`; `app.uuidv7()` shim; the three reference-code sequences; `ALTER ROLE … SET timezone='UTC'` and `SET pg_trgm.similarity_threshold` | — | 1 | Talha |
|
||
| 002 | `002_ref_core.sql` | `ref.lifecycle_status`, `ref.vocabulary_value` **with the composite-FK idiom documented in comments** (seeded including the `ats_override_reason` vocabulary that `012`'s `ats_result_override` FKs to — §20.7), `ref.currency`, `ref.department`, `ref.business_unit`, **`ref.region`** (created *before* `ref.location` in the same file, so `ref.location.region_id` is a same-file backward reference and not a forward one), `ref.location` (**with `region_id` FK, replacing the free-text `region` column** — §6), `ref.grade`, `ref.employment_type`, `ref.education_level`, `ref.pipeline_stage`, `ref.rejection_reason`, `ref.source_channel`, `ref.publish_platform`, `ref.assignment_role`, `ref.app_module`, `ref.permission_action`, `ref.tag`, `ref.skill`, `ref.skill_alias`, **`ref.non_identifying_contact`**, **`ref.retention_subject`** (17 seeds — 004 FKs to it) + **all seed data from BRD §9.2**. Seeds `ref.lifecycle_status` `job_application.superseded_by_merge` with `is_terminal = true, is_negative = false` (§15.1) | 001 | 1 | Ahmed |
|
||
| 003 | `003_identity.sql` | `app.app_user`, `app.role`, `app.permission` (all 250 rows), `app.role_permission` (the ~265 grants of `05` §2.9), `app.role_assignment` + `ix_role_assignment_user_period`, `app.user_session`, `app.api_idempotency_record`. **`access_scope`, `candidate_access_token` and `v_user_effective_scope` are NOT here** — see §33.1 | 002 | 1 | Talha |
|
||
| 004 | `004_audit_governance.sql` | `audit.audit_event` partitioned (incl. **`source_service` with `ck_audit_source`**, `row_hash`, `prev_hash`, `chain_seq`, `sealed_at`, `redacted_at`, `pre_redaction_row_hash`) + 14 months of partitions + the partition-creation function; `audit.partition_seal`; `audit.audit_event_redaction`; `audit.pii_classification` (incl. **`data_subject_kind` and `ck_pii_subject_kind`** — §28.2); `audit.retention_policy` (FK to `ref.retention_subject`) + all 17 seeds; `audit.retention_hold`; `audit.retention_action`; the generic audit trigger function; `audit.tg_audit_event_guard()` with its two named holes; the **sealing** function (advisory-locked, per partition) and the verifier | 003 | 1 | Talha |
|
||
| 005 | `005_files.sql` | `app.stored_file` | 003 | 1 | Ahmed |
|
||
| 006 | `006_jobs.sql` | `app.job` (incl. `close_reason_id`), `app.job_version` (incl. `salary_period` and the annualised columns), `app.job_requirement`, `app.job_status_history`, `app.job_vacancy`, `app.job_assignment` + `ex_job_primary_recruiter`; immutability grants + the `to_jsonb`-form triggers, **each as the `BEFORE UPDATE` / `BEFORE DELETE` pair §4.8 requires** (`tg_job_version_immutable` + `tg_job_version_no_delete`, and the same pair on `job_requirement`); **both** deferred weight-sum triggers (parent and child); the deferred current-version trigger; `app.v_job_live` | 004, 005 | 1 | Talha |
|
||
| 006a | `006a_job_posting.sql` | `app.job_posting` **only** — one row per published version for the careers portal. Split out of `027` because `job_application.job_posting_id` (created in `011`) is a Phase 1 FK and cannot forward-reference a Phase 4 table. `job_posting_metric` and all external-platform state stay in `027` | 006 | **1** | Ahmed |
|
||
| 007 | `007_pipeline.sql` | `app.pipeline_config`, `app.pipeline_config_version`, `app.pipeline_config_stage`, `app.pipeline_transition_rule`, `app.job_pipeline_assignment` + the default 7-stage config seed | 006 | 1 | Ahmed |
|
||
| 008 | `008_intake.sql` | `app.intake_channel` (+ the mandatory `manual_ui` row), `app.raw_intake`, `app.raw_intake_attachment`, `app.raw_intake_read`, `app.ingestion_run`, `app.ingestion_dead_letter` (`stored_file_id` + capped `raw_body_excerpt`, **no inline `raw_body`**), `app.integration_webhook_event` | 005, 006 | 1 | Talha |
|
||
| 009 | `009_candidate.sql` | `app.candidate` and all 11 non-derived child tables; the deferred contactability trigger; the `is_identifying` and `link_type_key` denormalisation triggers; the email/phone/link partial unique indexes **including the `suppressed_by_merge_id` and `is_identifying` predicates** (created now, so merge in 013 needs no index change); every generated normalisation column using `public.unaccent_immutable`; `app.candidate_status_history`; `app.v_candidate_live` | 008 | 1 | Talha |
|
||
| 010 | `010_parsing.sql` | `app.intake_parse_attempt` (incl. **`injection_signal` + `injection_signal_codes` + `ck_parse_injection`**, and both in the closing-column grant — §18.1) (+ the closing grant and `app.tg_parse_attempt_freeze()`), `app.parse_issue`, `app.candidate_document`. **`intake_resolution` moves to `011`** — see §33.1 | 009 | 1 | Talha |
|
||
| 011 | `011_application.sql` | `app.job_application` (incl. `status_is_terminal`/`status_is_negative`, the generated `state`, `current_overall_score`/`current_band`, `job_posting_id`), `app.job_application_stage_history`, `app.job_application_status_history`, `app.job_application_assignment`, `app.intake_resolution` (with its `job_application_id` FK; `matching_config_version_id` as a bare nullable `bigint`, given its `REFERENCES` in `013`), `app.access_scope` (with `department_id`/`business_unit_id`/`location_id`/**`region_id`**/`job_id`/`job_application_id` FKs — **and `ck_access_scope_target` carrying `region_id` in every branch plus its own unreachable `region` branch, and `scope_key`'s `coalesce` list carrying `region_id`, all three in this file; `'region'` in the `scope_type` CHECK is the one line pending OPEN-05** (§7.3); `talent_pool_id` added by `024`), `app.candidate_access_token` (with the `candidate_id` and `job_application_id` FKs; `interview_id`, `assessment_assignment_id` and `offer_id` added by `019`/`022`/`023`), **`app.access_grant`** (with `candidate_id`/`job_id`/`job_application_id`; `interview_id` added by `019` and `offer_id` by `023`, each re-declaring `ck_access_grant_subject`; + the revocation-column grant and the trigger pair `tg_access_grant_immutable` (`BEFORE UPDATE`) + `tg_access_grant_no_delete` (`BEFORE DELETE`) — §7.7), `app.v_user_effective_scope` **branches 1–2 and 4** (branch 4 without its `interview`/`offer` joins — §7.5), the plain `BEFORE INSERT` cooling-off trigger, both stage-in-pipeline constraint triggers, `app.v_job_application_live` | 010, 007, 006a | 1 | Talha |
|
||
| 011a | `011a_communications_core.sql` | `app.message_template(+_version)`, `app.message_thread`, `app.outbound_message` + the four CHECKs and the column-restricted grants of §22.1, `app.outbound_message_event`, `app.notification`; adds `message_thread_id` to `raw_intake`. **FK columns whose target does not exist yet are added by the migration that creates the target, which also re-declares the owning table's `num_nonnulls` subject CHECK** — `outbound_message.interview_id` / `offer_id` / `assessment_assignment_id`, `notification.interview_id` / `offer_id` / `approval_request_id`, and the AI links `outbound_message.ai_run_id` / `message_template_version.ai_suggestion_id` (back-filled by `016` with `ck_outbound_ai_human`). See §33.1 | 011 | **1** | Ahmed |
|
||
| 012 | `012_scoring.sql` | `app.scoring_config`, `app.scoring_config_version`, `app.scoring_config_criterion`, `app.job_scoring_assignment`, `app.ats_result` (incl. the immutable denormalised `job_id` and `reviewed_by_actor_kind`), `app.ats_result_criterion` (incl. **`match_state` NOT NULL with `ck_ats_criterion_match_state`, `ck_ats_criterion_unassessed` and `ix_ats_criterion_match`** — §20.5), `app.ats_result_skill`, **`app.ats_result_override`** (append-only + the `is_current`/`superseded_by_id` grant, `tg_ats_result_override_immutable` + `tg_ats_result_override_no_delete`, `uq_ats_result_override_current`, the composite FK to `ref.vocabulary_value`, and `tg_ats_override_target_current` — §20.7; `talentflow_worker` is granted **nothing** on it), **`app.setting`** (Phase 1, seeded with the score-visibility keys only — §28.4); the column-level grants; the `to_jsonb`-form immutability triggers, **each as a `BEFORE UPDATE` / `BEFORE DELETE` pair** (§4.8); **both** criterion weight-sum triggers; `tg_ats_exactly_one_current`; the trigger maintaining `job_application.current_overall_score`/`current_band` | 011, 002 | 1 | Talha |
|
||
| 013 | `013_duplicates_merge.sql` | `app.matching_config`, `app.matching_config_version`, `app.duplicate_candidate_pair`, `app.candidate_merge`, `app.candidate_merge_operation` (incl. `reversal_rank` and `ix_merge_op_replay`); the stack-discipline trigger; `ALTER TABLE app.intake_resolution ADD CONSTRAINT fk_resolution_matching_version FOREIGN KEY (matching_config_version_id) REFERENCES app.matching_config_version(id)` | 009, 011 | 1 detect / 2 merge UI | Talha |
|
||
| 014 | `014_search.sql` | `app.candidate_search_index`, `app.build_candidate_search_document()` (using `unaccent_immutable`), the GIN index, the five maintenance triggers, the trigram indexes, **`app.relevance_config` + `app.relevance_config_version`** with the sum-to-1.0 deferred trigger and one seeded version (§29.4) | 010, 011 | 1 | Ahmed |
|
||
| 015 | `015_currency_fx.sql` | `app.fx_rate`, the minor-unit rounding trigger, the money-pair CHECK helper conventions | 002 | 1 | Ahmed |
|
||
| 016 | `016_ai_core.sql` | `ai.ai_capability` (15 seeds with true availability), `ai.prompt_template(+_version)`, `ai.ai_model_config(+_version)`, `ai.ai_model_invocation` (with the closing-column grant and the `tg_run_no_reopen` / `tg_run_no_delete` pair — §27.2), `ai.ai_suggestion`, `ai.ai_review`, `ai.ai_feedback`; back-fills `ai_run_id`/`ai_suggestion_id` FKs on `ats_result`, `job_version`, `candidate_skill`, `job_application_stage_history`, **`outbound_message` (`ai_run_id`) and `message_template_version` (`ai_suggestion_id`)** — the last two because `011a` now creates those tables before the AI ledger exists, and `ck_outbound_ai_human` (§22.1) is declared here with the FK rather than in `011a` where `ai_run_id` would be a column with nothing to reference | 012 | 1 | Talha |
|
||
| 017a | `017a_approvals_single.sql` | `app.approval_request` (**without** `route_id` / `route_snapshot` — no route table exists yet; the resolved approver is stored directly on the request), `app.approval_decision`; adds `approval_request_id` to `job_version`; the `publish_version()` gate trigger | 006 | **1** | Talha (`07` T-17b — this migration carries the publish gate, so it is constraint work, not seed work) |
|
||
| 017b | `017b_approvals_routes.sql` | `app.approval_route`, `app.approval_route_step`; adds `route_id` + `route_snapshot jsonb` to `approval_request` as nullable, backfills existing rows with the default route and a snapshot reconstructed from the stored approver, then tightens `route_id` to NOT NULL. Phase 1 history is preserved rather than rewritten | 017a | 2 | Ahmed |
|
||
| 018 | `018_communications_pipeline.sql` | `app.notification_preference` and **`app.communication_suppression`** — the Communications tables Phase 1 does not write. The rest of the area moved to `011a` when the minimal send slice became Phase 1 work (§22, §33.1) | 011a, 009 | 2 | Ahmed |
|
||
| 019 | `019_interviews_scorecards.sql` | `app.interview`, `app.interview_participant` (with **both** double-booking `EXCLUDE` constraints and the `slot`/`interview_is_blocking` mirror trigger firing on `starts_at`, `ends_at`, `status_id` and `deleted_at`), `app.interview_slot`, `app.interview_status_history`, `app.user_availability_rule`, `app.user_availability_exception`, `app.scorecard_template(+_version, +_criterion)`, `app.scorecard`, `app.scorecard_criterion_score`; `ALTER TABLE app.candidate_access_token ADD COLUMN interview_id bigint NULL REFERENCES app.interview(id)` and re-declares `ck_token_subject`; **`ALTER TABLE app.access_grant ADD COLUMN interview_id bigint NULL REFERENCES app.interview(id)` and re-declares `ck_access_grant_subject`** (§7.7); `CREATE OR REPLACE VIEW` extends `v_user_effective_scope` with **branch 3 and branch 4's `interview` join + `coalesce` term** (§7.5) | 011 | 2 | Talha |
|
||
| 020 | `020_worklist.sql` | `app.task` (**without `offer_id`** — added by `023`), `app.saved_search`, `app.saved_report`, `app.report_run`; `app.v_activity_feed`. `app.setting` moved forward to `012` (§28.4) | 011, 004 | 2 | Ahmed |
|
||
| 021 | `021_analytics_views.sql` | Read-only reporting views: funnel, source performance, recruiter performance, **and the two KPI definitions `app.v_kpi_time_to_hire` / `app.v_kpi_time_to_fill` written as SQL once and nowhere else** (§9.6) | 011, 019, 006 | 2 | Talha |
|
||
| 021a | `021a_readonly_roles_rls.sql` | The Phase 2 half of §28.6: `ats_support_readonly` and `ats_report_reader` — column-scoped `GRANT SELECT` (contact columns, `candidate_document.extracted_text` and every money column **omitted**, generated from `audit.pii_classification`), `ats_report_reader` restricted to the `analytics` views of `021` and no base table, `ENABLE ROW LEVEL SECURITY` plus policies on `candidate`, `candidate_document`, `job_application`, `ats_result`, `interview` and the analytics views — **each paired in the same block with the `talentflow_app` pass-through policy** (§28.6), and the constraint test that asserts the application still reads its rows. `offer`/`offer_version` are **not** here — they do not exist until `023` | 021, 019, 012, 010, 004 | 2 | Talha |
|
||
| 022 | `022_assessments.sql` | `app.assessment_template(+_version)` (with `max_score > 0`), `app.assessment_assignment`, `app.assessment_result` (INSERT/SELECT grants only, null-safe `percentage`); adds `assessment_assignment_id` to `outbound_message` and to `candidate_access_token`, re-declaring both subject CHECKs | 011, 011a | 3 | Ahmed |
|
||
| 023 | `023_offers.sql` | `app.offer` (incl. `status_is_terminal`), `app.offer_version` (incl. the UTC-pinned `created_on` generated column and the annualisation columns), `app.offer_status_history`, `app.offer_response`; `ALTER TABLE app.task ADD COLUMN offer_id bigint NULL REFERENCES app.offer(id)`, `ALTER TABLE app.candidate_access_token ADD COLUMN offer_id …` and **`ALTER TABLE app.access_grant ADD COLUMN offer_id …`**, re-declaring all three `num_nonnulls` subject CHECKs; `CREATE OR REPLACE VIEW` extends `v_user_effective_scope` with **branch 4's `offer` join + `coalesce` term** (§7.5); **extends `021a`'s `ats_support_readonly` / `ats_report_reader` posture to the offer money columns** — the last tables §28.6's two Phase 2 roles need and the reason their policies are split across two files | 011, 015, 017a, 020, 021a | 3 | Talha |
|
||
| 024 | `024_talent_pool.sql` | `app.talent_pool`, `app.talent_pool_member`, `app.candidate_job_match` (incl. `relevance_config_version_id`); `ALTER TABLE app.access_scope ADD COLUMN talent_pool_id bigint NULL REFERENCES app.talent_pool(id)` and re-declares `ck_access_scope_target` | 012, 009, 014 | 3 | Ahmed |
|
||
| 025 | `025_fairness.sql` | `app.evaluation_dataset`, `app.evaluation_run`, `app.evaluation_metric`; adds `ck_activation_gate` **and `tg_scoring_activation_gate`** to `scoring_config_version` plus the `evaluation_run` status-lock trigger — the CHECK is a not-null guard, the triggers are the gate (§20.1) | 012 | 3 | Talha |
|
||
| 025a | `025a_erasure_requests.sql` | **`app.candidate_erasure_request`** — the candidate deletion/erasure workflow (§28.5) with its eight CHECKs, the UTC-pinned `received_on` generated column, the FK to `audit.retention_action` that makes execution checkable, the FK to `audit.retention_hold` behind `on_hold`, and `ix_erasure_open`. A new suffixed file rather than an extension of `004`, because `004` is Phase 1 and `audit`-only while this is Phase 3, in `app`, and references `app.candidate`; placed at `025a` so it sits after every Phase 3 migration whose objects it needs and still below the Phase 2 `026` | 009, 004, 003 | 3 | Talha |
|
||
| 026 | `026_chatbot.sql` | `ai.query_intent` (+ seeds), `ai.conversation`, `ai.conversation_message`, `ai.conversation_tool_invocation` | 016 | 2 read-only / 4 full | Talha |
|
||
| 027 | `027_publishing_external.sql` | `app.job_posting_metric` and the external-platform posting state (`publish_attempt`, per-platform external ids and cost bands). **`app.job_posting` itself is `006a`** | 006a | 4 | Ahmed |
|
||
| 027a | `027a_ats_ai_reader_rls.sql` | **CONDITIONAL — authored only if ADR 0010's revisit condition fires.** The Phase 4 half of §28.6: `ats_ai_reader`'s column-scoped `GRANT SELECT` excluding every `sensitive_personal` column (generated from `audit.pii_classification`), the RLS helper functions mirroring `scopes_for`, the policies keyed to `current_setting('app.actor_user_id')` on `candidate`/`job_application`/`ats_result`/`interview`/`offer`, and its own test suite. The role itself exists from `001`; nothing here is written speculatively, because writing it early means maintaining a second authorization implementation — the cost adr/0009 rejects Option B to avoid (adr/0009 V3; `05` §10 Phase 4) | 023, 021a | 4 (conditional) | Talha |
|
||
| 028 | `028_pgvector.sql` | `pgvector` extension; `app.candidate_embedding` + HNSW index | 010 | 2 | Talha |
|
||
| 029 | `029_materialised_analytics.sql` | Materialised views for funnel and time-to-hire + refresh function | 021 | 3 | Talha |
|
||
|
||
### 33.1 Critical-path notes
|
||
|
||
**The four forward references this sequence used to contain, and how each is fixed.** These were not
|
||
cosmetic — migration `003` could not execute at all, on the Phase 1 critical path.
|
||
|
||
| Was | Why it could not execute | Now |
|
||
|---|---|---|
|
||
| `003` created `app.candidate_access_token` | Its typed nullable subject FKs (§7.6) point at `candidate` (`009`), `job_application` (`011`), `interview` (`019`), `offer` (`023`) and `assessment_assignment` (`022`) — **none of which exists at `003`** | Created in **`011`** with only the two FKs whose targets exist (`candidate_id`, `job_application_id`); `interview_id`, `assessment_assignment_id` and `offer_id` are added by `ALTER TABLE … ADD COLUMN … REFERENCES` in `019`/`022`/`023`, each re-declaring `ck_token_subject`. **Three `ALTER`s the traceability document tried to avoid — accepted, because the alternative is a migration that does not run** |
|
||
| `003` created `app.access_scope` | Its FKs point at `app.job` (`006`), `app.job_application` (`011`) and `app.talent_pool` (`024`) | Created in **`011`**; `talent_pool_id` added by `ALTER` in `024`, re-declaring `ck_access_scope_target` |
|
||
| `003` created `app.v_user_effective_scope` "branches 1–2" | Branch 2 selects from `app.job_assignment`, created in `006`. A view over a non-existent relation fails at `CREATE VIEW` time | Branches 1–2 created in **`011`**; branch 3 added by `CREATE OR REPLACE VIEW` in **`019`** |
|
||
| `010` created `app.intake_resolution` | FKs to `job_application` (`011`) and `matching_config_version` (`013`) | Created in **`011`** — it is the resolution *of* an application, so that is also where it belongs conceptually. `matching_config_version_id` starts as a bare nullable `bigint` and gets its `REFERENCES` by `ALTER` in `013` |
|
||
| `020` created `app.task` with an `offer_id` FK | `app.offer` is `023` | `task.offer_id` added by `ALTER` in **`023`** |
|
||
| `027` (Phase 4) created `app.job_posting`, but `job_application.job_posting_id` is a Phase 1 FK in `011` | A Phase 1 migration cannot reference a Phase 4 table | `app.job_posting` split into **`006a`** (Phase 1, careers-portal postings only); `job_posting_metric` and external-platform state stay in `027`. This is the same split `08` GAP-25 recommends for §5's table-map row |
|
||
|
||
Also corrected: an earlier draft said branch 3 of `v_user_effective_scope` arrives in `016` in one line
|
||
and `019` in another. It is **`019`** — that is where `interview_participant` is created.
|
||
|
||
**The four GAP-27 objects with ordering consequences, and how each is placed.** The other five drop
|
||
into an existing migration as a column and need no note (`source_service` → `004`,
|
||
`data_subject_kind` → `004`, `injection_signal` → `010`, `match_state` → `012`,
|
||
`ref.location.region_id` → `002`). These four do not:
|
||
|
||
| Object | Ordering problem | Placement |
|
||
|---|---|---|
|
||
| `app.access_grant` | Its five typed nullable subject FKs (§7.7) point at `candidate` (`009`), `job` (`006`), `job_application` (`011`), `interview` (`019`) and `offer` (`023`) — the same shape that made `candidate_access_token` unbuildable at `003` | Created in **`011`** with the three FKs whose targets exist; `interview_id` added by `ALTER` in **`019`**, `offer_id` in **`023`**, each re-declaring `ck_access_grant_subject`. Between `011` and `019` a row with `subject_table = 'interview'` matches no branch of that CHECK and is rejected — the constraint enforces its own availability window, so no application code can get ahead of the schema |
|
||
| `v_user_effective_scope` **branch 4** | It reads `access_grant` (`011`), `permission` (`003`), and `LEFT JOIN`s `interview` (`019`) and `offer` (`023`). A view over a non-existent relation fails at `CREATE VIEW` time, exactly as branch 2 did at `003` | Branch 4 is created in **`011`** without the two `LEFT JOIN`s and without their `coalesce` terms; **`019`** adds the `interview` join in the same `CREATE OR REPLACE VIEW` that adds branch 3; **`023`** adds the `offer` join. All three replacements change only the body, never the column list — which is the only thing `CREATE OR REPLACE VIEW` permits, and the reason the column list (including `region_id`, `candidate_id` and `permission_key`) is declared complete at `011` |
|
||
| `ref.region` + `ref.location.region_id` | An FK from `ref.location` to `ref.region` inside `002`, where both are created | **`002`**, with `ref.region` written **before** `ref.location` in the file. Same-file ordering, not a cross-file dependency, so the forward-reference gate sees nothing to complain about — but statement order inside the file is load-bearing and a comment says so |
|
||
| `app.access_scope.region_id`, its CHECK branch and its `scope_key` entry | All three must land **with the table**, not after it: `ck_access_scope_target` is *tightened* by adding `region_id` to the other seven branches (a later `ALTER` would have to be validated against live rows), and `scope_key` is a generated column, which cannot be redefined in place — changing it later means `DROP COLUMN` + `ADD COLUMN` + rebuilding `uq_access_scope_key` on the table every authenticated request reads | All three in **`011`** with the table (§7.3). Only `'region'` in the `scope_type` CHECK is deferred, and that is a `DROP CONSTRAINT` / `ADD CONSTRAINT` pair — the cheapest possible unit of work, and it is the unit OPEN-05 gates |
|
||
|
||
**The read-only roles are split across three files, and the split is dependency-driven rather than
|
||
arbitrary.** §28.6's three roles are created inert in **`001`** so every later `GRANT` has a role to
|
||
name; their privileges and policies then follow the tables they cover. `021a` (Phase 2) can policy
|
||
everything up to and including the `021` analytics views. `offer` and `offer_version` do not exist until
|
||
`023`, so `ats_support_readonly`'s money-column exclusion is completed **in `023`** — the same "the
|
||
migration that creates the target adds the forward-dependent object" pattern the subject FKs use, and
|
||
the reason `023` now depends on `021a`. `ats_ai_reader`'s `027a` is **conditional**: the role sits at
|
||
zero privileges indefinitely, and the file is authored only if ADR 0010's revisit condition fires.
|
||
Two consequences worth stating plainly. First, **`021a` must ship the `talentflow_app` pass-through
|
||
policy in the same statement block as every `ENABLE ROW LEVEL SECURITY`** — `talentflow_migrate` owns
|
||
the tables, so `talentflow_app` is not the owner and is subject to RLS; omitting the pass-through is a
|
||
production lockout of the entire application, which is precisely the asymmetric failure adr/0009
|
||
rejects RLS-as-primary over. Second, a migration that enables RLS is the one class of migration whose
|
||
constraint test must connect as **`talentflow_app`** and assert that a known row is *still visible* —
|
||
every other constraint test asserts a forbidden write raises.
|
||
|
||
**The forward-reference CI gate.** The rule "a migration may only reference objects created in a
|
||
lower-numbered file" is now checked, because four violations survived several reviews of a document that
|
||
states the rule in its own opening paragraph:
|
||
|
||
```
|
||
For each migrations/NNN_*.sql, in ascending order:
|
||
1. Parse referenced relations: REFERENCES <rel>, FROM/JOIN <rel>, ALTER TABLE <rel>,
|
||
ON <rel>, and the argument of every regclass literal.
|
||
2. Maintain the cumulative set of relations CREATEd by files <= NNN.
|
||
3. Fail the build naming the file, the line and the relation for any reference
|
||
to a relation not yet in that set (schema-qualified; `queue.*` and `pg_*` exempt).
|
||
```
|
||
|
||
Cheap to write, and it is the difference between an ordering rule and an ordering *habit*. Assigned to
|
||
Ahmed with a Talha review; the four rows above are its regression fixtures.
|
||
|
||
**The `queue` schema is not ours, and the order matters.** §1.1 lists six schemas but this document
|
||
designs five — `queue` is created and owned by `procrastinate`'s own migrations. Migration `001`
|
||
therefore does `CREATE SCHEMA IF NOT EXISTS queue` **only** as a safety net, and the deployment runbook
|
||
runs procrastinate's migrations before any application migration that enqueues. This is load-bearing
|
||
rather than tidy: ADR 0004's entire justification is that an intake insert and its parse-job enqueue
|
||
commit in **one transaction**, so the first `enqueue` in the same transaction as an intake insert needs
|
||
somewhere to write. If `queue` does not exist at that moment, the transaction that was supposed to make
|
||
"no document is ever silently lost" true is the one that fails.
|
||
|
||
- **001 → 010 is the Phase 1 vertical slice and must land in order.** Nothing about intake, candidates
|
||
or applications can be demonstrated until migration 011 exists, and 011 depends transitively on
|
||
every file before it. That chain is the honest reason a one-month timebox yields Phase 0 plus one
|
||
vertical slice rather than a platform.
|
||
- **Migration 009 creates the merge-tolerant index predicates before merge exists.** Deliberate: adding
|
||
`AND suppressed_by_merge_id IS NULL` to a live unique index later requires a concurrent rebuild on a
|
||
table with hundreds of thousands of rows. Paying for the column and predicate up front costs nothing.
|
||
- **Communications is split across `011a` (Phase 1) and `018` (Phase 2), and the numbering is the
|
||
point.** Phase 1 owns inbound email and CV parsing, so it must be able to *reply* — the "send an
|
||
unprotected copy of your CV" path in `04` §4.3 — and an NDR cannot be classified without an
|
||
`outbound_message` row to attach it to (§22; `04` §9.1 row 5; `07` T-16b). **No file may carry a
|
||
number an earlier phase reaches if applying it early would break that earlier phase's writes — and
|
||
leaving the whole area at `018` with a Phase 1 label would have broken exactly that:** migrations
|
||
apply in numeric order, `017b` is Phase 2, and `017b` cannot simply be applied
|
||
early because it tightens `approval_request.route_id` to `NOT NULL` — do that during Phase 1 and
|
||
every Phase 1 `approval_request` insert fails, since route resolution does not exist until
|
||
Phase 2. So the Phase 1 objects take a number *below* `017b` rather than a phase label that lies
|
||
about apply order. `011a` is the right slot: everything it touches (`raw_intake`, `candidate`,
|
||
`job`, `job_application`) exists by `011`, and nothing in it depends on approvals.
|
||
- **That rule is about apply order, not about the phase column ascending — and the tail of §33's
|
||
table does not ascend.** Stating it as "the sequence must be monotone in phase" would be a rule the
|
||
table itself breaks in three places: `026` (Phase 2 read-only / 4 full) sits above the Phase 3
|
||
`025`/`025a`, `028` (`pgvector`, Phase 2) sits above the Phase 4 `027`/`027a`, and `029` (Phase 3)
|
||
sits last of all. Numbers running ahead of phases is harmless where the intervening files are
|
||
purely **additive** — new tables, an extension, materialised views — because a deployment that has
|
||
to reach one of them applies the files below it without tightening anything an earlier phase must
|
||
already satisfy. That covers `027`, `027a`, `028`, `029` and `025a`. **One tail inversion is the
|
||
`017b` shape rather than the harmless one, and is flagged here rather than silently renumbered:**
|
||
`025` adds `ck_activation_gate` and `tg_scoring_activation_gate` to `scoring_config_version`
|
||
(§20.1), so a deployment that reaches the Phase 2 `026` applies `025` on the way, and from that
|
||
point any `activated_at` write needs an `activation_evaluation_run_id` naming a `passed`
|
||
`evaluation_run` against a frozen dataset — objects Phase 1 and Phase 2 have no way to produce. If
|
||
Phase 1 or Phase 2 ever activates a scoring config version, the two available fixes are the two
|
||
`011a` chose between: give the gate a number above every file those phases reach, or move `026`
|
||
below `025`. It is recorded as a numbering question rather than resolved in place because §33's
|
||
file numbers are cited throughout `07` and `08`, so moving one is a package-wide edit and not a
|
||
§33 edit.
|
||
- **Forward subject FKs are added by the migration that creates the subject.** `outbound_message`
|
||
and `notification` each carry a set of mutually exclusive subject FKs (§22, §22.1), and four of
|
||
those targets are created after `011a`: `approval_request` (`017a`), `interview` (`019`),
|
||
`assessment_assignment` (`022`), `offer` (`023`). Each of those migrations adds its own nullable
|
||
column and **re-declares the owning table's `num_nonnulls(...) <= 1` CHECK**, which is the only
|
||
way that constraint stays true at every point in the sequence. Dropping and re-adding a CHECK is
|
||
cheap; a forward-declared FK to a table that does not exist is not expressible at all. The same
|
||
applies to the AI links: `outbound_message.ai_run_id` and `message_template_version.ai_suggestion_id`
|
||
target `ai` tables created in `016`, so `016` back-fills both FKs and declares
|
||
`ck_outbound_ai_human` (§22.1) at that point. **Consequence worth stating plainly: between `011a`
|
||
and `016` there is a window in which `outbound_message` has no AI-provenance constraint. That is
|
||
safe only because nothing AI-drafted can be sent before `016` exists** — the AI ledger *is* `016`.
|
||
Same reasoning as `016`'s AI back-fill below.
|
||
- **Do not "fix" this by putting the whole area back at `018` with a split phase label.** That was
|
||
the first attempt and it is wrong for the reason above: a phase label cannot reorder an apply
|
||
sequence. It is a numbering decision that costs minutes now and an already-applied, renumbered
|
||
migration later.
|
||
- **Migration 016 back-fills AI foreign keys** rather than 012 forward-declaring them, because
|
||
`ats_result` must exist before the AI ledger it points at can be sensibly modelled, and a circular
|
||
dependency between 012 and 016 would force one of them to be split. This is the only back-fill in
|
||
the sequence and it is deliberate.
|
||
- **Every migration that creates a trigger must, in the same file, create nothing else that depends on
|
||
that trigger having run.** Data seeding that must fire triggers goes in its own numbered file.
|
||
- **Constraint tests are numbered alongside migrations** (`tests/db/test_009_candidate.py` etc.), so a
|
||
migration and the test that proves its constraints hold arrive in the same pull request. A migration
|
||
merged without its constraint test is an incomplete change.
|
||
- **The purge job is enabled last, after one full cycle in `dry_run` mode.** The first real purge is
|
||
irreversible, and §31.2's retention periods are assumptions until legal confirms them.
|