HR-ATS-Portal/docs/architecture/adr/0001-modular-monolith-versus-microservices.md

ADR 0001 — Modular Monolith With a Separate Worker Process, Not Microservices

Status	Accepted — 2026-07-29
Scope	Backend runtime architecture for the Utopia Brands internal HR Recruitment & ATS platform
Owner	Talha Ahmed (senior)
Consistent with	_decisions.md Part 1 → Architecture style, Module boundary enforcement and dependency rules, Which processes are separate, Deployment topology
Related ADRs	0002 (one PostgreSQL), 0004 (in-database queue — the mechanism that makes the two-process split safe)

---

Context

The repository imposes no backend constraint at all. Direct inspection (_repo-findings.md §B) confirms the absence of package.json, requirements.txt, pyproject.toml, go.mod, Dockerfile, docker-compose.yml, any migration directory, any ORM or database driver, any API route or controller, any test file, and any CI configuration. What exists is a static browser-only prototype: index.html (287 lines), css/styles.css (1269 lines) and 22 unbundled scripts totalling ~4,780 lines, with zero network calls anywhere in js/ or index.html and all data generated in-browser by a seeded LCG (js/data.js:8-10).

The consequence is precise and worth stating so nobody re-litigates it later: the "do not replace the existing stack" rule constrains only the frontend and design system, not the backend. There is no backend stack to preserve. This ADR is therefore a greenfield choice bounded by three things — team size, actual load, and the platform's hard constraints.

Forces

Force	Evidence	Consequence for this decision
Two developers, one of them junior, one reviewer	_repo-findings.md §I	Operational surface per deployable is the dominant cost, not code elegance
66 named seats; ~20–25 realistic peak concurrency (assumption)	BRD §4 (2+4+15+12+8+24+1); 24 of the 66 are interviewers who touch only their own interviews	No scale argument for distribution exists
ONE master data model, ONE relational database, not multi-tenant	Platform constraint	Microservices would either share a database (anti-pattern) or split the model against an explicit constraint
The Phase 1 critical path crosses four module boundaries transactionally	intake → document_parsing → candidate → application → scoring; job_application.raw_intake_id and candidate.created_from_raw_intake_id are both NOT NULL against non-deferrable FKs (see _decisions.md Part 2)	In-process this is one COMMIT; across services it is a saga with compensations
Two workloads with genuinely different runtime properties	Parsing a scanned CV is CPU-bound and multi-second; a recruiter request is I/O-bound and sub-second. 200–600 documents/day at peak (assumption)	They must not share a request thread — this is the one split that is earned on day one
Unenforced modularity has already failed once in this repo	22 ordered <script> tags, everything on window, any file can reach any other (_repo-findings.md §C, index.html:264-285); the flat candidate array carrying jobId, stage, aiScore (js/data.js:117-127)	Boundaries must be machine-enforced, not conventional
Authorization must exist exactly once	Today the RBAC matrix is a display widget; clicking a cell mutates an in-memory array and nothing reads it (js/rbac.js:78, js/rbac.js:83-85); security settings are inert chrome (js/settings.js:148-154)	A single iam.can() chokepoint is a security requirement, especially for "the chatbot must never bypass access controls"
Forbidden without demonstrated need	Kubernetes, Kafka, microservices, Elasticsearch, multiple databases, a separate AI service in Phase 1	The architecture must be defensible without any of them

---

Options considered

Option A — Modular monolith: one codebase, one image, two processes

One Django project, 25 logical modules in five tiers, one container image, two runtime revisions differing only by entrypoint (web, worker), one PostgreSQL database.

Pros

• The four-module Phase 1 write path is a single database transaction. The raw-intake-before-candidate invariant is enforced by a NOT NULL FK rather than by a saga that must not be interrupted.
• One deploy pipeline, one log stream, one metrics dashboard, one backup story. Two people can actually hold this in their heads.
• One authorization implementation. ai_orchestration.invoke() calls the same iam.can() the REST API calls, with the human actor propagated — so the chatbot cannot see what the asking user cannot.
• Refactoring across module boundaries is a compile/type-check away, which matters because splitting candidate from application (js/data.js:117-127) will churn entity shapes through Phase 1.
• pgvector, FTS and trigram all live inside the same database, which is what makes "no separate AI service in Phase 1" practically achievable rather than aspirational.

Cons (stated, not minimised)

• Shared fate on deploy: a bad migration or a bad release stops recruiters and intake.
• One runtime and one dependency set for everything. An OCR or ML library that conflicts with a web-tier dependency blocks both.
• Horizontal scaling is coarse — you scale the whole image, not the hot module.
• Boundary discipline depends on CI tooling (import-linter) that somebody has to maintain and that a determined shortcut can be talked past in review.
• No independent module release cadence; prompt/model tweaks ride the domain release train.

Option B — Microservices (a service per domain area)

Pros — the real ones, not a strawman

• Genuine failure isolation: an OOM in parsing cannot degrade the recruiter UI at all.
• Independent scaling and independent release cadence per service; the AI service could deploy daily while domain code is release-gated.
• Hard, unbypassable module boundaries — a network call cannot accidentally import another module's models.py.
• Heterogeneous runtimes become possible (a non-Python model runtime, for example).
• Onboarding a third and fourth engineer later has a clean ownership story.

Cons

• Operationally disqualifying at this team size: 15–20 pipelines, 15–20 dashboards and alert routes, inter-service contract versioning, distributed tracing, and a local development story that involves running most of the estate. Two developers cannot own that and also ship 25 modules.
• Directly collides with the platform constraints. One master data model plus one relational database means the services share a database — the textbook anti-pattern — or the data model is fragmented against an explicit rule. There is no third answer.
• Transactional integrity on the critical path degrades into sagas plus compensating transactions. With one senior reviewer, the realistic outcome is orphaned candidates and applications with no raw intake, which is the exact invariant the brief calls non-negotiable.
• Authorization gets re-derived in every service. Twenty places to get can() wrong is a worse security posture than one.
• It drags in the forbidden infrastructure (an orchestrator) as a near-inevitability.
• Zero load justification: 66 seats, ~25 peak concurrent users.

Option C — Hybrid: monolith plus one extracted AI/parsing service

Pros

• Isolates the one genuinely different workload (CPU-bound, untrusted-file-handling, potentially GPU-bound later) behind a network boundary.
• Lets the AI/parsing component take a different dependency set and, eventually, different hardware.
• Keeps the domain model in one place, so it does not violate the one-database rule as long as the extracted service is stateless.

Cons

• The isolation it buys is already available as a process split inside one image and one codebase — same failure isolation for the web tier, without a second deploy surface, a second dependency graph, or an HTTP contract to version.
• It fractures the audit trail unless carefully engineered: AiRun rows must join to applications in the same database (BRD §6.2, §7.3), so an extracted AI service either writes to the shared database anyway or ships events that can be lost.
• Doubles the deploy and secret-management surface for two developers in exchange for a boundary CI can already enforce in-process.

Verdict: correct eventually, premature now. Kept as the named first split with measurable triggers (see Revisit conditions), which is materially better than either doing it now or leaving it undefined.

Option D — Serverless functions per module

Pros

• No servers to size or patch; scale-to-zero is genuinely cheap for a system idle outside business hours across six jurisdictions.
• Per-function scaling suits bursty CV batches well.

Cons

• Cold starts land directly on the interactive AI/chatbot path, which is the most latency-visible feature in the product.
• No shared connection pool against a single managed Postgres — connection exhaustion is the standard failure, and this design's connection budget is already tight (see ADR 0004).
• Execution ceilings (typically ~10 minutes) are wrong for OCR batches and full-requisition rescoring.
• Debuggability is poor for a junior developer, and local development diverges sharply from production.
• Fragments the transactional write path exactly as microservices do.

Option E — Single-process monolith with in-process background threads

Pros

• The simplest possible operational footprint: one process, one revision, nothing to coordinate. Cheapest hosting.
• No queue technology to learn.

Cons

• Puts multi-second OCR CPU load on the request path, degrading interactive latency unpredictably.
• No retries, no visibility, no job status. Work in flight is lost on every deploy — unacceptable when BRD §6.3 requires that no document is ever silently lost and BRD §8.3 requires retrievable async job status.
• No path to running untrusted-file parsing under a restricted OS user with no outbound network, which is a Phase 2 security requirement.

---

Decision

Modular monolith. One Django 5 / DRF codebase, one container image, exactly two runtime processes, one PostgreSQL 16 database. Module boundaries are logical — Python packages with a public service facade — not network boundaries.

graph LR
  subgraph IMAGE["ONE container image, ONE codebase"]
    WEB["web — uvicorn ASGI<br/>/api/v1/* + static"]
    WORKER["worker — queue consumer<br/>ingest · parse · score · ai · mail · maintenance"]
  end
  CDN["Platform CDN<br/>built React bundle"] --> WEB
  WEB --> PG[("PostgreSQL 16<br/>one logical database")]
  WORKER --> PG
  WEB --> REDIS[("Redis<br/>cache · rate limit · sessions")]
  WEB --> BLOB[["Object storage<br/>CV blobs"]]
  WORKER --> BLOB
  WORKER --> EXT["Microsoft Graph · AI provider · job boards"]

1. Processes

Process	Entrypoint	Contains	Sizing (per environment)
web	uvicorn ASGI	/api/v1/*, the built React bundle as static files, one async streaming endpoint for model output	2 vCPU / 4 GB, 2 uvicorn workers × 4 threads
worker	queue consumer	document parsing, all model invocations, batch rescoring, Graph mail polling, fairness evaluation, periodic/scheduled tasks	2 vCPU / 4 GB, concurrency 4

In Phase 2 the worker splits by queue, not by codebase: worker-default and worker-untrusted. The untrusted process runs parsing under a restricted OS user with no outbound network, a hard per-document CPU/wall timeout, and a memory cap. Same image, same repository, third revision.

2. Module tiers and the one-way dependency rule

Five tiers: surfaces → core domain → platform, and intelligence → core domain (read) + platform.

#	Rule	Enforcement
1	Every module is a Python package whose only public entry point is service.py; cross-module imports may touch only <module>.service and <module>.dto — never models, views or selectors	import-linter forbidden-import contract
2	No module reads or writes another module's tables. Sole exception: analytics, which owns read-only SQL views declared in migrations	Code review + import-linter; view ownership is explicit in migration files
3	Core domain modules must never import intelligence or surfaces. AI results are attached by the domain module accepting a suggestion	import-linter layered contract
4	identity, audit and config are ambient dependencies of every module	Layered contract exempts them
5	A violation is a failed build, not a review argument	Required CI job

Rule 3 is the load-bearing one. Because intelligence cannot import application, an AI module is physically unable to call application.transition() with a rejection. Combined with the guard that rejects any terminal-negative transition whose actor_kind <> 'user' — the exact literal, per _decisions.md RULING-01; actor_kind is ('user','system','integration','ai_agent') and has no human member — "AI must never auto-reject" becomes a property of the dependency graph and a database constraint rather than a paragraph in a policy document.

3. What is explicitly not built

No service mesh, no orchestrator, no per-module deployable, no separate AI service, no separate search service, no message broker outside Postgres (ADR 0004), no read replica in Phase 1.

---

Justification

Team size decides this, and it is not close. The cost of a deployable is not its code — it is its pipeline, its dashboards, its alert routing, its secrets, its contract versions and its local-dev story. Two developers with one reviewer have a budget for one or two of those, not fifteen. Every hour spent on inter-service plumbing is an hour not spent on the versioned requisitions, reversible merge and explainable scoring that are the actual product.

Load provides no counter-argument. 66 named seats, ~20–25 peak concurrent (assumption), 20k–60k applications/year and 200–600 documents/day at peak (assumptions). A single 2-vCPU web process is generously provisioned; a single 2-vCPU worker handles the document volume with headroom.

The hard constraints make microservices actively worse, not merely unnecessary. One master data model plus one relational database is incompatible with independent service datastores. Any microservice topology here converges on a shared database — which loses the isolation that was the entire point while keeping all the operational cost.

Transactional integrity is where the brief's invariants live. Raw intake must exist before a candidate; identity must be separate from applications; scores are per-application and pin exact versions. In-process these are foreign keys, CHECK constraints, partial unique indexes and one COMMIT. Distributed, they become eventual consistency plus compensations — and the failure mode is precisely the orphaned/duplicated state the brief exists to prevent.

Security improves rather than degrades. One iam.can() chokepoint, one audit writer, one place where the human actor is propagated into AI invocations. Twenty services each re-deriving authorization is twenty chances to leak candidate PII.

The one real loss — failure isolation — is bought back cheaply. The worker is already a separate process, so OCR and model calls cannot take the web tier down. The AI boundary carries a circuit breaker so the platform degrades gracefully with the model provider absent (BRD NFR-7, §8.3). That covers the overwhelming majority of the isolation microservices would have bought.

Judgement call, stated: we are choosing lower blast-radius isolation in exchange for correctness guarantees and operational tractability. For an internal, business-hours, 66-seat recruiting system where a few minutes of downtime costs a rescheduled screening call — not revenue — that trade is clearly right. It would be the wrong trade for a customer-facing system with an availability SLA, and this ADR should be reread if the platform is ever exposed externally.

---

Consequences

Positive

Consequence	Why it matters here
The Phase 1 critical path is one transaction	The raw-intake-before-candidate invariant is a NOT NULL FK, not a saga
One CI pipeline, one image, one deploy	The junior can deploy on day one without learning an orchestrator
One authorization decision point	Directly satisfies "the chatbot must never bypass access controls"
Cross-module refactoring stays cheap	Essential while the candidate/application split churns entity shapes
Facade-per-module is a natural test seam	Service-level tests per module give the junior varied, independently demonstrable work
AI governance is structural	Dependency direction + a transition guard, not a prompt instruction
pgvector, FTS and trigram in one engine	No separate AI or search service is needed to hit Phase 2 goals

Negative — costs we are accepting

Cost	Accepted because
Shared deploy fate. A bad release or migration stops recruiters and intake together	Business-hours internal tool; manual promote to production; PITR available
No HA. One app host means a platform restart is a few minutes of downtime	Explicitly accepted; the six-jurisdiction spread narrows the maintenance window and we schedule accordingly
One dependency graph. OCR/ML libraries share the image with the web tier; image size grows toward the ~2 GB trigger	Monitored as a hard split trigger; the split is pre-planned rather than emergent
Coarse scaling. You scale the image, not the module	At 66 seats this is theoretical; vertical scaling is the answer for years
Boundary enforcement is a tool, and tools rot. If import-linter contracts are weakened to unblock a PR, the boundaries silently stop existing	The contracts are a required check with no override path; weakening them requires a reviewed change to the contract file, which is visible in the diff
No independent release cadence for AI. Prompt/model changes ride the domain train	Named as a soft split trigger (>2 deploys/week needed)
A monolith invites accidental coupling under deadline pressure	Rules 1–3 are checkable mechanically because discipline does not scale with one reviewer

---

Risks

#	Risk	Likelihood	Impact	Mitigation
R1	The modular monolith degenerates into a ball of mud within ~6 months — the exact failure the prototype already shows (index.html:264-285)	Medium	High	import-linter layered + forbidden-import contracts as a required CI job; one service.py per module; no module touches another's tables
R2	A worker crash loop or OOM destabilises the shared host	Medium	Medium	Separate revision with its own memory cap; per-document CPU/wall timeout; queue-level isolation in Phase 2; "worker OOM took the host down twice in a quarter" is a named soft split trigger
R3	An untrusted-file parser is exploited and, being in the same image, has the web tier's credentials	Low–Medium	High	Phase 2 worker-untrusted queue under a restricted OS user, no outbound network, magic-byte allowlist, malware-scan gate before any parse attempt (ADR 0003). Named as a hard split trigger if a real sandbox is required
R4	An ML/OCR dependency becomes incompatible with the web stack	Medium	Medium	Hard split trigger at image >~2 GB or an irreconcilable conflict; the split is a new entrypoint, not a rewrite, because the codebase is already module-partitioned
R5	Someone later argues "we should have used microservices" from taste rather than evidence	High	Low	The Revisit conditions below are numeric; "AI feels like a different concern" and org-chart preference are explicitly not triggers
R6	Headcount grows and the single codebase becomes a merge-contention point	Low in Phase 1–2	Medium	Trigger T7 below; module facades mean extraction is mechanical when it is genuinely needed

---

Revisit conditions

Reviewed quarterly by Talha. Any single hard trigger justifies a separate deployable. Soft triggers require two sustained for 2+ weeks.

#	Trigger	Threshold	Hard?
T1	Dependency conflict	An ML/OCR dependency cannot coexist in the web image, or the image exceeds ~2 GB	Hard
T2	Hardware profile divergence	Parsing or inference requires a GPU, or >4 vCPU / >8 GB steady-state	Hard
T3	Runtime isolation	Untrusted-file handling requires a sandbox the worker process cannot provide, beyond the Phase 2 restricted queue	Hard
T4	Non-Python runtime	A required model runtime is not Python	Hard
T5	Queue starvation	p95 time-to-start for interactive AI tasks >10 s while web p95 <300 ms, after vertical scaling of the worker host is exhausted	Soft
T6	Release cadence conflict	Prompt/model changes require >2 deploys/week while domain code is release-gated	Soft
T7	Blast radius	Worker OOM or crash loops have taken the shared host down twice in one quarter	Soft
T8	Web tier saturation	Sustained web p95 >300 ms at 4 vCPU / 8 GB after query tuning, or CPU >70% for 2+ weeks in business hours	Soft
T9	Team shape	Engineering headcount ≥6 organised into two or more independently releasing teams	Soft
T10	Merge contention	>2 developer-days per month lost to cross-team merge conflicts or release-train blocking	Soft

Explicitly never a trigger: "AI feels like a different concern"; org-chart preference; résumé-driven architecture; multi-tenancy or regional data residency (both excluded by constraint — residency is answered by one region plus per-record retention and escalated to legal, not solved with topology).

If a trigger fires, the expected move is the cheapest one that clears it — in order: vertical scaling → a third revision from the same image with a different entrypoint/queue → extraction of document_parsing + ai_orchestration as one stateless service that still writes AiRun rows to the same database. Full domain decomposition remains off the table while the one-database and two-developer constraints hold.