diff --git a/Sync_read.md b/Sync_read.md new file mode 100644 index 0000000..6f24af0 --- /dev/null +++ b/Sync_read.md @@ -0,0 +1,228 @@ +# Read-status sync (`/sync/*`) + +Tracks **which messages got read or unread** — and which were deleted — without +re-downloading the mailbox. It sits on Microsoft Graph's **delta query**: Graph +hands you a cursor, and every later call with that cursor returns *only* what +changed since it was issued. + +Five of the six endpoints share one piece of state: a delta cursor per +**(signed-in user + folder)**, persisted to disk so a restart doesn't re-backfill +the whole folder. The sixth — the per-message lookup — is deliberately outside +that machinery: it reads one id live and touches no cursor. + +| Method | Path | Purpose | +| ------ | ---- | ------- | +| GET | `/sync/read-status` | Run one sync round **now** (synchronous) | +| GET | `/sync/read-status/changes` | Replay the last round's **full** result | +| GET | `/sync/read-status/message/{id}` | One message's status, by id — cursor-free | +| GET | `/sync/read-status/status` | Watcher health + cursor state | +| POST | `/sync/read-status/watch` | Start the background poller | +| DELETE | `/sync/read-status/watch` | Stop the background poller | + +All require the API bearer token, and act on the **signed-in user's** mailbox — +they answer `401` until device-code sign-in completes. + +--- + +## `GET /sync/read-status` + +The workhorse. Asks Graph "what changed in this folder since my cursor?", emits +the changes, and advances the cursor. + +The **first** call has no cursor, so it backfills the entire folder — an Inbox +with 4,700 messages is 47 pages of 100. Every call after that is incremental and +usually near-empty. + +| Param | Default | Meaning | +| ----- | ------- | ------- | +| `folder` | `inbox` | Well-known name (`inbox`, `sentitems`, …) or folder id. Graph delta is **folder-scoped** — there is no all-mail delta | +| `since` | – | ISO8601 lower bound, **initial sync only** (`receivedDateTime ge …`). The way to keep a first backfill small | +| `reset` | `false` | Discard the saved cursor and start a fresh baseline | +| `max_pages` | `10` | Cap on Graph pages (100 msgs each) fetched **per call** | +| `limit` | `10` | Cap on messages returned **in this response** | + +`max_pages` and `limit` are independent and easy to confuse: + +- **`max_pages` bounds the work.** Hit the cap and the call returns + `complete: false`, having saved its position; the next call resumes exactly + where it stopped. No changes are skipped, and no cursor is written until the + backfill genuinely finishes. +- **`limit` only trims the JSON.** It has no effect on how much is fetched. + `count` stays the true total, and the untruncated set is on + `/sync/read-status/changes`. + +```jsonc +{ + "synced_at": "2026-08-07T10:15:00Z", + "folder": "inbox", + "count": 1000, // changed messages this call actually fetched + "removed_count": 0, // deleted / moved out of the folder + "initial_sync": true, // this round is part of the first backfill + "complete": false, // hit max_pages — call again to continue + "pages": 10, // Graph pages fetched by this call + "truncated": true, // limit cut the lists below + "value": [ { "id": "AAMk…", "isRead": true, + "lastModifiedDateTime": "2026-08-07T10:14:52Z", + "subject": "Invoice #421" } ], + "removed": [ { "id": "AAMk…", "reason": "deleted" } ] +} +``` + +`value` is sorted newest-modified first before `limit` is applied, so a +truncated response shows the most recent changes rather than an arbitrary slice. +Only the four `$select` fields above come back — this endpoint is about *status*, +not content; use `GET /emails/{id}` for bodies. + +## `GET /sync/read-status/changes` + +Read-only replay of whatever the **last** round produced. No Graph call, cursor +untouched, safe to hit repeatedly. + +Two reasons it exists: + +1. It holds the **untruncated** lists — this is how you get the other 990 items + when `limit` trimmed the response. +2. It's the only way to collect what the **background watcher** found, since the + watcher has no caller to return to. + +`404` until some sync has run. One buffer, last-writer-wins: the next round +overwrites it, so with the watcher running you must read it faster than +`interval` or you will miss rounds. + +## `GET /sync/read-status/message/{message_id}` + +One message, one record — a point lookup rather than a batch: + +```jsonc +{ "id": "AAMk…", "isRead": true, + "lastModifiedDateTime": "2026-08-07T10:14:52Z", "subject": "Invoice #421" } +``` + +Identical shape to an entry in a sync `value` list, so both parse with the same +code. What makes it different from the endpoints above: + +- **Cursor-free.** Touches no delta cursor, no cached state, and advances + nothing. Call it as often as you like without affecting a sync in progress. +- **Live.** Reports the mailbox *now*, straight from Graph — not what the last + round happened to capture. That makes it the right tool for re-checking one + message ("has this been read yet?") and for confirming a status after the fact. +- **Any id.** Works whether or not the message appeared in a sync, and whatever + folder it lives in. + +It costs one Graph call per message, so it's a lookup, not a substitute for +delta — walking a mailbox with it would be far slower than a single sync round. + +```bash +curl -H "$A" "$B/sync/read-status/message/AAMkAGI2TG93..." +``` + +URL-encode the id. Ids containing `/`, `+`, or `=` are handled (the route uses a +`:path` converter), so an already-encoded `%2F` works too. Unknown or deleted +ids surface Graph's own `404 ErrorItemNotFound`. + +## `GET /sync/read-status/status` + +Health check for the whole subsystem. + +| Field | Meaning | +| ----- | ------- | +| `watching` / `interval` | Is the poller thread alive, and at what period | +| `folder` | Folder the cursor belongs to | +| `last_sync_at` | Timestamp of the most recent round | +| `last_change_count` / `last_removed_count` | Size of that round | +| `has_delta_link` | A real cursor exists ⇒ running incrementally | +| `backfill_in_progress` | Paused mid-backfill at the page cap ⇒ more rounds to go | +| `last_error` | Last Graph failure from the background thread, else `null` | + +`has_delta_link: false` + `backfill_in_progress: true` is the normal state +*during* a long first sync. + +## `POST /sync/read-status/watch` + +Starts a daemon thread that runs the same sync every `interval` seconds and +writes each change to stdout. + +```jsonc +{ "interval": 60, "folder": "inbox" } // interval min 10, both optional +``` + +- Idempotent — a second POST while running just answers + `{"message": "Already watching read-status changes"}`. +- While a backfill is still incomplete the loop continues immediately instead of + sleeping out the interval, so a big first sync finishes in consecutive chunks. +- Delivery is `_emit_read_status_changes()`, which prints. **That's the hook + point** — replace it to push to Slack, a webhook, or a queue. + +## `DELETE /sync/read-status/watch` + +Signals the thread to stop; `404` if nothing is running. The cursor survives, so +restarting the watcher resumes from where it left off rather than re-backfilling. + +--- + +## Typical first run + +```bash +export EMAIL_API_TOKEN=... +A="Authorization: Bearer $EMAIL_API_TOKEN" +B=http://localhost:5000 + +curl -X POST -H "$A" $B/auth/start # sign in once (see README) + +# Baseline. Keep calling while "complete": false. +curl -H "$A" "$B/sync/read-status?since=2026-08-01T00:00:00Z" + +# From here on, each call returns only what changed. +curl -H "$A" "$B/sync/read-status" + +# Or hand it to the background poller and read results out of /changes. +curl -X POST -H "$A" -H "Content-Type: application/json" \ + -d '{"interval":60,"folder":"inbox"}' $B/sync/read-status/watch +curl -H "$A" $B/sync/read-status/status +curl -H "$A" $B/sync/read-status/changes + +# Re-check one message any time — no cursor involved. +curl -H "$A" "$B/sync/read-status/message/AAMkAGI2TG93..." +``` + +## Which endpoint do I want? + +| You want | Use | +| -------- | --- | +| Everything that changed since last time | `GET /sync/read-status` | +| The full list a round produced (or the watcher's) | `GET /sync/read-status/changes` | +| The status of **one** message you already have an id for | `GET /sync/read-status/message/{id}` | +| Continuous tracking without calling in a loop | `POST /sync/read-status/watch` | +| Whether any of the above is healthy | `GET /sync/read-status/status` | + +Rule of thumb: **delta for "what changed", point lookup for "what about this +one".** Using the lookup in a loop over a mailbox works but costs one Graph call +per message — a single sync round does the same job in pages of 100. + +## How the cursor works + +- A finished round returns Graph's **deltaLink**, saved to + `.delta_cache.json` (override with `EMAIL_API_DELTA_CACHE`; in Docker it lives + on the `/data` volume beside the token cache). Keyed by user + folder — change + either and the cache is ignored rather than misapplied. +- A round stopped by `max_pages` has no deltaLink yet, so it saves Graph's + **nextLink** instead. That resume position takes priority over any older + deltaLink on the following call, which is what makes a capped backfill safe: + the cursor never advances past data you haven't received. +- Cursors expire. Graph answers `410 Gone`, and the sync automatically falls + back to a fresh baseline for that folder. +- `reset=true` throws the cursor away deliberately — expect a full backfill, and + pass `since` with it unless you want the whole history again. + +## Limits worth knowing + +- **Folder-scoped only.** `/me/messages/delta` is not supported by Graph. Watch + another folder by passing `folder=`, but each folder is its own cursor and the + disk cache holds one at a time — switching folders forces a re-backfill. +- **Polling, not push.** Latency floor is the poll `interval`. True push needs a + Graph change-notification subscription (public HTTPS endpoint, validation + handshake, ~3-day renewals) — and you'd keep delta anyway as the catch-up path + for dropped notifications. +- **Single worker.** Cursor, watcher thread, and the `last_changes` buffer are + in-memory per process, so this only behaves with one uvicorn worker (which is + what the Docker service runs, for the same reason auth needs it). diff --git a/Sync_write_request.md b/Sync_write_request.md new file mode 100644 index 0000000..c906988 --- /dev/null +++ b/Sync_write_request.md @@ -0,0 +1,86 @@ +# Email service — write read-status (`PATCH /sync/read-status/...`) + +Copy everything below the line into any LLM session (or hand it to whoever owns the +email microservice) before implementing the write endpoint. + +--- + +You are extending the **email microservice** that already exposes the read-status +delta and point-lookup APIs documented in `Sync_read.md`. Implement a **write** +path that marks a message read (or unread) in the signed-in user's Outlook mailbox +via Microsoft Graph. Mirror the existing `/sync/read-status/*` style exactly — +same bearer auth, same `:path` id handling, same response shape. + +## Why we need this + +The HR-ATS inbox app learns that a user opened a message before Outlook does. +Today that signal dies in our database: we have no Graph write permission and the +email service exposes no write endpoint. Without this PATCH, local mark-read and +Outlook drift permanently (and a later delta can even revert our flag). + +## Requested contract + +Mirror the existing read endpoints so both parse with one code path: + +``` +PATCH /sync/read-status/message/{id} +Authorization: Bearer +Content-Type: application/json + +{ "isRead": true } +``` + +**200 response** — identical shape to `GET /sync/read-status/message/{id}`: + +```jsonc +{ + "id": "AAMk…", + "isRead": true, + "lastModifiedDateTime": "2026-08-07T10:14:52Z", + "subject": "Invoice #421" +} +``` + +Same `:path` converter for Graph ids that contain `/`, `+`, or `=`. Same bearer +auth as every other `/sync/*` route. Answer `401` until device-code sign-in +completes. + +## Required behaviour + +- **Idempotent.** Re-PATCHing `isRead: true` when already true is a no-op `200` + with the current record. +- **Must not advance or disturb the delta cursor.** This is a point write, not a + sync round. Cursor, watcher, and `/changes` buffer stay untouched. +- **404 `ErrorItemNotFound`** for unknown or deleted ids (same as the GET). +- **403 surfaced distinctly** if the Graph scope is missing, so callers can tell + "not permitted" from "not found". + +## Graph scope prerequisite + +Needs `Mail.ReadWrite`. The service currently signs in read-only. Treat upgrading +the consent / device-code scopes as an explicit product decision before shipping +the route — not an implementation footnote. + +## Optional batch form + +For bulk reconcile without N round-trips: + +``` +PATCH /sync/read-status/messages +{ "ids": ["AAMk…", "AAMk…"], "isRead": true } +``` + +Return a list of the same per-message records (or per-id errors). Nice-to-have; +the single-id PATCH is the hard requirement. + +## What the caller will do with it + +HR-ATS will enqueue one Taskiq task per human mark-read, retried via existing +smart-retry middleware. Expected volume is low (opens, not sweeps). After this +lands we will stop treating local-only mark-read as a known divergence. + +## Out of scope for this request + +- Changing the delta `/sync/read-status` contract +- Push / Graph change-notification subscriptions +- Writing any field other than `isRead` diff --git a/backend/.env.example b/backend/.env.example index f299328..32f5440 100644 --- a/backend/.env.example +++ b/backend/.env.example @@ -4,6 +4,10 @@ DB_HOST= DB_PORT= DB_NAME= EMAIL_URL= +EMAIL_API_TOKEN= +EMAIL_SYNC_FOLDER=inbox +EMAIL_SYNC_SINCE= +EMAIL_SYNC_CRON=* * * * * JWT_SECRET_KEY= JWT_ALGORITHM=HS256 @@ -26,3 +30,25 @@ CONFIRM_TOKEN_RESEND_SECONDS=60 BUFFER_API= BUFFER_API_URL=https://api.buffer.com BUFFER_CHANNEL_ID= + +OPENAI_API_KEY= +OPENAI_MODEL=gpt-5.4-mini +# Blank omits the parameter, for reasoning models that reject it. +OPENAI_TEMPERATURE=0 +OPENAI_MAX_OUTPUT_TOKENS=4096 +OPENAI_TIMEOUT=60 +OPENAI_MAX_RETRIES=3 +OPENAI_CONNECT_RETRIES=3 +# Set only for Azure OpenAI or a gateway; blank uses api.openai.com. +OPENAI_BASE_URL= +OPENAI_ORGANIZATION= +OPENAI_PROJECT= + +REDIS_URL=redis://localhost:6379/0 +TASKIQ_QUEUE_NAME=inbox +TASKIQ_MAX_RETRIES=3 +TASKIQ_RETRY_DELAY=5 +TASKIQ_MAX_DELAY=120 +TASKIQ_DLQ_STREAM=taskiq:dlq +TASKIQ_IDLE_TIMEOUT_MS=600000 +APP_VERSION=dev diff --git a/backend/Dockerfile b/backend/Dockerfile new file mode 100644 index 0000000..e462a2e --- /dev/null +++ b/backend/Dockerfile @@ -0,0 +1,12 @@ +FROM python:3.12-slim + +WORKDIR /app + +COPY requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt + +COPY . . + +# Runs the Taskiq worker against taskiq_management.broker_setup. +# docker-compose overrides this command if needed. +CMD ["taskiq", "worker", "taskiq_management.broker_setup:broker", "inbox.tasks", "taskiq_management.tasks"] diff --git a/backend/README.md b/backend/README.md new file mode 100644 index 0000000..290a607 --- /dev/null +++ b/backend/README.md @@ -0,0 +1,676 @@ +# HR-ATS-Portal — Backend + +FastAPI service behind **TalentFlow**, an HR / Applicant-Tracking portal. It pulls candidate +applications out of a mailbox, decodes CV attachments, runs an LLM agent to match each CV +against live job posts, publishes job ads to social channels through Buffer, and serves the +whole thing to the frontend behind JWT auth and a database-driven RBAC model. + +Everything in this document describes `backend/` only. + +--- + +## Table of contents + +- [Architecture](#architecture) +- [Tech stack](#tech-stack) +- [Directory layout](#directory-layout) +- [House style — what lives in which file](#house-style--what-lives-in-which-file) +- [Domains](#domains) +- [Data model](#data-model) +- [API reference](#api-reference) +- [Authentication and RBAC](#authentication-and-rbac) +- [Background jobs](#background-jobs) +- [The matching agent](#the-matching-agent) +- [External integrations](#external-integrations) +- [Configuration](#configuration) +- [Running locally](#running-locally) +- [Database migrations](#database-migrations) +- [Docker](#docker) +- [Response conventions](#response-conventions) +- [Known gaps and gotchas](#known-gaps-and-gotchas) + +--- + +## Architecture + +```mermaid +flowchart TB + FE["Frontend (Vite, :5173)"] -->|JWT Bearer| API + + subgraph API["FastAPI — main.py :8000"] + R1["users / role / forget_password / notifications"] + R2["inbox"] + R3["job (job_post + candidate)"] + end + + API --> PG[("PostgreSQL — schema app")] + API -->|enqueue| REDIS[("Redis Streams")] + + REDIS --> W["Taskiq worker\ninbox.tasks + inbox.sync_tasks"] + SCHED["Taskiq scheduler\ncron"] --> REDIS + W --> PG + W --> AGENT["LangGraph agent\nagent/"] + AGENT --> OAI["OpenAI"] + + API -->|GET /emails, /sync/read-status| MAILAPI["Email API (MS Graph proxy)"] + W --> MAILAPI + API -->|multipart send| TEAMS["Teams Mail API"] + API -->|GraphQL| BUF["Buffer"] +``` + +**The application flow, end to end:** + +1. `GET /email/fetch` pulls messages from the external Email API, decodes PDF/DOC/DOCX + attachments to `inbox/decoded_attachments/`, and upserts them into `inbox_messages`. +2. When a message carries an attachment, the sender is linked to a `users` row — created with + the `candidate` role if new — through the `inbox` join table. +3. Any message with a stored attachment and no match result yet is enqueued onto Redis as an + `inbox.match_message` task. +4. The worker extracts the résumé text, hands it plus the active job posts to the LangGraph + agent, and writes `suggested_job_post_ids`, `match_summary`, `match_reasoning` and + `experience` back onto the row. +5. New candidate accounts land inactive and are mailed a confirmation link; the link is what + flips `is_active`. +6. A cron task sweeps Outlook read-status deltas back onto `inbox_messages.message_read`. +7. Recruiters read all of it through `/inbox/all-applications` and publish new roles with + `POST /job/post-job`, which renders the ad copy and pushes it to Buffer. + +--- + +## Tech stack + +| Concern | Choice | +|---|---| +| Web framework | FastAPI 0.136 + Uvicorn | +| ORM / models | SQLModel 0.0.38 on SQLAlchemy 2.0 (async, `asyncpg`) | +| Database | PostgreSQL, application objects in the `app` schema | +| Migrations | Alembic 1.18, driven by `alembic_setup.py` | +| Auth | PyJWT (HS256) access / refresh / reset tokens, `bcrypt` hashing | +| Task queue | Taskiq on Redis Streams, with a smart-retry + dead-letter middleware | +| LLM | OpenAI async client, orchestrated by LangGraph | +| PDF extraction | `pypdf` | +| HTTP client | `httpx` | +| Python | 3.12 (see `Dockerfile`) | + +--- + +## Directory layout + +``` +backend/ +├── main.py # FastAPI app, lifespan, CORS, router mounting +├── db_setup.py # Settings, async engine, sessions, init_db/lifespan +├── alembic_setup.py # Alembic scaffolding, autogenerate, migrate-on-boot +├── llm_setup.py # AsyncOpenAI client + llm_call helper +├── requirements.txt +├── Dockerfile # image for the Taskiq worker / scheduler +├── alembic.ini # generated by alembic_setup.py, not hand-written +├── migrations/ # generated env.py + versions/ +├── LLM_CONTEXT_PROMPT.md # house-style prompt to paste into an LLM before editing +│ +├── users/ # accounts, login, signup, RBAC enforcement +├── role/ # roles, permission bundles, permission tags +├── forget_password/ # reset-code request → verify → new password +├── notifications/ # email-confirmation tokens and mail +├── inbox/ # mailbox sync, attachments, applications +├── job/ +│ ├── app.py # routes for both sub-domains +│ ├── job_post/ # job ads + Buffer publishing +│ └── candidate/ # CV reading, candidate profile +├── agent/ # LangGraph CV → job-post matching agent +└── taskiq_management/ # broker, scheduler, DLQ middleware, smoke task +``` + +There are **no `__init__.py` files**. The service is run from `backend/`, so imports are +top-level (`from users.app import router`, `from db_setup import get_session`). + +--- + +## House style — what lives in which file + +Every domain package follows the same six-file shape. This is enforced by convention, and +`LLM_CONTEXT_PROMPT.md` is the canonical statement of it. + +| File | Owns | Must not do | +|---|---|---| +| `app.py` | Routes, inline request models, `JSONResponse`, dependency injection | Business rules, SQL | +| `views.py` | Business checks, calls models, raises `HTTPException` | Build login token envelopes | +| `models.py` | SQLModel table + `@classmethod async def` accessors | Import FastAPI, raise `HTTPException` | +| `serializers.py` | Hand-built `dict` builders (`serialize_*`) | Touch the DB or `Depends` | +| `plugins.py` | Pure helpers — hashing, JWT, HTTP calls to third parties | Import FastAPI | +| `permissions.py` | Bearer schemes and `Depends` aliases (auth domains only) | Hold route handlers | + +Additional rules that matter when you edit this code: + +- Request bodies are Pydantic models declared **inline in `app.py`**, never in `serializers.py`. +- There are **no Pydantic response models** — responses are hand-built dicts. +- Route paths are verb-in-path (`/users/create`, `/users/fetch`), not REST-resource-only, and + there is no `/api/v1` prefix. +- Non-DB config is module-level `load_dotenv()` + `os.getenv(...)`. Only database settings go + through `db_setup.Settings`. + +--- + +## Domains + +### `users/` +Signup, login, refresh, CRUD, role assignment, and the RBAC machinery every other domain +depends on. `users/permissions.py` defines the full `PermissionTag` vocabulary (13 modules × +8 actions = 104 tags) and the `require_permission(...)` dependency. A startup assertion +(`_assert_vocabulary_complete`) fails loudly if the tag list ever drifts from +`PermissionModule × PermissionAction`. + +Signup always assigns the `candidate` role, creates the account **inactive**, and sends a +confirmation email. Login rejects unconfirmed accounts. + +### `role/` +Three-level permission model: `permission_tags` (atomic `module.action` rows) → +`permissions` (named bundles holding a JSONB array of tag ids) → `roles` (holding a JSONB +array of bundle ids). `Roles.resolve_tags()` walks that chain and returns a flat tuple of tag +names; dangling or inactive ids simply contribute nothing rather than erroring. + +Eight system roles are seeded: `system_administrator`, `hr_administrator`, `recruiter`, +`hiring_manager`, `department_head`, `interviewer`, `ceo`, `candidate`. + +### `inbox/` +The heart of the ingestion pipeline. + +- `views.py::Email` talks to the external Email API, decodes attachments, upserts messages, + and enqueues matching work. +- `models.py` holds `Inbox_Messages` (the mail rows plus all agent output columns), + `Inbox_Alerts`, and `Inbox` — the join table linking a message to the candidate `Users` row + it came from. `_link_sender` creates the candidate account on first contact, skipping + `noreply@`-style senders. +- `file_decoder.py` turns Graph `contentBytes` into real PDF / DOCX / DOC files, validating + magic bytes for each format and stripping path traversal from filenames. +- `plugins.py` resolves attachment paths (handling Windows paths written by the host API but + read from a Linux worker), extracts résumé text, and calls the read-status sync endpoints. +- `tasks.py` / `sync_tasks.py` are the two Taskiq tasks. + +### `job/` +Two sub-domains behind one router: + +- **`job_post/`** — renders LinkedIn-shaped ad copy from a structured payload, resolves the + Buffer channel (by explicit id, by platform alias, or by the configured default), creates + the post over Buffer's GraphQL API, and records the mapped status. A queued post is recorded + as `scheduled`, not `published`; only Buffer reporting `sent` promotes it. +- **`candidate/`** — `FileRead` extracts text from an uploaded PDF (`pypdf`), and + `match_inbox_cv` force-requeues an existing inbox message for matching. `CandidateView` + reads the candidate profile through the `inbox` join. + +### `notifications/` and `forget_password/` +Two parallel token flows, deliberately kept separate so each owns its own mail copy and env +reads: + +- **Confirmation** — a 32-byte url-safe secret, bcrypt-hashed in + `email_confirmation_tokens`; the link carries `.` because a bcrypt hash + cannot be looked up. Replays (mail scanners, back button) are handled idempotently. +- **Password reset** — a short code mailed to the user, bcrypt-hashed in + `password_reset_codes`, with a resend cooldown and a max-attempts cap. Verifying the code + mints a `type=reset` JWT carrying the code row id (`crid`), which is the only thing that + authorises the new-password call. + +### `agent/` +LangGraph state machine — see [The matching agent](#the-matching-agent). + +### `taskiq_management/` +Broker, scheduler, DLQ middleware, and a `ping` smoke task. + +--- + +## Data model + +All tables live in the `app` schema (`DB_DEFAULT_SCHEMA`), with a shared naming convention for +indexes, constraints and foreign keys. `SQLModel.metadata` is pointed at `Base.metadata` in +`db_setup.py` so SQLModel and DeclarativeBase share one registry and Alembic sees everything. + +| Table | Key columns | Notes | +|---|---|---| +| `users` | `id` (uuid PK), `email` (unique), `role_id` → `roles.id`, `password`, `is_active`, `is_deleted` | Soft delete. `role` is `selectin`-loaded; lazy loads would raise `MissingGreenlet` under asyncio | +| `roles` | `id`, `role_name` (unique), `permissions` (JSONB int[]), `is_system` | | +| `permissions` | `id`, `name` (unique), `permission_tags` (JSONB int[]) | Named bundles | +| `permission_tags` | `id`, `tag_name` (unique), `module`, `action` | Unique on (`module`, `action`) | +| `inbox_messages` | `id` (uuid), `message_id` (upstream id, unique), `full_email_response` (JSONB), subject/body/from/to/cc/bcc, `message_read`, `attachment`, `file_name`, `file_path`, `application_status`, `resume_text`, `experience`, `suggested_job_post_ids` (JSONB), `match_summary`, `match_reasoning`, `match_status`, `match_error`, `matched_at` | One row per mail; agent output lands here | +| `inbox` | `id`, `user_id` → `users.id`, `message_id` → `inbox_messages.id`, `alert_id` | Join table linking a candidate to a message | +| `inbox_alerts` | `id`, `alert_sender_name`, `alert_sender_email`, `is_read` | | +| `job_posts` | `id` (uuid), `title`, `platform`, `channel_id`, `post_text`, `requirements`/`optional_skills` (JSON), `status`, `buffer_post_id`, `buffer_external_link`, `buffer_sent_at`, `buffer_error`, `created_by` → `users.id` | | +| `password_reset_codes` | `id`, `email`, `code_hash`, `expires_at`, `attempts`, `is_used`, `verified_at` | | +| `email_confirmation_tokens` | `id`, `user_id`, `email`, `token_hash`, `expires_at`, `is_used`, `confirmed_at` | | + +`application_status` is a `str` enum: `PROCESS`, `PENDING`, `APPROVED`, `REJECTED`, `ONHOLD`, +`CLOSED`. + +`match_status` is free-form text written by the worker: `processing`, `matched`, `skipped`, +`no_text`, `failed`, `dlq`. + +--- + +## API reference + +Base URL: `http://localhost:8000`. Interactive docs at `/docs`. + +### Auth — `users/app.py`, `notifications/app.py`, `forget_password/app.py` + +| Method | Path | Guard | Purpose | +|---|---|---|---| +| POST | `/users/signup` | public | Create a candidate account (inactive) and mail a confirmation link | +| POST | `/users/login` | public | Email/username + password → token envelope | +| POST | `/users/refresh` | public | Refresh token → new token pair | +| GET | `/users/me` | any authenticated user | Current user with resolved permissions | +| POST | `/users/confirm-email` | public | Consume a confirmation token, activate the account | +| POST | `/users/confirm-email/resend` | public | Re-issue a confirmation link (cooldown enforced) | +| POST | `/users/forget-password` | public | Mail a reset code | +| POST | `/users/forget-password/verify-code` | public | Verify the code → `type=reset` JWT | +| POST | `/users/forget-password/new-password` | reset JWT | Set the new password | + +### Users — `users/app.py` + +| Method | Path | Required tag | +|---|---|---| +| GET | `/users/fetch` | `rbac_users.view` | +| POST | `/users/create` | `rbac_users.create` | +| PUT | `/users/update?record_id=` | `rbac_users.edit` | +| PUT | `/users/assign-role?record_id=` | `rbac_users.edit` (+ `rbac_users.manage` inside the service) | +| PUT | `/users/remove-role?record_id=` | `rbac_users.edit` (+ `rbac_users.manage`) | +| DELETE | `/users/delete?record_id=` | `rbac_users.delete` | + +### Roles and permissions — `role/app.py` + +| Method | Path | Required tag | +|---|---|---| +| GET | `/roles/fetch` | `rbac_users.view` | +| POST | `/roles/create` | `rbac_users.create` | +| PUT | `/roles/update?record_id=` | `rbac_users.edit` | +| DELETE | `/roles/delete?record_id=` | `rbac_users.delete` | +| GET | `/permissions/fetch` | `rbac_users.view` | +| POST | `/permissions/create` | `rbac_users.manage` | +| PUT | `/permissions/update?record_id=` | `rbac_users.manage` | +| GET | `/permission-tags/fetch` | `rbac_users.view` | + +### Inbox — `inbox/app.py` + +| Method | Path | Guard | Purpose | +|---|---|---|---| +| GET | `/email/fetch` | upstream token only | Pull from the Email API, decode attachments, upsert, enqueue matching. `test_on=true` (default) returns raw payloads and skips the account-setup mails | +| GET | `/inbox/fetch` | none | Stored messages, with attachments inlined as base64 | +| GET | `/inbox/all-applications` | `inbox.view` | The Applications tab. Filters: `application_status`, `isread`, `search`, `record_id`, `top`, `skip` | +| POST | `/inbox/{record_id}/match` | `inbox.edit` | Force a re-match of one message | +| POST | `/inbox/{record_id}/read` | `inbox.edit` | Mark read locally | +| GET | `/inbox/{record_id}/read-status` | `inbox.edit` | Re-pull read status from upstream for one message | + +### Jobs and candidates — `job/app.py` + +| Method | Path | Required tag | Purpose | +|---|---|---|---| +| GET | `/jobs/alias` | public | Accepted platform shorthands (`fb`, `ig`, `li`, `x`, …) | +| POST | `/job/post-job` | `job_board.create` | Render the ad, create the Buffer post, persist the result | +| GET | `/job/buffer/channels` | `job_board.view` | Connected Buffer channels across all organizations | +| POST | `/candidate/cv_upload` | `candidates.create` | Upload a PDF, get extracted text back | +| POST | `/candidate/inbox-match?inbox_message_id=` | `candidates.edit` | Queue a forced re-match for a stored message | +| GET | `/candidate/fetch?user_id=` | `candidates.view` | Candidate profile via the `inbox` join | + +`POST /job/post-job` takes `mode` ∈ `addToQueue` | `shareNow` | `customScheduled`; the +`customScheduled` mode requires `scheduler_date` (and optionally `scheduler_time`), which the +route combines into a UTC `due_at`. + +--- + +## Authentication and RBAC + +**Tokens.** PyJWT, HS256, with a `type` claim that `decode_token(..., expected_type=...)` +rejects on mismatch. Three types: + +| Type | Lifetime (default) | Extra claims | +|---|---|---| +| `access` | 30 min | `email`, `role_id` | +| `refresh` | 7 days | — | +| `reset` | 10 min | `crid` (reset-code row id) | + +`iat` / `exp` are always timezone-aware UTC. Every token carries a `jti`. + +**Login response** puts the OAuth2 fields at the root so Swagger's Authorize button can read +them: + +```json +{ + "access_token": "...", + "refresh_token": "...", + "token_type": "bearer", + "expires_in": 1800, + "data": { "id": "...", "email": "...", "role_id": 3, "role_name": "recruiter", "...": "..." }, + "status_code": 200 +} +``` + +**Passwords** use `bcrypt` directly rather than `passlib` — passlib 1.7.4 reads +`bcrypt.__about__.__version__`, which bcrypt dropped in 4.1, and the failed probe makes it +reject every password as over 72 bytes. Input is truncated to 72 bytes on a character +boundary before hashing. + +**Authorization.** `get_current_user` decodes the access token, loads the user by `sub`, +rejects missing / deleted / inactive accounts, resolves the role's tags, and returns a +serialized user dict with a `permissions` list. `require_permission(*tags, require_all=True)` +is the dependency that guards routes; a user with no role assigned gets a 403 before any tag +check runs. + +Role assignment is additionally guarded in `users/views.py`: you cannot grant a role holding +permissions you do not yourself hold. + +--- + +## Background jobs + +Taskiq over **Redis Streams**, with a result backend and a Redis-backed schedule source. + +**Middleware order matters** — `DeadLetterMiddleware` sits before `SmartRetryMiddleware` so a +`PermanentTaskError` can set `retry_on_error=False` before retry logic runs. + +| Task | Trigger | What it does | +|---|---|---| +| `inbox.match_message` | enqueued by `/email/fetch`, `/inbox/{id}/match`, `/candidate/inbox-match` | Extract résumé text → run the agent → write match results | +| `inbox.sync_read_status` | cron, `EMAIL_SYNC_CRON` (default every minute) | Pull read-status deltas from the Email API and apply them | +| `ping` | manual | Framework smoke test | + +**Retries.** `SmartRetryMiddleware` with jitter and exponential delay, `TASKIQ_MAX_RETRIES` +attempts, capped at `TASKIQ_MAX_DELAY`. Raising `PermanentTaskError` (missing record, no +attachment, blank `record_id`) skips retries entirely. + +**Dead letter queue.** Exhausted or permanently-failed tasks are written to the +`taskiq:dlq` Redis stream as a JSON payload. For `inbox.match_message` specifically, the +middleware also stamps `match_status="dlq"` on the row so the failure is visible in the UI +rather than only in Redis. + +**The read-status latch.** `apply_read_status` only ever applies `false → true`. Nothing +pushes local reads back to Outlook, so upstream keeps reporting `isRead=false`; without the +latch the every-minute sweep would un-read a mail the user just opened. The trade-off is that +un-reading a mail in Outlook no longer propagates here. `sync_read_status` holds a Redis lock +(`inbox:sync_read_status:lock`, 300s TTL) so overlapping cron ticks cannot double-run, and +pages at most 10 rounds per tick. + +--- + +## The matching agent + +A small LangGraph `StateGraph` over an `AgentState` TypedDict. + +``` +START → prepare → (conditional) → match_jobs → END + └→ END (when resume_text or job_posts is empty) +``` + +- **`prepare_context`** trims the subject and résumé text and normalizes the job posts down to + the fields the model needs. Empty résumé text or no active posts short-circuits to + `status="skipped"`. +- **`match_jobs`** calls `llm_call(prompt(), user_prompt(state), json_mode=True)` and parses + the response. + +The parser (`agent/decorators.py::parse_match_response`) is deliberately strict: a suggested +id must be present in the posts that were actually sent, must be a valid UUID, and duplicates +are dropped. The model cannot invent a job post. `reasoning` returned as a list is joined into +a string; non-string fields fall back to `""`. + +The graph is compiled once per process (`init_agent()` from the FastAPI lifespan, or lazily on +first use inside the worker) and the OpenAI client is created lazily and closed on shutdown. + +Startup is **fault-tolerant**: if the broker or the LLM/agent fails to initialize, `main.py` +logs a warning and the API still serves. Only the database is a hard requirement. + +--- + +## External integrations + +| Service | Used by | Contract | +|---|---|---| +| **Email API** (a Microsoft Graph proxy) | `inbox/` | `GET {EMAIL_URL}/emails`, `GET {EMAIL_URL}/emails/{id}`, `GET {EMAIL_URL}/sync/read-status`, `GET {EMAIL_URL}/sync/read-status/message/{id}` — Bearer `EMAIL_API_TOKEN` | +| **Teams Mail API** | `notifications/`, `forget_password/` | multipart POST to `TEAMS_MAIL_API_URL`; success is HTTP **202**, anything else raises | +| **Buffer** | `job/job_post/` | GraphQL against `BUFFER_API_URL` — `createPost` mutation, `account { organizations }` and `channels` queries | +| **OpenAI** | `agent/`, `llm_setup.py` | Chat Completions with `response_format: json_object` | + +Attachments are written to `backend/inbox/decoded_attachments/`. In Docker this directory is +bind-mounted into the worker so both processes see the same files. + +--- + +## Configuration + +Copy `.env.example` to `.env` and fill it in. `.env` is git-ignored; `.env.example` is not. +`db_setup.Settings` reads `backend/.env` or the repo-root `.env`; every other module reads its +own keys with `os.getenv`. + +### Database + +| Variable | Default | Notes | +|---|---|---| +| `DB_USERNAME`, `DB_PASSWORD`, `DB_HOST`, `DB_PORT`, `DB_NAME` | — | **Required.** `DB_PORT` must be an integer | +| `DATABASE_URL` | — | Full DSN; wins over the parts above | +| `DB_SSLMODE` | — | e.g. `require` on Azure; translated to asyncpg's `ssl` | +| `DB_SCHEMAS` | `app` | Comma-separated; created on startup | +| `DB_DEFAULT_SCHEMA` | `app` | Schema for models that declare none | +| `DB_ECHO` | `false` | SQL logging | +| `DB_POOL_SIZE` / `DB_MAX_OVERFLOW` / `DB_POOL_RECYCLE` | `5` / `10` / `1800` | | +| `DB_CONNECT_RETRIES` | `10` | Startup wait-for-Postgres | +| `DB_AUTO_MIGRATE` | `true` | Run `upgrade head` on startup | +| `DB_AUTOGENERATE` | `true` | Write a revision when models drift | +| `APP_NAME` | `hr-ats-portal` | Postgres `application_name` | + +### Auth + +| Variable | Default | +|---|---| +| `JWT_SECRET_KEY` | — (**required**) | +| `JWT_ALGORITHM` | `HS256` | +| `JWT_ACCESS_TOKEN_EXPIRE_MINUTES` | `30` | +| `JWT_REFRESH_TOKEN_EXPIRE_DAYS` | `7` | +| `JWT_RESET_TOKEN_EXPIRE_MINUTES` | `10` | + +### Email ingestion + +| Variable | Default | +|---|---| +| `EMAIL_URL`, `EMAIL_API_TOKEN` | — | +| `EMAIL_SYNC_FOLDER` | `inbox` | +| `EMAIL_SYNC_SINCE` | — | +| `EMAIL_SYNC_CRON` | `* * * * *` | +| `BACKEND_URL` | `http://localhost:8000` (used for the internal confirmation-resend call) | +| `DEFAULT_CANDIDATE_PASSWORD` | `Utopia!@#` — placeholder only; the account is inactive until confirmed | + +### Mail out + +| Variable | Default | +|---|---| +| `TEAMS_MAIL_API_URL`, `TEAMS_API_TOKEN` | — | +| `FRONTEND_URL` | `http://localhost:5173` | +| `CONFIRM_EMAIL_PATH` | `/auth/confirm-email` | +| `CONFIRM_TOKEN_TTL_SECONDS` | `86400` | +| `CONFIRM_TOKEN_RESEND_SECONDS` | `60` | +| `RESET_CODE_TTL_SECONDS` | `60` | +| `RESET_CODE_RESEND_SECONDS` | `30` | +| `RESET_CODE_MAX_ATTEMPTS` | `5` | + +### Buffer + +| Variable | Default | +|---|---| +| `BUFFER_API` | — (access token) | +| `BUFFER_API_URL` | `https://api.buffer.com` | +| `BUFFER_CHANNEL_ID` | — (fallback channel) | + +### OpenAI + +| Variable | Default | +|---|---| +| `OPENAI_API_KEY` | — | +| `OPENAI_MODEL` | `gpt-5.4-mini` | +| `OPENAI_TEMPERATURE` | `0` — leave blank to omit the parameter for models that reject it | +| `OPENAI_MAX_OUTPUT_TOKENS` | `32768` (`.env.example` ships `4096`) | +| `OPENAI_TIMEOUT` / `OPENAI_MAX_RETRIES` / `OPENAI_CONNECT_RETRIES` | `60` / `3` / `3` | +| `OPENAI_BASE_URL`, `OPENAI_ORGANIZATION`, `OPENAI_PROJECT` | — (set only for Azure or a gateway) | + +### Taskiq / Redis + +| Variable | Default | +|---|---| +| `REDIS_URL` | `redis://localhost:6379/0` | +| `TASKIQ_QUEUE_NAME` | `inbox` | +| `TASKIQ_CONSUMER_GROUP` | `taskiq` | +| `TASKIQ_MAX_RETRIES` | `3` | +| `TASKIQ_RETRY_DELAY` | `5` | +| `TASKIQ_MAX_DELAY` | `120` | +| `TASKIQ_IDLE_TIMEOUT_MS` | `600000` | +| `TASKIQ_DLQ_STREAM` | `taskiq:dlq` | +| `TASKIQ_WORKER_NAME` | falls back to `HOSTNAME` | +| `APP_VERSION` | `dev` | + +--- + +## Running locally + +**Prerequisites:** Python 3.12, PostgreSQL, Redis. + +```bash +cd backend + +python -m venv .venv +source .venv/bin/activate # Windows: .venv\Scripts\activate +pip install -r requirements.txt + +cp .env.example .env # then fill it in +``` + +All commands must be run from `backend/` — the import paths depend on it. + +**API:** + +```bash +uvicorn main:app --reload --port 8000 +``` + +Startup connects to Postgres (retrying with backoff), creates the configured schemas, runs +migrations to head, then starts the broker, the OpenAI client and the agent graph. Broker and +LLM failures are logged and skipped; the API still comes up. + +**Worker** (needs Redis): + +```bash +taskiq worker taskiq_management.broker_setup:broker \ + inbox.tasks inbox.sync_tasks taskiq_management.tasks +``` + +**Scheduler** (cron ticks for `inbox.sync_read_status`): + +```bash +taskiq scheduler taskiq_management.broker_setup:scheduler inbox.sync_tasks +``` + +Docs: + +--- + +## Database migrations + +`alembic_setup.py` wraps Alembic so the plain `alembic` CLI and the app's own +migrate-on-startup share one configuration. It scaffolds `alembic.ini`, `migrations/env.py` +and `script.py.mako` on first use and never overwrites them. Model modules are discovered +automatically — every `/models.py` under `backend/` is imported before the metadata is +diffed. + +```bash +python alembic_setup.py migrate # upgrade to head, then autogenerate any drift +python alembic_setup.py revision -m "add x" # write a revision if the models have drifted +python alembic_setup.py upgrade -r head +python alembic_setup.py downgrade -r -1 +python alembic_setup.py current +python alembic_setup.py head +``` + +Migrations run under a Postgres advisory lock, so several workers booting at once cannot +migrate concurrently. Empty revisions are suppressed. Alembic's own `alembic_version` table is +excluded from autogenerate, as is anything outside the configured schemas. + +The module is named `alembic_setup` rather than `alembic` because `backend/` is on `sys.path` +and a module called `alembic.py` would shadow the installed package. + +--- + +## Docker + +The repo-root `docker-compose.yml` runs Redis plus the two Taskiq processes; the API itself is +expected to run on the host (the compose file points the containers at +`host.docker.internal` for the database). + +```bash +docker compose up -d # from the repo root +docker compose logs -f taskiq-worker +``` + +`backend/Dockerfile` builds a `python:3.12-slim` image whose default command is the Taskiq +worker. `backend/inbox/decoded_attachments` is bind-mounted so the worker can read the +attachments the API wrote. + +--- + +## Response conventions + +Every handler wraps its body in the same try/except: + +```python +try: + service=Email(session=session) + data=await service.some_method(...) + return JSONResponse(content={"data":data,"status_code":200}) +except HTTPException: + raise +except Exception as e: + raise HTTPException(status_code=500,detail=str(e)) +``` + +| Shape | Response | +|---|---| +| List | `{"data": [...], "total": , "status_code": 200}` | +| Single record | `{"data": {...}, "total": 1, "status_code": 200}` | +| Login / refresh | OAuth2 fields at the root, user under `data` | +| Error | FastAPI's `{"detail": "..."}` with the real status code | + +Serializers always `str()` UUIDs, `.isoformat()` datetimes, and never emit `password`. + +--- + +## Known gaps and gotchas + +- **`DB_PORT` must be set.** `db_setup.Settings` evaluates `int(os.getenv("DB_PORT"))` at class + definition time, so a missing value raises `TypeError` on import rather than a friendly + config error. +- **CORS is fully open** (`allow_origins=["*"]` with credentials). Fine for development, needs + tightening before production. +- **`/email/fetch` and `/inbox/fetch` carry no permission guard.** `/email/fetch` authenticates + only against the upstream Email API token. +- **`.doc` / `.docx` résumés are decoded and stored but not parsed.** `extract_resume_text` + handles PDFs only and reports `no PDF attachment to extract` for the rest. +- **`serialize_application` returns `null` for `ats_score`, `phone`, `recruiter` and + `duplicate`** — `inbox_messages` has no columns for them yet, and `processing` is derived + from `message_read` alone, so it is only ever `"Read"` or `"Unread"`. +- **`Inbox.get_candidate_profile` filters on `cls.user.role_id`**, which is a relationship + attribute rather than a joined column; the candidate-profile query needs a join before it + behaves as intended. +- **Attachment paths may be Windows absolutes** written by the host API but read by a Linux + worker. `resolve_attachment_path` normalizes separators and falls back to the basename under + the mounted `decoded_attachments` directory. +- **Read status is a one-way latch** — see [Background jobs](#background-jobs). +- **There is no test suite** in `backend/` at present. + +--- + +## Editing this codebase + +Before changing anything here, read [`LLM_CONTEXT_PROMPT.md`](LLM_CONTEXT_PROMPT.md). It states +the house style in full and is the reference used to keep new code indistinguishable from +`users/` and `inbox/`. The short version: mirror the neighbouring file, keep the layer duties +intact, add no new layers, and do not reformat code you did not otherwise need to touch. + +Adding an endpoint, in order: + +1. Model accessor in `models.py` (if it touches the DB). +2. Service method in `views.py`. +3. `serialize_*` in `serializers.py` if the shape is new. +4. Route in `app.py` with the standard try/except and `JSONResponse`. +5. `CurrentUser` or `Depends(require_permission(...))` if the route is protected. diff --git a/backend/agent/agent_setup.py b/backend/agent/agent_setup.py new file mode 100644 index 0000000..2bd9bba --- /dev/null +++ b/backend/agent/agent_setup.py @@ -0,0 +1,51 @@ +"""LangGraph agent framework setup for HR-ATS workflows. + +Pure module: no FastAPI imports and no HTTPException. +This file only owns graph construction and lifecycle: + + init_agent() -> get_graph() -> build_graph() -> graph.compile() + +LLM client/config lives in llm_setup. Nodes live in agent.views. +Run entrypoint lives in agent.execute_agent. +""" + +from __future__ import annotations + +import logging + +from langgraph.graph import END,START,StateGraph + +from agent.models import AgentState +from agent.views import match_jobs,prepare_context,route_after_prepare + +logger=logging.getLogger("agent") + +_graph=None + + +def build_graph(): + graph=StateGraph(AgentState) + graph.add_node("prepare",prepare_context) + graph.add_node("match_jobs",match_jobs) + graph.add_edge(START,"prepare") + graph.add_conditional_edges("prepare",route_after_prepare) + graph.add_edge("match_jobs",END) + return graph.compile() + + +def get_graph(): + global _graph + if _graph is None: + _graph=build_graph() + logger.info("langgraph compiled") + return _graph + + +async def init_agent(): + get_graph() + + +async def close_agent(): + global _graph + _graph=None + logger.info("agent graph closed") diff --git a/backend/agent/decorators.py b/backend/agent/decorators.py new file mode 100644 index 0000000..5cdbbeb --- /dev/null +++ b/backend/agent/decorators.py @@ -0,0 +1,72 @@ +"""Agent response parsers and input normalizers. + +Pure module: no FastAPI imports, no HTTPException, and no module-level state. +Mirrors job/candidate/decorators.py — helpers that clean/shape data before or +after the graph nodes run. +""" + +from __future__ import annotations + +import uuid + + +def normalize_job_posts(job_posts) -> list[dict]: + if not job_posts: + return [] + normalized=[] + for item in job_posts: + if not isinstance(item,dict): + continue + job_id=item.get("id") + if job_id is None: + continue + normalized.append({ + "id":str(job_id), + "title":item.get("title") or "", + "description":item.get("description") or "", + "post_text":item.get("post_text") or "", + "requirements":item.get("requirements") or [], + "optional_skills":item.get("optional_skills") or [], + "location":item.get("location") or "", + "employment_type":item.get("employment_type") or "", + }) + return normalized + + +def parse_match_response(data,allowed_ids) -> tuple[list[str],str,str,str]: + if not isinstance(data,dict): + raise RuntimeError(f"model did not return a JSON object: {data!r}") + + allowed=set(allowed_ids or []) + raw_ids=data.get("suggested_job_post_ids") or [] + if not isinstance(raw_ids,list): + raw_ids=[] + + suggested=[] + seen=set() + for raw_id in raw_ids: + job_id=str(raw_id).strip() + if not job_id or job_id not in allowed or job_id in seen: + continue + try: + uuid.UUID(job_id) + except ValueError: + continue + seen.add(job_id) + suggested.append(job_id) + + summary=data.get("summary") + if not isinstance(summary,str): + summary="" + + reasoning=data.get("reasoning") + if isinstance(reasoning,list): + reasoning="\n".join(str(item) for item in reasoning) + if not isinstance(reasoning,str): + reasoning="" + + experience=data.get("experience") + if not isinstance(experience,str): + experience="" + + return suggested,summary.strip(),reasoning.strip(),experience.strip() diff --git a/backend/agent/execute_agent.py b/backend/agent/execute_agent.py new file mode 100644 index 0000000..f7e0c03 --- /dev/null +++ b/backend/agent/execute_agent.py @@ -0,0 +1,19 @@ +"""Agent entrypoint — run the compiled graph. + +Pure module: no FastAPI imports and no HTTPException. +""" + +from __future__ import annotations + +from agent.agent_setup import get_graph +from agent.serializers import serialize_agent_result + + +async def run_agent(*,subject="",resume_text="",job_posts=None) -> dict: + final_state=await get_graph().ainvoke({ + "subject":subject or "", + "resume_text":resume_text or "", + "job_posts":job_posts or [], + "status":"pending", + }) + return serialize_agent_result(final_state) diff --git a/backend/agent/models.py b/backend/agent/models.py new file mode 100644 index 0000000..01d15de --- /dev/null +++ b/backend/agent/models.py @@ -0,0 +1,20 @@ +"""LangGraph agent state for HR-ATS workflows. + +Pure module: no FastAPI imports and no HTTPException. +""" + +from __future__ import annotations + +from typing import Literal,TypedDict + + +class AgentState(TypedDict,total=False): + subject:str + resume_text:str + experience:str + job_posts:list[dict] + suggested_job_post_ids:list[str] + summary:str + reasoning:str + error:str + status:Literal["pending","ready","matched","skipped","failed"] diff --git a/backend/agent/prompt.py b/backend/agent/prompt.py new file mode 100644 index 0000000..0484963 --- /dev/null +++ b/backend/agent/prompt.py @@ -0,0 +1,43 @@ +"""Agent prompt builders. + +Pure module: no FastAPI imports and no HTTPException. +""" + +from __future__ import annotations + +import json + + +def prompt(): + return """You are an HR-ATS recruiting assistant. + +You are given a candidate email subject, CV/resume text extracted from an +attachment, and a list of active job posts (id, title, description, requirements). + +Identify which job posts the candidate is most likely applying for. + +Rules: +- Only suggest job_post_id values that appear in the provided job_posts list. +- A candidate may match zero, one, or multiple posts. +- Base matches on skills, role title, experience, and the subject line — not guesses. +- If confidence is low, return an empty list rather than forcing a match. + +Respond with JSON only: +{ + "suggested_job_post_ids": ["uuid", "..."], + "summary": "one short sentence for the recruiter", + "reasoning": "brief bullet-style explanation per suggested match", + "experience": "the relevant experience of the candidate in years for the suggested match" +} +""" + + +def user_prompt(state) -> str: + return json.dumps( + { + "subject": state.get("subject") or "", + "resume_text": state.get("resume_text") or "", + "job_posts": state.get("job_posts") or [], + }, + ensure_ascii=False, + ) diff --git a/backend/agent/serializers.py b/backend/agent/serializers.py new file mode 100644 index 0000000..833aac7 --- /dev/null +++ b/backend/agent/serializers.py @@ -0,0 +1,17 @@ +"""Agent result serializers. + +Pure module: no FastAPI imports and no HTTPException. +""" + +from __future__ import annotations + + +def serialize_agent_result(state:dict) -> dict: + return { + "suggested_job_post_ids":state.get("suggested_job_post_ids") or [], + "summary":state.get("summary") or "", + "reasoning":state.get("reasoning") or "", + "experience":state.get("experience") or "", + "status":state.get("status") or "failed", + "error":state.get("error") or "", + } diff --git a/backend/agent/views.py b/backend/agent/views.py new file mode 100644 index 0000000..a128e70 --- /dev/null +++ b/backend/agent/views.py @@ -0,0 +1,68 @@ +"""Agent graph node logic. + +Pure module: no FastAPI imports and no HTTPException. +LLM client/config lives in llm_setup — nodes call llm_call only. +""" + +from __future__ import annotations + +import logging +from typing import Literal + +from langgraph.graph import END + +from agent.decorators import normalize_job_posts,parse_match_response +from agent.models import AgentState +from agent.prompt import prompt,user_prompt +from llm_setup import llm_call + +logger=logging.getLogger("agent") + + +async def prepare_context(state:AgentState) -> dict: + subject=(state.get("subject") or "").strip() + resume_text=(state.get("resume_text") or "").strip() + job_posts=normalize_job_posts(state.get("job_posts")) + + if not resume_text: + return {"status":"skipped","error":"resume_text is empty","suggested_job_post_ids":[]} + if not job_posts: + return {"status":"skipped","error":"no active job posts to match against","suggested_job_post_ids":[]} + + return { + "subject":subject, + "resume_text":resume_text, + "job_posts":job_posts, + "status":"ready", + "error":"", + } + + +def route_after_prepare(state:AgentState) -> Literal["match_jobs","__end__"]: + if state.get("status")=="ready": + return "match_jobs" + return END + + +async def match_jobs(state:AgentState) -> dict: + try: + data=await llm_call(prompt(),user_prompt(state),json_mode=True) + allowed_ids={item["id"] for item in state.get("job_posts") or []} + suggested,summary,reasoning,experience=parse_match_response(data,allowed_ids) + return { + "status":"matched", + "suggested_job_post_ids":suggested, + "summary":summary, + "reasoning":reasoning, + "experience":experience, + } + except Exception as e: + logger.exception("agent match_jobs failed") + return { + "status":"failed", + "error":str(e), + "suggested_job_post_ids":[], + "summary":"", + "reasoning":"", + "experience":"", + } diff --git a/backend/inbox/app.py b/backend/inbox/app.py index c817262..6d4321e 100644 --- a/backend/inbox/app.py +++ b/backend/inbox/app.py @@ -2,29 +2,43 @@ from fastapi import APIRouter,Depends, Query from fastapi.responses import JSONResponse from fastapi import HTTPException from db_setup import get_session +from inbox.enums import Candidate_application_Status from sqlalchemy.ext.asyncio import AsyncSession from inbox.views import Email +from users.permissions import PermissionTag, require_permission from dotenv import load_dotenv load_dotenv() router = APIRouter() @router.get("/email/fetch") -async def fetch_email(top:int=Query(100),skip:int=Query(0,ge=0),token=Query(...),session: AsyncSession = Depends(get_session)): +async def fetch_email( + top:int=Query(100), + skip:int=Query(0,ge=0), + test_on: bool = Query(True), + token: str | None = Query(None), + session: AsyncSession = Depends(get_session), +): try: - if not token: - raise HTTPException(status_code=401,detail="Unauthorized") service=Email(session=session,token=token) + if not service.token: + raise HTTPException(status_code=401,detail="Unauthorized") data=await service.service_email(top,skip) value=data.get("value") items_lst=[] for item in value: message_id=item.get("id") - service_per_email=await service.get_email_by_id(message_id) + service_per_email=await service.get_email_by_id(message_id,test_on) items_lst.append({"message_id":message_id,"email_contents":service_per_email}) + if service.pending_match_ids: + await service.enqueue_matching(list(service.pending_match_ids),force=False) - return JSONResponse(content={"data":items_lst,"status_code":200}) + account_setup=[] + if service.pending_confirmation_emails: + account_setup=await service.send_account_setup(list(service.pending_confirmation_emails)) + + return JSONResponse(content={"data":items_lst,"account_setup":account_setup,"status_code":200}) except HTTPException: raise @@ -53,3 +67,87 @@ async def fetch_inbox( raise except Exception as e: raise HTTPException(status_code=500,detail=str(e)) + + +@router.post("/inbox/{record_id}/match") +async def rematch_inbox( + record_id: str, + current_user: dict = Depends(require_permission(PermissionTag.INBOX_EDIT)), + session: AsyncSession = Depends(get_session), +): + try: + service=Email(session=session) + queued_id=await service.queue_rematch(record_id) + task_ids=await service.enqueue_matching([queued_id], force=True) + return JSONResponse(content={"data":{"queued":True,"task_ids":task_ids},"status_code":200}) + except HTTPException: + raise + except Exception as e: + raise HTTPException(status_code=500,detail=str(e)) + + +@router.post("/inbox/{record_id}/read") +async def mark_inbox_read( + record_id: str, + current_user: dict = Depends(require_permission(PermissionTag.INBOX_EDIT)), + session: AsyncSession = Depends(get_session), +): + try: + service=Email(session=session) + data=await service.mark_read(record_id) + return JSONResponse(content={"data":data,"total":1,"status_code":200}) + except HTTPException: + raise + except Exception as e: + raise HTTPException(status_code=500,detail=str(e)) + + +@router.get("/inbox/{record_id}/read-status") +async def get_inbox_read_status( + record_id: str, + token: str | None = Query(None), + current_user: dict = Depends(require_permission(PermissionTag.INBOX_EDIT)), + session: AsyncSession = Depends(get_session), +): + try: + service=Email(session=session,token=token) + data=await service.refresh_read_status(record_id) + return JSONResponse(content={"data":data,"total":1,"status_code":200}) + except HTTPException: + raise + except Exception as e: + raise HTTPException(status_code=500,detail=str(e)) + +@router.get("/inbox/all-applications") +async def get_all_applications( + record_id: str | None = Query(None), + application_status: Candidate_application_Status = Query(default=Candidate_application_Status.CLOSED), + isread: bool = Query(default=True), + search: str | None = Query(None), + top: int | None = Query(None), + skip: int = Query(0, ge=0), + current_user: dict = Depends(require_permission(PermissionTag.INBOX_VIEW)), + session: AsyncSession = Depends(get_session), +): + try: + service=Email(session=session) + + if application_status == Candidate_application_Status.PROCESS or application_status==Candidate_application_Status.REJECTED: + items=await service.get_all_applications(top, skip, search, application_status=application_status) + total=await service.count_inbox_messages(search, application_status=application_status) + return JSONResponse(content={"data":items,"total":total,"status_code":200}) + if isread==False: + items=await service.get_all_applications(top, skip, search, isread=False) + total=await service.count_inbox_messages(search, isread=False) + return JSONResponse(content={"data":items,"total":total,"status_code":200}) + if record_id: + item=await service.get_application_by_id(record_id) + return JSONResponse(content={"data":item,"total":1,"status_code":200}) + + items=await service.get_all_applications(top,skip,search) + total=await service.count_inbox_messages(search) + return JSONResponse(content={"data":items,"total":total,"status_code":200}) + except HTTPException: + raise + except Exception as e: + raise HTTPException(status_code=500,detail=str(e)) \ No newline at end of file diff --git a/backend/inbox/enums.py b/backend/inbox/enums.py new file mode 100644 index 0000000..992c23b --- /dev/null +++ b/backend/inbox/enums.py @@ -0,0 +1,11 @@ +from enum import Enum + +# (str, Enum), like EnumRoles and PermissionTag: a bare Enum member is not JSON +# serializable, so JSONResponse raises the moment a serializer emits this field. +class Candidate_application_Status(str, Enum): + PROCESS="PROCESS" + PENDING="PENDING" + APPROVED="APPROVED" + REJECTED="REJECTED" + ONHOLD="ONHOLD" + CLOSED="CLOSED" \ No newline at end of file diff --git a/backend/inbox/file_decoder.py b/backend/inbox/file_decoder.py index a7b117f..368383f 100644 --- a/backend/inbox/file_decoder.py +++ b/backend/inbox/file_decoder.py @@ -1,5 +1,7 @@ """Decode Graph fileAttachment contentBytes into PDF / DOC / DOCX files.""" - +# this file is decoding the pdf and also calling in the flow of first fetch of email if i do use func from this file rather then touchjing the email flow and create a bg task from here that can call the llm re i add param of subject and readc the file of pdf to get +#the location to the llm_call thne it's probable that without touching the real flow i can use background task without stopping or delaying the real result and add a column in Inbox_Messages that i can later update the file recorby using filename to pdate the answer or suggeswtions from the lmm that i can later or get from get api so user/recruiter can see and map the candidate to it's real final job_post_id that then can be linked with job_post_id +# as job_post_id is already linked by created_by and llm_call would require to read job_post of every recruiter and user ever posted only the posts that are still active it must read all post content and then finalize that this candidate might inlcude one of or more then one job_post_id : Note use list[uuid] to map with job_post_id inside Inbox_Messages table from __future__ import annotations import asyncio diff --git a/backend/inbox/models.py b/backend/inbox/models.py index 8b7fdf9..97f9699 100644 --- a/backend/inbox/models.py +++ b/backend/inbox/models.py @@ -1,13 +1,31 @@ +import logging +import os import uuid -from datetime import datetime +from datetime import datetime, timezone from typing import Any, Optional -from sqlalchemy import Column, func, or_ +from dotenv import load_dotenv +from fastapi import HTTPException +from inbox.enums import Candidate_application_Status +from role.models import EnumRoles, Roles +from sqlalchemy import Column, DateTime, func, or_, update from sqlalchemy.dialects.postgresql import JSONB +from sqlalchemy.exc import IntegrityError from sqlalchemy.ext.asyncio import AsyncSession -from sqlmodel import Field, Relationship, SQLModel, select +from sqlmodel import Field, Relationship, SQLModel, select, true from users.models import Users +from users.plugins import hash_password + +load_dotenv() +logger = logging.getLogger("inbox.models") + +# Placeholder only. The account lands inactive and the candidate is mailed a +# confirmation link; the real password comes from the reset flow afterwards. +DEFAULT_CANDIDATE_PASSWORD = os.getenv("DEFAULT_CANDIDATE_PASSWORD", "Utopia!@#") +CANDIDATE_ROLE_ID_FALLBACK = 8 # mirrors users/views.py:signup_user +SKIP_SENDER_PREFIXES = ("noreply", "no-reply", "donotreply", "do-not-reply", + "mailer-daemon", "postmaster", "bounce") class Inbox(SQLModel, table=True): @@ -49,6 +67,7 @@ class Inbox_Messages(SQLModel, table=True): full_email_response: dict[str, Any] | None = Field( default=None, sa_column=Column(JSONB) ) + application_status: Candidate_application_Status = Field(default=Candidate_application_Status.CLOSED) message_subject: str message_body: str message_sent_time: str @@ -62,6 +81,14 @@ class Inbox_Messages(SQLModel, table=True): message_reply: str | None = Field(default=None) file_name: str | None = Field(default=None) file_path: str | None = Field(default=None) + resume_text: str | None = Field(default=None) + experience: str | None = Field(default=None) + suggested_job_post_ids: list[str] | None = Field(default=None, sa_column=Column(JSONB)) + match_summary: str | None = Field(default=None) + match_reasoning: str | None = Field(default=None) + match_status: str | None = Field(default=None) + match_error: str | None = Field(default=None) + matched_at: datetime | None = Field(default=None, sa_type=DateTime(timezone=True)) inbox: list[Inbox] = Relationship(back_populates="messages") @@ -74,6 +101,58 @@ class Inbox_Messages(SQLModel, table=True): return body return email_data.get("bodyPreview") or "" + # @classmethod + # async def get_candidate_profile(cls,session:AsyncSession,user_id:uuid.UUID|None=None): + # try: + # qryy=select(cls,Users).join(cls,cls.) + # if user_id + + # except Exception as e: + # raise HTTPException(status_code=500,detail=str(e)) + + @classmethod + async def get_all_applications(cls,session:AsyncSession,message_id:uuid.UUID|int|None=None): + try: + qry=select(cls.message_id,cls.full_email_response,cls.message_subject,cls.message_from,cls.message_to,cls.message_sent_time,cls.message_read,cls.attachment) + if message_id: + qry=qry.where(cls.message_id==message_id) + result=await session.execute(qry) + return result.scalars().all() + + except Exception as e: + raise HTTPException(status_code=500,detail=str(e)) + @classmethod + async def set_match_result( + cls, + session: AsyncSession, + record_id, + *, + resume_text=None, + experience=None, + suggested_job_post_ids=None, + summary="", + reasoning="", + status="", + error="", + ): + """Persist agent output onto one inbox row; returns the row or None.""" + row = await cls.get_inbox_message_by_id(session, record_id) + if not row: + return None + if resume_text is not None: + row.resume_text = resume_text + row.suggested_job_post_ids = suggested_job_post_ids + row.match_summary = summary or None + row.match_reasoning = reasoning or None + row.match_status = status or None + row.match_error = error or None + row.experience = experience or None + row.matched_at = datetime.now(timezone.utc) + session.add(row) + await session.commit() + await session.refresh(row) + return row + @classmethod def _fields_from_email(cls, email_data: dict, file_path: list[str] | None = None) -> dict: return { @@ -106,6 +185,72 @@ class Inbox_Messages(SQLModel, table=True): "full_email_response": email_data, } + @classmethod + def _sender_address(cls, email_data: dict) -> str: + return ( + email_data.get("from", {}) + .get("emailAddress", {}) + .get("address", "") + or "" + ).strip().lower() + + @classmethod + def _sender_display_name(cls, email_data: dict, address: str) -> str: + name = ( + email_data.get("from", {}) + .get("emailAddress", {}) + .get("name") + or "" + ).strip() + if name: + return name + return address.split("@", 1)[0] if address else "candidate" + + @classmethod + def _is_linkable_sender(cls, address: str) -> bool: + if not address or "@" not in address: + return False + local = address.split("@", 1)[0] + return not local.startswith(SKIP_SENDER_PREFIXES) + + @classmethod + async def _link_sender(cls,session:AsyncSession,email_data:dict,email): + address=cls._sender_address(email_data) + if not cls._is_linkable_sender(address): + return None + try: + user=(await session.execute( + select(Users).where(func.lower(Users.email)==address) + )).scalars().first() + + if not user: + role=await Roles.get_role_by_name(session,EnumRoles.CANDIDATE.value) + user=Users( + name=cls._sender_display_name(email_data,address), + email=address, + role_id=role.id if role else CANDIDATE_ROLE_ID_FALLBACK, + password=hash_password(DEFAULT_CANDIDATE_PASSWORD), + ) + session.add(user) + session.add(Inbox(user_id=user.id,message_id=email.id)) + await session.commit() + return address + + link=(await session.execute( + select(Inbox).where(Inbox.message_id==email.id,Inbox.user_id==user.id) + )).scalars().first() + if not link: + session.add(Inbox(user_id=user.id,message_id=email.id)) + await session.commit() + return None + except IntegrityError: + await session.rollback() + return None + except Exception as e: + await session.rollback() + logger.warning("sender link failed for %s: %s",address,e) + return None + @classmethod async def insert_email( cls, @@ -113,9 +258,11 @@ class Inbox_Messages(SQLModel, table=True): email_data: dict, file_path: list[str] | None = None, ): + """Returns (row, new_user_email). new_user_email is set only when this call + created the sender's Users row.""" fields = cls._fields_from_email(email_data, file_path) external_id = fields.get("message_id") - + link_user=None if external_id: existing = ( await session.execute( @@ -128,12 +275,18 @@ class Inbox_Messages(SQLModel, table=True): session.add(existing) await session.commit() await session.refresh(existing) - return existing + + if fields.get("attachment"): + link_user=await cls._link_sender(session, email_data, existing) + return existing, link_user email = cls(**fields) session.add(email) await session.commit() - return email + + if fields.get("attachment"): + link_user=await cls._link_sender(session, email_data, email) + return email, link_user @classmethod def _search_filter(cls, search: str): @@ -146,15 +299,23 @@ class Inbox_Messages(SQLModel, table=True): @classmethod async def get_inbox_messages( - cls, session: AsyncSession, top: int | None, skip: int, search: str | None + cls, session: AsyncSession, top: int | None, skip: int, search: str | None, isread: bool=True, application_status: Candidate_application_Status=Candidate_application_Status.CLOSED ): statement = select(cls).order_by(cls.message_received_time.desc()) if search: statement = statement.where(cls._search_filter(search)) + + if application_status == Candidate_application_Status.PROCESS or application_status==Candidate_application_Status.REJECTED: + statement = statement.where(cls.application_status==application_status) + if skip: statement = statement.offset(skip) + if top is not None: statement = statement.limit(top) + + if isread==False: + statement = statement.where(cls.message_read==False) result = await session.execute(statement) return result.scalars().all() @@ -168,9 +329,46 @@ class Inbox_Messages(SQLModel, table=True): return result.scalars().first() @classmethod - async def count_inbox_messages(cls, session: AsyncSession, search: str | None): + async def count_inbox_messages(cls, session: AsyncSession, search: str | None, isread: bool=True, application_status: Candidate_application_Status=Candidate_application_Status.CLOSED): statement = select(func.count()).select_from(cls) if search: statement = statement.where(cls._search_filter(search)) + if application_status == Candidate_application_Status.PROCESS or application_status==Candidate_application_Status.REJECTED: + statement = statement.where(cls.application_status==application_status) + if isread==False: + statement = statement.where(cls.message_read==False) result = await session.execute(statement) return result.scalar_one() + + @classmethod + async def apply_read_status(cls, session: AsyncSession, changes) -> int: + """[{id, isRead, ...}] -> bulk UPDATE message_read. Returns rows touched. + + read is a ONE-WAY LATCH: only false -> true is applied, never the reverse. + mark_message_read writes the local column only — nothing pushes the state + back to Outlook — so upstream keeps reporting isRead=false and the + every-minute sync_read_status sweep would otherwise revert a mail the user + just opened. Cost of the latch: un-reading a mail in Outlook no longer + propagates here. + """ + if not changes: + return 0 + read_ids=[c.get("id") for c in changes if c.get("id") and c.get("isRead")] + if not read_ids: + return 0 + result=await session.execute( + update(cls).where(cls.message_id.in_(read_ids)).values(message_read=True) + ) + await session.commit() + return result.rowcount or 0 + + @classmethod + async def mark_message_read(cls, session: AsyncSession, record_id): + row=await cls.get_inbox_message_by_id(session,record_id) + if not row: + return None + row.message_read=True + session.add(row) + await session.commit() + await session.refresh(row) + return row diff --git a/backend/inbox/plugins.py b/backend/inbox/plugins.py index b0795ed..5176f49 100644 --- a/backend/inbox/plugins.py +++ b/backend/inbox/plugins.py @@ -1,32 +1,147 @@ -"""Inbox helpers — attachment loading and other non-routing checks.""" +"""Inbox helpers — attachment loading, resume text extraction, read-status sync.""" from __future__ import annotations import base64 +import os from pathlib import Path +from urllib.parse import quote + +import httpx +from dotenv import load_dotenv from inbox.models import Inbox_Messages +from job.candidate.views import FileRead + +load_dotenv() + +EMAIL_URL=os.getenv("EMAIL_URL") +EMAIL_API_TOKEN=os.getenv("EMAIL_API_TOKEN") +BACKEND_URL=os.getenv("BACKEND_URL","http://localhost:8000") + +_ATTACHMENTS_DIR=Path(__file__).resolve().parent/"decoded_attachments" -def load_message_files(message: Inbox_Messages) -> list[dict]: - """Read files from file_path when they exist on disk.""" +async def request_email_confirmation(email): + """POST /users/confirm-email/resend on this same service -> status code. + + Goes through the endpoint rather than importing Confirmation so the token row, + resend cooldown and mail send stay on one code path. + """ + async with httpx.AsyncClient(timeout=20.0) as client: + response=await client.post( + f"{BACKEND_URL.rstrip('/')}/users/confirm-email/resend", + json={"email":email}, + ) + return response.status_code + + +async def fetch_read_status_delta(folder, since=None, limit=1000, max_pages=10, token=None): + """GET /sync/read-status -> the raw round dict.""" + if not EMAIL_URL: + raise RuntimeError("EMAIL_URL must be set") + auth_token=token or EMAIL_API_TOKEN + if not auth_token: + raise RuntimeError("EMAIL_API_TOKEN must be set") + params={"folder":folder,"limit":limit,"max_pages":max_pages} + if since: + params["since"]=since + async with httpx.AsyncClient(timeout=15.0) as client: + response=await client.get( + f"{EMAIL_URL.rstrip('/')}/sync/read-status", + params=params, + headers={"Authorization":f"Bearer {auth_token}"}, + ) + if response.status_code>=400: + raise httpx.HTTPStatusError( + response.text, + request=response.request, + response=response, + ) + return response.json() + + +async def fetch_message_read_status(message_id, token=None): + """GET /sync/read-status/message/{id} -> record dict, or None on 404.""" + if not EMAIL_URL: + raise RuntimeError("EMAIL_URL must be set") + auth_token=token or EMAIL_API_TOKEN + if not auth_token: + raise RuntimeError("EMAIL_API_TOKEN must be set") + encoded_id=quote(str(message_id),safe="") + async with httpx.AsyncClient(timeout=15.0) as client: + response=await client.get( + f"{EMAIL_URL.rstrip('/')}/sync/read-status/message/{encoded_id}", + headers={"Authorization":f"Bearer {auth_token}"}, + ) + if response.status_code==404: + return None + if response.status_code>=400: + raise httpx.HTTPStatusError( + response.text, + request=response.request, + response=response, + ) + return response.json() + + +def resolve_attachment_path(path_str:str) -> Path: + """Prefer stored path; fall back to basename under decoded_attachments. + + Stored paths may be Windows absolutes written by the host API. The Taskiq + worker runs in Linux, where ``Path(r"D:\\...\\file.pdf").name`` is the + whole string (backslash is not a separator), so normalize separators before + taking the basename for the mounted attachments dir. + """ + raw=path_str.strip() + path=Path(raw) + if path.is_file(): + return path + basename=Path(raw.replace("\\","/")).name + fallback=_ATTACHMENTS_DIR/basename + if fallback.is_file(): + return fallback + return path + + +def load_message_files(message:Inbox_Messages) -> list[dict]: if not message.file_path: return [] - - files: list[dict] = [] + files=[] for path_str in message.file_path.split(","): - path = Path(path_str.strip()) + path=resolve_attachment_path(path_str) if not path.is_file(): continue try: - raw = path.read_bytes() + raw=path.read_bytes() except OSError: continue - files.append( - { - "file_name": path.name, - "content_base64": base64.b64encode(raw).decode("ascii"), - "size": len(raw), - } - ) + files.append({ + "file_name":path.name, + "content_base64":base64.b64encode(raw).decode("ascii"), + "size":len(raw), + }) return files + + +async def extract_resume_text(file_paths:list[str]) -> tuple[str,str]: + candidates=[resolve_attachment_path(p) for p in (file_paths or []) if p and p.strip()] + existing=[p for p in candidates if p.is_file() and p.suffix.lower()==".pdf"] + if not existing: + return "","no PDF attachment to extract (.doc/.docx not supported)" + + texts=[] + errors=[] + for path in existing: + try: + raw=path.read_bytes() + result=await FileRead(session=None,filename=path.name,file=raw).read_file() + text=(result.get("text") or "").strip() + if text: + texts.append(text) + except Exception as exc: + errors.append(f"{path.name}: {exc}") + + if not texts: + return "","; ".join(errors) if errors else "no text extracted from PDF" + return "\n\n---\n\n".join(texts),"" diff --git a/backend/inbox/serializers.py b/backend/inbox/serializers.py index 712db4b..4419fd6 100644 --- a/backend/inbox/serializers.py +++ b/backend/inbox/serializers.py @@ -2,9 +2,19 @@ from pathlib import Path from inbox.models import Inbox_Messages +# match_status (inbox/tasks.py) -> the resume badge the inbox tabs render. +_RESUME_STATUS = { + "processing": "Parsing", + "matched": "Parsed", + "no_text": "Failed", + "failed": "Failed", + "dlq": "Failed", + "skipped": "Pending", +} -def serialize_message(message: Inbox_Messages) -> dict: - """inbox_messages row -> the shape the #inbox Email tab renders.""" + +def _sender_name(message: Inbox_Messages) -> str: + """Graph's display name when the payload carries one, else the raw address.""" sender_name = message.message_from full = message.full_email_response if isinstance(full, dict): @@ -15,16 +25,26 @@ def serialize_message(message: Inbox_Messages) -> dict: name = email_address.get("name") if name: sender_name = name + return sender_name - attachment_name = None + +def _attachment_name(message: Inbox_Messages) -> str | None: if message.file_name: - attachment_name = message.file_name.split(",")[0].strip() or None - elif message.file_path: - attachment_name = Path(message.file_path.split(",")[0].strip()).name or None + return message.file_name.split(",")[0].strip() or None + if message.file_path: + return Path(message.file_path.split(",")[0].strip()).name or None + return None + + +def serialize_message(message: Inbox_Messages) -> dict: + """inbox_messages row -> the shape the #inbox Email tab renders.""" + sender_name = _sender_name(message) + attachment_name = _attachment_name(message) return { "id": str(message.id), "message_id": str(message.message_id) if message.message_id else None, + "full_email_response": message.full_email_response, "sender_name": sender_name, "fromEmail": message.message_from, "subject": message.message_subject, @@ -40,4 +60,44 @@ def serialize_message(message: Inbox_Messages) -> dict: "message_sent_time": message.message_sent_time, "message_reply": message.message_reply, "file_path": message.file_path, + "suggested_job_post_ids": list(message.suggested_job_post_ids or []), + "match_summary": message.match_summary, + "match_reasoning": message.match_reasoning, + "match_status": message.match_status, + "match_error": message.match_error, + "matched_at": message.matched_at.isoformat() if message.matched_at else None, + } + + +def serialize_application(message: Inbox_Messages) -> dict: + """inbox_messages row -> the shape the #inbox All Applications tab renders. + + `position` is the mail subject and `source` is the To address, which is where + the board tag (Rozee, Mustakbil, Employee Referral, ...) lands. + + The tab also wants ats_score, phone, experience, recruiter, duplicate and a + processing state beyond read/unread. inbox_messages has no columns for any of + those, so they come back null instead of invented — see the note in + inbox/file_decoder.py. `processing` is derived from message_read alone, so it + is only ever "Unread" or "Read"; Imported/Processed/Rejected need a column. + """ + return { + "id": str(message.id), + "name": _sender_name(message), + "email": message.message_from, + "position": message.message_subject, + "source": message.message_to, + "received": message.message_received_time, + "unread": not message.message_read, + "processing": "Read" if message.message_read else "Unread", + "application_status": message.application_status, + "resume_status": _RESUME_STATUS.get(message.match_status, "Pending"), + "attachment": _attachment_name(message), + "has_attachment": message.attachment, + "resume_text": message.resume_text, + "ats_score": None, + "phone": None, + "experience": message.experience or "", + "recruiter": None, + "duplicate": None, } diff --git a/backend/inbox/sync_tasks.py b/backend/inbox/sync_tasks.py new file mode 100644 index 0000000..42dc6ac --- /dev/null +++ b/backend/inbox/sync_tasks.py @@ -0,0 +1,82 @@ +"""Inbox Taskiq tasks — Outlook read-status delta sweep.""" + +from __future__ import annotations + +import logging +import os + +import httpx +import redis.asyncio as redis +from dotenv import load_dotenv + +from db_setup import session_scope +from inbox.models import Inbox_Messages +from inbox.plugins import fetch_read_status_delta +from taskiq_management.broker_setup import broker + +load_dotenv() + +logger=logging.getLogger("inbox.sync") + +EMAIL_SYNC_FOLDER=os.getenv("EMAIL_SYNC_FOLDER","inbox") +EMAIL_SYNC_SINCE=os.getenv("EMAIL_SYNC_SINCE") or None +EMAIL_SYNC_CRON=os.getenv("EMAIL_SYNC_CRON","* * * * *") +REDIS_URL=os.getenv("REDIS_URL","redis://localhost:6379/0") + +_LOCK_KEY="inbox:sync_read_status:lock" +_LOCK_TTL=300 +_MAX_ROUNDS=10 + + +@broker.task(task_name="inbox.sync_read_status",schedule=[{"cron":EMAIL_SYNC_CRON}]) +async def sync_read_status() -> dict: + client=redis.from_url(REDIS_URL,decode_responses=True) + try: + acquired=await client.set(_LOCK_KEY,"1",nx=True,ex=_LOCK_TTL) + if not acquired: + logger.info("sync_read_status skipped — lock held") + return {"skipped":"locked"} + + try: + rounds=0 + applied_total=0 + removed_total=0 + since=EMAIL_SYNC_SINCE + + while rounds<_MAX_ROUNDS: + rounds+=1 + try: + round_data=await fetch_read_status_delta( + EMAIL_SYNC_FOLDER, + since=since if rounds==1 else None, + limit=1000, + max_pages=10, + ) + except httpx.HTTPStatusError as e: + if e.response.status_code==401: + logger.warning("sync_read_status 401 — device-code sign-in required") + return {"error":"unauthorized","status_code":401} + raise + + changes=round_data.get("value") or [] + removed=round_data.get("removed") or [] + removed_total+=len(removed) + if removed: + logger.info("sync_read_status removed=%s",len(removed)) + + async with session_scope() as session: + applied=await Inbox_Messages.apply_read_status(session,changes) + applied_total+=applied + + if round_data.get("complete",True): + break + + return { + "rounds":rounds, + "applied":applied_total, + "removed":removed_total, + } + finally: + await client.delete(_LOCK_KEY) + finally: + await client.aclose() diff --git a/backend/inbox/tasks.py b/backend/inbox/tasks.py new file mode 100644 index 0000000..2d59cc9 --- /dev/null +++ b/backend/inbox/tasks.py @@ -0,0 +1,77 @@ +"""Inbox Taskiq tasks — CV → job-post matching.""" + +from __future__ import annotations + +import logging +from datetime import datetime,timezone + +from agent.execute_agent import run_agent +from db_setup import session_scope +from inbox.models import Inbox_Messages +from inbox.plugins import extract_resume_text +from job.job_post.models import JobPosts +from job.job_post.serializers import serialize_job_post +from taskiq_management.broker_setup import MAX_RETRIES,RETRY_DELAY,broker +from taskiq_management.middleware import PermanentTaskError + +logger=logging.getLogger("inbox.tasks") +_DONE=frozenset({"matched","skipped","no_text","failed","dlq"}) + + +@broker.task( + task_name="inbox.match_message", + retry_on_error=True, + max_retries=MAX_RETRIES, + delay=RETRY_DELAY, +) +async def match_inbox_message(record_id:str,force:bool=False) -> dict: + if not record_id or not str(record_id).strip(): + raise PermanentTaskError("record_id is required") + record_id=str(record_id).strip() + + async with session_scope() as session: + row=await Inbox_Messages.get_inbox_message_by_id(session,record_id) + if not row: + raise PermanentTaskError(f"inbox message {record_id} not found") + if not force and row.match_status in _DONE: + return {"status":row.match_status,"skipped":True} + if not row.attachment or not row.file_path: + raise PermanentTaskError("message has no attachment to match") + + paths=[p.strip() for p in row.file_path.split(",") if p.strip()] + subject=row.message_subject or "" + row.match_status="processing" + row.match_error=None + row.matched_at=datetime.now(timezone.utc) + session.add(row) + await session.commit() + + posts=await JobPosts.get_active_job_posts(session) + job_posts=[serialize_job_post(p) for p in posts] + + text,extract_err=await extract_resume_text(paths) + if not text: + async with session_scope() as session: + await Inbox_Messages.set_match_result( + session,record_id,status="no_text",error=extract_err or "no text extracted", + ) + return {"status":"no_text","error":extract_err} + + result=await run_agent(subject=subject,resume_text=text,job_posts=job_posts) + status=result.get("status") or "failed" + if status=="failed": + raise RuntimeError(result.get("error") or "agent returned failed status") + + async with session_scope() as session: + await Inbox_Messages.set_match_result( + session, + record_id, + resume_text=text, + experience=result.get("experience") or "", + suggested_job_post_ids=result.get("suggested_job_post_ids") or [], + summary=result.get("summary") or "", + reasoning=result.get("reasoning") or "", + status=status, + error=result.get("error") or "", + ) + return {"status":status,"suggested_job_post_ids":result.get("suggested_job_post_ids") or []} diff --git a/backend/inbox/views.py b/backend/inbox/views.py index e561a44..8c6092d 100644 --- a/backend/inbox/views.py +++ b/backend/inbox/views.py @@ -1,19 +1,41 @@ +import logging import httpx,os from fastapi import HTTPException +from inbox.enums import Candidate_application_Status from inbox.models import Inbox_Messages -from inbox.file_decoder import decode_attachment, AttachmentDecodeError -from inbox.serializers import serialize_message -from inbox.plugins import load_message_files +from inbox.file_decoder import decode_attachment +from inbox.serializers import serialize_application, serialize_message +from inbox.plugins import ( + EMAIL_API_TOKEN, + fetch_message_read_status, + load_message_files, + request_email_confirmation, +) from dotenv import load_dotenv load_dotenv() from sqlalchemy.ext.asyncio import AsyncSession -from pydantic import BaseModel +from datetime import datetime,timezone + +logger=logging.getLogger("inbox.match") + class Email: def __init__(self,session:AsyncSession,token=None): self.session=session self.get_url=os.getenv("EMAIL_URL") - self.token=token + self.token=token or EMAIL_API_TOKEN + self.pending_match_ids:list[str]=[] + self.pending_confirmation_emails:list[str]=[] + + # async def get_all_applications(self,app_id=None): + # try: + # if app_id: + # application_lst=await Inbox_Messages.get_all_applications(self.session,message_id=app_id) + # else: + # application_lst=await Inbox_Messages.get_all_applications(self.session) + # return application_lst + # except Exception as e: + # raise HTTPException(status_code=500,detail=str(e)) async def service_email(self,top,skip): async with httpx.AsyncClient() as client: @@ -38,8 +60,12 @@ class Email: if response.status_code==200: data=response.json() re_create_file=await decode_attachment(data.get("attachments")) - insert_func=await Inbox_Messages.insert_email(session=self.session,email_data=data,file_path=re_create_file) - return response.json() + row,new_user_email=await Inbox_Messages.insert_email(session=self.session,email_data=data,file_path=re_create_file) + if row.attachment and row.file_path and row.match_status is None: + self.pending_match_ids.append(str(row.id)) + if new_user_email: + self.pending_confirmation_emails.append(new_user_email) + return data else: raise HTTPException(status_code=response.status_code,detail=response.text) except Exception as e: @@ -66,5 +92,81 @@ class Email: item["files"]=files return item - async def count_inbox_messages(self,search=None): - return await Inbox_Messages.count_inbox_messages(self.session,search) + async def get_all_applications(self,top,skip,search=None,isread:bool=True,application_status:Candidate_application_Status=Candidate_application_Status.CLOSED): + if application_status == Candidate_application_Status.PROCESS or application_status==Candidate_application_Status.REJECTED: + messages=await Inbox_Messages.get_inbox_messages(self.session,top,skip,search,application_status=application_status) + elif isread==False: + messages=await Inbox_Messages.get_inbox_messages(self.session,top,skip,search,isread) + else: + messages=await Inbox_Messages.get_inbox_messages(self.session,top,skip,search) + return [serialize_application(m) for m in messages] + + async def get_application_by_id(self,record_id): + message=await Inbox_Messages.get_inbox_message_by_id(self.session,record_id) + if not message: + raise HTTPException(status_code=404,detail="Application not found") + return serialize_application(message) + + async def queue_rematch(self,record_id): + message=await Inbox_Messages.get_inbox_message_by_id(self.session,record_id) + if not message: + raise HTTPException(status_code=404,detail="Message not found") + if not message.attachment or not message.file_path: + raise HTTPException(status_code=400,detail="Message has no attachment to match") + return str(message.id) + + async def enqueue_matching(self,inbox_ids,force=False): + from inbox.tasks import match_inbox_message + task_ids=[] + for record_id in inbox_ids or []: + created_at=datetime.now(timezone.utc).isoformat() + task=await match_inbox_message.kicker().with_labels( + created_at=created_at, + correlation_id=str(record_id), + queue="inbox", + ).kiq(str(record_id),force=force) + task_ids.append(task.task_id) + return task_ids + + async def send_account_setup(self,emails): + results=[] + for email in emails or []: + try: + status=await request_email_confirmation(email) + results.append({"email":email,"sent":status==200}) + except Exception as e: + logger.warning("confirmation request failed for %s: %s",email,e) + results.append({"email":email,"sent":False}) + return results + + async def count_inbox_messages(self,search=None,isread:bool=True,application_status:Candidate_application_Status=Candidate_application_Status.CLOSED): + if application_status == Candidate_application_Status.PROCESS or application_status==Candidate_application_Status.REJECTED: + return await Inbox_Messages.count_inbox_messages(self.session,search,application_status=application_status) + elif isread==False: + return await Inbox_Messages.count_inbox_messages(self.session,search,isread=False) + else: + return await Inbox_Messages.count_inbox_messages(self.session,search) + + async def mark_read(self,record_id): + message=await Inbox_Messages.mark_message_read(self.session,record_id) + if not message: + raise HTTPException(status_code=404,detail="Message not found") + return serialize_message(message) + + async def refresh_read_status(self,record_id): + message=await Inbox_Messages.get_inbox_message_by_id(self.session,record_id) + if not message: + raise HTTPException(status_code=404,detail="Message not found") + if not message.message_id: + raise HTTPException(status_code=400,detail="Message has no upstream id") + try: + status=await fetch_message_read_status(message.message_id,token=self.token) + except httpx.HTTPStatusError as e: + raise HTTPException(status_code=e.response.status_code,detail=e.response.text) + except Exception as e: + raise HTTPException(status_code=500,detail=str(e)) + if status is None: + raise HTTPException(status_code=404,detail="Message not found upstream") + await Inbox_Messages.apply_read_status(self.session,[status]) + refreshed=await Inbox_Messages.get_inbox_message_by_id(self.session,record_id) + return serialize_message(refreshed) diff --git a/backend/job/app.py b/backend/job/app.py index af14011..682bb75 100644 --- a/backend/job/app.py +++ b/backend/job/app.py @@ -1,4 +1,4 @@ -from fastapi import APIRouter,Depends +from fastapi import APIRouter,Depends,Query from fastapi.responses import JSONResponse from fastapi import HTTPException from db_setup import get_session @@ -40,7 +40,7 @@ async def cv_upload( file_content = await file.read() logger.info(f"Received file: {file.filename} ({len(file_content)} bytes)") service=FileRead(session=session,filename=file.filename,file=file_content) - data=await service.read_file(file_content, file.filename) + data=await service.read_file() return JSONResponse(content={"data":data,"status_code":200}) except HTTPException: @@ -49,6 +49,22 @@ async def cv_upload( raise HTTPException(status_code=500,detail=str(e)) +@router.post("/candidate/inbox-match") +async def candidate_inbox_match( + inbox_message_id: str = Query(...), + current_user: dict = Depends(require_permission(PermissionTag.CANDIDATES_EDIT)), + session: AsyncSession = Depends(get_session), +): + try: + service=FileRead(session=session) + data=await service.match_inbox_cv(inbox_message_id) + return JSONResponse(content={"data":data,"status_code":200}) + except HTTPException: + raise + except Exception as e: + raise HTTPException(status_code=500,detail=str(e)) + + @router.post("/job/post-job") async def post_job( payload: JobPostCreate, @@ -85,3 +101,22 @@ async def buffer_channels( raise except Exception as e: raise HTTPException(status_code=500,detail=str(e)) + +# @router.get("/candidate/fetch") +# async def fetch_candidate( +# user_id:str=Query(None), +# current_user: dict = Depends(require_permission(PermissionTag.CANDIDATES_VIEW)), +# session: AsyncSession = Depends(get_session), +# ): +# try: +# service=CandidateView(session=session) +# if user_id: +# data=await service.get_candidate(user_id=user_id) +# else: +# data=await service.get_candidate() +# return JSONResponse(content={"data":data,"status_code":200}) +# except HTTPException: +# raise +# except Exception as e: +# raise HTTPException(status_code=500,detail=str(e)) + \ No newline at end of file diff --git a/backend/job/candidate/models.py b/backend/job/candidate/models.py index e69de29..482916b 100644 --- a/backend/job/candidate/models.py +++ b/backend/job/candidate/models.py @@ -0,0 +1,6 @@ +# from sqlmodel import SQLModel, Field +# from uuid import UUID, uuid4 +# from datetime import datetime +# from enum import Enum + +# class CV_extraction(SQLModel,table=True): diff --git a/backend/job/candidate/plugins.py b/backend/job/candidate/plugins.py index b0cb8ef..f545b15 100644 --- a/backend/job/candidate/plugins.py +++ b/backend/job/candidate/plugins.py @@ -27,3 +27,4 @@ def normalize_spaced_text(text) -> str: return "" lines = [re.sub(r" {2,}", " ", line).strip() for line in text.splitlines()] return re.sub(r"\n{3,}", "\n\n", "\n".join(lines)).strip() + diff --git a/backend/job/candidate/views.py b/backend/job/candidate/views.py index a59ddd5..6b10aa3 100644 --- a/backend/job/candidate/views.py +++ b/backend/job/candidate/views.py @@ -1,8 +1,11 @@ from sqlalchemy.ext.asyncio import AsyncSession import os,logging,io +from datetime import datetime,timezone from fastapi import HTTPException from pypdf import PdfReader from sqlalchemy import select +from sqlmodel import true +from inbox.models import Inbox_Messages from job.candidate.plugins import normalize_spaced_text class FileRead: @@ -25,4 +28,52 @@ class FileRead: except HTTPException: raise except Exception as e: - raise HTTPException(400, str(e)) \ No newline at end of file + raise HTTPException(400, str(e)) + + async def match_inbox_cv(self,inbox_message_id): + from inbox.plugins import resolve_attachment_path + from inbox.tasks import match_inbox_message + + row=await Inbox_Messages.get_inbox_message_by_id(self.session,inbox_message_id) + if not row: + raise HTTPException(status_code=404,detail="Message not found") + if not row.attachment or not row.file_path: + raise HTTPException(status_code=400,detail="your file isnt in the system") + + found=None + for path_str in (p.strip() for p in row.file_path.split(",") if p.strip()): + path=resolve_attachment_path(path_str) + if path.is_file(): + found=path + break + if found is None: + raise HTTPException(status_code=400,detail="your file isnt in the system") + + created_at=datetime.now(timezone.utc).isoformat() + task=await match_inbox_message.kicker().with_labels( + created_at=created_at, + correlation_id=str(row.id), + queue="inbox", + ).kiq(str(row.id),force=True) + + file_name=(row.file_name or "").split(",")[0].strip() or found.name + return { + "queued":True, + "inbox_message_id":str(row.id), + "file_name":file_name, + "task_id":task.task_id, + } + # async def get_intention(self,input): + # try: + # get_subject=Inbox_Messages.candidate_x_inbox(self.session,self.candidate_id) + # get_file= + +# class CandidateView: +# def __init__(self,session:AsyncSession): +# self.session=session + +# async def get_candidate(self,user_id=None): +# try: +# call_func=Inbox_Messages.get_candidate_profile(user_id=user_id) +# except Exception as e: +# raise HTTPException(status_code=500,detail=str(e)) \ No newline at end of file diff --git a/backend/job/job_post/models.py b/backend/job/job_post/models.py index 0c00b49..d87c564 100644 --- a/backend/job/job_post/models.py +++ b/backend/job/job_post/models.py @@ -62,6 +62,13 @@ class JobPosts(SQLModel, table=True): result = await session.execute(select(cls).where(cls.id == uid)) return result.scalars().first() + @classmethod + async def get_active_job_posts(cls, session: AsyncSession): + result = await session.execute( + select(cls).where(cls.is_active == True, cls.is_deleted == False) # noqa: E712 + ) + return result.scalars().all() + @classmethod async def insert_job_post(cls, session: AsyncSession, fields: dict): row = cls(**fields) diff --git a/backend/llm_setup.py b/backend/llm_setup.py new file mode 100644 index 0000000..4b94063 --- /dev/null +++ b/backend/llm_setup.py @@ -0,0 +1,142 @@ +"""OpenAI async client and a single llm_call helper. + +Pure module: no FastAPI imports and no HTTPException. + +Config is module-level `os.getenv` (house style for non-DB secrets); the client is +lazy, created on first use like `db_setup.get_engine()`. + + text = await llm_call(system, user) + data = await llm_call(system, user, json_mode=True) + +`init_llm()` confirms the key on startup and `close_llm()` disposes of the connection +pool, so both can hang off the FastAPI lifespan beside `init_db()` / `close_db()`. +""" + +from __future__ import annotations + +import asyncio +import json +import logging +import os + +from dotenv import load_dotenv +from openai import APIError, APIStatusError, AsyncOpenAI + +load_dotenv() + +logger = logging.getLogger("llm") + +OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") +OPENAI_BASE_URL = os.getenv("OPENAI_BASE_URL") or None +OPENAI_ORGANIZATION = os.getenv("OPENAI_ORGANIZATION") or None +OPENAI_PROJECT = os.getenv("OPENAI_PROJECT") or None +OPENAI_MODEL = os.getenv("OPENAI_MODEL", "gpt-5.4-mini") +OPENAI_MAX_OUTPUT_TOKENS = int(os.getenv("OPENAI_MAX_OUTPUT_TOKENS") or 32768) +OPENAI_TIMEOUT = float(os.getenv("OPENAI_TIMEOUT") or 60) +OPENAI_MAX_RETRIES = int(os.getenv("OPENAI_MAX_RETRIES") or 3) +OPENAI_CONNECT_RETRIES = int(os.getenv("OPENAI_CONNECT_RETRIES") or 3) + +# Blank OPENAI_TEMPERATURE means omit the param (some models reject it). +_raw_temp = (os.getenv("OPENAI_TEMPERATURE") or "").strip() +OPENAI_TEMPERATURE = float(_raw_temp) if _raw_temp else None + +_client: AsyncOpenAI | None = None + + +def get_client() -> AsyncOpenAI: + """The process-wide AsyncOpenAI client, created on first use.""" + global _client + if _client is None: + if not OPENAI_API_KEY: + raise RuntimeError("OPENAI_API_KEY is not configured") + _client = AsyncOpenAI( + api_key=OPENAI_API_KEY, + base_url=OPENAI_BASE_URL, + organization=OPENAI_ORGANIZATION, + project=OPENAI_PROJECT, + timeout=OPENAI_TIMEOUT, + max_retries=OPENAI_MAX_RETRIES, + ) + return _client + + +async def llm_call(system, user, *, model=None, temperature=None, json_mode=False): + """One system+user turn. Returns text, or a parsed dict when json_mode=True. + + With json_mode the prompt must mention JSON somewhere or the API rejects the call. + """ + kwargs = { + "model": model or OPENAI_MODEL, + "max_completion_tokens": OPENAI_MAX_OUTPUT_TOKENS, + "messages": [ + {"role": "system", "content": system}, + {"role": "user", "content": user}, + ], + } + resolved = OPENAI_TEMPERATURE if temperature is None else temperature + if resolved is not None: + kwargs["temperature"] = resolved + if json_mode: + kwargs["response_format"] = {"type": "json_object"} + + response = await get_client().chat.completions.create(**kwargs) + content = (response.choices[0].message.content or "").strip() + if not json_mode: + return content + try: + return json.loads(content) + except json.JSONDecodeError as exc: + raise RuntimeError(f"model did not return valid JSON: {content[:200]}") from exc + + +async def check_connection(retries=None, delay=1.0): + """Confirm the key works, retrying with a capped backoff.""" + attempts = OPENAI_CONNECT_RETRIES if retries is None else retries + for attempt in range(1, max(attempts, 1) + 1): + try: + await get_client().models.list() + logger.info("openai reachable, default model %s", OPENAI_MODEL) + return + except APIStatusError as exc: + if exc.status_code in (401, 403): + raise RuntimeError(f"OPENAI_API_KEY rejected ({exc.status_code})") from exc + if attempt >= attempts: + raise + logger.warning("openai not ready (%s/%s): %s", attempt, attempts, exc) + await asyncio.sleep(delay) + delay = min(delay * 2, 10.0) + except APIError as exc: + if attempt >= attempts: + raise + logger.warning("openai not ready (%s/%s): %s", attempt, attempts, exc) + await asyncio.sleep(delay) + delay = min(delay * 2, 10.0) + + +async def init_llm(*, verify=True): + """Build the client and, unless told otherwise, confirm the key is live.""" + get_client() + if verify: + await check_connection() + + +async def close_llm(): + """Close the underlying httpx pool and reset the cached client.""" + global _client + if _client is not None: + await _client.close() + logger.info("openai client closed") + _client = None + + +# if __name__ == "__main__": +# logging.basicConfig(level=logging.INFO, format="%(levelname)-8s %(name)s: %(message)s") + +# async def _main(): +# try: +# await init_llm() +# print(await llm_call("You are terse.", "Reply with the single word: ready")) +# finally: +# await close_llm() + +# asyncio.run(_main()) diff --git a/backend/main.py b/backend/main.py index 2c07775..87ec33d 100644 --- a/backend/main.py +++ b/backend/main.py @@ -1,22 +1,59 @@ import logging +from contextlib import asynccontextmanager -import fastapi from fastapi.middleware.cors import CORSMiddleware -from pydantic import BaseModel -from fastapi import FastAPI,APIRouter -from db_setup import lifespan +from fastapi import FastAPI +from db_setup import lifespan as db_lifespan from inbox.app import router as inbox_router from users.app import router as users_router from role.app import router as role_router from forget_password.app import router as forget_password_router from job.app import router as candidate_router from notifications.app import router as confirmation_router -# Without this the db/migration logs have no handler and are swallowed under uvicorn. -logging.basicConfig(level=logging.INFO, format="%(levelname)-8s %(name)s: %(message)s") -# lifespan connects to Postgres and brings migrations up to head on startup, -# and disposes of the connection pool on shutdown. -app = FastAPI(lifespan=lifespan) +logging.basicConfig(level=logging.INFO,format="%(levelname)-8s %(name)s: %(message)s") +logger=logging.getLogger("main") + + +@asynccontextmanager +async def lifespan(app): + async with db_lifespan(app): + broker_ready=False + llm_ready=False + agent_ready=False + close_llm=None + close_agent=None + broker=None + try: + from taskiq_management.broker_setup import broker as _broker + broker=_broker + await broker.startup() + broker_ready=True + except Exception as exc: + logger.warning("taskiq broker startup skipped: %s",exc) + try: + from llm_setup import init_llm,close_llm as _close_llm + from agent.agent_setup import init_agent,close_agent as _close_agent + close_llm=_close_llm + close_agent=_close_agent + await init_llm() + llm_ready=True + await init_agent() + agent_ready=True + except Exception as exc: + logger.warning("llm/agent startup skipped: %s",exc) + try: + yield + finally: + if agent_ready and close_agent is not None: + await close_agent() + if llm_ready and close_llm is not None: + await close_llm() + if broker_ready and broker is not None: + await broker.shutdown() + + +app=FastAPI(lifespan=lifespan) app.add_middleware( CORSMiddleware, allow_origins=["*"], @@ -30,4 +67,4 @@ app.include_router(users_router) app.include_router(role_router) app.include_router(forget_password_router) app.include_router(confirmation_router) -app.include_router(candidate_router) \ No newline at end of file +app.include_router(candidate_router) diff --git a/backend/requirements.txt b/backend/requirements.txt index 0ef2d18..63ad39f 100644 --- a/backend/requirements.txt +++ b/backend/requirements.txt @@ -29,3 +29,12 @@ bcrypt==5.0.0 # password hashing in users/plugins.py # --- PDF extraction -------------------------------------------------------- pypdf==5.1.0 + +# --- task queue ------------------------------------------------------------ +taskiq>=0.11,<0.12 # broker + worker/scheduler CLI (taskiq_management/) +taskiq-redis>=1.0,<2.0 # RedisStreamBroker / result backend / schedule source +redis>=5.0,<6.0 # DLQ middleware (taskiq_management/middleware.py) async client + +# --- LLM ------------------------------------------------------------------- +openai==2.53.0 # AsyncOpenAI client in llm_setup.py +langgraph==1.2.10 # StateGraph agent framework in agent/agent_setup.py diff --git a/backend/taskiq_management/broker_setup.py b/backend/taskiq_management/broker_setup.py new file mode 100644 index 0000000..931634c --- /dev/null +++ b/backend/taskiq_management/broker_setup.py @@ -0,0 +1,59 @@ +"""Taskiq broker — Redis Streams + smart retry + DLQ. + +Worker: taskiq worker taskiq_management.broker_setup:broker inbox.tasks inbox.sync_tasks taskiq_management.tasks +Scheduler: taskiq scheduler taskiq_management.broker_setup:scheduler +""" + +from __future__ import annotations + +import os + +from dotenv import load_dotenv +from taskiq import TaskiqScheduler +from taskiq.middlewares import SmartRetryMiddleware +from taskiq.schedule_sources import LabelScheduleSource +from taskiq_redis import ( + ListRedisScheduleSource, + RedisAsyncResultBackend, + RedisStreamBroker, +) + +from taskiq_management.middleware import DeadLetterMiddleware + +load_dotenv() + +REDIS_URL=os.getenv("REDIS_URL","redis://localhost:6379/0") +QUEUE_NAME=os.getenv("TASKIQ_QUEUE_NAME","inbox") +# 2 retries after first failure → max_retries=3 +MAX_RETRIES=int(os.getenv("TASKIQ_MAX_RETRIES","3")) +RETRY_DELAY=float(os.getenv("TASKIQ_RETRY_DELAY","5")) + +result_backend=RedisAsyncResultBackend(redis_url=REDIS_URL) +schedule_source=ListRedisScheduleSource(url=REDIS_URL,prefix="taskiq:schedule") + +broker=( + RedisStreamBroker( + url=REDIS_URL, + queue_name=QUEUE_NAME, + consumer_group_name=os.getenv("TASKIQ_CONSUMER_GROUP","taskiq"), + idle_timeout=int(os.getenv("TASKIQ_IDLE_TIMEOUT_MS","600000")), + ) + .with_result_backend(result_backend) + .with_middlewares( + DeadLetterMiddleware(redis_url=REDIS_URL), + SmartRetryMiddleware( + default_retry_count=MAX_RETRIES, + default_retry_label=True, + default_delay=RETRY_DELAY, + use_jitter=True, + use_delay_exponent=True, + max_delay_exponent=float(os.getenv("TASKIQ_MAX_DELAY","120")), + schedule_source=schedule_source, + ), + ) +) + +scheduler=TaskiqScheduler( + broker=broker, + sources=[schedule_source,LabelScheduleSource(broker)], +) diff --git a/backend/taskiq_management/middleware.py b/backend/taskiq_management/middleware.py new file mode 100644 index 0000000..a031dc0 --- /dev/null +++ b/backend/taskiq_management/middleware.py @@ -0,0 +1,102 @@ +"""PermanentTaskError + Redis Stream DLQ middleware for Taskiq. + +Middleware order: DeadLetterMiddleware before SmartRetryMiddleware so +permanent failures can set retry_on_error=False before SmartRetry runs. + +Pure module: no FastAPI imports. +""" + +from __future__ import annotations + +import json +import logging +from typing import Any + +import redis.asyncio as redis +from taskiq import TaskiqMiddleware +from taskiq.message import TaskiqMessage +from taskiq.result import TaskiqResult + +from taskiq_management.models import DLQ_STREAM +from taskiq_management.serializers import serialize_dlq_payload + +logger=logging.getLogger("taskiq.dlq") + + +class PermanentTaskError(Exception): + """Validation / business failure — DLQ immediately, no retries.""" + + +class DeadLetterMiddleware(TaskiqMiddleware): + def __init__(self,redis_url:str,stream:str=DLQ_STREAM): + super().__init__() + self.redis_url=redis_url + self.stream=stream + self._redis:redis.Redis|None=None + + async def startup(self) -> None: + self._redis=redis.from_url(self.redis_url,decode_responses=True) + + async def shutdown(self) -> None: + if self._redis is not None: + await self._redis.aclose() + self._redis=None + + def _client(self) -> redis.Redis: + if self._redis is None: + self._redis=redis.from_url(self.redis_url,decode_responses=True) + return self._redis + + async def on_error( + self, + message:TaskiqMessage, + result:TaskiqResult[Any], + exception:BaseException, + ) -> None: + retries=int(message.labels.get("_retries",0)) + max_retries=int(message.labels.get("max_retries",2)) + is_permanent=isinstance(exception,PermanentTaskError) + retries_exhausted=(retries+1)>=max_retries + + if is_permanent: + message.labels["retry_on_error"]=False + + if not is_permanent and not retries_exhausted: + return + + queue=getattr(self.broker,"queue_name",None) + payload=serialize_dlq_payload(message,exception,retries=retries+1,queue=queue) + try: + await self._client().xadd(self.stream,{"payload":json.dumps(payload,ensure_ascii=False,default=str)}) + logger.error( + "task %s (%s) sent to DLQ after %s", + message.task_name, + message.task_id, + "permanent failure" if is_permanent else f"{retries+1} attempts", + ) + except Exception: + logger.exception("failed to write DLQ entry for %s",message.task_id) + + await self._mark_inbox_dlq(message,exception) + + async def _mark_inbox_dlq(self,message:TaskiqMessage,exception:BaseException) -> None: + if message.task_name!="inbox.match_message": + return + record_id=(message.kwargs or {}).get("record_id") + if not record_id and message.args: + record_id=message.args[0] + if not record_id: + return + try: + from db_setup import session_scope + from inbox.models import Inbox_Messages + + async with session_scope() as session: + await Inbox_Messages.set_match_result( + session, + record_id, + status="dlq", + error=f"{type(exception).__name__}: {exception}", + ) + except Exception: + logger.exception("failed to mark inbox %s as dlq",record_id) diff --git a/backend/taskiq_management/models.py b/backend/taskiq_management/models.py new file mode 100644 index 0000000..f66669b --- /dev/null +++ b/backend/taskiq_management/models.py @@ -0,0 +1,15 @@ +"""Taskiq constants — DLQ stream + app version defaults. + +Pure module: no FastAPI imports. +""" + +from __future__ import annotations + +import os + +from dotenv import load_dotenv + +load_dotenv() + +DLQ_STREAM=os.getenv("TASKIQ_DLQ_STREAM","taskiq:dlq") +APP_VERSION=os.getenv("APP_VERSION","dev") diff --git a/backend/taskiq_management/serializers.py b/backend/taskiq_management/serializers.py new file mode 100644 index 0000000..497b8be --- /dev/null +++ b/backend/taskiq_management/serializers.py @@ -0,0 +1,44 @@ +"""DLQ payload serializers for Taskiq dead-letter entries. + +Pure module: no FastAPI imports. +""" + +from __future__ import annotations + +import os +import socket +import sys +import traceback +from datetime import datetime, timezone + +from taskiq.message import TaskiqMessage + +from taskiq_management.models import APP_VERSION + + +def serialize_dlq_payload( + message:TaskiqMessage, + exception:BaseException, + *, + retries:int, + queue:str|None=None, +) -> dict: + now=datetime.now(timezone.utc).isoformat() + return { + "task_name":message.task_name, + "task_id":message.task_id, + "kwargs":message.kwargs or {}, + "args":list(message.args or []), + "exception":type(exception).__name__, + "message":str(exception), + "traceback":"".join(traceback.format_exception(type(exception),exception,exception.__traceback__)), + "retry_count":retries, + "worker":os.getenv("TASKIQ_WORKER_NAME") or os.getenv("HOSTNAME") or socket.gethostname(), + "queue":message.labels.get("queue") or queue or "taskiq", + "created_at":message.labels.get("created_at") or now, + "failed_at":now, + "correlation_id":message.labels.get("correlation_id") or message.task_id, + "hostname":socket.gethostname(), + "python_version":sys.version.split()[0], + "app_version":APP_VERSION, + } diff --git a/backend/taskiq_management/tasks.py b/backend/taskiq_management/tasks.py new file mode 100644 index 0000000..ad5702c --- /dev/null +++ b/backend/taskiq_management/tasks.py @@ -0,0 +1,10 @@ +"""Framework smoke tasks for Taskiq — domain tasks stay in their packages.""" + +from __future__ import annotations + +from taskiq_management.broker_setup import broker + + +@broker.task(task_name="ping") +async def ping() -> str: + return "pong" diff --git a/docker-compose.yml b/docker-compose.yml index b2b1c23..be849d0 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,61 +1,64 @@ services: - minio: - image: minio/minio:RELEASE.2025-04-22T22-12-26Z - container_name: hrms-minio - command: server /data --console-address ":9001" - environment: - MINIO_ROOT_USER: ${MINIO_ROOT_USER:-minioadmin} - MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD:-minioadmin} + redis: + image: redis:7-alpine + container_name: hrms-redis + command: ["redis-server", "--appendonly", "yes"] ports: - - "9000:9000" # S3 API - - "9001:9001" # web console + - "${REDIS_PORT:-6379}:6379" volumes: - - minio-data:/data + - redis-data:/data healthcheck: - test: ["CMD", "mc", "ready", "local"] + test: ["CMD", "redis-cli", "ping"] interval: 10s timeout: 5s retries: 5 - start_period: 10s restart: unless-stopped - # One-shot: creates the attachments bucket, then exits. - minio-init: - image: minio/mc:RELEASE.2025-04-16T18-13-26Z - container_name: hrms-minio-init - depends_on: - minio: - condition: service_healthy + taskiq-worker: + build: + context: ./backend + container_name: hrms-taskiq-worker + command: + [ + "taskiq", + "worker", + "taskiq_management.broker_setup:broker", + "inbox.tasks", + "inbox.sync_tasks", + "taskiq_management.tasks", + "--workers", + "1", + ] + env_file: + - ./backend/.env environment: - MINIO_ROOT_USER: ${MINIO_ROOT_USER:-minioadmin} - MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD:-minioadmin} - MINIO_BUCKET: ${MINIO_BUCKET:-hrms-attachments} - entrypoint: > - /bin/sh -c " - mc alias set local http://minio:9000 \"$$MINIO_ROOT_USER\" \"$$MINIO_ROOT_PASSWORD\" && - mc mb --ignore-existing local/\"$$MINIO_BUCKET\" && - mc version enable local/\"$$MINIO_BUCKET\" && - echo 'bucket ready: '\"$$MINIO_BUCKET\" - " - - postgres: - image: postgres:16-alpine - container_name: hrms-postgres - environment: - POSTGRES_USER: ${DB_USERNAME:-postgres} - POSTGRES_PASSWORD: ${DB_PASSWORD:-postgres} - POSTGRES_DB: ${DB_NAME:-hrms} - ports: - - "${DB_PORT:-5432}:5432" + REDIS_URL: redis://redis:6379/0 + TASKIQ_QUEUE_NAME: inbox + TASKIQ_WORKER_NAME: worker-01 + DB_HOST: host.docker.internal + extra_hosts: + - "host.docker.internal:host-gateway" volumes: - - postgres-data:/var/lib/postgresql/data - healthcheck: - test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME:-postgres} -d ${DB_NAME:-hrms}"] - interval: 10s - timeout: 5s - retries: 5 + - ./backend/inbox/decoded_attachments:/app/inbox/decoded_attachments + depends_on: + redis: + condition: service_healthy + restart: unless-stopped + + taskiq-scheduler: + build: + context: ./backend + container_name: hrms-taskiq-scheduler + command: ["taskiq", "scheduler", "taskiq_management.broker_setup:scheduler", "inbox.sync_tasks"] + env_file: + - ./backend/.env + environment: + REDIS_URL: redis://redis:6379/0 + TASKIQ_QUEUE_NAME: inbox + depends_on: + redis: + condition: service_healthy restart: unless-stopped volumes: - minio-data: - postgres-data: + redis-data: diff --git a/frontend/src/api/inbox.js b/frontend/src/api/inbox.js index 8ad8e13..beee5c6 100644 --- a/frontend/src/api/inbox.js +++ b/frontend/src/api/inbox.js @@ -11,7 +11,41 @@ export function listMessages() { return request('/inbox/fetch') } +/** + * Persisted applications — the shape the All Applications tab renders. + * + * Unlike /inbox/fetch this one IS permissioned server-side + * (require_permission(INBOX_VIEW)), so a caller without the tag gets a 403. + */ +export function listApplications({ search, top, skip, recordId, isread, applicationStatus } = {}) { + return request('/inbox/all-applications', { + // `isread` is tri-valued on the wire: omit it for every tab (server defaults + // to true = no filter), send false for the Unread tab only. buildUrl drops + // undefined but keeps false, so `isread: undefined` sends no param at all. + // Same for `application_status`: omit for every tab (server defaults to + // CLOSED = no filter), send PROCESS / REJECTED for those tabs only. + params: { search, top, skip, record_id: recordId, isread, application_status: applicationStatus }, + }) +} + +/** + * One persisted message by id — the detail behind an inbox row. + * + * `record_id` is the inbox_messages PRIMARY KEY, not the Graph message_id: + * get_inbox_message_by_id runs uuid.UUID(record_id) and matches on `id`, so the + * external string id would fail the parse and 404. The `id` field on both + * /inbox/fetch and /inbox/all-applications rows is already that primary key. + */ +export function getMessage(recordId) { + return request('/inbox/fetch', { params: { record_id: recordId } }) +} + /** Triggers the Graph proxy to pull new mail and persist it. */ export function syncMailbox({ token, top, skip } = {}) { return request('/email/fetch', { params: { token, top, skip } }) } + +/** Marks one persisted inbox row read (local DB only). */ +export function markRead(recordId) { + return request(`/inbox/${recordId}/read`, { method: 'POST' }) +} diff --git a/frontend/src/lib/queryKeys.js b/frontend/src/lib/queryKeys.js index eb9464b..2a4d330 100644 --- a/frontend/src/lib/queryKeys.js +++ b/frontend/src/lib/queryKeys.js @@ -19,6 +19,8 @@ export const qk = { mailbox: { all: () => ['mailbox'], messages: () => ['mailbox', 'messages'], + applications: (p = {}) => ['mailbox', 'applications', p], + message: (id) => ['mailbox', 'message', id], }, // --- seed-backed buckets --- diff --git a/frontend/src/screens/Inbox.jsx b/frontend/src/screens/Inbox.jsx index fd29bde..89772ea 100644 --- a/frontend/src/screens/Inbox.jsx +++ b/frontend/src/screens/Inbox.jsx @@ -11,7 +11,7 @@ import { useMemo, useState } from 'react' import { useNavigate } from 'react-router-dom' -import { useQuery, useQueryClient } from '@tanstack/react-query' +import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query' import Modal from '../ui/Modal' import { Tabs } from '../ui/Tabs' @@ -23,30 +23,60 @@ import { friendlyAuthError } from '../lib/errors' import * as inboxApi from '../api/inbox' import { atsRecommendationClass, avatarColor, companies, fmtDate, fmtShort, getJob, - initials as initialsOf, int, locations, pick, relTime, skillsPool, TODAY, + initials as initialsOf, inboxSources, int, locations, pick, relTime, sourceMeta, + TODAY, } from '../data/seed' const TABS = ['All Applications', 'Unread', 'Imported', 'Processed', 'Rejected', 'Duplicates', 'Email'] +/** + * Server-side filters for the tabs that /inbox/all-applications can narrow. + * Unfiltered tabs (and countsQuery) pass `{}` so the backend defaults apply — + * isread=true and application_status=CLOSED both mean "no filter". + */ +const TAB_FILTERS = { + Unread: { isread: false }, + Processed: { applicationStatus: 'PROCESS' }, + Rejected: { applicationStatus: 'REJECTED' }, +} + /** The prototype computed "time ago" against a fixed 2026-07-09T20:00. */ const NOW = new Date('2026-07-09T20:00') -function resumeText(i) { - return `${i.name.toUpperCase()} -${i.email} · ${i.phone} -${'—'.repeat(30)} -PROFESSIONAL SUMMARY -${i.experience} years of experience. Applied for ${i.position} via ${i.source}. +/** + * The seed candidate record importEmail() writes needs a number. The agent + * returns a verdict, not a score, so there is nothing on the wire to use — + * named here so the fabricated value is visible at its point of use instead of + * arriving disguised as a server field on every message. + */ +const SEED_ATS_SCORE = 70 -EXPERIENCE -• ${pick(companies)} — Senior role (2021–Present) -• ${pick(companies)} — Associate (2018–2021) +/** + * message_received_time / message_sent_time are plain string columns + * (backend/inbox/models.py:54-56), not timestamps. An unparseable value yields + * an Invalid Date that every fmt* helper renders as the literal "Invalid Date", + * so return null instead and let the call sites decide what to show. + */ +function parseDate(value) { + if (!value) return null + const d = new Date(value) + return Number.isNaN(d.getTime()) ? null : d +} -EDUCATION -• Bachelor's Degree, Computer Science - -SKILLS -• ${pick(skillsPool)}, ${pick(skillsPool)}, ${pick(skillsPool)}, ${pick(skillsPool)}` +/** + * `source` arrives as the raw To address, because that is where the board tag + * lands — careers-rozee@, employee-referral@, mustakbil@ and so on. Strip + * everything but letters from both sides so "Employee Referral" still matches + * "employee-referral@", and keep the brand colour SourceChip paints from. + * Nothing matches -> show the first recipient verbatim rather than guess. + */ +function sourceFrom(messageTo) { + const raw = (messageTo || '').trim() + if (!raw) return { source: 'Unknown', sourceMeta: null } + const flat = raw.toLowerCase().replace(/[^a-z]/g, '') + const hit = inboxSources.find((s) => flat.includes(s.toLowerCase().replace(/[^a-z]/g, ''))) + if (hit) return { source: hit, sourceMeta: sourceMeta[hit] } + return { source: raw.split(',')[0].trim(), sourceMeta: null } } function SourceChip({ item }) { @@ -60,10 +90,155 @@ function SourceChip({ item }) { ) } +/** + * Graph delivers the body as text/html, so rendering it verbatim as text — which + * is what keeps it XSS-safe — prints the raw markup at the user. + * + * DOMParser builds a DETACHED document: it is never adopted into the live DOM, so + * scripts do not run and never fires. Reading textContent off it is + * therefore both safe and readable, and needs no dangerouslySetInnerHTML. + */ +function htmlToText(value) { + const raw = (value || '').trim() + if (!raw) return '' + if (!/<[a-z!/]/i.test(raw)) return raw // already plain text + // textContent ignores block boundaries, so

a

b

would collapse to + // "ab". Turn breaks and closing block tags into newlines BEFORE parsing. + const withBreaks = raw + .replace(//gi, '\n') + .replace(/<\/(p|div|li|tr|h[1-6]|blockquote|table)\s*>/gi, '\n') + const doc = new DOMParser().parseFromString(withBreaks, 'text/html') + doc.querySelectorAll('script, style, head').forEach((n) => n.remove()) + return (doc.body?.textContent || '').replace(/[ \t]+\n/g, '\n').replace(/\n{3,}/g, '\n\n').trim() +} + +/** match_status -> the resume badge, mirroring _RESUME_STATUS in inbox/serializers.py. */ +const RESUME_STATUS = { + processing: 'Parsing', matched: 'Parsed', no_text: 'Failed', + failed: 'Failed', dlq: 'Failed', skipped: 'Pending', +} + +/** + * GET /inbox/fetch?record_id= -> the detail behind one application row. + * + * Returns serialize_message, a different shape from serialize_application, so it + * is remapped onto the row shape here and OVERLAID on the list row rather than + * replacing it: serialize_message carries the body and the real decoded + * attachments, but omits resume_text, so the list row keeps supplying that. + * suggested_job_post_ids is dropped, same as everywhere else on this page. + */ +async function fetchMessageDetail(recordId) { + const res = await inboxApi.getMessage(recordId) + const row = res?.data + if (!row) return null + const name = row.sender_name || row.fromEmail || 'Unknown' + return { + id: String(row.id), + name, + initials: initialsOf(name), + color: avatarColor(name), + email: row.fromEmail || '', + position: row.subject || '(no subject)', + ...sourceFrom(row.message_to), + received: parseDate(row.when) ?? parseDate(row.message_sent_time), + unread: Boolean(row.unread), + processing: row.unread ? 'Unread' : 'Read', + resumeStatus: RESUME_STATUS[row.match_status] ?? 'Pending', + attachment: row.attachment_name, + hasAttachment: Boolean(row.attachment), + body: htmlToText(row.body), + cc: row.message_cc || '', + bcc: row.message_bcc || '', + sentAt: parseDate(row.message_sent_time), + files: Array.isArray(row.files) ? row.files : [], + matchStatus: row.match_status || null, + matchSummary: row.match_summary || '', + matchReasoning: row.match_reasoning || '', + matchError: row.match_error || '', + matchedAt: parseDate(row.matched_at), + } +} + +/** + * GET /inbox/all-applications -> the shape the application tabs render. + * + * READ-ONLY: inbox_messages has no columns for duplicates, recruiter, phone, + * experience or an ATS score, so those arrive null and every mutating action + * on these tabs is disabled until the endpoints exist. `processing` is derived + * from message_read alone (Read/Unread). Processed / Rejected tabs filter on + * `application_status` (PROCESS / REJECTED); Imported / Duplicates stay empty + * with no backing columns. + */ +async function fetchApplications(params) { + const res = await inboxApi.listApplications(params) + const rows = Array.isArray(res?.data) ? res.data : [] + return rows.map((row) => { + const name = row.name || row.email || 'Unknown' + return { + id: String(row.id), + name, + initials: initialsOf(name), + color: avatarColor(name), + email: row.email || '', + position: row.position || '(no subject)', + ...sourceFrom(row.source), + received: parseDate(row.received), + unread: Boolean(row.unread), + processing: row.processing || 'Unread', + applicationStatus: row.application_status || null, + resumeStatus: row.resume_status || 'Pending', + attachment: row.attachment, + hasAttachment: Boolean(row.has_attachment), + resumeText: row.resume_text || '', + atsScore: row.ats_score, + phone: row.phone, + experience: row.experience, + recruiter: row.recruiter, + duplicate: Boolean(row.duplicate), + } + }) +} + +/** + * POST /inbox/{record_id}/read — flips message_read false -> true for one row. + * + * Optimistic, so the row un-bolds on click instead of after the round trip, and + * rolls back if the server rejects. Both mailbox caches hold {id, unread} rows, + * so one setQueriesData over qk.mailbox.all() covers the Email tab and the + * application tabs at once; `processing` is derived from the same column, so it + * moves with it. + * + * NOTE: the route requires INBOX_EDIT while the lists only require INBOX_VIEW, + * so a view-only user gets a 403 here and the row snaps back to unread. + */ +function useMarkRead(toast) { + const qc = useQueryClient() + return useMutation({ + mutationFn: (recordId) => inboxApi.markRead(recordId), + onMutate: async (recordId) => { + await qc.cancelQueries({ queryKey: qk.mailbox.all() }) + const previous = qc.getQueriesData({ queryKey: qk.mailbox.all() }) + qc.setQueriesData({ queryKey: qk.mailbox.all() }, (rows) => ( + Array.isArray(rows) + ? rows.map((r) => (r.id === recordId + ? { ...r, unread: false, processing: r.processing === 'Unread' ? 'Read' : r.processing } + : r)) + : rows + )) + return { previous } + }, + onError: (err, _recordId, ctx) => { + for (const [key, data] of ctx?.previous ?? []) qc.setQueryData(key, data) + toast(friendlyAuthError(err, 'Could not mark as read.'), 'error') + }, + onSettled: () => qc.invalidateQueries({ queryKey: qk.mailbox.all() }), + }) +} + export default function Inbox() { const { toast } = useToast() const navigate = useNavigate() - const { data: inbox = [] } = useQuery(seedQuery('inbox')) + const qc = useQueryClient() const { data: jobs = [] } = useQuery(seedQuery('jobs')) const { data: recruiters = [] } = useQuery(seedQuery('recruiters')) const updateInbox = useSeedMutation('inbox') @@ -76,6 +251,32 @@ export default function Inbox() { const [assigning, setAssigning] = useState(null) const [noting, setNoting] = useState(null) + // Tabs with a server-side filter pass their params; everything else (and + // countsQuery) passes `{}` so the backend defaults mean "no filter". + const tabFilter = TAB_FILTERS[tab] ?? {} + + const applicationsQuery = useQuery({ + queryKey: qk.mailbox.applications(tabFilter), + queryFn: () => fetchApplications(tabFilter), + enabled: tab !== 'Email', + }) + + /** + * The tab badges need whole-table counts, which a server-filtered response + * cannot give — and there is no counts endpoint. So the unfiltered set stays + * loaded for them. On every tab without a TAB_FILTERS entry this resolves to + * the SAME query key as the list above, so React Query serves both from one + * request. + */ + const countsQuery = useQuery({ + queryKey: qk.mailbox.applications({}), + queryFn: () => fetchApplications({}), + enabled: tab !== 'Email', + }) + + const inbox = applicationsQuery.data ?? [] + const allApplications = countsQuery.data ?? [] + const emailsQuery = useQuery({ queryKey: qk.mailbox.messages(), queryFn: async () => { @@ -87,11 +288,18 @@ export default function Inbox() { fromEmail: row.fromEmail || '', subject: row.subject || '', body: row.body || '', - when: row.when ? new Date(row.when) : new Date(), + when: parseDate(row.when) ?? parseDate(row.message_sent_time), unread: Boolean(row.unread), attachment: row.attachment_name || 'Resume.pdf', attachmentSize: '—', - atsScore: 70, + // The agent's verdict, straight off backend/inbox/serializers.py:44-48. + // suggested_job_post_ids is deliberately NOT carried: job posts stay + // dark to the inbox. + matchStatus: row.match_status || null, + matchSummary: row.match_summary || '', + matchReasoning: row.match_reasoning || '', + matchError: row.match_error || '', + matchedAt: parseDate(row.matched_at), imported: false, })) }, @@ -99,34 +307,57 @@ export default function Inbox() { }) const counts = useMemo( + // Counted off the UNFILTERED set — `inbox` is server-filtered on Unread / + // Processed / Rejected, so counting it there would report that tab's total + // for every badge. () => ({ - 'All Applications': inbox.length, - Unread: inbox.filter((i) => i.processing === 'Unread').length, - Imported: inbox.filter((i) => i.processing === 'Imported').length, - Processed: inbox.filter((i) => i.processing === 'Processed').length, - Rejected: inbox.filter((i) => i.processing === 'Rejected').length, - Duplicates: inbox.filter((i) => i.duplicate).length, + 'All Applications': allApplications.length, + Unread: allApplications.filter((i) => i.processing === 'Unread').length, + Imported: allApplications.filter((i) => i.processing === 'Imported').length, + Processed: allApplications.filter((i) => i.applicationStatus === 'PROCESS').length, + Rejected: allApplications.filter((i) => i.applicationStatus === 'REJECTED').length, + Duplicates: allApplications.filter((i) => i.duplicate).length, Email: (emailsQuery.data ?? []).filter((e) => e.unread).length, }), - [inbox, emailsQuery.data], + [allApplications, emailsQuery.data], ) const list = useMemo(() => { let l = inbox + // Unread / Processed / Rejected are already filtered server-side; re-applying + // client-side keeps the optimistic mark-read drop-off for Unread, and keeps + // Processed/Rejected coherent if a stale cache briefly holds mixed rows. if (tab === 'Unread') l = l.filter((i) => i.processing === 'Unread') else if (tab === 'Imported') l = l.filter((i) => i.processing === 'Imported') - else if (tab === 'Processed') l = l.filter((i) => i.processing === 'Processed') - else if (tab === 'Rejected') l = l.filter((i) => i.processing === 'Rejected') + else if (tab === 'Processed') l = l.filter((i) => i.applicationStatus === 'PROCESS') + else if (tab === 'Rejected') l = l.filter((i) => i.applicationStatus === 'REJECTED') else if (tab === 'Duplicates') l = l.filter((i) => i.duplicate) if (q) l = l.filter((i) => (i.name + i.position + i.source).toLowerCase().includes(q.toLowerCase())) return l }, [inbox, tab, q]) - const selected = inbox.find((i) => i.id === selectedId) + // Clicking a row fetches that one record from /inbox/fetch. The list row is + // kept as the base and the detail is overlaid, so the pane paints instantly + // from cached list data and fills in body/attachments when the fetch lands. + const detailQuery = useQuery({ + queryKey: qk.mailbox.message(selectedId), + queryFn: () => fetchMessageDetail(selectedId), + enabled: tab !== 'Email' && Boolean(selectedId), + }) + + const selectedRow = inbox.find((i) => i.id === selectedId) + const selected = selectedRow || detailQuery.data + ? { ...selectedRow, ...(detailQuery.data ?? {}) } + : null + + // The one mutation these tabs CAN persist — everything else on them is + // disabled until the endpoints exist. + const markRead = useMarkRead(toast) function select(id) { setSelectedId(id) - updateInbox((items) => items.map((i) => (i.id === id ? { ...i, unread: false } : i))) + const item = inbox.find((i) => i.id === id) + if (item?.unread) markRead.mutate(id) } function makeCandidate(item, job, cs) { @@ -223,7 +454,15 @@ export default function Inbox() {
- {list.length === 0 ? ( + {applicationsQuery.isPending && ( + Fetching applications from the server. + )} + {applicationsQuery.isError && ( + + {friendlyAuthError(applicationsQuery.error, 'Request failed')} + + )} + {applicationsQuery.isSuccess && list.length === 0 ? ( No applications in this view. ) : ( list.map((i) => ( @@ -241,11 +480,23 @@ export default function Inbox() { )}
{i.position}
-
{i.processing}
+
+ {i.processing} + {i.applicationStatus && i.applicationStatus !== 'CLOSED' && ( + {i.applicationStatus} + )} +
-
{relTime(Math.round((NOW - i.received) / 60000))}
-
+
+ {i.received ? relTime(Math.round((NOW - i.received) / 60000)) : '—'} +
+ {/* No ATS score exists server-side — the agent returns a + verdict, not a number. The chip stays off rather than + rendering a placeholder that reads as a real score. */} + {i.atsScore != null && ( +
+ )}
)) @@ -260,9 +511,16 @@ export default function Inbox() { Choose an item from the list to view details and take action. + ) : detailQuery.isError ? ( +
+ + {friendlyAuthError(detailQuery.error, 'Request failed')} + +
) : ( setPreviewing(selected)} onImport={() => importItem(selected)} onParse={() => parseResume(selected)} @@ -289,13 +547,17 @@ export default function Inbox() { } > - 
{resumeText(previewing)}
+ 
+            {previewing.resumeText || 'Resume text not extracted yet — the matching task has not run for this application.'}
+
)} @@ -336,9 +598,17 @@ export default function Inbox() { ) } -function ApplicationDetail({ item: i, onPreview, onImport, onParse, onAssign, onMove, onNote, onReject }) { +/** Fields inbox_messages has no column for come back null; show a dash, not "null". */ +function orDash(value, suffix = '') { + return value == null || value === '' ? '—' : `${value}${suffix}` +} + +function ApplicationDetail({ item: i, loading, onPreview, onImport, onParse, onAssign, onMove, onNote, onReject }) { const recLabel = i.atsScore >= 82 ? 'Strong Match' : i.atsScore >= 65 ? 'Potential Match' : 'Weak Match' const ringColor = i.atsScore >= 82 ? 'var(--success)' : i.atsScore >= 65 ? 'var(--warning)' : 'var(--danger)' + // Every action below writes to a table column or an endpoint that does not + // exist yet, so they are disabled rather than silently dropping the click. + const noBackend = 'Needs a backend endpoint — not implemented yet' return (
@@ -349,48 +619,103 @@ function ApplicationDetail({ item: i, onPreview, onImport, onParse, onAssign, on
{i.position}
{i.processing}{' '} + {i.applicationStatus && i.applicationStatus !== 'CLOSED' && ( + <>{i.applicationStatus}{' '} + )} {i.resumeStatus} - + {' '} + {loading && Loading details…}
-
-
-
{i.atsScore}
+ {i.atsScore != null && ( +
+
+
{i.atsScore}
+
+
ATS Score
-
ATS Score
-
+ )}
-
Email
{i.email}
-
Phone
{i.phone}
-
Experience
{i.experience} years
-
Assigned Recruiter
{i.recruiter}
-
Received
{fmtDate(i.received)}
+
Email
{orDash(i.email)}
+
Phone
{orDash(i.phone)}
+
Experience
{orDash(i.experience, ' years')}
+
Assigned Recruiter
{orDash(i.recruiter)}
-
Match
-
{recLabel}
+
Received
+
{i.received ? fmtDate(i.received) : '—'}
+ {/* Only present once GET /inbox/fetch?record_id= has resolved — the list + endpoint carries none of these. */} + {i.sentAt && ( +
Sent
{fmtDate(i.sentAt)}
+ )} + {i.cc &&
CC
{i.cc}
} + {i.bcc &&
BCC
{i.bcc}
} + {i.atsScore != null && ( +
+
Match
+
{recLabel}
+
+ )}
-
-
-
-
{i.attachment}
- -
- 
{resumeText(i)}
+ {/* Body arrives only from GET /inbox/fetch?record_id= — the list endpoint + does not carry it. Already run through htmlToText, and still rendered as + TEXT: inbound mail is attacker-supplied. A body that is only an empty + HTML skeleton flattens to '' and the block is skipped entirely. */} + {!loading && ( +
+ {i.body || This email has no message body.}
-
+ )} + + {i.hasAttachment && ( +
+
+
+
+ {orDash(i.attachment)} + {i.files?.[0]?.size != null && ( + · {Math.round(i.files[0].size / 1024)} KB + )} +
+ +
+ {/* The real extracted PDF text (inbox_messages.resume_text), written + by the matching task. Empty until that task has run. */} + 
+              {i.resumeText || 'Resume text not extracted yet — the matching task has not run for this application.'}
+
+
+
+ )}
- - - - - - + + + + +
@@ -436,13 +761,20 @@ function EmailTab({ query, jobs, updateCandidates, toast }) { const selected = emails.find((e) => e.id === selectedId) const unread = emails.filter((e) => e.unread).length - async function sync() { - toast('Fetching from Outlook…', 'info') - const res = await qc.refetchQueries({ queryKey: qk.mailbox.messages() }) - if (query.isError) toast('Sync failed', 'error') - else toast('Mailbox synced', 'success') - return res - } + const markRead = useMarkRead(toast) + + // Refetching the list alone only re-reads rows already in our DB. GET + // /email/fetch is the Graph proxy pull that inserts new mail and enqueues the + // matching agent, so it has to run FIRST — then the list is invalidated to + // pick up whatever it wrote. + const sync = useMutation({ + mutationFn: () => inboxApi.syncMailbox(), + onSuccess: async () => { + await qc.invalidateQueries({ queryKey: qk.mailbox.all() }) + toast('Mailbox synced', 'success') + }, + onError: (err) => toast(friendlyAuthError(err, 'Sync failed'), 'error'), + }) function importEmail(e) { const job = jobs[0] @@ -455,12 +787,12 @@ function EmailTab({ query, jobs, updateCandidates, toast }) { jobId: job.id, jobTitle: job.title, department: job.department, experience: int(2, 10), currentCompany: pick(companies), currentTitle: job.title, location: pick(locations), stage: 'Applied', status: 'Applied', - aiScore: e.atsScore, source: 'Microsoft Outlook', recruiter: job.recruiter, recruiterId: '', + aiScore: SEED_ATS_SCORE, source: 'Microsoft Outlook', recruiter: job.recruiter, recruiterId: '', applied: new Date(TODAY), education: "Bachelor's Degree", skills: job.skills.slice(0, 4), rating: '4.0', salary: 130000, matchedSkills: job.skills.slice(0, 3), missingSkills: job.skills.slice(3), recommendation: 'Potential Match', - subScores: { skills: e.atsScore, experience: 80, education: 80, keywords: e.atsScore, location: 100, salary: 90 }, + subScores: { skills: SEED_ATS_SCORE, experience: 80, education: 80, keywords: SEED_ATS_SCORE, location: 100, salary: 90 }, noticePeriod: '1 month', availability: '2 weeks', certifications: [], favorite: false, interviewStatus: 'Not Scheduled', }, @@ -472,6 +804,11 @@ function EmailTab({ query, jobs, updateCandidates, toast }) { const isImported = (e) => imported.has(e.id) + function selectEmail(e) { + setSelectedId(e.id) + if (e.unread) markRead.mutate(e.id) + } + return ( <>
@@ -479,8 +816,13 @@ function EmailTab({ query, jobs, updateCandidates, toast }) { {query.isPending ? 'Loading…' : query.isError ? 'Sync failed' : `${emails.length} messages · ${unread} unread`} -
@@ -498,8 +840,8 @@ function EmailTab({ query, jobs, updateCandidates, toast }) { {query.isSuccess && emails.map((e) => (
setSelectedId(e.id)} + className={`inbox-item${e.unread ? ' unread' : ''}${selectedId === e.id ? ' active' : ''}`} + onClick={() => selectEmail(e)} >
@@ -512,7 +854,7 @@ function EmailTab({ query, jobs, updateCandidates, toast }) { {isImported(e) && Imported}
-
{fmtShort(e.when)}
+
{e.when ? fmtShort(e.when) : '—'}
))} @@ -534,7 +876,9 @@ function EmailTab({ query, jobs, updateCandidates, toast }) {
{selected.from}
-
{selected.fromEmail} · {fmtDate(selected.when)}
+
+ {selected.fromEmail} · {selected.when ? fmtDate(selected.when) : 'Date unavailable'} +
@@ -551,7 +895,6 @@ function EmailTab({ query, jobs, updateCandidates, toast }) {
{selected.attachmentSize} · PDF
-