diff --git a/.env b/.env new file mode 100644 index 0000000..94335dc --- /dev/null +++ b/.env @@ -0,0 +1,5 @@ +# Compose pointer only — NO secrets here. +# Makes `docker compose up -d --build` read ${FRONTEND_PORT}, ${BACKEND_PORT}, … +# from backend/.env without typing --env-file every time. +# All credentials and app config live in backend/.env. +COMPOSE_ENV_FILES=./backend/.env diff --git a/.gitignore b/.gitignore index 2275f4b..9acc410 100644 --- a/.gitignore +++ b/.gitignore @@ -34,10 +34,13 @@ __pycache__/ venv/ env/ -# Environment / secrets -.env +# Environment / secrets — app secrets only in backend/.env (never commit it). +# Root .env is a Compose pointer (COMPOSE_ENV_FILES) with no secrets; keep it. +backend/.env .env.* +!.env !.env.example +!backend/.env.example !frontend/.env.development !frontend/.env.production @@ -62,4 +65,5 @@ Utopia-ai-hr-ats-portal 1.pem # Local-only Compose overrides (never deployed) docker.local.env -tests/** \ No newline at end of file +tests/** +**/.env** \ No newline at end of file diff --git a/DOCKER.md b/DOCKER.md index 501bbeb..47a8711 100644 --- a/DOCKER.md +++ b/DOCKER.md @@ -1,10 +1,25 @@ # Docker +```bash +docker compose up -d --build +``` + +Root `.env` is a **pointer only** (`COMPOSE_ENV_FILES=./backend/.env`) so Compose +interpolates `${FRONTEND_PORT}`, `${BACKEND_PORT}`, … from **`backend/.env`**. +All secrets and app config live in `backend/.env` (also injected into containers +via `env_file`). + ## How the browser reaches the API -The SPA is on **http://127.0.0.1:5173** (nginx → `backend-api` on the Compose -network). The API is also published on **http://127.0.0.1:8000** for host tools -and `npm run dev` (`VITE_API_BASE=http://127.0.0.1:8000`). +| Surface | URL | +|---|---| +| SPA | http://127.0.0.1:5173 | +| API (host) | http://127.0.0.1:8000 | + +nginx on `:5173` also proxies API paths to `backend-api` (same-origin when +`VITE_API_BASE` is empty). + +In `backend/.env`: ```env FRONTEND_PORT=5173 @@ -12,17 +27,8 @@ BACKEND_PORT=8000 FRONTEND_URL=http://127.0.0.1:5173 ``` -Sole env file: **`backend/.env`** (no repo-root `.env`). Always pass it for -Compose variable substitution: - -```bash -docker compose --env-file ./backend/.env up -d --build -``` - ## Local (host Postgres) -In `backend/.env`: - ```env PROD_ENV=false DB_USERNAME=... @@ -31,115 +37,90 @@ DB_HOST=localhost DB_PORT=5432 DB_NAME=hrms DB_SSLMODE= -FRONTEND_PORT=8080 +FRONTEND_PORT=5173 +BACKEND_PORT=8000 +FRONTEND_URL=http://127.0.0.1:5173 ``` Containers set `IN_DOCKER=1`. With `PROD_ENV=false`, `db_setup` rewrites `localhost` / `127.0.0.1` → `host.docker.internal` for the connection URL only -(SSL off unless `DB_SSLMODE` is set). Host Postgres must accept Docker-bridge -clients (`listen_addresses`, `pg_hba`). +(SSL off unless `DB_SSLMODE` is set). ```bash cp backend/.env.example backend/.env # set JWT, OpenAI, DB_*, PROD_ENV=false -docker compose --env-file ./backend/.env up -d --build +docker compose up -d --build ``` | Service | Host access | |---|---| -| `frontend` | `${FRONTEND_PORT:-80}` (all interfaces) | -| `backend-api` | Compose network only (`backend-api:8000`); nginx proxies | -| `ats-engine` | Compose network only (`ats-engine:8100`) | -| `redis` | Compose network only (`redis:6379`) | +| `frontend` | `${FRONTEND_PORT:-5173}` | +| `backend-api` | `${BACKEND_PORT:-8000}` | +| `ats-engine` / `redis` | Compose network (optional host-ports overlay) | | `postgres` | not started (optional `--profile postgres`) | -Optional loopback publishes for host tools (curl / redis-cli / Postman): +Optional loopback publishes for ATS / Redis: ```bash -docker compose --env-file ./backend/.env -f docker-compose.yml -f docker-compose.host-ports.yml up -d +docker compose -f docker-compose.yml -f docker-compose.host-ports.yml up -d ``` -If bind fails on Windows because Cursor/VS Code still holds `:80` / `:8100` / -`:6379` after a previous run, clear **Ports** in the IDE or set free values in -`backend/.env` (`FRONTEND_PORT`, and with the overlay `ATS_PORT` / `REDIS_PORT` / -`BACKEND_PORT`). - Optional live-reload / bind mounts: ```bash -docker compose --env-file ./backend/.env -f docker-compose.yml -f docker-compose.dev.yml up -d --build +docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build ``` Optional Compose Postgres (empty volume — not host data): ```bash -docker compose --env-file ./backend/.env --profile postgres up -d postgres +docker compose --profile postgres up -d postgres # set DB_HOST=postgres in backend/.env, then recreate backend services ``` ## Production (RDS) -In `backend/.env`, set `PROD_ENV=true` and point plain `DB_*` at RDS (no -prefixed credential sets). Blank `DB_SSLMODE` → SSL `require`. Host is never -rewritten. +In `backend/.env`, set `PROD_ENV=true` and point plain `DB_*` at RDS. Blank +`DB_SSLMODE` → SSL `require` (or set `DB_SSLMODE=require` explicitly). ```bash -cp backend/.env.example backend/.env -# Edit backend/.env: PROD_ENV=true, DB_* = RDS, JWT_SECRET_KEY, FRONTEND_PORT=80, … -docker compose --env-file ./backend/.env up -d --build -docker compose --env-file ./backend/.env ps +docker compose up -d --build +docker compose ps ``` -Browser → `http:///` → nginx (same-origin) → `backend-api:8000`. -CV files live in the `attachments-data` named volume (shared by API + workers). +First boot against RDS can take a few minutes while Alembic applies drift; the +API healthcheck `start_period` is 180s so Compose does not mark it unhealthy too early. ### Schema / migrations (automatic) On every `backend-api` start: 1. Fresh empty Postgres → create all tables from models and stamp a marker. -2. Otherwise → `alembic upgrade head` if any revision files exist in the image - (they normally do not — versions stay gitignored and are excluded from builds). +2. Otherwise → `alembic upgrade head` if any revision files exist in the image. 3. If `DB_AUTOGENERATE=true` → detect ORM drift and apply DDL **in-memory**. -4. Apply any pending `backend/migrations/manual/*.sql` (seed/RBAC batches only). +4. Apply any pending `backend/migrations/manual/*.sql`. -Toggle in `backend/.env`: `DB_AUTO_MIGRATE` / `DB_AUTOGENERATE` (default `true`). +Toggle in `backend/.env`: `DB_AUTO_MIGRATE` / `DB_AUTOGENERATE`. ### Verify ```bash -docker compose --env-file ./backend/.env config -curl -sf http://127.0.0.1:${FRONTEND_PORT:-8080}/health -curl -sf -o /dev/null -w "%{http_code}\n" http://127.0.0.1:${FRONTEND_PORT:-8080}/ -docker compose --env-file ./backend/.env logs -f backend-api +docker compose config +curl -sf http://127.0.0.1:5173/health +curl -sf http://127.0.0.1:8000/health +docker compose logs -f backend-api ``` ### Secrets -- Never bake `.env` into images (`.dockerignore` already excludes them). -- Require a strong `DB_PASSWORD` and `JWT_SECRET_KEY` before any real deploy. -- Only `backend/.env` holds app + Compose substitution values. +- Never bake `backend/.env` into images. +- Root `.env` must stay a pointer (`COMPOSE_ENV_FILES`) — no passwords there. - Do not put `DB_HOST` under Compose `environment:` (empty override blanks RDS). -### TLS - -This stack serves HTTP on the frontend port. Terminate TLS at a reverse proxy or -cloud load balancer in front of that port. - -## Data migration - -The optional Compose Postgres volume starts empty. To move an existing host database: - -```bash -pg_dump -Fc hrms > hrms.dump -pg_restore -h 127.0.0.1 -p 5433 -U postgres -d hrms --clean --if-exists hrms.dump -``` - ## Useful commands ```bash -docker compose --env-file ./backend/.env logs -f backend-api -docker compose --env-file ./backend/.env logs -f taskiq-worker -docker compose --env-file ./backend/.env restart backend-api -docker compose --env-file ./backend/.env down # keep volumes -docker compose --env-file ./backend/.env down -v # wipe volumes +docker compose logs -f backend-api +docker compose restart backend-api +docker compose down +docker compose down -v ``` diff --git a/backend/.env b/backend/.env new file mode 100644 index 0000000..2cca649 --- /dev/null +++ b/backend/.env @@ -0,0 +1,105 @@ + +# true → RDS over SSL (asyncpg). false → local Postgres over asyncpg (no SSH). +PROD_ENV=false + +DB_USERNAME=postgres +DB_PASSWORD=ambaig123 +DB_HOST=localhost +DB_PORT=5432 +DB_NAME=dev_hrms +# Blank: require when PROD_ENV=true, off when local. Override only if needed. +DB_SSLMODE= +# DB_USERNAME=utopiaaiadmin +# DB_PASSWORD=cOIYBbjeRpb5rZigZBYX +# DB_HOST=utopia-ai-hr-ats-portal-db.co658idowpql.us-east-2.rds.amazonaws.com +# DB_PORT=5432 +# DB_NAME=dev_hrms +# DB_SSLMODE=require + +EMAIL_URL=http://172.16.204.191:5000 +EMAIL_API_TOKEN=vn1a9sx6tI3L5LVXgblOiVpxn3gAiJXTVYvO5GICUNo +EMAIL_SYNC_FOLDER=inbox +EMAIL_SYNC_SINCE= +EMAIL_SYNC_CRON=* * * * * +JWT_SECRET_KEY=8B6lep0TuSrGp__2SFDCqKGyEluuUWV7SxvobhG5ECQYvN4o4P0OakRW0VWmiflZ6hgNK7RGdOVmi4H8LnSLwA +JWT_ALGORITHM=HS256 +JWT_ACCESS_TOKEN_EXPIRE_MINUTES=30 +JWT_REFRESH_TOKEN_EXPIRE_DAYS=7 + +TEAMS_MAIL_API_URL=http://ec2-13-222-174-68.compute-1.amazonaws.com:8000/mail/send +TEAMS_API_TOKEN=wt139Dg1bm0lXoYszzwCcMmkjQrkuP +RESET_CODE_TTL_SECONDS=60 +RESET_CODE_RESEND_SECONDS=30 +RESET_CODE_MAX_ATTEMPTS=5 +JWT_RESET_TOKEN_EXPIRE_MINUTES=10 + +# Vite / email links. Docker SPA publishes on FRONTEND_PORT (5173). +FRONTEND_URL=http://127.0.0.1:5173 +CONFIRM_EMAIL_PATH=/auth/confirm-email +CONFIRM_TOKEN_TTL_SECONDS=86400 +CONFIRM_TOKEN_RESEND_SECONDS=60 + +BUFFER_API=1aF1d6NQ3qTsjtfA_mdNwqfhK6myTqXK0vwSySiPRrK +BUFFER_API_URL=https://api.buffer.com +BUFFER_CHANNEL_ID=6a73328399afb4434907444f +CLIENT_ID=6a7331dc2d90e07071891bfc + +# OpenAI API Key +OPENAI_API_KEY=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 +OPENAI_BASE_URL=https://api.openai.com/v1 +OPENAI_MAX_OUTPUT_TOKENS=32768 +OPENAI_MODEL=gpt-5.4-mini +OPENAI_TIMEOUT=60 +OPENAI_MAX_RETRIES=3 +OPENAI_CONNECT_RETRIES=3 +OPENAI_TEMPERATURE=0.3 + +REDIS_URL=redis://redis:6379/0 +TASKIQ_QUEUE_NAME=inbox +TASKIQ_MAX_RETRIES=3 +TASKIQ_RETRY_DELAY=5 +TASKIQ_MAX_DELAY=120 +TASKIQ_DLQ_STREAM=taskiq:dlq +TASKIQ_IDLE_TIMEOUT_MS=600000 +APP_VERSION=dev + + +# Google Sheet +SPREADSHEET_NAME=Product Hiring +SPREADSHEET_ID=1yjRueJjA1NbW8rS-KsSCJxxnkqh6cPiJjwcmXbTLusk +SPREADSHEET_URL=https://docs.google.com/spreadsheets/d/1yjRueJjA1NbW8rS-KsSCJxxnkqh6cPiJjwcmXbTLusk/edit + +# Google Cloud project +GOOGLE_CLOUD_PROJECT=hrms-ats-portal +GOOGLE_ACCOUNT=ahmed.mujtaba@utopiabrands.com + +# OAuth client (Desktop) +GOOGLE_CLIENT_ID=679334897177-ufal3rogbg8cgm3qcren6pv20phqd7nl.apps.googleusercontent.com +GOOGLE_CLIENT_SECRET=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 +GOOGLE_OAUTH_CLIENT_ID_FILE=credentials/client_secret.json + +# Application Default Credentials (used by google-auth) +GOOGLE_APPLICATION_CREDENTIALS=credentials/application_default_credentials.json +GOOGLE_REFRESH_TOKEN=1//038IfgCu3D42fCgYIARAAGAMSNwF-L9IrYAZ_DJUqwC9ETwLtH23D46j61gWMwFQjRWPklFZIiLmv7Q3-TOgcNxTrjO30jgkeYOo +GOOGLE_CREDENTIALS_TYPE=authorized_user + + + +BACKEND_URL=http://backend-api:8000 +DB_AUTOGENERATE=false + +# --------------------------------------------------------------------------- + +DB_AUTO_MIGRATE=false + +# Compose host ports (docker compose --env-file ./backend/.env …). +FRONTEND_PORT=5173 +BACKEND_PORT=8000 +ATS_PORT=8100 +REDIS_PORT=6379 +POSTGRES_PORT=5433 + +UVICORN_WORKERS=2 + +# Empty = same-origin; frontend nginx proxies API paths to backend-api. +VITE_API_BASE= diff --git a/docker-compose.yml b/docker-compose.yml index 9d0a3d4..b13b9dd 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,26 +1,24 @@ # HR-ATS-Portal — single Compose file for local and production. # -# Sole env file: backend/.env (no repo-root .env). Pass it for Compose -# variable substitution (${FRONTEND_PORT}, ${DB_*}, …): +# docker compose up -d --build # -# docker compose --env-file ./backend/.env up -d --build -# docker compose --env-file ./backend/.env ps -# docker compose --env-file ./backend/.env logs -f backend-api +# Root `.env` only sets COMPOSE_ENV_FILES=./backend/.env so ${FRONTEND_PORT}, +# ${BACKEND_PORT}, ${UVICORN_WORKERS}, … interpolate from backend/.env. +# Secrets and app config live solely in backend/.env (loaded into containers +# via env_file as well). # # Local (PROD_ENV=false, DB_HOST=localhost in backend/.env): # Containers reach host Postgres via host.docker.internal (db_setup rewrite -# when IN_DOCKER=1). API / ATS / Redis stay on the Compose network by default -# (avoids IDE/Cursor stale port-forwards fighting Docker on Windows). Opt in: -# docker compose --env-file ./backend/.env -f docker-compose.yml -f docker-compose.host-ports.yml up -d +# when IN_DOCKER=1). Frontend :5173, API :8000 by default. # # Prod (PROD_ENV=true, DB_* = RDS in backend/.env — edit manually): # Same command. No host rewrite; SSL require when DB_SSLMODE is blank. # -# Optional Compose Postgres (empty volume, not host/RDS data): -# docker compose --env-file ./backend/.env --profile postgres up -d postgres +# Optional Compose Postgres: +# docker compose --profile postgres up -d postgres # -# Optional live-reload / bind mounts (not required day-to-day): -# docker compose --env-file ./backend/.env -f docker-compose.yml -f docker-compose.dev.yml up -d --build +# Optional live-reload / bind mounts: +# docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build # # See DOCKER.md for env checklist and verification. @@ -164,8 +162,9 @@ services: ] interval: 15s timeout: 5s - retries: 5 - start_period: 40s + retries: 12 + # RDS / first-boot Alembic can run well past 40s before /health answers. + start_period: 180s # --- bulk ATS scoring engine (standalone service form of app/) -------------------- ats-engine: