From 3b5ba425e87949a9c772579d4668e618e6008023 Mon Sep 17 00:00:00 2001 From: Talha Ahmed Date: Thu, 3 Sep 2026 18:22:59 +0500 Subject: [PATCH] CI: verify the build before it ships, and fix two dead lines in the deploy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The pipeline deployed whatever was on main without checking any of it. No build, no tests, no lint. A frontend that failed to compile zipped and uploaded exactly like a working one, and the first sign of trouble would have been the running site. The checks now gate the deploy. `deploy` needs a `checks` job, so a red main never reaches the bucket. A second workflow runs the same script on branches and pull requests, so the answer arrives before the merge rather than after it. main is excluded there because the deploy workflow already covers it. Both call scripts/ci-checks.sh rather than inlining the commands twice, which is the only way the two cannot drift into disagreeing about what passing means. What gates, and what deliberately does not: - frontend: npm ci then npm run verify. All four suites are jsdom, so no browser is needed on the runner. Node 22 matches frontend/Dockerfile. - bulk-ats: ruff check, ruff format --check, mypy, pytest. Scoped to app and tests. Repo-wide, ruff reports 926 errors and would reformat 126 files — the backend is written to another style and demanding a rewrite of it is not this change's business. The scoped set is clean today. - backend/tests: passes and gates, minus test_candidate_forms.py and test_employment_agent.py. Those nine failures predate this change and are unrelated to it. They are named in the script, not silently skipped, so the exclusion stays visible and someone can delete the two lines. Two dead lines in the deploy, found while reading it: - The step named "Configure AWS credentials" set three variables and then only echoed a message. `env:` is scoped to its own step, so the values were discarded before anything could use them. It did nothing while reading as though credentials were configured globally. Removed; the upload step sets them where they are actually used. - `-x ".gitignore/*"` excludes a *directory* named .gitignore, which does not exist, so the file was never excluded. Now `-x ".gitignore"`. The zip still ships frontend/node_modules, about 90 MB and most of the artifact. Left alone on purpose: nothing in this repo says what unpacks the object — there is no appspec and no deploy script here — so if that side runs without installing dependencies, dropping it would break the deploy. The workflow now carries a comment saying so and the one-line change to make once that is confirmed. .gitattributes pins *.sh to LF. The script is bash on a Linux runner; committed with CRLF from a machine with core.autocrlf=false it would fail on line one with `$'\r': command not found`, which reads as a broken pipeline rather than a line-ending problem. Verified by running scripts/ci-checks.sh locally end to end, exit 0. Only the npm ci line was skipped, because it would rewrite the 5,230 committed node_modules files; frontend/Dockerfile already builds that way. Both workflow files were parsed and their job graph inspected. Co-Authored-By: Claude Opus 5 (1M context) --- .gitattributes | 9 ++++ .gitea/workflows/ci.yml | 36 +++++++++++++++ .gitea/workflows/deploy-to-s3.yml | 73 ++++++++++++++++++++++--------- scripts/ci-checks.sh | 58 ++++++++++++++++++++++++ 4 files changed, 155 insertions(+), 21 deletions(-) create mode 100644 .gitattributes create mode 100644 .gitea/workflows/ci.yml create mode 100755 scripts/ci-checks.sh diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..51a702b --- /dev/null +++ b/.gitattributes @@ -0,0 +1,9 @@ +# Shell scripts must be LF in the repository, whatever a contributor's +# core.autocrlf happens to be. +# +# scripts/ci-checks.sh is executed by bash on the Gitea runner. Committed with +# CRLF it fails there with `$'\r': command not found` on the first line, which +# reads as a broken pipeline rather than a line-ending problem. This machine +# has core.autocrlf=true and normalises correctly on its own; that is a local +# setting, not a property of the repo, so it is pinned here instead. +*.sh text eol=lf diff --git a/.gitea/workflows/ci.yml b/.gitea/workflows/ci.yml new file mode 100644 index 0000000..7db53b6 --- /dev/null +++ b/.gitea/workflows/ci.yml @@ -0,0 +1,36 @@ +name: CI + +# Same checks deploy-to-s3.yml gates on, run before a change reaches main. +# main itself is excluded because the deploy workflow already runs them there; +# without branches-ignore every merge would run the suite twice. +on: + push: + branches-ignore: + - main + pull_request: + +jobs: + checks: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v3 + + - name: Set up Node + uses: actions/setup-node@v4 + with: + node-version: '22' + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.11' + + - name: Install Python dependencies + run: | + python -m pip install --upgrade pip + pip install -e ".[dev]" + pip install -r backend/requirements.txt + + - name: Run checks + run: bash scripts/ci-checks.sh diff --git a/.gitea/workflows/deploy-to-s3.yml b/.gitea/workflows/deploy-to-s3.yml index 76500c8..b4102e0 100644 --- a/.gitea/workflows/deploy-to-s3.yml +++ b/.gitea/workflows/deploy-to-s3.yml @@ -1,25 +1,58 @@ name: Deploy to S3 +# main only. Everything else is covered by ci.yml, which runs the same checks +# without deploying. on: push: - branches: + branches: - main jobs: - deploy: + # Nothing was verified before this existed: a frontend that failed to compile + # would zip and ship exactly like a working one. `deploy` now needs this job, + # so a red main does not reach the bucket. + checks: runs-on: ubuntu-latest steps: - name: Checkout repository uses: actions/checkout@v3 - - name: Configure AWS credentials - env: - AWS_ACCESS_KEY_ID: ${{ secrets.DEVOPS_USER_AWS_ACCESS_KEY_ID }} - AWS_SECRET_ACCESS_KEY: ${{ secrets.DEVOPS_USER_AWS_SECRET_ACCESS_KEY }} - AWS_DEFAULT_REGION: us-east-1 - run: | - echo "AWS credentials configured" + # 22 to match frontend/Dockerfile, so CI resolves the same tree the + # production image builds from. + - name: Set up Node + uses: actions/setup-node@v4 + with: + node-version: '22' + # 3.11 is the floor in pyproject.toml and the version the project's conda + # env runs. + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.11' + + - name: Install Python dependencies + run: | + python -m pip install --upgrade pip + pip install -e ".[dev]" + pip install -r backend/requirements.txt + + - name: Run checks + run: bash scripts/ci-checks.sh + + deploy: + needs: checks + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v3 + + # NOTE: this zip still contains frontend/node_modules, roughly 90 MB and + # the bulk of the artifact. It is left in deliberately. Nothing in this + # repo says what unpacks the zip on the other side — there is no appspec + # file and no deploy script here — so if that side runs the app without + # installing dependencies, dropping node_modules would break the deploy. + # Confirm what consumes the bucket object, then add -x "frontend/node_modules/*". - name: Archive project run: | apt-get update -y @@ -27,8 +60,8 @@ jobs: zip -r utopia-ai-hr-ats-portal.zip . \ -x ".git/*" \ -x ".gitea/*" \ - -x ".gitignore/*" \ - -x "*.DS_Store" + -x ".gitignore" \ + -x "*.DS_Store" - name: Install AWS CLI run: | @@ -37,20 +70,18 @@ jobs: curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip -q awscliv2.zip ./aws/install - aws --version + aws --version + # The credentials live only on this step. There used to be a separate + # "Configure AWS credentials" step above that set the same three variables + # and then only echoed a message — env: is scoped to its own step, so + # those values were discarded before anything could use them. It was doing + # nothing, and it read as though credentials were set up globally. - name: Upload files to S3 env: AWS_ACCESS_KEY_ID: ${{ secrets.DEVOPS_USER_AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.DEVOPS_USER_AWS_SECRET_ACCESS_KEY }} AWS_DEFAULT_REGION: us-east-1 run: | - echo "Uploading repo contents to S3..." - aws s3 cp utopia-ai-hr-ats-portal.zip s3://utopia-ai-s3-repo-bucket/utopia-ai-hr-ats-portal.zip - - - - - - - + echo "Uploading repo contents to S3..." + aws s3 cp utopia-ai-hr-ats-portal.zip s3://utopia-ai-s3-repo-bucket/utopia-ai-hr-ats-portal.zip diff --git a/scripts/ci-checks.sh b/scripts/ci-checks.sh new file mode 100755 index 0000000..c820a4e --- /dev/null +++ b/scripts/ci-checks.sh @@ -0,0 +1,58 @@ +#!/usr/bin/env bash +# +# Everything CI gates on, in one place. +# +# Two workflows call this: deploy-to-s3.yml runs it before shipping, and ci.yml +# runs it on branches and pull requests. Keeping the commands here rather than +# inline in both YAML files is the only way those two can't drift into +# disagreeing about what "passing" means. +# +# Runnable locally, from the repo root: bash scripts/ci-checks.sh +# +set -euo pipefail + +cd "$(dirname "$0")/.." + +step() { printf '\n\033[1m=== %s ===\033[0m\n' "$1"; } + +# ---------------------------------------------------------------- frontend +# npm ci, not npm install: it installs the lockfile exactly and fails if +# package.json and the lock have drifted apart. frontend/Dockerfile already +# builds this way, so CI and the production image resolve identical trees. +# +# It also wipes node_modules first, which matters here because node_modules is +# committed to this repo. CI gets a clean tree regardless of what was checked in. +step "frontend — build, 31 route renders, token, theme, inbox" +( + cd frontend + npm ci --no-audit --no-fund + npm run verify +) + +# ---------------------------------------------------------------- bulk-ats +# Scoped to `app` and `tests` on purpose. `ruff check .` over the whole repo +# reports 926 errors and `ruff format --check .` would rewrite 126 files: the +# backend was written to a different style and reformatting it is not CI's job +# to demand. These two directories are the bulk-ats package that CLAUDE.md sets +# the standard for, and they are clean today, so they can gate. +step "bulk-ats package — lint, format, types, tests" +ruff check app tests +ruff format --check app tests +mypy app +pytest tests -q + +# ---------------------------------------------------------------- backend +# The rest of backend/tests passes and gates normally. These two files do not, +# on main, independently of any change in this repo: +# +# test_candidate_forms.py 6 failures +# test_employment_agent.py 3 failures +# +# They are named here rather than silently skipped so the exclusion stays +# visible and temporary. Fix them and delete these two lines. +step "backend suite — minus two files that fail on main today" +pytest backend/tests -q \ + --ignore=backend/tests/test_candidate_forms.py \ + --ignore=backend/tests/test_employment_agent.py + +printf '\n\033[1mAll CI checks passed\033[0m\n'