What a release used to be
Until 26 October, production shipped when I opened the Vercel dashboard and pressed Create Deployment. About fourteen commits reach main on a typical day. I released five times between 4 and 19 October, and the last version tag was v0.4.1, from 30 September. If you had asked which version was live, the honest answer was a commit SHA from /api/version and whatever I remembered about it.
Clinical systems often have the same shape. Each change is reviewed and tested on its own, but the release, the moment changes reach the people using them, rests on one person's memory. Change control asks a release four questions: what changed, who decided it should ship, what evidence showed it worked, and how do we get back. A pull request answers them for one change. Nothing answered them for the batch that went live.
Merges never deploy
The first rule is that a merge does nothing to production. vercel.json sets git.deploymentEnabled to false, so merges and branch pushes never start a build. Production ships from one GitHub Actions workflow, the weekly release train, every Sunday evening at 7 PM Central. I can also start it by hand.
The train checks its gates in order and stops at the first that fails:
- It runs only while a repository variable says it is on, and not at all while any open issue carries the
release:holdlabel. - The candidate is the head of main. If the live site already serves it, there is nothing to release.
- The repository's required check, Merge Gate, must have passed on that exact commit.
- A script reads every database migration added between the live commit and the candidate and flags statements that would break the running app: dropped tables or columns, renames, new NOT NULL constraints, type changes. A finding holds the release so that I ship it by hand.
Test the commit you ship
This gap was the one I did not see coming. Every pull request passes the full browser suite before it merges. A squash merge then creates a new commit, and CI on main skips the browser jobs, so the commit that ships has never been in a browser. It is almost always the same code as the pull request. "Almost always" is a poor thing to write in a release record.
So the train builds the candidate itself, with the same bundle budgets as CI, and runs the full Chromium Playwright suite against it in three shards, in the same digest-pinned container image. Those jobs get a read-only token. If they fail, nothing ships and the train opens an issue for me. That issue does not hold next week's train, because production has not changed.
Publish after production answers
With the gates passed, the train drafts a GitHub Release. The version comes from the commit messages since the last tag: a feature bumps the minor number, anything else the patch. It checks that main still points at the candidate, then POSTs a Vercel Deploy Hook. The hook can build main and do nothing else, and its URL is the only production-related value GitHub holds. Database credentials stay in Vercel. The train never holds a Vercel token, never runs a migration and never builds what ships.
Then it waits up to 40 minutes for /api/version to report the release commit, runs the synthetic probes against the live site, and only after they pass publishes the Release, which creates the tag. If any of that fails, the Release stays a draft and the train opens a release:hold issue assigned to me, which also stops next week's run. A version tag on this repository now records a commit that passed in production.
v1.0.0 shipped this way on 27 October, from a manual run. Every gate passed, and the Deploy Hook built production while Git deployments were switched off. Vercel does not document that those two work together. The first real run is how we found out, and the decision record says what to do if Vercel changes it.
The dry run that probed nothing
The train has a dry-run mode that probes the live site without drafting or deploying anything. The first dry run finished green. Its smoke-test job had been skipped.
The probe job's if condition had no status function, so GitHub Actions added an implicit success(), which is false whenever any upstream job was skipped. A dry run always skips the release-candidate jobs. The fix was an explicit condition starting with !cancelled(), and a test now pins it (#3027). A skipped job shows as a grey icon in a green run, and nobody reads grey icons. A gate needs a test that proves it ran.
What I would keep anywhere
- Separate merging from releasing, and give each release its own record: a version, notes, a compare link and the evidence.
- Test the artifact you ship. If your merge strategy creates a new commit, test that commit.
- Publish the release record after production has answered, so it describes what happened.
- Make holding a release cheaper than overriding a check. Here a label on an issue stops the train, and removing it is a deliberate act.
- Keep the deploy credential narrow. A URL that can only rebuild one branch can do far less harm than a token that can do anything.
Rollback is still a person, using Vercel's Instant Rollback. None of this makes the site a validated system. It does mean every week's changes have a record a reviewer can follow from the Release to each pull request in it.