Skip to content

Rollback strategy

Status: Draft for ops use — refine as Epic F deploy automation lands. Cross-link from hardening / incident path: #76.

Companion: Deployment strategy · Release strategy

Decide first: redeploy previous vs hotfix

Situation Prefer
Bad config / bad image; last version known-good Redeploy previous SHA (fastest)
Data migration already applied / irreversible Hotfix forward on hotfix/TASK-*
Partial frontend-only breakage Rollback Amplify app only
Partial API-only breakage Rollback ECS service image only

When in doubt: restore service health with previous artifacts, then fix forward under a new Project task.

API / Frappe (ECS via CodePipeline)

  1. Identify last known-good image tag (sha-… or prior release tag).
  2. Redeploy that image to the prod ECS service (pipeline previous execution, or update service task definition to previous image — follow aws-infra runbooks).
  3. Verify health: ALB health, /api/method/... smoke, Desk login if applicable.
  4. Open / update incident issue; file hotfix/TASK-* if forward fix needed.
  5. Do not leave prod on an untagged ad-hoc image — retag/record the SHA.

Workers / scheduler share the same image family — keep them consistent with web unless the runbook says otherwise.

Frontends (Amplify)

  1. In Amplify console (or CLI): redeploy previous successful job for the app (admin / venuplus / customer) or redeploy a known-good commit SHA.
  2. Confirm the app hostname serves the rolled-back build.
  3. If API and UI must move together, coordinate versions (same release train).

After rollback

  • Incident / Project issue updated with from→to SHAs
  • Stakeholders notified if customer-facing
  • Root cause task created (feature/TASK-* or fix/TASK-*)
  • If hotfix shipped: back-merge to develop per release strategy

What not to do

  • Force-push main or release/* to “undo” history
  • Deploy latest without recording the digest
  • Roll back through the payment processor (card voids/refunds are a separate finance path — not a git rollback)