Hover a node · click to focus · ←/→ steps
Promote only after eval. Canary a traffic slice on K8s; auto-rollback on SLO breach; GitOps is the audit trail.
When you need this
- Manual kubectl deploys with no history
- Model config drift between staging and prod
- Need change approval for enterprise releases
- Rollback depends on one engineer’s memory
Prerequisites
- Git repo for manifests (separate or monorepo)
- K8s cluster with Argo CD
- CI that can open PRs with image tags
Tools
Works best once model configs live in Git alongside app manifests.
Start with registry + experiment tracking before full deployment automation.
Steps
- 1
Store model manifests in Git
Image tag, env vars, resource limits, and feature flags as YAML. One folder (or overlay) per environment.
- 2
Configure Argo CD applications
Sync staging on merge; production requires manual sync or an approval policy / ApplicationSet progressive sync.
- 3
Link to MLflow registry version
CI updates the manifest image tag only when registry promotion succeeds — registry remains source of truth for model identity.
- 4
Practice rollback
Revert the Git commit and sync. Time a rollback drill quarterly; document who can approve prod sync.
Adoption pitfalls
- Argo auto-sync to prod without human gate for high-risk models
- Secrets in Git instead of ExternalSecrets / Sealed Secrets
- Manifest repo out of sync with registry for weeks
Adoption checklist
- No manual prod deploys
- Manifest change = PR with reviewer
- Rollback tested in last 90 days
- Registry version matches live manifest
SEER REAL assessment / sprint
Assessment checks deploy path and change control. Sprint often migrates one model family to Argo with staging auto-sync and prod approval.