What is a canary analysis, and how do you automate the go/no-go decision?
CI/CD & GitOps · Advanced level
Answer
Canary analysis compares the new version against the baseline using metrics such as error rate, latency, saturation, logs, and business KPIs. Automation makes a go/no-go decision by evaluating thresholds over a window and either promotes, pauses, or rolls back.
Technical explanation
Traffic shifting should be separate from artifact build so the same artifact can be promoted or rolled back.
Use readiness checks, error budgets, and automated metrics before increasing production exposure.
Safe release design assumes rollback may be needed and separates code deployment, data changes, and user exposure.
Compatibility windows and observability are mandatory when application and database versions overlap.
A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.
Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.
Hands-on example
1. Design an advanced delivery exercise for: What is a canary analysis, and how do you automate the go/no-go decision using one service, one Git repository, one artifact registry, and one Kubernetes environment.
2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.
3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.
4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.
5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.
Check how well your resume matches the role with our free resume checker— match score, ATS check, and the skills you're missing.
More CI/CD & GitOps interview questions
- What is CI/CD, and what is the difference between continuous delivery and continuous deployment?
- What are the goals of a CI pipeline beyond just running tests?
- What is Jenkins, and what is the difference between a controller and an agent?
- What is the difference between a freestyle job and a pipeline job in Jenkins?
- What is the difference between a declarative and a scripted Jenkins pipeline?
- What is a Jenkinsfile, and why keep your pipeline as code in the repo?
- Explain the structure of a declarative pipeline (agent, stages, steps, post).
- What is the post section used for, and what are its conditions (success, failure, always)?