CI/CD & GitOps interview questions & answers

What is CI/CD, and what is the difference between continuous delivery and continuous deployment?Basic

Answer

CI/CD is the automation discipline that moves code from commit to production safely and repeatably. CI continuously builds, tests, and validates every change. Continuous delivery means the software is always releasable but production deployment may still require approval. Continuous deployment means every change that passes the pipeline is automatically deployed to production.

Technical explanation

Continuous delivery still has a human or business decision before production; continuous deployment removes that manual production step when all gates pass.

Both require strong automated testing, artifact traceability, rollback strategy, and environment consistency.

CI/CD should optimize both speed and safety: fast feedback for developers, controlled promotion for environments, and clear evidence for operations.

The deployable artifact should be built once, versioned, scanned, and promoted by immutable version or digest rather than rebuilt per environment.

Use automated gates for tests, policy, security, and health checks; use human approval only for risk decisions that automation cannot make reliably.

Track delivery health with lead time, deployment frequency, change-failure rate, MTTR, pipeline duration, queue time, and flaky failure rate.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is CI/CD, and what is the difference between continuous delivery and continuous deployment.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What are the goals of a CI pipeline beyond just running tests?Basic

Answer

A CI pipeline should give fast confidence, not just run tests. It should compile/build, run unit and integration checks, validate style and policy, scan dependencies and images, produce traceable artifacts, publish reports, and block bad changes before they reach shared branches or deployment environments.

Technical explanation

CI/CD should optimize both speed and safety: fast feedback for developers, controlled promotion for environments, and clear evidence for operations.

The deployable artifact should be built once, versioned, scanned, and promoted by immutable version or digest rather than rebuilt per environment.

Use automated gates for tests, policy, security, and health checks; use human approval only for risk decisions that automation cannot make reliably.

Track delivery health with lead time, deployment frequency, change-failure rate, MTTR, pipeline duration, queue time, and flaky failure rate.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What are the goals of a CI pipeline beyond just running tests.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is Jenkins, and what is the difference between a controller and an agent?Basic

Answer

Jenkins is an automation server commonly used for CI/CD. The controller manages configuration, jobs, queues, credentials, plugins, and pipeline orchestration. Agents are the execution workers where builds actually run, so heavy build tasks should run on agents rather than on the controller.

Technical explanation

The controller schedules and coordinates; agents execute workload-specific commands and should be replaceable.

Do not overload the controller with compilers, Docker builds, or long-running test processes.

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is Jenkins, and what is the difference between a controller and an agent.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is the difference between a freestyle job and a pipeline job in Jenkins?Basic

Answer

A freestyle job is a UI-configured job with build steps and post-build actions. A pipeline job is defined as code, usually in a Jenkinsfile, and supports stages, approvals, parallel execution, retries, shared libraries, and version-controlled delivery logic. For serious CI/CD, I prefer pipeline jobs.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is the difference between a freestyle job and a pipeline job in Jenkins.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is the difference between a declarative and a scripted Jenkins pipeline?Basic

Answer

Declarative Pipeline is a structured Jenkins syntax with blocks like pipeline, agent, stages, steps, environment, and post. Scripted Pipeline is Groovy-driven and more flexible but easier to make inconsistent. I usually choose Declarative for standard delivery flows and use Scripted blocks only when needed.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is the difference between a declarative and a scripted Jenkins pipeline.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is a Jenkinsfile, and why keep your pipeline as code in the repo?Basic

Answer

A Jenkinsfile is the pipeline definition stored with the application code. Keeping it in the repo makes the pipeline versioned, reviewable, reproducible, branch-aware, and easier to evolve with the application instead of hiding delivery logic in Jenkins UI configuration.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is a Jenkinsfile, and why keep your pipeline as code in the repo.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

Explain the structure of a declarative pipeline (agent, stages, steps, post).Basic

Answer

A Declarative Pipeline normally starts with pipeline, chooses an agent, defines environment/options/parameters when needed, then lists stages with steps. The post section handles cleanup and notifications after success, failure, unstable, aborted, or always conditions.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: Explain the structure of a declarative pipeline (agent, stages, steps, post)..

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is the post section used for, and what are its conditions (success, failure, always)?Basic

Answer

The post section defines actions that run after a stage or the full pipeline completes. Common conditions are always for cleanup, success for publish or notify success, failure for alerts, unstable for test-quality issues, aborted for cancelled runs, and cleanup for final cleanup actions.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is the post section used for, and what are its conditions (success, failure, always).

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How do you run pipeline stages in parallel, and why would you?Basic

Answer

I run independent stages in parallel when they do not depend on each other, for example unit tests, linting, security scans, and image build checks. Parallelism shortens feedback time, but each branch must have isolated workspaces, deterministic inputs, and clear fail-fast behavior.

Technical explanation

Parallel stages must not write to the same output path unless isolated or synchronized.

Fail-fast behavior is useful for PR feedback but can hide secondary failures if not configured intentionally.

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How do you run pipeline stages in parallel, and why would you.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How do you securely handle credentials in Jenkins (credentials binding, masking)?Basic

Answer

Jenkins credentials should be stored in the credentials store and injected only inside a withCredentials block or a supported credentials binding. Secrets must be masked in logs, scoped to the minimum job/folder, never echoed, and preferably replaced with short-lived tokens or cloud identity federation where possible.

Technical explanation

Masking is a safety net, not a complete control; scripts can still leak secrets through files, command arguments, debug output, or third-party tools.

Prefer scoped, short-lived credentials and store only the minimum secrets needed for that job or environment.

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How do you securely handle credentials in Jenkins (credentials binding, masking).

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is the difference between environment variables and credentials in a pipeline?Basic

Answer

Environment variables are general runtime configuration values. Credentials are sensitive values managed by Jenkins with access control, masking, and scoped injection. A value such as APP_ENV can be a normal environment variable, but a token, password, SSH key, or cloud secret must be treated as a credential.

Technical explanation

Masking is a safety net, not a complete control; scripts can still leak secrets through files, command arguments, debug output, or third-party tools.

Prefer scoped, short-lived credentials and store only the minimum secrets needed for that job or environment.

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is the difference between environment variables and credentials in a pipeline.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is a Jenkins shared library, and why use one?Basic

Answer

A Jenkins shared library is reusable Pipeline code stored in a separate repository and loaded by Jenkins. I use it to standardize common functions such as build, scan, test, publish, deploy, notifications, and policy gates across many repositories without copy-pasting Jenkinsfiles.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is a Jenkins shared library, and why use one.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How do Jenkins agents get provisioned, and what are ephemeral/Kubernetes agents?Basic

Answer

Jenkins agents can be static machines, cloud instances, containers, or ephemeral Kubernetes Pods. Ephemeral agents are created for a build and destroyed afterward, which improves isolation, reduces snowflake build machines, and lets teams pin tools in container images.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How do Jenkins agents get provisioned, and what are ephemeral/Kubernetes agents.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How would you run builds in isolated containers using the Kubernetes plugin?Basic

Answer

With the Jenkins Kubernetes plugin, I define a Pod template for the build, choose containers for tools such as Maven, Docker/BuildKit, kubectl, or Helm, and run stages inside those containers. Each build gets an isolated Pod and the Pod is removed after completion.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How would you run builds in isolated containers using the Kubernetes plugin.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is a multibranch pipeline, and how does it handle branches and PRs?Basic

Answer

A multibranch pipeline automatically discovers branches and pull requests, reads the Jenkinsfile from each branch, and creates branch-specific jobs. This lets feature branches validate their own pipeline version while main, release branches, and PRs can use different gating behavior.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is a multibranch pipeline, and how does it handle branches and PRs.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How do you trigger a Jenkins pipeline (webhook, poll SCM, cron, upstream)?Basic

Answer

A Jenkins pipeline can be triggered by Git webhooks, SCM polling, cron schedules, upstream job completion, manual starts, API calls, or multibranch indexing. I prefer webhooks for commit-driven builds because they are faster and less wasteful than polling.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How do you trigger a Jenkins pipeline (webhook, poll SCM, cron, upstream).

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is the difference between a webhook trigger and polling SCM?Basic

Answer

A webhook is event-driven: GitHub or Git sends Jenkins an HTTP event when something changes. Polling SCM is Jenkins periodically asking the repository whether anything changed. Webhooks are more immediate and scalable; polling is a fallback when webhook connectivity is not available.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is the difference between a webhook trigger and polling SCM.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How do you implement an approval/manual gate before a production deploy?Basic

Answer

I implement a production manual gate with an input step, environment approval, or a change-management integration. The gate should appear after automated tests and staging validation, be limited to authorized approvers, have a timeout, and record who approved the production deployment.

Technical explanation

A gate should be placed after objective validation, not used as a substitute for testing.

Approval metadata should be retained with the pipeline run, change request, and deployment audit trail.

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How do you implement an approval/manual gate before a production deploy.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Add a production gate: timeout(time: 30, unit: 'MINUTES') { input message: 'Deploy to prod?', ok: 'Deploy', submitter: 'sre-leads,release-managers' }; record the submitter in the deployment metadata.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How would you fail a build when a security scan finds a critical vulnerability?Basic

Answer

I fail the build by defining an explicit vulnerability threshold, such as no critical findings in runtime dependencies or container images. The scanner should emit machine-readable results, the pipeline should evaluate severity and exceptions, and the build should stop before artifact promotion or deployment.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How would you fail a build when a security scan finds a critical vulnerability.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How do you cache dependencies to speed up Jenkins builds?Basic

Answer

I cache dependencies by separating dependency download from source compilation and using a cache keyed by lockfiles or build descriptors. In Jenkins this can be a persistent workspace, mounted cache volume, repository proxy like Nexus, or tool-specific cache such as Maven, Gradle, npm, pip, or Docker layer cache.

Technical explanation

Cache keys should include lockfiles or dependency descriptors so stale dependencies do not hide build problems.

A cache miss must not fail the build; the pipeline should still work from a clean environment.

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How do you cache dependencies to speed up Jenkins builds.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Mount or restore a dependency cache keyed by the lockfile hash, for example ~/.m2 for Maven or ~/.npm for npm, and verify a clean build still works when the cache is empty.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How would you reduce the run time of a slow pipeline (as you did at Intuit)?Basic

Answer

To reduce a slow pipeline, I first measure stage duration and queue time, then remove unnecessary serial work, parallelize independent checks, cache dependencies, use smaller test scopes for PRs, split long integration tests, and move slow non-blocking checks to scheduled or post-merge pipelines.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How would you reduce the run time of a slow pipeline (as you did at Intuit).

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is a Jenkins agent label, and how do you target specific agents?Basic

Answer

A Jenkins agent label is metadata used to select where a build runs. A pipeline can target an agent label such as linux, docker, gpu, arm64, or k8s so the job runs only on workers with the required OS, tools, network access, or hardware.

Technical explanation

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is a Jenkins agent label, and how do you target specific agents.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How do you handle build artifacts and where do you store them (Nexus, ECR)?Basic

Answer

Build artifacts should be stored in a durable artifact repository or registry, not only in a Jenkins workspace. Examples are Nexus or Artifactory for packages, ECR for container images, S3 for build outputs, and Jenkins archiveArtifacts only for short-term logs or diagnostics.

Technical explanation

Artifacts should be immutable and addressed by version or digest, with metadata linking them to commit SHA and pipeline run.

Jenkins archived artifacts are convenient for diagnostics, but production deployment should consume from a controlled artifact repository or registry.

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How do you handle build artifacts and where do you store them (Nexus, ECR).

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is the difference between archiving artifacts and publishing to a registry?Basic

Answer

Archiving artifacts keeps files attached to a Jenkins build for traceability or debugging. Publishing to a registry makes a versioned artifact available for downstream deployment or consumption. For production, the deployable artifact should be published to a controlled repository or registry.

Technical explanation

Artifacts should be immutable and addressed by version or digest, with metadata linking them to commit SHA and pipeline run.

Jenkins archived artifacts are convenient for diagnostics, but production deployment should consume from a controlled artifact repository or registry.

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is the difference between archiving artifacts and publishing to a registry.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How do you keep a pipeline idempotent and safely re-runnable?Basic

Answer

A pipeline is idempotent when rerunning it with the same inputs produces the same result without unsafe duplicate side effects. I make pipelines re-runnable by using immutable versions, detecting existing artifacts, applying declarative deployment tools, making migrations safe, and avoiding hidden mutable state.

Technical explanation

CI/CD should optimize both speed and safety: fast feedback for developers, controlled promotion for environments, and clear evidence for operations.

The deployable artifact should be built once, versioned, scanned, and promoted by immutable version or digest rather than rebuilt per environment.

Use automated gates for tests, policy, security, and health checks; use human approval only for risk decisions that automation cannot make reliably.

Track delivery health with lead time, deployment frequency, change-failure rate, MTTR, pipeline duration, queue time, and flaky failure rate.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How do you keep a pipeline idempotent and safely re-runnable.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How do you implement blue-green or canary deployment from a pipeline?Basic

Answer

A pipeline can implement blue-green or canary by deploying a new version beside the current one, validating health and metrics, then shifting traffic gradually or all at once. The pipeline should include automated checks and a rollback path before increasing exposure.

Technical explanation

Traffic shifting should be separate from artifact build so the same artifact can be promoted or rolled back.

Use readiness checks, error budgets, and automated metrics before increasing production exposure.

CI/CD should optimize both speed and safety: fast feedback for developers, controlled promotion for environments, and clear evidence for operations.

The deployable artifact should be built once, versioned, scanned, and promoted by immutable version or digest rather than rebuilt per environment.

Use automated gates for tests, policy, security, and health checks; use human approval only for risk decisions that automation cannot make reliably.

Track delivery health with lead time, deployment frequency, change-failure rate, MTTR, pipeline duration, queue time, and flaky failure rate.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How do you implement blue-green or canary deployment from a pipeline.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Deploy the new artifact beside the old version, run smoke checks, shift 5% traffic, evaluate error rate and p95 latency for 10 minutes, then either increase traffic or rollback to the previous digest.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is the difference between blue-green, rolling, and canary deployments?Basic

Answer

Rolling deployment replaces instances gradually in the same environment. Blue-green creates a full new environment and switches traffic after validation. Canary deploys to a small subset first and expands based on metrics. Canary gives the most controlled exposure, while blue-green gives fast cutover and rollback.

Technical explanation

Traffic shifting should be separate from artifact build so the same artifact can be promoted or rolled back.

Use readiness checks, error budgets, and automated metrics before increasing production exposure.

CI/CD should optimize both speed and safety: fast feedback for developers, controlled promotion for environments, and clear evidence for operations.

The deployable artifact should be built once, versioned, scanned, and promoted by immutable version or digest rather than rebuilt per environment.

Use automated gates for tests, policy, security, and health checks; use human approval only for risk decisions that automation cannot make reliably.

Track delivery health with lead time, deployment frequency, change-failure rate, MTTR, pipeline duration, queue time, and flaky failure rate.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What is the difference between blue-green, rolling, and canary deployments.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Deploy the new artifact beside the old version, run smoke checks, shift 5% traffic, evaluate error rate and p95 latency for 10 minutes, then either increase traffic or rollback to the previous digest.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

How would you implement automatic rollback if a deployment is unhealthy?Basic

Answer

Automatic rollback should be driven by health checks, deployment status, and runtime metrics. The pipeline or rollout controller watches readiness, errors, latency, saturation, and business signals; if thresholds fail, it reverts traffic or rolls back the Deployment, Helm release, or GitOps commit.

Technical explanation

CI/CD should optimize both speed and safety: fast feedback for developers, controlled promotion for environments, and clear evidence for operations.

The deployable artifact should be built once, versioned, scanned, and promoted by immutable version or digest rather than rebuilt per environment.

Use automated gates for tests, policy, security, and health checks; use human approval only for risk decisions that automation cannot make reliably.

Track delivery health with lead time, deployment frequency, change-failure rate, MTTR, pipeline duration, queue time, and flaky failure rate.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: How would you implement automatic rollback if a deployment is unhealthy.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Deploy the new artifact beside the old version, run smoke checks, shift 5% traffic, evaluate error rate and p95 latency for 10 minutes, then either increase traffic or rollback to the previous digest.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What metrics define a healthy CI/CD system (lead time, deployment frequency, change-fail rate, MTTR)?Basic

Answer

Healthy CI/CD is measured by both flow and reliability. Key metrics include lead time for changes, deployment frequency, change-failure rate, MTTR, pipeline duration, queue time, flaky test rate, rollback frequency, and percentage of deployments that are fully automated and traceable.

Technical explanation

CI/CD should optimize both speed and safety: fast feedback for developers, controlled promotion for environments, and clear evidence for operations.

The deployable artifact should be built once, versioned, scanned, and promoted by immutable version or digest rather than rebuilt per environment.

Use automated gates for tests, policy, security, and health checks; use human approval only for risk decisions that automation cannot make reliably.

Track delivery health with lead time, deployment frequency, change-failure rate, MTTR, pipeline duration, queue time, and flaky failure rate.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What metrics define a healthy CI/CD system (lead time, deployment frequency, change-fail rate, MTTR).

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What are the DORA metrics, and how have you used them?Basic

Answer

DORA metrics are deployment frequency, lead time for changes, change-failure rate, and time to restore service. I use them to see whether delivery speed is improving without sacrificing reliability, then connect pipeline improvements to real outcomes like faster feedback and lower rollback or incident rates.

Technical explanation

DORA metrics are outcome metrics; they should be interpreted with service context rather than used as vanity numbers.

Improvement work should connect pipeline bottlenecks to business impact, such as faster recovery and smaller change batches.

CI/CD should optimize both speed and safety: fast feedback for developers, controlled promotion for environments, and clear evidence for operations.

The deployable artifact should be built once, versioned, scanned, and promoted by immutable version or digest rather than rebuilt per environment.

Use automated gates for tests, policy, security, and health checks; use human approval only for risk decisions that automation cannot make reliably.

Track delivery health with lead time, deployment frequency, change-failure rate, MTTR, pipeline duration, queue time, and flaky failure rate.

Hands-on example

1. Create or update a Jenkinsfile for the scenario: What are the DORA metrics, and how have you used them.

2. Use a Declarative Pipeline skeleton: pipeline { agent { label 'linux && docker' } options { timestamps(); disableConcurrentBuilds() } stages { stage('Checkout') { steps { checkout scm } } stage('Test') { parallel { stage('Unit') { steps { sh 'make unit' } } stage('Lint') { steps { sh 'make lint' } } } } } post { always { junit 'reports/*.xml'; cleanWs() } failure { echo 'notify team' } } }.

3. Inject secrets only in the narrowest stage, for example withCredentials([string(credentialsId: 'scanner-token', variable: 'TOKEN')]) { sh 'scanner --fail-on critical' }; do not echo TOKEN or write it into archived artifacts.

4. Publish the immutable result: tag the image with the Git SHA, push to ECR/Nexus, archive test reports, and record build URL, commit SHA, artifact digest, approver, and deployment status.

5. Prove the design by rerunning the same commit twice: the second run should reuse safe caches, produce the same artifact version or detect it already exists, and avoid duplicate side effects.

What is GitOps, and what are its core principles?Basic

Answer

GitOps is an operating model where the desired state of systems is stored declaratively in Git and an agent continuously reconciles the runtime environment to match Git. The core ideas are declarative configuration, versioned immutable history, pull-based automation, and continuous reconciliation.

Technical explanation

The desired state should be declarative enough for an agent to converge without manual runbook steps.

Git history becomes the audit trail for who requested, reviewed, and approved an environment change.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: What is GitOps, and what are its core principles in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

How does GitOps differ from a traditional push-based CI/CD deploy?Basic

Answer

Traditional CI/CD often pushes changes directly from a pipeline into the cluster. GitOps makes the pipeline update Git, and a cluster-side controller pulls from Git and applies the desired state. That reduces deployment credentials in CI and gives a cleaner audit trail.

Technical explanation

The desired state should be declarative enough for an agent to converge without manual runbook steps.

Git history becomes the audit trail for who requested, reviewed, and approved an environment change.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: How does GitOps differ from a traditional push-based CI/CD deploy in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

What does it mean that Git is the single source of truth in GitOps?Basic

Answer

Git as the single source of truth means the intended application and environment state should be visible in Git. If the cluster differs from Git, the cluster is considered drifted. Changes should be made through commits and pull requests, not through untracked manual kubectl changes.

Technical explanation

The desired state should be declarative enough for an agent to converge without manual runbook steps.

Git history becomes the audit trail for who requested, reviewed, and approved an environment change.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: What does it mean that Git is the single source of truth in GitOps in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

What is ArgoCD, and how does it implement GitOps?Intermediate

Answer

Argo CD is a Kubernetes GitOps controller. It watches a Git repository or Helm/Kustomize source, compares the desired manifests with live cluster state, reports sync and health status, and can manually or automatically reconcile the cluster back to the declared state.

Technical explanation

The desired state should be declarative enough for an agent to converge without manual runbook steps.

Git history becomes the audit trail for who requested, reviewed, and approved an environment change.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: What is ArgoCD, and how does it implement GitOps in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

How does ArgoCD detect and reconcile drift between Git and the cluster?Intermediate

Answer

Argo CD detects drift by rendering the configured source, reading live Kubernetes objects, comparing desired versus actual state, and marking the application OutOfSync when they differ. With automated sync or self-heal enabled, it can apply Git state back to the cluster.

Technical explanation

Drift can be intentional during emergency response, but it should be time-bound and reconciled back into Git.

Self-heal is powerful, but it can fight other controllers if ownership boundaries are unclear.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: How does ArgoCD detect and reconcile drift between Git and the cluster in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

What is an ArgoCD Application, and what does it define?Intermediate

Answer

An Argo CD Application defines what to deploy, from where, and to which cluster/namespace. It includes the source repository, revision, path or chart, destination cluster and namespace, project, sync policy, and optional Helm or Kustomize settings.

Technical explanation

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: What is an ArgoCD Application, and what does it define in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

What is the difference between automated and manual sync in ArgoCD?Intermediate

Answer

Manual sync means Argo CD shows drift but waits for a user or automation to press sync. Automated sync means Argo CD applies changes when Git changes or when live drift is detected, depending on options such as prune and selfHeal.

Technical explanation

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: What is the difference between automated and manual sync in ArgoCD in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

What are ArgoCD sync waves and sync hooks?Intermediate

Answer

Sync waves and hooks control ordering during Argo CD sync. Hooks run resources at phases such as PreSync, Sync, PostSync, or SyncFail. Waves let resources apply in numeric order, which is useful for CRDs, namespaces, operators, migrations, and dependent workloads.

Technical explanation

Hooks are useful for one-off operations such as migrations, but they must be idempotent and have cleanup policies.

Waves order resources, but health and readiness still matter for safe dependent rollout.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: What are ArgoCD sync waves and sync hooks in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Annotate dependent resources, for example argocd.argoproj.io/sync-wave: '-1' on CRDs and PreSync on a migration Job; verify the sync order in Argo CD operation details.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

What is the app-of-apps pattern in ArgoCD?Intermediate

Answer

The app-of-apps pattern uses one parent Argo CD Application to create and manage multiple child Applications. It is useful for bootstrapping a cluster or platform because one root app can define all environment, infrastructure, and service apps.

Technical explanation

The parent app controls child Application objects, which can bootstrap an entire environment from one root Git path.

Avoid making the root app too broad without RBAC and project boundaries, because mistakes can affect many services.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: What is the app-of-apps pattern in ArgoCD in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

How does ArgoCD determine the health status of resources?Intermediate

Answer

Argo CD determines health using built-in health checks for common Kubernetes resources and custom health logic for other kinds. Sync status answers whether live state matches Git; health answers whether the live resources are operationally ready.

Technical explanation

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: How does ArgoCD determine the health status of resources in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

What is a self-heal in ArgoCD, and when would you disable it?Intermediate

Answer

Self-heal means Argo CD automatically corrects live changes that drift from Git. I would disable or limit it when debugging, during emergency manual intervention, when another controller intentionally mutates fields, or when automatic correction could cause operational risk.

Technical explanation

Drift can be intentional during emergency response, but it should be time-bound and reconciled back into Git.

Self-heal is powerful, but it can fight other controllers if ownership boundaries are unclear.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: What is a self-heal in ArgoCD, and when would you disable it in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

How does ArgoCD handle secrets, and why is it tricky in GitOps?Intermediate

Answer

Secrets are tricky in GitOps because Git wants declarative state, but plain Kubernetes Secret YAML should not contain raw secrets. Common patterns are SOPS-encrypted secrets, Sealed Secrets, External Secrets Operator, cloud secret managers, or runtime injection from a vault system.

Technical explanation

Masking is a safety net, not a complete control; scripts can still leak secrets through files, command arguments, debug output, or third-party tools.

Prefer scoped, short-lived credentials and store only the minimum secrets needed for that job or environment.

Artifacts should be immutable and addressed by version or digest, with metadata linking them to commit SHA and pipeline run.

Jenkins archived artifacts are convenient for diagnostics, but production deployment should consume from a controlled artifact repository or registry.

The desired state should be declarative enough for an agent to converge without manual runbook steps.

Git history becomes the audit trail for who requested, reviewed, and approved an environment change.

Hands-on example

1. Model the desired state for: How does ArgoCD handle secrets, and why is it tricky in GitOps in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Store only encrypted or external-secret references in Git, for example SOPS-encrypted YAML or an ExternalSecret pointing to AWS Secrets Manager; never commit raw secret values.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

How would you manage multiple environments (dev/staging/prod) in ArgoCD?Intermediate

Answer

I manage multiple environments by separating environment-specific values and promotion controls while keeping common templates reusable. Typical approaches are folders per environment, overlays with Kustomize, Helm values per environment, separate Argo CD Applications, and tighter approval policies for production.

Technical explanation

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: How would you manage multiple environments (dev/staging/prod) in ArgoCD in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Keep common manifests in base/ and environment differences in overlays or values-dev.yaml, values-staging.yaml, and values-prod.yaml; production changes require a separate PR and approval.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

What is the difference between ArgoCD and Flux?Intermediate

Answer

Argo CD and Flux are both Kubernetes GitOps tools. Argo CD is well known for its UI, Application model, RBAC, and operational visibility. Flux is toolkit-oriented, controller-native, and highly composable. The best choice depends on team workflow, governance model, and platform standards.

Technical explanation

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: What is the difference between ArgoCD and Flux in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

How does ArgoCD integrate with Helm and Kustomize?Intermediate

Answer

Argo CD can render plain YAML, Kustomize overlays, Helm charts, Jsonnet, and plugin-based sources. For Helm it can render a chart with values; for Kustomize it can build overlays. Argo CD then compares the rendered manifests to live cluster state.

Technical explanation

Argo CD renders the source first, then compares the rendered result to live objects.

Keep environment-specific differences in values or overlays rather than forking templates unnecessarily.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: How does ArgoCD integrate with Helm and Kustomize in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

How do you roll back a bad change in a GitOps workflow?Intermediate

Answer

In GitOps, rollback is normally a Git operation: revert the bad commit, restore a previous tag or commit, or update the desired version back to a known-good artifact. Argo CD then reconciles the cluster to that corrected desired state.

Technical explanation

The desired state should be declarative enough for an agent to converge without manual runbook steps.

Git history becomes the audit trail for who requested, reviewed, and approved an environment change.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: How do you roll back a bad change in a GitOps workflow in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

How do you prevent configuration drift from manual kubectl changes with ArgoCD?Intermediate

Answer

To prevent manual drift, I use Argo CD auto-sync with self-heal where appropriate, RBAC to restrict direct kubectl access, admission policies to block unsafe changes, alerts for OutOfSync apps, and a team rule that production changes go through pull requests.

Technical explanation

Drift can be intentional during emergency response, but it should be time-bound and reconciled back into Git.

Self-heal is powerful, but it can fight other controllers if ownership boundaries are unclear.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: How do you prevent configuration drift from manual kubectl changes with ArgoCD in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

What are the advantages of pull-based deployment for security?Intermediate

Answer

Pull-based deployment is safer because CI does not need broad cluster credentials or inbound access to the cluster. The cluster-side agent pulls approved desired state, which reduces credential blast radius, improves auditability, and works better for private clusters behind firewalls.

Technical explanation

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: What are the advantages of pull-based deployment for security in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

How do you structure Git repositories for GitOps (mono vs multi, app vs config repo)?Intermediate

Answer

GitOps repository structure can be mono-repo, multi-repo, app repo plus config repo, or environment repo. I choose based on ownership boundaries, promotion workflow, blast radius, audit needs, and how often application code and deployment configuration change together.

Technical explanation

The desired state should be declarative enough for an agent to converge without manual runbook steps.

Git history becomes the audit trail for who requested, reviewed, and approved an environment change.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: How do you structure Git repositories for GitOps (mono vs multi, app vs config repo) in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Create an Argo CD Application that points to repoURL, targetRevision, path or chart, destination server, namespace, and project; render with Helm/Kustomize before merging.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

How would you promote a change from staging to production in GitOps?Intermediate

Answer

Promotion from staging to production should be a controlled Git change. The pipeline promotes the same artifact digest or version, updates the production manifest or values file, opens a pull request, runs policy checks, gets approval, and lets Argo CD sync production.

Technical explanation

The desired state should be declarative enough for an agent to converge without manual runbook steps.

Git history becomes the audit trail for who requested, reviewed, and approved an environment change.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Model the desired state for: How would you promote a change from staging to production in GitOps in a GitOps repository, for example environments/staging/apps/payments and environments/prod/apps/payments.

2. Keep common manifests in base/ and environment differences in overlays or values-dev.yaml, values-staging.yaml, and values-prod.yaml; production changes require a separate PR and approval.

3. Open a pull request that changes only the desired version or values, require review and policy checks, then merge to let Argo CD detect OutOfSync state.

4. Run argocd app get payments and argocd app diff payments, then sync manually or let automated sync reconcile; verify sync status, health status, events, and Kubernetes rollout status.

5. Test rollback by reverting the Git commit or promoting the previous artifact digest, then watch Argo CD reconcile the cluster back to the known-good desired state.

What is GitHub Enterprise, and how does it differ from github.com?Intermediate

Answer

GitHub Enterprise is GitHub's enterprise offering, available as GitHub Enterprise Cloud or Server, with organization controls, SSO/SAML, audit logs, enterprise policies, advanced security features, private networking options, and compliance controls that go beyond a normal github.com personal workflow.

Technical explanation

GitHub delivery controls combine repository settings, branch protection, required status checks, environments, CODEOWNERS, and workflow permissions.

Use least-privilege permissions for the GITHUB_TOKEN and prefer OIDC federation over long-lived cloud access keys.

Separate trusted and untrusted workflow contexts, especially pull_request from forks, and avoid exposing secrets to unreviewed code.

Make merge policy explicit: linear history, squash, merge commits, signed commits, or code-owner approvals should match audit and release requirements.

Hands-on example

1. Implement the control for: What is GitHub Enterprise, and how does it differ from github.com in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Add branch protection on main requiring the CI workflow, at least one approval, CODEOWNERS review for protected paths, conversation resolution, and no direct pushes.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

What is the difference between GitHub Actions and Jenkins?Intermediate

Answer

GitHub Actions is tightly integrated into GitHub with YAML workflows, hosted or self-hosted runners, and repository-native events. Jenkins is a standalone automation server with a large plugin ecosystem and deep customizability. Actions is often simpler for GitHub-centric workflows; Jenkins can fit complex legacy or cross-tool environments.

Technical explanation

Workflow files live under .github/workflows and are triggered by events such as push, pull_request, workflow_dispatch, schedule, or release.

Runners should be ephemeral or carefully isolated because workflow code can execute arbitrary commands.

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Implement the control for: What is the difference between GitHub Actions and Jenkins in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Add branch protection on main requiring the CI workflow, at least one approval, CODEOWNERS review for protected paths, conversation resolution, and no direct pushes.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

What is a GitHub Actions workflow, job, step, and runner?Intermediate

Answer

A GitHub Actions workflow is a YAML automation file triggered by events. A job is a group of steps running on a runner. A step is an individual shell command or action. A runner is the machine or container executing the job.

Technical explanation

Workflow files live under .github/workflows and are triggered by events such as push, pull_request, workflow_dispatch, schedule, or release.

Runners should be ephemeral or carefully isolated because workflow code can execute arbitrary commands.

GitHub delivery controls combine repository settings, branch protection, required status checks, environments, CODEOWNERS, and workflow permissions.

Use least-privilege permissions for the GITHUB_TOKEN and prefer OIDC federation over long-lived cloud access keys.

Separate trusted and untrusted workflow contexts, especially pull_request from forks, and avoid exposing secrets to unreviewed code.

Make merge policy explicit: linear history, squash, merge commits, signed commits, or code-owner approvals should match audit and release requirements.

Hands-on example

1. Implement the control for: What is a GitHub Actions workflow, job, step, and runner in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Run the same workflow once on ubuntu-latest and once on a hardened self-hosted runner; compare network access, installed tools, cleanup behavior, and patch ownership.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

What is the difference between a GitHub-hosted and a self-hosted runner?Intermediate

Answer

GitHub-hosted runners are managed by GitHub, ephemeral, and easy to use. Self-hosted runners are managed by the organization and can access private networks or custom tools, but they require patching, scaling, isolation, and hardening.

Technical explanation

Workflow files live under .github/workflows and are triggered by events such as push, pull_request, workflow_dispatch, schedule, or release.

Runners should be ephemeral or carefully isolated because workflow code can execute arbitrary commands.

GitHub delivery controls combine repository settings, branch protection, required status checks, environments, CODEOWNERS, and workflow permissions.

Use least-privilege permissions for the GITHUB_TOKEN and prefer OIDC federation over long-lived cloud access keys.

Separate trusted and untrusted workflow contexts, especially pull_request from forks, and avoid exposing secrets to unreviewed code.

Make merge policy explicit: linear history, squash, merge commits, signed commits, or code-owner approvals should match audit and release requirements.

Hands-on example

1. Implement the control for: What is the difference between a GitHub-hosted and a self-hosted runner in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Run the same workflow once on ubuntu-latest and once on a hardened self-hosted runner; compare network access, installed tools, cleanup behavior, and patch ownership.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

How do you store and use secrets in GitHub Actions?Intermediate

Answer

GitHub Actions secrets are stored at repository, environment, or organization scope and injected into workflows through the secrets context. I scope secrets narrowly, prefer OIDC short-lived cloud credentials, avoid printing secrets, and combine secrets with environment protection rules for production.

Technical explanation

Masking is a safety net, not a complete control; scripts can still leak secrets through files, command arguments, debug output, or third-party tools.

Prefer scoped, short-lived credentials and store only the minimum secrets needed for that job or environment.

Artifacts should be immutable and addressed by version or digest, with metadata linking them to commit SHA and pipeline run.

Jenkins archived artifacts are convenient for diagnostics, but production deployment should consume from a controlled artifact repository or registry.

Workflow files live under .github/workflows and are triggered by events such as push, pull_request, workflow_dispatch, schedule, or release.

Runners should be ephemeral or carefully isolated because workflow code can execute arbitrary commands.

Hands-on example

1. Implement the control for: How do you store and use secrets in GitHub Actions in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Move cloud deployment from static secrets to OIDC: grant id-token: write to the job, configure cloud trust on repository/ref/environment claims, and remove long-lived keys from repository secrets.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

What is a branch protection rule, and what would you enforce on main?Intermediate

Answer

A branch protection rule defines requirements before code can merge into a protected branch such as main. I would enforce pull requests, required reviews, CODEOWNERS where needed, required status checks, no direct pushes, signed commits if required, and up-to-date branches for critical repos.

Technical explanation

Branch protection is effective only when admins cannot bypass it casually and required checks are stable.

CODEOWNERS review works best with clear ownership boundaries and small, maintainable path patterns.

GitHub delivery controls combine repository settings, branch protection, required status checks, environments, CODEOWNERS, and workflow permissions.

Use least-privilege permissions for the GITHUB_TOKEN and prefer OIDC federation over long-lived cloud access keys.

Separate trusted and untrusted workflow contexts, especially pull_request from forks, and avoid exposing secrets to unreviewed code.

Make merge policy explicit: linear history, squash, merge commits, signed commits, or code-owner approvals should match audit and release requirements.

Hands-on example

1. Implement the control for: What is a branch protection rule, and what would you enforce on main in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Add branch protection on main requiring the CI workflow, at least one approval, CODEOWNERS review for protected paths, conversation resolution, and no direct pushes.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

What is the difference between merge commit, squash, and rebase merging?Intermediate

Answer

A merge commit preserves full branch history with a merge node. Squash combines the PR into one commit for a clean main history. Rebase replays commits on top of the target branch for linear history. I choose based on audit needs and team history policy.

Technical explanation

Use commands that preserve team auditability on shared branches; rewrite only local or explicitly coordinated history.

Always run the relevant test suite after conflict resolution or cherry-picking because code may compile but behavior can change.

GitHub delivery controls combine repository settings, branch protection, required status checks, environments, CODEOWNERS, and workflow permissions.

Use least-privilege permissions for the GITHUB_TOKEN and prefer OIDC federation over long-lived cloud access keys.

Separate trusted and untrusted workflow contexts, especially pull_request from forks, and avoid exposing secrets to unreviewed code.

Make merge policy explicit: linear history, squash, merge commits, signed commits, or code-owner approvals should match audit and release requirements.

Hands-on example

1. Implement the control for: What is the difference between merge commit, squash, and rebase merging in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Add branch protection on main requiring the CI workflow, at least one approval, CODEOWNERS review for protected paths, conversation resolution, and no direct pushes.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

What is a pull request review workflow, and what gates would you require?Intermediate

Answer

A pull request review workflow is the controlled path for changing shared code. I require automated tests, security scans, required reviewers, CODEOWNERS for sensitive areas, conversation resolution, branch protection, and clear approval before merging to main.

Technical explanation

A gate should be placed after objective validation, not used as a substitute for testing.

Approval metadata should be retained with the pipeline run, change request, and deployment audit trail.

Branch protection is effective only when admins cannot bypass it casually and required checks are stable.

CODEOWNERS review works best with clear ownership boundaries and small, maintainable path patterns.

GitHub delivery controls combine repository settings, branch protection, required status checks, environments, CODEOWNERS, and workflow permissions.

Use least-privilege permissions for the GITHUB_TOKEN and prefer OIDC federation over long-lived cloud access keys.

Hands-on example

1. Implement the control for: What is a pull request review workflow, and what gates would you require in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Add branch protection on main requiring the CI workflow, at least one approval, CODEOWNERS review for protected paths, conversation resolution, and no direct pushes.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

What are required status checks, and how do they protect main?Intermediate

Answer

Required status checks are CI checks that must pass before a pull request can merge. They protect main by ensuring builds, tests, scans, policy checks, and deployment previews pass consistently before the branch accepts new code.

Technical explanation

Branch protection is effective only when admins cannot bypass it casually and required checks are stable.

CODEOWNERS review works best with clear ownership boundaries and small, maintainable path patterns.

GitHub delivery controls combine repository settings, branch protection, required status checks, environments, CODEOWNERS, and workflow permissions.

Use least-privilege permissions for the GITHUB_TOKEN and prefer OIDC federation over long-lived cloud access keys.

Separate trusted and untrusted workflow contexts, especially pull_request from forks, and avoid exposing secrets to unreviewed code.

Make merge policy explicit: linear history, squash, merge commits, signed commits, or code-owner approvals should match audit and release requirements.

Hands-on example

1. Implement the control for: What are required status checks, and how do they protect main in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Add branch protection on main requiring the CI workflow, at least one approval, CODEOWNERS review for protected paths, conversation resolution, and no direct pushes.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

Explain a typical Git branching strategy (trunk-based, Git flow, GitHub flow).Intermediate

Answer

Trunk-based development keeps changes small and frequently merged to main. GitHub Flow uses short-lived branches and PRs into main, often with deployment from main. Git Flow uses long-lived develop, release, and hotfix branches. I prefer trunk-based or GitHub Flow for faster CI/CD.

Technical explanation

GitHub delivery controls combine repository settings, branch protection, required status checks, environments, CODEOWNERS, and workflow permissions.

Use least-privilege permissions for the GITHUB_TOKEN and prefer OIDC federation over long-lived cloud access keys.

Separate trusted and untrusted workflow contexts, especially pull_request from forks, and avoid exposing secrets to unreviewed code.

Make merge policy explicit: linear history, squash, merge commits, signed commits, or code-owner approvals should match audit and release requirements.

Hands-on example

1. Implement the control for: Explain a typical Git branching strategy (trunk-based, Git flow, GitHub flow). in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Add branch protection on main requiring the CI workflow, at least one approval, CODEOWNERS review for protected paths, conversation resolution, and no direct pushes.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

What is the difference between trunk-based development and Git flow?Intermediate

Answer

Trunk-based development minimizes long-lived branches and integrates continuously into main, usually with feature flags. Git Flow separates development, release, and hotfix branches, which can suit scheduled releases but often slows feedback and increases merge complexity.

Technical explanation

GitHub delivery controls combine repository settings, branch protection, required status checks, environments, CODEOWNERS, and workflow permissions.

Use least-privilege permissions for the GITHUB_TOKEN and prefer OIDC federation over long-lived cloud access keys.

Separate trusted and untrusted workflow contexts, especially pull_request from forks, and avoid exposing secrets to unreviewed code.

Make merge policy explicit: linear history, squash, merge commits, signed commits, or code-owner approvals should match audit and release requirements.

Hands-on example

1. Implement the control for: What is the difference between trunk-based development and Git flow in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Add branch protection on main requiring the CI workflow, at least one approval, CODEOWNERS review for protected paths, conversation resolution, and no direct pushes.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

What is a CODEOWNERS file, and how does it enforce review ownership?Intermediate

Answer

A CODEOWNERS file maps repository paths to responsible reviewers. When branch protection requires code owner review, changes to owned paths cannot merge until an owner approves, which enforces domain ownership for platform, security, infrastructure, or critical service files.

Technical explanation

Branch protection is effective only when admins cannot bypass it casually and required checks are stable.

CODEOWNERS review works best with clear ownership boundaries and small, maintainable path patterns.

GitHub delivery controls combine repository settings, branch protection, required status checks, environments, CODEOWNERS, and workflow permissions.

Use least-privilege permissions for the GITHUB_TOKEN and prefer OIDC federation over long-lived cloud access keys.

Separate trusted and untrusted workflow contexts, especially pull_request from forks, and avoid exposing secrets to unreviewed code.

Make merge policy explicit: linear history, squash, merge commits, signed commits, or code-owner approvals should match audit and release requirements.

Hands-on example

1. Implement the control for: What is a CODEOWNERS file, and how does it enforce review ownership in a GitHub repository that contains a simple service and .github/workflows/ci.yml.

2. Create a workflow with on: [pull_request], jobs: build, test, scan; set permissions: contents: read by default and grant write only to jobs that truly need it.

3. Add CODEOWNERS entries such as /terraform/ @platform-team and /.github/workflows/ @devops-team, then enable required code-owner review in branch protection.

4. Use environments for staging/prod with required reviewers and environment secrets; prefer OIDC cloud login over storing AWS/Azure/GCP access keys.

5. Validate by opening a PR that fails one required check and confirm GitHub blocks merge until the check passes and required reviewers approve.

How do you sign commits, and why might that be required?Intermediate

Answer

Commit signing cryptographically proves that a commit or tag was created by a trusted identity. It can be done with GPG, SSH signing, or S/MIME. Teams require it to improve provenance, reduce impersonation risk, and strengthen audit trails.

Technical explanation

Security gates should be policy-driven with documented exception handling; otherwise teams will bypass noisy checks.

Provenance should connect repository, commit, workflow identity, artifact digest, signer identity, and deployment environment.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: How do you sign commits, and why might that be required in a throwaway repository with main, feature, release, and hotfix branches.

2. Create commits on multiple branches, then run the relevant commands: git merge, git rebase main, git cherry-pick <sha>, git revert <sha>, git tag -a v1.2.3, or git reset --hard only on a private branch.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

What is the difference between git merge and git rebase, and when do you use each?Intermediate

Answer

git merge combines histories and preserves the branch relationship. git rebase rewrites local commit history by replaying commits on a new base. I use merge for shared branch integration and rebase for cleaning my own local branch before opening or updating a PR.

Technical explanation

Use commands that preserve team auditability on shared branches; rewrite only local or explicitly coordinated history.

Always run the relevant test suite after conflict resolution or cherry-picking because code may compile but behavior can change.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: What is the difference between git merge and git rebase, and when do you use each in a throwaway repository with main, feature, release, and hotfix branches.

2. Create commits on multiple branches, then run the relevant commands: git merge, git rebase main, git cherry-pick <sha>, git revert <sha>, git tag -a v1.2.3, or git reset --hard only on a private branch.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

What is a fast-forward merge?Intermediate

Answer

A fast-forward merge happens when the target branch has no new commits since the feature branch was created, so Git can just move the target pointer forward. It creates a linear history and no merge commit.

Technical explanation

Use commands that preserve team auditability on shared branches; rewrite only local or explicitly coordinated history.

Always run the relevant test suite after conflict resolution or cherry-picking because code may compile but behavior can change.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: What is a fast-forward merge in a throwaway repository with main, feature, release, and hotfix branches.

2. Create commits on multiple branches, then run the relevant commands: git merge, git rebase main, git cherry-pick <sha>, git revert <sha>, git tag -a v1.2.3, or git reset --hard only on a private branch.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

How do you resolve a merge conflict, and how do you avoid them?Intermediate

Answer

To resolve a merge conflict, I identify conflicting files, edit them to the intended final content, run tests, stage the resolved files, and continue the merge or rebase. To avoid conflicts, I keep branches short-lived, pull frequently, and split large changes.

Technical explanation

Use commands that preserve team auditability on shared branches; rewrite only local or explicitly coordinated history.

Always run the relevant test suite after conflict resolution or cherry-picking because code may compile but behavior can change.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: How do you resolve a merge conflict, and how do you avoid them in a throwaway repository with main, feature, release, and hotfix branches.

2. Create commits on multiple branches, then run the relevant commands: git merge, git rebase main, git cherry-pick <sha>, git revert <sha>, git tag -a v1.2.3, or git reset --hard only on a private branch.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

What is git cherry-pick, and when is it useful?Advanced

Answer

git cherry-pick applies a specific commit from one branch onto another. It is useful for backporting a hotfix to a release branch, moving a small fix without merging unrelated work, or applying a known-good commit to a support branch.

Technical explanation

Use commands that preserve team auditability on shared branches; rewrite only local or explicitly coordinated history.

Always run the relevant test suite after conflict resolution or cherry-picking because code may compile but behavior can change.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: What is git cherry-pick, and when is it useful in a throwaway repository with main, feature, release, and hotfix branches.

2. Create commits on multiple branches, then run the relevant commands: git merge, git rebase main, git cherry-pick <sha>, git revert <sha>, git tag -a v1.2.3, or git reset --hard only on a private branch.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

What is git revert versus git reset, and which is safe on a shared branch?Advanced

Answer

git revert creates a new commit that undoes a previous commit, so it is safe on a shared branch. git reset moves branch pointers and can rewrite history, so it is dangerous on shared branches unless coordinated. For main, revert is usually the safe option.

Technical explanation

Use commands that preserve team auditability on shared branches; rewrite only local or explicitly coordinated history.

Always run the relevant test suite after conflict resolution or cherry-picking because code may compile but behavior can change.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: What is git revert versus git reset, and which is safe on a shared branch in a throwaway repository with main, feature, release, and hotfix branches.

2. Create commits on multiple branches, then run the relevant commands: git merge, git rebase main, git cherry-pick <sha>, git revert <sha>, git tag -a v1.2.3, or git reset --hard only on a private branch.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

What is a detached HEAD state?Advanced

Answer

A detached HEAD means HEAD points directly to a commit instead of a branch. You can inspect or test that commit, but new commits may be lost unless you create a branch or tag before moving away.

Technical explanation

Use commands that preserve team auditability on shared branches; rewrite only local or explicitly coordinated history.

Always run the relevant test suite after conflict resolution or cherry-picking because code may compile but behavior can change.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: What is a detached HEAD state in a throwaway repository with main, feature, release, and hotfix branches.

2. Create commits on multiple branches, then run the relevant commands: git merge, git rebase main, git cherry-pick <sha>, git revert <sha>, git tag -a v1.2.3, or git reset --hard only on a private branch.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

What is a Git tag, and how do you use tags for releases?Advanced

Answer

A Git tag is a named pointer to a commit, commonly used to mark a release. Lightweight tags are simple pointers; annotated tags include metadata and can be signed. CI can build release artifacts from tags such as v1.4.2.

Technical explanation

Use commands that preserve team auditability on shared branches; rewrite only local or explicitly coordinated history.

Always run the relevant test suite after conflict resolution or cherry-picking because code may compile but behavior can change.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: What is a Git tag, and how do you use tags for releases in a throwaway repository with main, feature, release, and hotfix branches.

2. Create commits on multiple branches, then run the relevant commands: git merge, git rebase main, git cherry-pick <sha>, git revert <sha>, git tag -a v1.2.3, or git reset --hard only on a private branch.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

What is semantic versioning, and how do you apply it to releases?Advanced

Answer

Semantic versioning uses MAJOR.MINOR.PATCH. A patch version is backward-compatible bug fix, a minor version adds backward-compatible functionality, and a major version introduces breaking changes. I tie release notes and automated versioning to commit or PR semantics.

Technical explanation

Automated versioning should be deterministic and based on agreed signals such as conventional commits or release labels.

Tag creation, changelog generation, artifact publishing, and deployment promotion should be part of the same release trail.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: What is semantic versioning, and how do you apply it to releases in a throwaway repository with main, feature, release, and hotfix branches.

2. Create commits on multiple branches, then run the relevant commands: git merge, git rebase main, git cherry-pick <sha>, git revert <sha>, git tag -a v1.2.3, or git reset --hard only on a private branch.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

How would you implement automated semantic version bumping in CI?Advanced

Answer

Automated semantic version bumping can read commit messages, PR labels, or conventional commits, determine major/minor/patch, update changelog and version files, create a tag, and publish artifacts. The pipeline must protect release branches and prevent duplicate tags.

Technical explanation

Automated versioning should be deterministic and based on agreed signals such as conventional commits or release labels.

Tag creation, changelog generation, artifact publishing, and deployment promotion should be part of the same release trail.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: How would you implement automated semantic version bumping in CI in a throwaway repository with main, feature, release, and hotfix branches.

2. Create commits on multiple branches, then run the relevant commands: git merge, git rebase main, git cherry-pick <sha>, git revert <sha>, git tag -a v1.2.3, or git reset --hard only on a private branch.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

What is a monorepo versus polyrepo, and what are the trade-offs?Advanced

Answer

A monorepo stores multiple projects in one repository, which simplifies cross-project changes and shared tooling but requires selective builds and ownership controls. A polyrepo uses separate repositories, which improves isolation but can make cross-service coordination and dependency updates harder.

Technical explanation

Affected-build logic needs a dependency graph, not only file path matching, because shared libraries can impact many services.

Owners, path filters, and cache strategy become critical as repository size grows.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: What is a monorepo versus polyrepo, and what are the trade-offs in a throwaway repository with main, feature, release, and hotfix branches.

2. Add service folders such as services/api and services/worker plus a shared lib; use git diff --name-only origin/main...HEAD to map changed paths to affected builds.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

How do you only build/test the changed parts of a monorepo?Advanced

Answer

In a monorepo, I build and test only changed parts by calculating the diff against the base branch, mapping paths to services or packages, and running affected builds plus dependency-impact checks. Tools like Bazel, Nx, Turborepo, Gradle, or custom path filters can help.

Technical explanation

Affected-build logic needs a dependency graph, not only file path matching, because shared libraries can impact many services.

Owners, path filters, and cache strategy become critical as repository size grows.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: How do you only build/test the changed parts of a monorepo in a throwaway repository with main, feature, release, and hotfix branches.

2. Add service folders such as services/api and services/worker plus a shared lib; use git diff --name-only origin/main...HEAD to map changed paths to affected builds.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

What is a webhook, and how does it connect GitHub to your CI?Advanced

Answer

A webhook is an HTTP callback sent by GitHub when an event occurs, such as push, pull_request, tag creation, or release. Jenkins or another CI system receives the event, verifies it, and starts the matching pipeline quickly without polling.

Technical explanation

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Tags, release branches, signed commits, and changelogs connect source history to released artifacts and operational traceability.

Prefer safe, reviewable operations on shared branches: revert bad changes, open PRs for backports, and avoid force-pushes unless the team explicitly coordinates them.

Hands-on example

1. Practice the Git operation for: What is a webhook, and how does it connect GitHub to your CI in a throwaway repository with main, feature, release, and hotfix branches.

2. Configure a GitHub webhook for push and pull_request events, set a shared secret, expose the CI endpoint, and verify the received payload signature before triggering the job.

3. Inspect the result with git log --oneline --graph --decorate --all and verify whether history was preserved, replayed, tagged, or safely undone.

4. Push through a pull request with required CI checks, run tests after conflict resolution or cherry-pick, and document why that operation was chosen.

5. For release scenarios, build from a signed tag, publish the artifact with the commit SHA and version, and confirm the release notes match the merged changes.

How do you secure a CI/CD pipeline against supply-chain attacks?Advanced

Answer

I secure CI/CD against supply-chain attacks with least-privilege tokens, protected branches, pinned dependencies and actions, isolated runners, secret scanning, SAST/SCA/container scanning, SBOM generation, artifact signing, provenance, and promotion of immutable artifacts by digest.

Technical explanation

Security gates should be policy-driven with documented exception handling; otherwise teams will bypass noisy checks.

Provenance should connect repository, commit, workflow identity, artifact digest, signer identity, and deployment environment.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How do you secure a CI/CD pipeline against supply-chain attacks using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

What is an SBOM, and how would you generate and gate on one in CI?Advanced

Answer

An SBOM is a software bill of materials: an inventory of components, packages, versions, and sometimes licenses and hashes. In CI I generate it during the build, attach it to the artifact, scan it for vulnerabilities and policy issues, and use it for incident response.

Technical explanation

A gate should be placed after objective validation, not used as a substitute for testing.

Approval metadata should be retained with the pipeline run, change request, and deployment audit trail.

Security gates should be policy-driven with documented exception handling; otherwise teams will bypass noisy checks.

Provenance should connect repository, commit, workflow identity, artifact digest, signer identity, and deployment environment.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

Hands-on example

1. Design an advanced delivery exercise for: What is an SBOM, and how would you generate and gate on one in CI using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you sign and verify build artifacts or container images?Advanced

Answer

I sign artifacts or container images with a tool such as cosign, attach provenance or attestations, and verify signatures before deployment. Verification should check the expected identity, issuer, digest, and policy so only trusted build outputs reach production.

Technical explanation

Artifacts should be immutable and addressed by version or digest, with metadata linking them to commit SHA and pipeline run.

Jenkins archived artifacts are convenient for diagnostics, but production deployment should consume from a controlled artifact repository or registry.

Security gates should be policy-driven with documented exception handling; otherwise teams will bypass noisy checks.

Provenance should connect repository, commit, workflow identity, artifact digest, signer identity, and deployment environment.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

Hands-on example

1. Design an advanced delivery exercise for: How do you sign and verify build artifacts or container images using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

What is artifact promotion, and why not rebuild between environments?Advanced

Answer

Artifact promotion means the exact same built artifact moves from dev to staging to production. I do not rebuild between environments because rebuilds can introduce different dependencies, timestamps, base images, or compiler output, breaking reproducibility and weakening auditability.

Technical explanation

Artifacts should be immutable and addressed by version or digest, with metadata linking them to commit SHA and pipeline run.

Jenkins archived artifacts are convenient for diagnostics, but production deployment should consume from a controlled artifact repository or registry.

Security gates should be policy-driven with documented exception handling; otherwise teams will bypass noisy checks.

Provenance should connect repository, commit, workflow identity, artifact digest, signer identity, and deployment environment.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

Hands-on example

1. Design an advanced delivery exercise for: What is artifact promotion, and why not rebuild between environments using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

Why should the same artifact move through environments rather than rebuilding per stage?Advanced

Answer

The same artifact should move through environments so every test result applies to the exact artifact deployed to production. Rebuilding per stage means production may run code that was never tested, especially if dependencies, base images, or build tools changed.

Technical explanation

Artifacts should be immutable and addressed by version or digest, with metadata linking them to commit SHA and pipeline run.

Jenkins archived artifacts are convenient for diagnostics, but production deployment should consume from a controlled artifact repository or registry.

Security gates should be policy-driven with documented exception handling; otherwise teams will bypass noisy checks.

Provenance should connect repository, commit, workflow identity, artifact digest, signer identity, and deployment environment.

Git history is both collaboration state and audit evidence, so choose merge, rebase, revert, and reset based on whether history is shared.

Short-lived branches reduce merge conflicts and make CI feedback meaningful; long-lived branches increase drift and integration risk.

Hands-on example

1. Design an advanced delivery exercise for: Why should the same artifact move through environments rather than rebuilding per stage using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you implement secrets scanning to prevent credentials in commits?Advanced

Answer

Secrets scanning detects credentials before they land in Git history or before they reach main. I use pre-commit hooks, server-side scanning, GitHub secret scanning, CI checks, and automated revocation or alerting for exposed tokens.

Technical explanation

Masking is a safety net, not a complete control; scripts can still leak secrets through files, command arguments, debug output, or third-party tools.

Prefer scoped, short-lived credentials and store only the minimum secrets needed for that job or environment.

Artifacts should be immutable and addressed by version or digest, with metadata linking them to commit SHA and pipeline run.

Jenkins archived artifacts are convenient for diagnostics, but production deployment should consume from a controlled artifact repository or registry.

Security gates should be policy-driven with documented exception handling; otherwise teams will bypass noisy checks.

Provenance should connect repository, commit, workflow identity, artifact digest, signer identity, and deployment environment.

Hands-on example

1. Design an advanced delivery exercise for: How do you implement secrets scanning to prevent credentials in commits using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How would you integrate SAST and dependency scanning into a pipeline?Advanced

Answer

I integrate SAST and dependency scanning as early PR gates and deeper scheduled scans. SAST checks source-code patterns; dependency scanning checks known vulnerable packages and licenses. Results should be severity-thresholded, deduplicated, and tied to remediation ownership.

Technical explanation

Security gates should be policy-driven with documented exception handling; otherwise teams will bypass noisy checks.

Provenance should connect repository, commit, workflow identity, artifact digest, signer identity, and deployment environment.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How would you integrate SAST and dependency scanning into a pipeline using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you handle database migrations safely within a deployment pipeline?Advanced

Answer

Database migrations should be backward-compatible, observable, and separated from risky application rollout when needed. I prefer expand-contract patterns, pre-deploy additive changes, app deploy, validation, and later cleanup. The pipeline should block irreversible changes without explicit approval.

Technical explanation

Safe release design assumes rollback may be needed and separates code deployment, data changes, and user exposure.

Compatibility windows and observability are mandatory when application and database versions overlap.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How do you handle database migrations safely within a deployment pipeline using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Implement an expand-contract migration: add nullable column/table first, deploy app compatible with old and new schema, backfill safely, then remove old schema in a later release.

3. Run the migration in a staging clone, measure duration and locks, add a pre-deploy backup checkpoint, and define a forward-fix plan for non-reversible steps.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you implement feature flags, and how do they decouple deploy from release?Advanced

Answer

Feature flags put release control into configuration rather than deployment. Code can be deployed disabled, enabled for internal users or a percentage, and rolled back instantly by changing the flag. This decouples deploy from release and reduces deployment risk.

Technical explanation

Safe release design assumes rollback may be needed and separates code deployment, data changes, and user exposure.

Compatibility windows and observability are mandatory when application and database versions overlap.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How do you implement feature flags, and how do they decouple deploy from release using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Add a feature flag with default disabled, enable it for internal users, then for 1% of traffic or one region, and watch error rate, latency, and business metrics.

3. Rollback by disabling the flag first; only redeploy if the underlying code or infrastructure is unhealthy.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

What is the difference between deployment and release?Advanced

Answer

Deployment means putting a version of software into an environment. Release means exposing functionality to users or business traffic. With feature flags, canaries, and traffic controls, a team can deploy code without releasing it broadly.

Technical explanation

Safe release design assumes rollback may be needed and separates code deployment, data changes, and user exposure.

Compatibility windows and observability are mandatory when application and database versions overlap.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: What is the difference between deployment and release using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Add a feature flag with default disabled, enable it for internal users, then for 1% of traffic or one region, and watch error rate, latency, and business metrics.

3. Rollback by disabling the flag first; only redeploy if the underlying code or infrastructure is unhealthy.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you ensure a pipeline is fast enough to give quick feedback?Advanced

Answer

A fast pipeline gives quick feedback by running cheap checks first, parallelizing independent work, caching dependencies, using affected tests, avoiding shared bottlenecks, and pushing long-running or low-signal tests to scheduled or post-merge workflows.

Technical explanation

Separate signal from noise: flaky tests erode trust and should have owners, quarantine rules, and fix SLAs.

Measure queue time, execution time, retry rate, failure class, and runner health instead of only pass/fail count.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How do you ensure a pipeline is fast enough to give quick feedback using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Export pipeline duration, queue time, retry count, failure reason, and flaky-test history into a dashboard by repository and stage.

3. Quarantine known flaky tests with owner and SLA, run fast PR smoke tests first, and schedule the full integration suite nightly or before release.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How would you design a pipeline for a microservices architecture?Advanced

Answer

For microservices, I design pipelines with reusable templates, service-specific build/test/deploy stages, dependency-aware integration tests, container image publishing by digest, environment promotion, and GitOps or progressive delivery per service. Shared platform controls enforce consistency.

Technical explanation

Security gates should be policy-driven with documented exception handling; otherwise teams will bypass noisy checks.

Provenance should connect repository, commit, workflow identity, artifact digest, signer identity, and deployment environment.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How would you design a pipeline for a microservices architecture using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you test infrastructure changes (Terraform) in a pipeline?Advanced

Answer

Terraform changes should be formatted, validated, linted, security-scanned, planned, reviewed, and applied through controlled approvals. The plan artifact should be tied to the commit, and production apply should use remote state locking, least-privilege credentials, and drift detection.

Technical explanation

Terraform plans should be reviewed as artifacts tied to a commit, but apply should re-check state and use locking.

Use policy-as-code and drift detection to catch risky infrastructure changes before apply.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How do you test infrastructure changes (Terraform) in a pipeline using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Run terraform fmt -check, terraform validate, tflint, security scan, and terraform plan -out=tfplan against remote state with locking enabled.

3. Attach the plan to the PR, require review, then apply only the reviewed commit in the target environment using short-lived credentials.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

What is a deployment freeze, and how do you enforce one?Advanced

Answer

A deployment freeze is a temporary block on production changes, usually for business-critical periods or incident stabilization. I enforce it through environment rules, pipeline gates, change-calendar checks, branch protections, or policy-as-code exceptions requiring leadership approval.

Technical explanation

A freeze should have explicit scope, start/end time, exception process, and audit trail.

Automation should enforce the policy rather than relying only on verbal coordination.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: What is a deployment freeze, and how do you enforce one using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How would you measure and improve pipeline reliability (flaky builds)?Advanced

Answer

Pipeline reliability is improved by measuring failures, classifying flakes, isolating unstable tests, fixing shared infrastructure issues, retrying only known transient operations, tracking queue time and runner health, and making ownership visible through dashboards and alerts.

Technical explanation

Separate signal from noise: flaky tests erode trust and should have owners, quarantine rules, and fix SLAs.

Measure queue time, execution time, retry rate, failure class, and runner health instead of only pass/fail count.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How would you measure and improve pipeline reliability (flaky builds) using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Export pipeline duration, queue time, retry count, failure reason, and flaky-test history into a dashboard by repository and stage.

3. Quarantine known flaky tests with owner and SLA, run fast PR smoke tests first, and schedule the full integration suite nightly or before release.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you handle long-running integration tests without slowing every commit?Advanced

Answer

I keep long-running integration tests out of every commit path by running smoke tests on PRs, targeted integration tests for affected services, full suites nightly or pre-release, and ephemeral test environments for changes that need deeper validation.

Technical explanation

Separate signal from noise: flaky tests erode trust and should have owners, quarantine rules, and fix SLAs.

Measure queue time, execution time, retry rate, failure class, and runner health instead of only pass/fail count.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How do you handle long-running integration tests without slowing every commit using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Export pipeline duration, queue time, retry count, failure reason, and flaky-test history into a dashboard by repository and stage.

3. Quarantine known flaky tests with owner and SLA, run fast PR smoke tests first, and schedule the full integration suite nightly or before release.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

What is a canary analysis, and how do you automate the go/no-go decision?Advanced

Answer

Canary analysis compares the new version against the baseline using metrics such as error rate, latency, saturation, logs, and business KPIs. Automation makes a go/no-go decision by evaluating thresholds over a window and either promotes, pauses, or rolls back.

Technical explanation

Traffic shifting should be separate from artifact build so the same artifact can be promoted or rolled back.

Use readiness checks, error budgets, and automated metrics before increasing production exposure.

Safe release design assumes rollback may be needed and separates code deployment, data changes, and user exposure.

Compatibility windows and observability are mandatory when application and database versions overlap.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

Hands-on example

1. Design an advanced delivery exercise for: What is a canary analysis, and how do you automate the go/no-go decision using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you roll out a change to a subset of users or regions first?Advanced

Answer

I roll out to a subset by using canary traffic weights, feature flags, region-based deployment, ring deployments, tenant allowlists, or service-mesh routing. I start with low-risk users or one region, watch metrics, then expand progressively.

Technical explanation

Safe release design assumes rollback may be needed and separates code deployment, data changes, and user exposure.

Compatibility windows and observability are mandatory when application and database versions overlap.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How do you roll out a change to a subset of users or regions first using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Add a feature flag with default disabled, enable it for internal users, then for 1% of traffic or one region, and watch error rate, latency, and business metrics.

3. Rollback by disabling the flag first; only redeploy if the underlying code or infrastructure is unhealthy.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you audit who deployed what and when?Advanced

Answer

I audit deployments by recording commit SHA, artifact digest, version, environment, approver, pipeline run, Argo CD sync operation, Kubernetes change events, and production release notes. The audit trail should connect who approved, what changed, and when it reached production.

Technical explanation

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How do you audit who deployed what and when using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you manage pipeline configuration across dozens of repositories?Advanced

Answer

Across many repositories, I manage pipeline configuration with reusable workflows, Jenkins shared libraries, versioned templates, central policy checks, golden runner images, and automated update PRs. Teams can override allowed parameters without copying every pipeline detail.

Technical explanation

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How do you manage pipeline configuration across dozens of repositories using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How would you migrate a team from Jenkins to GitOps with ArgoCD?Advanced

Answer

To migrate from Jenkins to GitOps with Argo CD, I separate build from deploy. Jenkins or GitHub Actions keeps building, testing, scanning, and publishing images, then updates a GitOps repo. Argo CD takes over cluster reconciliation, deployment visibility, and rollback through Git.

Technical explanation

The desired state should be declarative enough for an agent to converge without manual runbook steps.

Git history becomes the audit trail for who requested, reviewed, and approved an environment change.

Keep build execution away from the controller; agents should be disposable, labeled, and sized for the workload.

Treat the pipeline definition as production code: peer review it, test changes, version shared libraries, and avoid hidden UI-only job logic.

Use least-privilege credentials, immutable artifacts, deterministic versions, and clear post-build cleanup to make pipelines repeatable and auditable.

Design stages around fast feedback: fail cheap checks early, isolate workspaces, parallelize independent work, and publish evidence such as test reports and build metadata.

Hands-on example

1. Design an advanced delivery exercise for: How would you migrate a team from Jenkins to GitOps with ArgoCD using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

What is progressive delivery, and what tools enable it (Argo Rollouts, Flagger)?Advanced

Answer

Progressive delivery is controlled rollout with automated validation, such as canary, blue-green, traffic shifting, and feature-flag rollout. Tools include Argo Rollouts, Flagger, service meshes, ingress controllers, and observability systems that feed promotion decisions.

Technical explanation

Safe release design assumes rollback may be needed and separates code deployment, data changes, and user exposure.

Compatibility windows and observability are mandatory when application and database versions overlap.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Design an advanced delivery exercise for: What is progressive delivery, and what tools enable it (Argo Rollouts, Flagger) using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you handle rollback when a release includes a non-reversible DB migration?Advanced

Answer

If a release includes a non-reversible database migration, rollback cannot simply redeploy the old app. I avoid that with expand-contract design. If unavoidable, I use backups, forward-fix plans, compatibility windows, manual approval, feature flags, and a clear incident playbook.

Technical explanation

Safe release design assumes rollback may be needed and separates code deployment, data changes, and user exposure.

Compatibility windows and observability are mandatory when application and database versions overlap.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How do you handle rollback when a release includes a non-reversible DB migration using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Implement an expand-contract migration: add nullable column/table first, deploy app compatible with old and new schema, backfill safely, then remove old schema in a later release.

3. Run the migration in a staging clone, measure duration and locks, add a pre-deploy backup checkpoint, and define a forward-fix plan for non-reversible steps.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

What recent CI/CD or GitOps practice have you adopted, and what improved?Advanced

Answer

A recent CI/CD practice I would highlight is replacing long-lived deployment secrets with OIDC-based short-lived credentials and adding artifact signing/SBOM gates. It improves security by reducing static secret exposure and improves traceability from source commit to deployed artifact.

Technical explanation

The desired state should be declarative enough for an agent to converge without manual runbook steps.

Git history becomes the audit trail for who requested, reviewed, and approved an environment change.

GitOps separates build from deploy: CI produces immutable artifacts, while the GitOps controller reconciles declarative desired state into the cluster.

Argo CD status has two dimensions: sync status indicates desired versus live state; health status indicates whether live resources appear operationally healthy.

Use projects, RBAC, repository allowlists, destination restrictions, sync windows, and admission policies to constrain what an Application may deploy.

Prefer reviewed Git changes over direct kubectl changes; direct changes create drift and bypass audit, policy, and promotion workflow.

Hands-on example

1. Design an advanced delivery exercise for: What recent CI/CD or GitOps practice have you adopted, and what improved using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build once from a protected branch, generate test reports, SBOM, vulnerability scan results, provenance metadata, and a container image tagged by both semantic version and digest.

3. Sign the image or artifact, verify the signature in the deployment pipeline, and promote the same digest through dev, staging, and production without rebuilding.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

How do you keep build agents and runner images patched and secure over time?Advanced

Answer

Build agents and runner images should be treated as production infrastructure. I keep them patched with immutable golden images, automated rebuilds, vulnerability scanning, base-image refresh schedules, least-privilege configuration, ephemeral runners, and monitoring for drift or outdated tooling.

Technical explanation

Runner images should be rebuilt from a controlled base and scanned before use.

Long-lived self-hosted runners need stronger isolation because workspace residue and credentials can survive across jobs.

A secure pipeline protects source, build environment, dependencies, artifacts, deployment credentials, and runtime promotion gates as one chain.

Immutable artifacts, SBOMs, signatures, provenance, vulnerability gates, and environment promotion reduce ambiguity about what was built and deployed.

CI runners are high-value targets; isolate untrusted jobs, patch runner images, remove persistent credentials, and prefer ephemeral execution where possible.

Release safety depends on both automation and observability: use canaries, feature flags, rollback plans, and automated metric-based decisions.

Hands-on example

1. Design an advanced delivery exercise for: How do you keep build agents and runner images patched and secure over time using one service, one Git repository, one artifact registry, and one Kubernetes environment.

2. Build a golden runner image weekly from a patched base, install pinned tool versions, scan the image, and publish it only if the scan passes policy.

3. Use ephemeral self-hosted runners for jobs, destroy the VM/Pod after each run, and alert on images older than the approved patch window.

4. Use progressive exposure where relevant: feature flag off by default, canary 5%, automated metric check for error rate and latency, then expand or rollback.

5. Record audit evidence: PR, approver, pipeline run, artifact digest, SBOM location, signature verification result, deployment event, and rollback or forward-fix decision.

Source Note for Current CI/CD and GitOps Items

Most answers are based on stable CI/CD, Jenkins Pipeline, GitOps, Argo CD, GitHub Actions, Git, and supply-chain security concepts. Validate tool behavior against your exact Jenkins plugin versions, Argo CD version, GitHub plan, runner model, and organization security policy before implementing in production.

Jenkins Pipeline syntax reference: https://www.jenkins.io/doc/book/pipeline/syntax/

Jenkins Pipeline overview: https://www.jenkins.io/doc/book/pipeline/

Argo CD automated sync policy: https://argo-cd.readthedocs.io/en/stable/user-guide/auto_sync/

Argo CD sync phases and waves: https://argo-cd.readthedocs.io/en/stable/user-guide/sync-waves/

GitHub Actions OIDC cloud-provider guidance: https://docs.github.com/en/actions/how-tos/secure-your-work/security-harden-deployments/oidc-in-cloud-providers

OpenGitOps principles: https://github.com/open-gitops/documents/blob/main/PRINCIPLES.md