Companies › Meta › DevOps / SRE
Meta DevOps / SRE interview questions
100 real Meta DevOps / SRE interview questions with model answers, key talking points, and common pitfalls — free prep for your Meta interview.
Paste the job description and your resume into SkillFitly's free resume checker to see your match score and missing skills, then practice with timed interview quizzes.
1For Meta Platforms's feed ranking requests, how would you design SLOs and error budget management so the platform remains reliable during misinformation spread? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on SLOs and error budget management, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Start by defining Service Level Indicators (SLIs) that reflect the user's experience — typically availability (successful requests / total) and latency (proportion of requests under a threshold). Turn each SLI into a Service Level Objective (SLO), e.g. 99.9% of requests succeed over a rolling 30 days. The error budget is simply 100% minus the SLO: at 99.9% you may 'spend' 0.1% of requests failing (~43 minutes/month). When the budget is healthy you ship fast; when it's exhausted you freeze risky changes and prioritize reliability.
Measure SLIs from real telemetry, not host health. A request-based availability SLI in Prometheus looks like this:
# 30-day availability SLI (ratio of good requests)
sum(rate(http_requests_total{code!~"5.."}[30d]))
/
sum(rate(http_requests_total[30d]))
# Fast + slow burn-rate alert on the error budget (multi-window)
(
sum(rate(http_requests_total{code=~"5.."}[1h]))
/ sum(rate(http_requests_total[1h]))
) > (14.4 * 0.001) # 14.4x burn of a 0.1% budget => pageUse multi-window, multi-burn-rate alerts (fast window to catch acute outages, slow window to catch slow burns) so you page on symptoms that actually threaten the budget, not on every blip. Assign each SLO a clear owner and a runbook. The tradeoff: tighter SLOs (more nines) cost exponentially more in redundancy and toil, so set them from real user need, not vanity.
2Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for SLOs and error budget management in ads delivery platforms, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on SLOs and error budget management, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Start by defining Service Level Indicators (SLIs) that reflect the user's experience — typically availability (successful requests / total) and latency (proportion of requests under a threshold). Turn each SLI into a Service Level Objective (SLO), e.g. 99.9% of requests succeed over a rolling 30 days. The error budget is simply 100% minus the SLO: at 99.9% you may 'spend' 0.1% of requests failing (~43 minutes/month). When the budget is healthy you ship fast; when it's exhausted you freeze risky changes and prioritize reliability.
Measure SLIs from real telemetry, not host health. A request-based availability SLI in Prometheus looks like this:
# 30-day availability SLI (ratio of good requests)
sum(rate(http_requests_total{code!~"5.."}[30d]))
/
sum(rate(http_requests_total[30d]))
# Fast + slow burn-rate alert on the error budget (multi-window)
(
sum(rate(http_requests_total{code=~"5.."}[1h]))
/ sum(rate(http_requests_total[1h]))
) > (14.4 * 0.001) # 14.4x burn of a 0.1% budget => pageUse multi-window, multi-burn-rate alerts (fast window to catch acute outages, slow window to catch slow burns) so you page on symptoms that actually threaten the budget, not on every blip. Assign each SLO a clear owner and a runbook. The tradeoff: tighter SLOs (more nines) cost exponentially more in redundancy and toil, so set them from real user need, not vanity.
3A production incident affects syncing a VR device: latency is rising, error rates are elevated, and ad-delivery regression is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on SLOs and error budget management, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Start by defining Service Level Indicators (SLIs) that reflect the user's experience — typically availability (successful requests / total) and latency (proportion of requests under a threshold). Turn each SLI into a Service Level Objective (SLO), e.g. 99.9% of requests succeed over a rolling 30 days. The error budget is simply 100% minus the SLO: at 99.9% you may 'spend' 0.1% of requests failing (~43 minutes/month). When the budget is healthy you ship fast; when it's exhausted you freeze risky changes and prioritize reliability.
Measure SLIs from real telemetry, not host health. A request-based availability SLI in Prometheus looks like this:
# 30-day availability SLI (ratio of good requests)
sum(rate(http_requests_total{code!~"5.."}[30d]))
/
sum(rate(http_requests_total[30d]))
# Fast + slow burn-rate alert on the error budget (multi-window)
(
sum(rate(http_requests_total{code=~"5.."}[1h]))
/ sum(rate(http_requests_total[1h]))
) > (14.4 * 0.001) # 14.4x burn of a 0.1% budget => pageUse multi-window, multi-burn-rate alerts (fast window to catch acute outages, slow window to catch slow burns) so you page on symptoms that actually threaten the budget, not on every blip. Assign each SLO a clear owner and a runbook. The tradeoff: tighter SLOs (more nines) cost exponentially more in redundancy and toil, so set them from real user need, not vanity.
4What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying SLOs and error budget management to Meta Platforms's content moderation pipelines?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on SLOs and error budget management, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Start by defining Service Level Indicators (SLIs) that reflect the user's experience — typically availability (successful requests / total) and latency (proportion of requests under a threshold). Turn each SLI into a Service Level Objective (SLO), e.g. 99.9% of requests succeed over a rolling 30 days. The error budget is simply 100% minus the SLO: at 99.9% you may 'spend' 0.1% of requests failing (~43 minutes/month). When the budget is healthy you ship fast; when it's exhausted you freeze risky changes and prioritize reliability.
Measure SLIs from real telemetry, not host health. A request-based availability SLI in Prometheus looks like this:
# 30-day availability SLI (ratio of good requests)
sum(rate(http_requests_total{code!~"5.."}[30d]))
/
sum(rate(http_requests_total[30d]))
# Fast + slow burn-rate alert on the error budget (multi-window)
(
sum(rate(http_requests_total{code=~"5.."}[1h]))
/ sum(rate(http_requests_total[1h]))
) > (14.4 * 0.001) # 14.4x burn of a 0.1% budget => pageUse multi-window, multi-burn-rate alerts (fast window to catch acute outages, slow window to catch slow burns) so you page on symptoms that actually threaten the budget, not on every blip. Assign each SLO a clear owner and a runbook. The tradeoff: tighter SLOs (more nines) cost exponentially more in redundancy and toil, so set them from real user need, not vanity.
5Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize SLOs and error budget management for social graph updates.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on SLOs and error budget management, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Start by defining Service Level Indicators (SLIs) that reflect the user's experience — typically availability (successful requests / total) and latency (proportion of requests under a threshold). Turn each SLI into a Service Level Objective (SLO), e.g. 99.9% of requests succeed over a rolling 30 days. The error budget is simply 100% minus the SLO: at 99.9% you may 'spend' 0.1% of requests failing (~43 minutes/month). When the budget is healthy you ship fast; when it's exhausted you freeze risky changes and prioritize reliability.
Measure SLIs from real telemetry, not host health. A request-based availability SLI in Prometheus looks like this:
# 30-day availability SLI (ratio of good requests)
sum(rate(http_requests_total{code!~"5.."}[30d]))
/
sum(rate(http_requests_total[30d]))
# Fast + slow burn-rate alert on the error budget (multi-window)
(
sum(rate(http_requests_total{code=~"5.."}[1h]))
/ sum(rate(http_requests_total[1h]))
) > (14.4 * 0.001) # 14.4x burn of a 0.1% budget => pageUse multi-window, multi-burn-rate alerts (fast window to catch acute outages, slow window to catch slow burns) so you page on symptoms that actually threaten the budget, not on every blip. Assign each SLO a clear owner and a runbook. The tradeoff: tighter SLOs (more nines) cost exponentially more in redundancy and toil, so set them from real user need, not vanity.
6For Meta Platforms's messaging delivery, how would you design incident response and major-incident command so the platform remains reliable during messaging delay? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on incident response and major-incident command, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Effective incident response separates roles: an Incident Commander (IC) owns coordination and decisions, an Operations lead drives the technical fix, and a Communications lead handles stakeholders. The goal is to restore service first (mitigate) and diagnose root cause later. Declare an incident early with a clear severity, open a dedicated channel/bridge, and keep a running timeline.
Mitigation usually means the fastest safe reversal: roll back the last deploy, shift traffic away from a bad region, or disable a feature flag. A rollback via a deployment tool:
# Roll back a Kubernetes deployment to the previous known-good revision
kubectl rollout undo deployment/checkout-api
kubectl rollout status deployment/checkout-api --timeout=120s
# Or shift traffic away from an unhealthy region at the LB/DNS layer
aws elbv2 modify-target-group-attributes \
--target-group-arn $TG --attributes Key=deregistration_delay.timeout_seconds,Value=5After recovery, run a blameless postmortem: timeline, contributing factors, what detected it, and concrete action items with owners. The tradeoff to manage is speed vs. certainty — mitigate on symptoms quickly rather than waiting for a full root cause during a customer-facing outage.
7Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for incident response and major-incident command in messaging reliability systems, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on incident response and major-incident command, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Effective incident response separates roles: an Incident Commander (IC) owns coordination and decisions, an Operations lead drives the technical fix, and a Communications lead handles stakeholders. The goal is to restore service first (mitigate) and diagnose root cause later. Declare an incident early with a clear severity, open a dedicated channel/bridge, and keep a running timeline.
Mitigation usually means the fastest safe reversal: roll back the last deploy, shift traffic away from a bad region, or disable a feature flag. A rollback via a deployment tool:
# Roll back a Kubernetes deployment to the previous known-good revision
kubectl rollout undo deployment/checkout-api
kubectl rollout status deployment/checkout-api --timeout=120s
# Or shift traffic away from an unhealthy region at the LB/DNS layer
aws elbv2 modify-target-group-attributes \
--target-group-arn $TG --attributes Key=deregistration_delay.timeout_seconds,Value=5After recovery, run a blameless postmortem: timeline, contributing factors, what detected it, and concrete action items with owners. The tradeoff to manage is speed vs. certainty — mitigate on symptoms quickly rather than waiting for a full root cause during a customer-facing outage.
8A production incident affects loading an Instagram feed: latency is rising, error rates are elevated, and privacy-sensitive social graph exposure is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on incident response and major-incident command, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Effective incident response separates roles: an Incident Commander (IC) owns coordination and decisions, an Operations lead drives the technical fix, and a Communications lead handles stakeholders. The goal is to restore service first (mitigate) and diagnose root cause later. Declare an incident early with a clear severity, open a dedicated channel/bridge, and keep a running timeline.
Mitigation usually means the fastest safe reversal: roll back the last deploy, shift traffic away from a bad region, or disable a feature flag. A rollback via a deployment tool:
# Roll back a Kubernetes deployment to the previous known-good revision
kubectl rollout undo deployment/checkout-api
kubectl rollout status deployment/checkout-api --timeout=120s
# Or shift traffic away from an unhealthy region at the LB/DNS layer
aws elbv2 modify-target-group-attributes \
--target-group-arn $TG --attributes Key=deregistration_delay.timeout_seconds,Value=5After recovery, run a blameless postmortem: timeline, contributing factors, what detected it, and concrete action items with owners. The tradeoff to manage is speed vs. certainty — mitigate on symptoms quickly rather than waiting for a full root cause during a customer-facing outage.
9What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying incident response and major-incident command to Meta Platforms's Reality Labs device services?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on incident response and major-incident command, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Effective incident response separates roles: an Incident Commander (IC) owns coordination and decisions, an Operations lead drives the technical fix, and a Communications lead handles stakeholders. The goal is to restore service first (mitigate) and diagnose root cause later. Declare an incident early with a clear severity, open a dedicated channel/bridge, and keep a running timeline.
Mitigation usually means the fastest safe reversal: roll back the last deploy, shift traffic away from a bad region, or disable a feature flag. A rollback via a deployment tool:
# Roll back a Kubernetes deployment to the previous known-good revision
kubectl rollout undo deployment/checkout-api
kubectl rollout status deployment/checkout-api --timeout=120s
# Or shift traffic away from an unhealthy region at the LB/DNS layer
aws elbv2 modify-target-group-attributes \
--target-group-arn $TG --attributes Key=deregistration_delay.timeout_seconds,Value=5After recovery, run a blameless postmortem: timeline, contributing factors, what detected it, and concrete action items with owners. The tradeoff to manage is speed vs. certainty — mitigate on symptoms quickly rather than waiting for a full root cause during a customer-facing outage.
10Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize incident response and major-incident command for ad impressions.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on incident response and major-incident command, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Effective incident response separates roles: an Incident Commander (IC) owns coordination and decisions, an Operations lead drives the technical fix, and a Communications lead handles stakeholders. The goal is to restore service first (mitigate) and diagnose root cause later. Declare an incident early with a clear severity, open a dedicated channel/bridge, and keep a running timeline.
Mitigation usually means the fastest safe reversal: roll back the last deploy, shift traffic away from a bad region, or disable a feature flag. A rollback via a deployment tool:
# Roll back a Kubernetes deployment to the previous known-good revision
kubectl rollout undo deployment/checkout-api
kubectl rollout status deployment/checkout-api --timeout=120s
# Or shift traffic away from an unhealthy region at the LB/DNS layer
aws elbv2 modify-target-group-attributes \
--target-group-arn $TG --attributes Key=deregistration_delay.timeout_seconds,Value=5After recovery, run a blameless postmortem: timeline, contributing factors, what detected it, and concrete action items with owners. The tradeoff to manage is speed vs. certainty — mitigate on symptoms quickly rather than waiting for a full root cause during a customer-facing outage.
11For Meta Platforms's social graph updates, how would you design observability with metrics, logs, traces, and profiling so the platform remains reliable during ad-delivery regression? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on observability with metrics, logs, traces, and profiling, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Observability rests on three pillars: metrics (cheap, aggregate time-series for alerting and dashboards), logs (high-cardinality event detail for debugging), and traces (request flow across services to find where latency is spent). Profiling adds a fourth dimension — CPU/memory hot paths inside a process. The art is choosing the right pillar per question: alert on metrics, debug with traces, then drill into logs or profiles.
Instrument with OpenTelemetry so you emit all three from one SDK. A minimal trace + metric in code:
from opentelemetry import trace, metrics
tracer = trace.get_tracer("checkout")
meter = metrics.get_meter("checkout")
latency = meter.create_histogram("checkout.latency", unit="ms")
with tracer.start_as_current_span("process_order") as span:
span.set_attribute("order.id", order_id)
t0 = time.time()
result = charge_and_fulfill(order)
latency.record((time.time() - t0) * 1000, {"status": result.status})Control cost with sampling (tail-based sampling keeps slow/error traces) and cardinality limits on labels — an unbounded label like user_id will blow up your metrics store. Correlate the pillars with a shared trace_id so you can jump from a latency spike to the exact slow trace to its logs.
12Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for observability with metrics, logs, traces, and profiling in content moderation pipelines, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on observability with metrics, logs, traces, and profiling, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Observability rests on three pillars: metrics (cheap, aggregate time-series for alerting and dashboards), logs (high-cardinality event detail for debugging), and traces (request flow across services to find where latency is spent). Profiling adds a fourth dimension — CPU/memory hot paths inside a process. The art is choosing the right pillar per question: alert on metrics, debug with traces, then drill into logs or profiles.
Instrument with OpenTelemetry so you emit all three from one SDK. A minimal trace + metric in code:
from opentelemetry import trace, metrics
tracer = trace.get_tracer("checkout")
meter = metrics.get_meter("checkout")
latency = meter.create_histogram("checkout.latency", unit="ms")
with tracer.start_as_current_span("process_order") as span:
span.set_attribute("order.id", order_id)
t0 = time.time()
result = charge_and_fulfill(order)
latency.record((time.time() - t0) * 1000, {"status": result.status})Control cost with sampling (tail-based sampling keeps slow/error traces) and cardinality limits on labels — an unbounded label like user_id will blow up your metrics store. Correlate the pillars with a shared trace_id so you can jump from a latency spike to the exact slow trace to its logs.
13A production incident affects sending a WhatsApp message: latency is rising, error rates are elevated, and abuse traffic spike is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on observability with metrics, logs, traces, and profiling, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Observability rests on three pillars: metrics (cheap, aggregate time-series for alerting and dashboards), logs (high-cardinality event detail for debugging), and traces (request flow across services to find where latency is spent). Profiling adds a fourth dimension — CPU/memory hot paths inside a process. The art is choosing the right pillar per question: alert on metrics, debug with traces, then drill into logs or profiles.
Instrument with OpenTelemetry so you emit all three from one SDK. A minimal trace + metric in code:
from opentelemetry import trace, metrics
tracer = trace.get_tracer("checkout")
meter = metrics.get_meter("checkout")
latency = meter.create_histogram("checkout.latency", unit="ms")
with tracer.start_as_current_span("process_order") as span:
span.set_attribute("order.id", order_id)
t0 = time.time()
result = charge_and_fulfill(order)
latency.record((time.time() - t0) * 1000, {"status": result.status})Control cost with sampling (tail-based sampling keeps slow/error traces) and cardinality limits on labels — an unbounded label like user_id will blow up your metrics store. Correlate the pillars with a shared trace_id so you can jump from a latency spike to the exact slow trace to its logs.
14What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying observability with metrics, logs, traces, and profiling to Meta Platforms's social feed ranking services?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on observability with metrics, logs, traces, and profiling, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Observability rests on three pillars: metrics (cheap, aggregate time-series for alerting and dashboards), logs (high-cardinality event detail for debugging), and traces (request flow across services to find where latency is spent). Profiling adds a fourth dimension — CPU/memory hot paths inside a process. The art is choosing the right pillar per question: alert on metrics, debug with traces, then drill into logs or profiles.
Instrument with OpenTelemetry so you emit all three from one SDK. A minimal trace + metric in code:
from opentelemetry import trace, metrics
tracer = trace.get_tracer("checkout")
meter = metrics.get_meter("checkout")
latency = meter.create_histogram("checkout.latency", unit="ms")
with tracer.start_as_current_span("process_order") as span:
span.set_attribute("order.id", order_id)
t0 = time.time()
result = charge_and_fulfill(order)
latency.record((time.time() - t0) * 1000, {"status": result.status})Control cost with sampling (tail-based sampling keeps slow/error traces) and cardinality limits on labels — an unbounded label like user_id will blow up your metrics store. Correlate the pillars with a shared trace_id so you can jump from a latency spike to the exact slow trace to its logs.
15Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize observability with metrics, logs, traces, and profiling for content moderation queues.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on observability with metrics, logs, traces, and profiling, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Observability rests on three pillars: metrics (cheap, aggregate time-series for alerting and dashboards), logs (high-cardinality event detail for debugging), and traces (request flow across services to find where latency is spent). Profiling adds a fourth dimension — CPU/memory hot paths inside a process. The art is choosing the right pillar per question: alert on metrics, debug with traces, then drill into logs or profiles.
Instrument with OpenTelemetry so you emit all three from one SDK. A minimal trace + metric in code:
from opentelemetry import trace, metrics
tracer = trace.get_tracer("checkout")
meter = metrics.get_meter("checkout")
latency = meter.create_histogram("checkout.latency", unit="ms")
with tracer.start_as_current_span("process_order") as span:
span.set_attribute("order.id", order_id)
t0 = time.time()
result = charge_and_fulfill(order)
latency.record((time.time() - t0) * 1000, {"status": result.status})Control cost with sampling (tail-based sampling keeps slow/error traces) and cardinality limits on labels — an unbounded label like user_id will blow up your metrics store. Correlate the pillars with a shared trace_id so you can jump from a latency spike to the exact slow trace to its logs.
16For Meta Platforms's ad impressions, how would you design capacity planning and demand forecasting so the platform remains reliable during privacy-sensitive social graph exposure? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on capacity planning and demand forecasting, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Capacity planning turns a demand forecast into a resourcing plan with headroom. Start from historical utilization (CPU, memory, QPS, connections), model growth (organic + known events like launches or seasonal peaks), and add a safety margin so you can absorb spikes and failover load. The core equation: required capacity = peak forecast demand / target utilization, plus N+1 (or N+2) redundancy for failures.
A simple forecast fits a trend to historical peaks and projects forward:
import numpy as np
# weekly peak QPS over the last 12 weeks
peaks = np.array([120,128,131,140,138,150,159,165,171,180,188,196])
weeks = np.arange(len(peaks))
m, b = np.polyfit(weeks, peaks, 1) # linear trend
forecast_next = m * (len(peaks) + 4) + b # 4 weeks out
target_util = 0.6 # keep 40% headroom
capacity_qps = forecast_next / target_util
print(f"Provision for {capacity_qps:.0f} QPS")Validate the plan with load tests at forecast+margin, and prefer autoscaling for elastic tiers so you don't statically over-provision. The tradeoff is cost vs. risk: too much headroom wastes money, too little risks saturation during the exact peak you were planning for.
17Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for capacity planning and demand forecasting in Reality Labs device services, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on capacity planning and demand forecasting, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Capacity planning turns a demand forecast into a resourcing plan with headroom. Start from historical utilization (CPU, memory, QPS, connections), model growth (organic + known events like launches or seasonal peaks), and add a safety margin so you can absorb spikes and failover load. The core equation: required capacity = peak forecast demand / target utilization, plus N+1 (or N+2) redundancy for failures.
A simple forecast fits a trend to historical peaks and projects forward:
import numpy as np
# weekly peak QPS over the last 12 weeks
peaks = np.array([120,128,131,140,138,150,159,165,171,180,188,196])
weeks = np.arange(len(peaks))
m, b = np.polyfit(weeks, peaks, 1) # linear trend
forecast_next = m * (len(peaks) + 4) + b # 4 weeks out
target_util = 0.6 # keep 40% headroom
capacity_qps = forecast_next / target_util
print(f"Provision for {capacity_qps:.0f} QPS")Validate the plan with load tests at forecast+margin, and prefer autoscaling for elastic tiers so you don't statically over-provision. The tradeoff is cost vs. risk: too much headroom wastes money, too little risks saturation during the exact peak you were planning for.
18A production incident affects serving a targeted ad: latency is rising, error rates are elevated, and misinformation spread is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on capacity planning and demand forecasting, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Capacity planning turns a demand forecast into a resourcing plan with headroom. Start from historical utilization (CPU, memory, QPS, connections), model growth (organic + known events like launches or seasonal peaks), and add a safety margin so you can absorb spikes and failover load. The core equation: required capacity = peak forecast demand / target utilization, plus N+1 (or N+2) redundancy for failures.
A simple forecast fits a trend to historical peaks and projects forward:
import numpy as np
# weekly peak QPS over the last 12 weeks
peaks = np.array([120,128,131,140,138,150,159,165,171,180,188,196])
weeks = np.arange(len(peaks))
m, b = np.polyfit(weeks, peaks, 1) # linear trend
forecast_next = m * (len(peaks) + 4) + b # 4 weeks out
target_util = 0.6 # keep 40% headroom
capacity_qps = forecast_next / target_util
print(f"Provision for {capacity_qps:.0f} QPS")Validate the plan with load tests at forecast+margin, and prefer autoscaling for elastic tiers so you don't statically over-provision. The tradeoff is cost vs. risk: too much headroom wastes money, too little risks saturation during the exact peak you were planning for.
19What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying capacity planning and demand forecasting to Meta Platforms's ads delivery platforms?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on capacity planning and demand forecasting, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Capacity planning turns a demand forecast into a resourcing plan with headroom. Start from historical utilization (CPU, memory, QPS, connections), model growth (organic + known events like launches or seasonal peaks), and add a safety margin so you can absorb spikes and failover load. The core equation: required capacity = peak forecast demand / target utilization, plus N+1 (or N+2) redundancy for failures.
A simple forecast fits a trend to historical peaks and projects forward:
import numpy as np
# weekly peak QPS over the last 12 weeks
peaks = np.array([120,128,131,140,138,150,159,165,171,180,188,196])
weeks = np.arange(len(peaks))
m, b = np.polyfit(weeks, peaks, 1) # linear trend
forecast_next = m * (len(peaks) + 4) + b # 4 weeks out
target_util = 0.6 # keep 40% headroom
capacity_qps = forecast_next / target_util
print(f"Provision for {capacity_qps:.0f} QPS")Validate the plan with load tests at forecast+margin, and prefer autoscaling for elastic tiers so you don't statically over-provision. The tradeoff is cost vs. risk: too much headroom wastes money, too little risks saturation during the exact peak you were planning for.
20Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize capacity planning and demand forecasting for feed ranking requests.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on capacity planning and demand forecasting, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Capacity planning turns a demand forecast into a resourcing plan with headroom. Start from historical utilization (CPU, memory, QPS, connections), model growth (organic + known events like launches or seasonal peaks), and add a safety margin so you can absorb spikes and failover load. The core equation: required capacity = peak forecast demand / target utilization, plus N+1 (or N+2) redundancy for failures.
A simple forecast fits a trend to historical peaks and projects forward:
import numpy as np
# weekly peak QPS over the last 12 weeks
peaks = np.array([120,128,131,140,138,150,159,165,171,180,188,196])
weeks = np.arange(len(peaks))
m, b = np.polyfit(weeks, peaks, 1) # linear trend
forecast_next = m * (len(peaks) + 4) + b # 4 weeks out
target_util = 0.6 # keep 40% headroom
capacity_qps = forecast_next / target_util
print(f"Provision for {capacity_qps:.0f} QPS")Validate the plan with load tests at forecast+margin, and prefer autoscaling for elastic tiers so you don't statically over-provision. The tradeoff is cost vs. risk: too much headroom wastes money, too little risks saturation during the exact peak you were planning for.
21For Meta Platforms's content moderation queues, how would you design progressive delivery, canary releases, and safe rollbacks so the platform remains reliable during abuse traffic spike? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on progressive delivery, canary releases, and safe rollbacks, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Progressive delivery reduces blast radius by rolling a change to a small slice of traffic first (canary), watching health signals, then widening only if the canary is healthy. Combined with automated rollback, a bad release affects 1-5% of users for minutes instead of everyone. The key is defining objective promotion criteria — error rate, latency, and business KPIs — that gate each step.
An Argo Rollouts canary spec encodes the steps and analysis:
apiVersion: argoproj.io/v1alpha1
kind: Rollout
spec:
strategy:
canary:
steps:
- setWeight: 5
- pause: {duration: 5m}
- analysis:
templates: [{templateName: success-rate}]
- setWeight: 25
- pause: {duration: 10m}
- setWeight: 100
# analysis fails -> automatic rollback to stable ReplicaSetPair canaries with feature flags for instant kill-switches independent of deploys. The tradeoff: progressive delivery adds pipeline complexity and slows full rollout, but that latency is cheap insurance against a full-fleet outage.
22Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for progressive delivery, canary releases, and safe rollbacks in social feed ranking services, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on progressive delivery, canary releases, and safe rollbacks, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Progressive delivery reduces blast radius by rolling a change to a small slice of traffic first (canary), watching health signals, then widening only if the canary is healthy. Combined with automated rollback, a bad release affects 1-5% of users for minutes instead of everyone. The key is defining objective promotion criteria — error rate, latency, and business KPIs — that gate each step.
An Argo Rollouts canary spec encodes the steps and analysis:
apiVersion: argoproj.io/v1alpha1
kind: Rollout
spec:
strategy:
canary:
steps:
- setWeight: 5
- pause: {duration: 5m}
- analysis:
templates: [{templateName: success-rate}]
- setWeight: 25
- pause: {duration: 10m}
- setWeight: 100
# analysis fails -> automatic rollback to stable ReplicaSetPair canaries with feature flags for instant kill-switches independent of deploys. The tradeoff: progressive delivery adds pipeline complexity and slows full rollout, but that latency is cheap insurance against a full-fleet outage.
23A production incident affects moderating harmful content: latency is rising, error rates are elevated, and messaging delay is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on progressive delivery, canary releases, and safe rollbacks, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Progressive delivery reduces blast radius by rolling a change to a small slice of traffic first (canary), watching health signals, then widening only if the canary is healthy. Combined with automated rollback, a bad release affects 1-5% of users for minutes instead of everyone. The key is defining objective promotion criteria — error rate, latency, and business KPIs — that gate each step.
An Argo Rollouts canary spec encodes the steps and analysis:
apiVersion: argoproj.io/v1alpha1
kind: Rollout
spec:
strategy:
canary:
steps:
- setWeight: 5
- pause: {duration: 5m}
- analysis:
templates: [{templateName: success-rate}]
- setWeight: 25
- pause: {duration: 10m}
- setWeight: 100
# analysis fails -> automatic rollback to stable ReplicaSetPair canaries with feature flags for instant kill-switches independent of deploys. The tradeoff: progressive delivery adds pipeline complexity and slows full rollout, but that latency is cheap insurance against a full-fleet outage.
24What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying progressive delivery, canary releases, and safe rollbacks to Meta Platforms's messaging reliability systems?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on progressive delivery, canary releases, and safe rollbacks, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Progressive delivery reduces blast radius by rolling a change to a small slice of traffic first (canary), watching health signals, then widening only if the canary is healthy. Combined with automated rollback, a bad release affects 1-5% of users for minutes instead of everyone. The key is defining objective promotion criteria — error rate, latency, and business KPIs — that gate each step.
An Argo Rollouts canary spec encodes the steps and analysis:
apiVersion: argoproj.io/v1alpha1
kind: Rollout
spec:
strategy:
canary:
steps:
- setWeight: 5
- pause: {duration: 5m}
- analysis:
templates: [{templateName: success-rate}]
- setWeight: 25
- pause: {duration: 10m}
- setWeight: 100
# analysis fails -> automatic rollback to stable ReplicaSetPair canaries with feature flags for instant kill-switches independent of deploys. The tradeoff: progressive delivery adds pipeline complexity and slows full rollout, but that latency is cheap insurance against a full-fleet outage.
25Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize progressive delivery, canary releases, and safe rollbacks for messaging delivery.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on progressive delivery, canary releases, and safe rollbacks, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Progressive delivery reduces blast radius by rolling a change to a small slice of traffic first (canary), watching health signals, then widening only if the canary is healthy. Combined with automated rollback, a bad release affects 1-5% of users for minutes instead of everyone. The key is defining objective promotion criteria — error rate, latency, and business KPIs — that gate each step.
An Argo Rollouts canary spec encodes the steps and analysis:
apiVersion: argoproj.io/v1alpha1
kind: Rollout
spec:
strategy:
canary:
steps:
- setWeight: 5
- pause: {duration: 5m}
- analysis:
templates: [{templateName: success-rate}]
- setWeight: 25
- pause: {duration: 10m}
- setWeight: 100
# analysis fails -> automatic rollback to stable ReplicaSetPair canaries with feature flags for instant kill-switches independent of deploys. The tradeoff: progressive delivery adds pipeline complexity and slows full rollout, but that latency is cheap insurance against a full-fleet outage.
26For Meta Platforms's feed ranking requests, how would you design container orchestration and Kubernetes operations so the platform remains reliable during misinformation spread? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on container orchestration and Kubernetes operations, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Kubernetes schedules containers against declared desired state and continuously reconciles reality toward it. Operationally you care about resource requests/limits (for scheduling and stability), probes (liveness restarts a hung pod, readiness gates traffic), and disruption controls (PodDisruptionBudgets) so maintenance doesn't take down a service. Getting requests/limits right is the single biggest lever for reliability and cost.
A production-grade deployment sets these explicitly:
spec:
containers:
- name: api
resources:
requests: {cpu: "250m", memory: "256Mi"}
limits: {cpu: "1", memory: "512Mi"}
readinessProbe:
httpGet: {path: /healthz, port: 8080}
periodSeconds: 5
livenessProbe:
httpGet: {path: /livez, port: 8080}
initialDelaySeconds: 15
---
apiVersion: policy/v1
kind: PodDisruptionBudget
spec: {minAvailable: 2, selector: {matchLabels: {app: api}}}Watch for OOMKills (raise memory limits or fix leaks) and CPU throttling (limits set too low). The tradeoff with tight limits is bin-packing efficiency vs. the risk of throttling under burst — set requests from p50 and limits from p99 of observed usage.
27Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for container orchestration and Kubernetes operations in ads delivery platforms, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on container orchestration and Kubernetes operations, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Kubernetes schedules containers against declared desired state and continuously reconciles reality toward it. Operationally you care about resource requests/limits (for scheduling and stability), probes (liveness restarts a hung pod, readiness gates traffic), and disruption controls (PodDisruptionBudgets) so maintenance doesn't take down a service. Getting requests/limits right is the single biggest lever for reliability and cost.
A production-grade deployment sets these explicitly:
spec:
containers:
- name: api
resources:
requests: {cpu: "250m", memory: "256Mi"}
limits: {cpu: "1", memory: "512Mi"}
readinessProbe:
httpGet: {path: /healthz, port: 8080}
periodSeconds: 5
livenessProbe:
httpGet: {path: /livez, port: 8080}
initialDelaySeconds: 15
---
apiVersion: policy/v1
kind: PodDisruptionBudget
spec: {minAvailable: 2, selector: {matchLabels: {app: api}}}Watch for OOMKills (raise memory limits or fix leaks) and CPU throttling (limits set too low). The tradeoff with tight limits is bin-packing efficiency vs. the risk of throttling under burst — set requests from p50 and limits from p99 of observed usage.
28A production incident affects syncing a VR device: latency is rising, error rates are elevated, and ad-delivery regression is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on container orchestration and Kubernetes operations, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Start by defining Service Level Indicators (SLIs) that reflect the user's experience — typically availability (successful requests / total) and latency (proportion of requests under a threshold). Turn each SLI into a Service Level Objective (SLO), e.g. 99.9% of requests succeed over a rolling 30 days. The error budget is simply 100% minus the SLO: at 99.9% you may 'spend' 0.1% of requests failing (~43 minutes/month). When the budget is healthy you ship fast; when it's exhausted you freeze risky changes and prioritize reliability.
Measure SLIs from real telemetry, not host health. A request-based availability SLI in Prometheus looks like this:
# 30-day availability SLI (ratio of good requests)
sum(rate(http_requests_total{code!~"5.."}[30d]))
/
sum(rate(http_requests_total[30d]))
# Fast + slow burn-rate alert on the error budget (multi-window)
(
sum(rate(http_requests_total{code=~"5.."}[1h]))
/ sum(rate(http_requests_total[1h]))
) > (14.4 * 0.001) # 14.4x burn of a 0.1% budget => pageUse multi-window, multi-burn-rate alerts (fast window to catch acute outages, slow window to catch slow burns) so you page on symptoms that actually threaten the budget, not on every blip. Assign each SLO a clear owner and a runbook. The tradeoff: tighter SLOs (more nines) cost exponentially more in redundancy and toil, so set them from real user need, not vanity.
29What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying container orchestration and Kubernetes operations to Meta Platforms's content moderation pipelines?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on container orchestration and Kubernetes operations, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Kubernetes schedules containers against declared desired state and continuously reconciles reality toward it. Operationally you care about resource requests/limits (for scheduling and stability), probes (liveness restarts a hung pod, readiness gates traffic), and disruption controls (PodDisruptionBudgets) so maintenance doesn't take down a service. Getting requests/limits right is the single biggest lever for reliability and cost.
A production-grade deployment sets these explicitly:
spec:
containers:
- name: api
resources:
requests: {cpu: "250m", memory: "256Mi"}
limits: {cpu: "1", memory: "512Mi"}
readinessProbe:
httpGet: {path: /healthz, port: 8080}
periodSeconds: 5
livenessProbe:
httpGet: {path: /livez, port: 8080}
initialDelaySeconds: 15
---
apiVersion: policy/v1
kind: PodDisruptionBudget
spec: {minAvailable: 2, selector: {matchLabels: {app: api}}}Watch for OOMKills (raise memory limits or fix leaks) and CPU throttling (limits set too low). The tradeoff with tight limits is bin-packing efficiency vs. the risk of throttling under burst — set requests from p50 and limits from p99 of observed usage.
30Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize container orchestration and Kubernetes operations for social graph updates.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on container orchestration and Kubernetes operations, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Kubernetes schedules containers against declared desired state and continuously reconciles reality toward it. Operationally you care about resource requests/limits (for scheduling and stability), probes (liveness restarts a hung pod, readiness gates traffic), and disruption controls (PodDisruptionBudgets) so maintenance doesn't take down a service. Getting requests/limits right is the single biggest lever for reliability and cost.
A production-grade deployment sets these explicitly:
spec:
containers:
- name: api
resources:
requests: {cpu: "250m", memory: "256Mi"}
limits: {cpu: "1", memory: "512Mi"}
readinessProbe:
httpGet: {path: /healthz, port: 8080}
periodSeconds: 5
livenessProbe:
httpGet: {path: /livez, port: 8080}
initialDelaySeconds: 15
---
apiVersion: policy/v1
kind: PodDisruptionBudget
spec: {minAvailable: 2, selector: {matchLabels: {app: api}}}Watch for OOMKills (raise memory limits or fix leaks) and CPU throttling (limits set too low). The tradeoff with tight limits is bin-packing efficiency vs. the risk of throttling under burst — set requests from p50 and limits from p99 of observed usage.
31For Meta Platforms's messaging delivery, how would you design CI/CD pipeline reliability and artifact promotion so the platform remains reliable during messaging delay? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on CI/CD pipeline reliability and artifact promotion, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
A reliable pipeline is deterministic and promotes a single immutable artifact through environments rather than rebuilding per stage. Build once, tag by content/digest, run tests, then promote the exact same image from dev to staging to prod. This eliminates 'works in staging, breaks in prod' drift and makes rollbacks a matter of re-pointing to a prior digest.
A promotion flow pins the image by digest, not a mutable tag:
# GitHub Actions: build once, promote by digest
jobs:
build:
steps:
- id: push
run: |
docker build -t $REG/api:$SHA .
docker push $REG/api:$SHA
echo "digest=$(docker inspect --format='{{index .RepoDigests 0}}' $REG/api:$SHA)" >> $GITHUB_OUTPUT
deploy-prod:
needs: build
environment: production # requires approval
steps:
- run: kubectl set image deploy/api api=${{ needs.build.outputs.digest }}Add flake control: retry only known-flaky integration tests, quarantine chronic offenders, and require green required-checks before merge. The tradeoff is speed vs. safety — parallelize and cache to keep pipelines fast so engineers don't route around them.
32Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for CI/CD pipeline reliability and artifact promotion in messaging reliability systems, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on CI/CD pipeline reliability and artifact promotion, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
A reliable pipeline is deterministic and promotes a single immutable artifact through environments rather than rebuilding per stage. Build once, tag by content/digest, run tests, then promote the exact same image from dev to staging to prod. This eliminates 'works in staging, breaks in prod' drift and makes rollbacks a matter of re-pointing to a prior digest.
A promotion flow pins the image by digest, not a mutable tag:
# GitHub Actions: build once, promote by digest
jobs:
build:
steps:
- id: push
run: |
docker build -t $REG/api:$SHA .
docker push $REG/api:$SHA
echo "digest=$(docker inspect --format='{{index .RepoDigests 0}}' $REG/api:$SHA)" >> $GITHUB_OUTPUT
deploy-prod:
needs: build
environment: production # requires approval
steps:
- run: kubectl set image deploy/api api=${{ needs.build.outputs.digest }}Add flake control: retry only known-flaky integration tests, quarantine chronic offenders, and require green required-checks before merge. The tradeoff is speed vs. safety — parallelize and cache to keep pipelines fast so engineers don't route around them.
33A production incident affects loading an Instagram feed: latency is rising, error rates are elevated, and privacy-sensitive social graph exposure is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on CI/CD pipeline reliability and artifact promotion, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Effective incident response separates roles: an Incident Commander (IC) owns coordination and decisions, an Operations lead drives the technical fix, and a Communications lead handles stakeholders. The goal is to restore service first (mitigate) and diagnose root cause later. Declare an incident early with a clear severity, open a dedicated channel/bridge, and keep a running timeline.
Mitigation usually means the fastest safe reversal: roll back the last deploy, shift traffic away from a bad region, or disable a feature flag. A rollback via a deployment tool:
# Roll back a Kubernetes deployment to the previous known-good revision
kubectl rollout undo deployment/checkout-api
kubectl rollout status deployment/checkout-api --timeout=120s
# Or shift traffic away from an unhealthy region at the LB/DNS layer
aws elbv2 modify-target-group-attributes \
--target-group-arn $TG --attributes Key=deregistration_delay.timeout_seconds,Value=5After recovery, run a blameless postmortem: timeline, contributing factors, what detected it, and concrete action items with owners. The tradeoff to manage is speed vs. certainty — mitigate on symptoms quickly rather than waiting for a full root cause during a customer-facing outage.
34What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying CI/CD pipeline reliability and artifact promotion to Meta Platforms's Reality Labs device services?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on CI/CD pipeline reliability and artifact promotion, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
A reliable pipeline is deterministic and promotes a single immutable artifact through environments rather than rebuilding per stage. Build once, tag by content/digest, run tests, then promote the exact same image from dev to staging to prod. This eliminates 'works in staging, breaks in prod' drift and makes rollbacks a matter of re-pointing to a prior digest.
A promotion flow pins the image by digest, not a mutable tag:
# GitHub Actions: build once, promote by digest
jobs:
build:
steps:
- id: push
run: |
docker build -t $REG/api:$SHA .
docker push $REG/api:$SHA
echo "digest=$(docker inspect --format='{{index .RepoDigests 0}}' $REG/api:$SHA)" >> $GITHUB_OUTPUT
deploy-prod:
needs: build
environment: production # requires approval
steps:
- run: kubectl set image deploy/api api=${{ needs.build.outputs.digest }}Add flake control: retry only known-flaky integration tests, quarantine chronic offenders, and require green required-checks before merge. The tradeoff is speed vs. safety — parallelize and cache to keep pipelines fast so engineers don't route around them.
35Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize CI/CD pipeline reliability and artifact promotion for ad impressions.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on CI/CD pipeline reliability and artifact promotion, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
A reliable pipeline is deterministic and promotes a single immutable artifact through environments rather than rebuilding per stage. Build once, tag by content/digest, run tests, then promote the exact same image from dev to staging to prod. This eliminates 'works in staging, breaks in prod' drift and makes rollbacks a matter of re-pointing to a prior digest.
A promotion flow pins the image by digest, not a mutable tag:
# GitHub Actions: build once, promote by digest
jobs:
build:
steps:
- id: push
run: |
docker build -t $REG/api:$SHA .
docker push $REG/api:$SHA
echo "digest=$(docker inspect --format='{{index .RepoDigests 0}}' $REG/api:$SHA)" >> $GITHUB_OUTPUT
deploy-prod:
needs: build
environment: production # requires approval
steps:
- run: kubectl set image deploy/api api=${{ needs.build.outputs.digest }}Add flake control: retry only known-flaky integration tests, quarantine chronic offenders, and require green required-checks before merge. The tradeoff is speed vs. safety — parallelize and cache to keep pipelines fast so engineers don't route around them.
36For Meta Platforms's social graph updates, how would you design infrastructure as code and configuration drift control so the platform remains reliable during ad-delivery regression? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on infrastructure as code and configuration drift control, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Infrastructure as Code (IaC) makes infrastructure declarative, versioned, and reviewable. Drift — when live infrastructure diverges from code due to manual changes — is the enemy; you detect it by regularly running a plan/diff and alerting on any non-empty diff, and you prevent it by removing console write access and routing all changes through the pipeline.
A Terraform drift check you can run in CI on a schedule:
terraform init -input=false
# -detailed-exitcode: 0=no changes, 2=drift detected, 1=error
terraform plan -detailed-exitcode -out=tfplan
code=$?
if [ $code -eq 2 ]; then
echo "::warning::Infrastructure drift detected"
terraform show -no-color tfplan # post the diff to Slack/PR
exit 1
fiKeep state locked (remote backend with locking) so concurrent applies can't corrupt it, and use modules for reuse. The tradeoff: strict IaC slows one-off emergency fixes, so provide a documented break-glass path that still reconciles back into code afterward.
37Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for infrastructure as code and configuration drift control in content moderation pipelines, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on infrastructure as code and configuration drift control, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Infrastructure as Code (IaC) makes infrastructure declarative, versioned, and reviewable. Drift — when live infrastructure diverges from code due to manual changes — is the enemy; you detect it by regularly running a plan/diff and alerting on any non-empty diff, and you prevent it by removing console write access and routing all changes through the pipeline.
A Terraform drift check you can run in CI on a schedule:
terraform init -input=false
# -detailed-exitcode: 0=no changes, 2=drift detected, 1=error
terraform plan -detailed-exitcode -out=tfplan
code=$?
if [ $code -eq 2 ]; then
echo "::warning::Infrastructure drift detected"
terraform show -no-color tfplan # post the diff to Slack/PR
exit 1
fiKeep state locked (remote backend with locking) so concurrent applies can't corrupt it, and use modules for reuse. The tradeoff: strict IaC slows one-off emergency fixes, so provide a documented break-glass path that still reconciles back into code afterward.
38A production incident affects sending a WhatsApp message: latency is rising, error rates are elevated, and abuse traffic spike is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on infrastructure as code and configuration drift control, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Observability rests on three pillars: metrics (cheap, aggregate time-series for alerting and dashboards), logs (high-cardinality event detail for debugging), and traces (request flow across services to find where latency is spent). Profiling adds a fourth dimension — CPU/memory hot paths inside a process. The art is choosing the right pillar per question: alert on metrics, debug with traces, then drill into logs or profiles.
Instrument with OpenTelemetry so you emit all three from one SDK. A minimal trace + metric in code:
from opentelemetry import trace, metrics
tracer = trace.get_tracer("checkout")
meter = metrics.get_meter("checkout")
latency = meter.create_histogram("checkout.latency", unit="ms")
with tracer.start_as_current_span("process_order") as span:
span.set_attribute("order.id", order_id)
t0 = time.time()
result = charge_and_fulfill(order)
latency.record((time.time() - t0) * 1000, {"status": result.status})Control cost with sampling (tail-based sampling keeps slow/error traces) and cardinality limits on labels — an unbounded label like user_id will blow up your metrics store. Correlate the pillars with a shared trace_id so you can jump from a latency spike to the exact slow trace to its logs.
39What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying infrastructure as code and configuration drift control to Meta Platforms's social feed ranking services?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on infrastructure as code and configuration drift control, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Infrastructure as Code (IaC) makes infrastructure declarative, versioned, and reviewable. Drift — when live infrastructure diverges from code due to manual changes — is the enemy; you detect it by regularly running a plan/diff and alerting on any non-empty diff, and you prevent it by removing console write access and routing all changes through the pipeline.
A Terraform drift check you can run in CI on a schedule:
terraform init -input=false
# -detailed-exitcode: 0=no changes, 2=drift detected, 1=error
terraform plan -detailed-exitcode -out=tfplan
code=$?
if [ $code -eq 2 ]; then
echo "::warning::Infrastructure drift detected"
terraform show -no-color tfplan # post the diff to Slack/PR
exit 1
fiKeep state locked (remote backend with locking) so concurrent applies can't corrupt it, and use modules for reuse. The tradeoff: strict IaC slows one-off emergency fixes, so provide a documented break-glass path that still reconciles back into code afterward.
40Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize infrastructure as code and configuration drift control for content moderation queues.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on infrastructure as code and configuration drift control, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Infrastructure as Code (IaC) makes infrastructure declarative, versioned, and reviewable. Drift — when live infrastructure diverges from code due to manual changes — is the enemy; you detect it by regularly running a plan/diff and alerting on any non-empty diff, and you prevent it by removing console write access and routing all changes through the pipeline.
A Terraform drift check you can run in CI on a schedule:
terraform init -input=false
# -detailed-exitcode: 0=no changes, 2=drift detected, 1=error
terraform plan -detailed-exitcode -out=tfplan
code=$?
if [ $code -eq 2 ]; then
echo "::warning::Infrastructure drift detected"
terraform show -no-color tfplan # post the diff to Slack/PR
exit 1
fiKeep state locked (remote backend with locking) so concurrent applies can't corrupt it, and use modules for reuse. The tradeoff: strict IaC slows one-off emergency fixes, so provide a documented break-glass path that still reconciles back into code afterward.
41For Meta Platforms's ad impressions, how would you design cloud networking, DNS, CDN, and edge routing so the platform remains reliable during privacy-sensitive social graph exposure? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on cloud networking, DNS, CDN, and edge routing, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Edge architecture layers DNS (resolves names, supports geo/weighted/failover routing), a CDN (caches static and cacheable dynamic content near users), and a load balancer (health-checks and distributes to origins). Latency and availability both improve when you terminate TLS and serve cache hits at the edge, and when DNS health checks steer traffic away from a failed region automatically.
Weighted + health-checked DNS failover in Route 53:
{
"Name": "api.example.com",
"Type": "A",
"SetIdentifier": "us-east-primary",
"Failover": "PRIMARY",
"AliasTarget": {"DNSName": "lb-use1.example.com", "EvaluateTargetHealth": true},
"HealthCheckId": "hc-primary-123"
}Mind DNS TTLs: low TTL enables fast failover but increases resolver query load; high TTL caches better but slows recovery. Set cache-control headers deliberately so the CDN caches what's safe and revalidates what isn't.
42Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for cloud networking, DNS, CDN, and edge routing in Reality Labs device services, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on cloud networking, DNS, CDN, and edge routing, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Edge architecture layers DNS (resolves names, supports geo/weighted/failover routing), a CDN (caches static and cacheable dynamic content near users), and a load balancer (health-checks and distributes to origins). Latency and availability both improve when you terminate TLS and serve cache hits at the edge, and when DNS health checks steer traffic away from a failed region automatically.
Weighted + health-checked DNS failover in Route 53:
{
"Name": "api.example.com",
"Type": "A",
"SetIdentifier": "us-east-primary",
"Failover": "PRIMARY",
"AliasTarget": {"DNSName": "lb-use1.example.com", "EvaluateTargetHealth": true},
"HealthCheckId": "hc-primary-123"
}Mind DNS TTLs: low TTL enables fast failover but increases resolver query load; high TTL caches better but slows recovery. Set cache-control headers deliberately so the CDN caches what's safe and revalidates what isn't.
43A production incident affects serving a targeted ad: latency is rising, error rates are elevated, and misinformation spread is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on cloud networking, DNS, CDN, and edge routing, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Capacity planning turns a demand forecast into a resourcing plan with headroom. Start from historical utilization (CPU, memory, QPS, connections), model growth (organic + known events like launches or seasonal peaks), and add a safety margin so you can absorb spikes and failover load. The core equation: required capacity = peak forecast demand / target utilization, plus N+1 (or N+2) redundancy for failures.
A simple forecast fits a trend to historical peaks and projects forward:
import numpy as np
# weekly peak QPS over the last 12 weeks
peaks = np.array([120,128,131,140,138,150,159,165,171,180,188,196])
weeks = np.arange(len(peaks))
m, b = np.polyfit(weeks, peaks, 1) # linear trend
forecast_next = m * (len(peaks) + 4) + b # 4 weeks out
target_util = 0.6 # keep 40% headroom
capacity_qps = forecast_next / target_util
print(f"Provision for {capacity_qps:.0f} QPS")Validate the plan with load tests at forecast+margin, and prefer autoscaling for elastic tiers so you don't statically over-provision. The tradeoff is cost vs. risk: too much headroom wastes money, too little risks saturation during the exact peak you were planning for.
44What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying cloud networking, DNS, CDN, and edge routing to Meta Platforms's ads delivery platforms?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on cloud networking, DNS, CDN, and edge routing, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Edge architecture layers DNS (resolves names, supports geo/weighted/failover routing), a CDN (caches static and cacheable dynamic content near users), and a load balancer (health-checks and distributes to origins). Latency and availability both improve when you terminate TLS and serve cache hits at the edge, and when DNS health checks steer traffic away from a failed region automatically.
Weighted + health-checked DNS failover in Route 53:
{
"Name": "api.example.com",
"Type": "A",
"SetIdentifier": "us-east-primary",
"Failover": "PRIMARY",
"AliasTarget": {"DNSName": "lb-use1.example.com", "EvaluateTargetHealth": true},
"HealthCheckId": "hc-primary-123"
}Mind DNS TTLs: low TTL enables fast failover but increases resolver query load; high TTL caches better but slows recovery. Set cache-control headers deliberately so the CDN caches what's safe and revalidates what isn't.
45Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize cloud networking, DNS, CDN, and edge routing for feed ranking requests.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on cloud networking, DNS, CDN, and edge routing, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Edge architecture layers DNS (resolves names, supports geo/weighted/failover routing), a CDN (caches static and cacheable dynamic content near users), and a load balancer (health-checks and distributes to origins). Latency and availability both improve when you terminate TLS and serve cache hits at the edge, and when DNS health checks steer traffic away from a failed region automatically.
Weighted + health-checked DNS failover in Route 53:
{
"Name": "api.example.com",
"Type": "A",
"SetIdentifier": "us-east-primary",
"Failover": "PRIMARY",
"AliasTarget": {"DNSName": "lb-use1.example.com", "EvaluateTargetHealth": true},
"HealthCheckId": "hc-primary-123"
}Mind DNS TTLs: low TTL enables fast failover but increases resolver query load; high TTL caches better but slows recovery. Set cache-control headers deliberately so the CDN caches what's safe and revalidates what isn't.
46For Meta Platforms's content moderation queues, how would you design multi-region disaster recovery and failover so the platform remains reliable during abuse traffic spike? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on multi-region disaster recovery and failover, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
DR design is driven by two numbers: RPO (how much data loss is acceptable) and RTO (how fast you must recover). These dictate the architecture — active-active (near-zero RPO/RTO, highest cost/complexity), active-passive with warm standby (minutes), or backup-restore (hours). You replicate data across regions and provide a mechanism to redirect traffic when a region fails.
A failover runbook automates promotion of the standby and traffic shift:
# Promote the standby database replica in the DR region
aws rds promote-read-replica --db-instance-identifier orders-dr
# Wait until available, then repoint the app's DNS/endpoint
aws rds wait db-instance-available --db-instance-identifier orders-dr
# Shift traffic: flip the weighted/failover DNS record to the DR LB
aws route53 change-resource-record-sets --hosted-zone-id $ZONE \
--change-batch file://failover-to-dr.jsonTest DR regularly with game days — an untested DR plan is a liability. The central tradeoff is cost vs. RPO/RTO: active-active doubles infra spend but survives a full region loss with no downtime.
47Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for multi-region disaster recovery and failover in social feed ranking services, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on multi-region disaster recovery and failover, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
DR design is driven by two numbers: RPO (how much data loss is acceptable) and RTO (how fast you must recover). These dictate the architecture — active-active (near-zero RPO/RTO, highest cost/complexity), active-passive with warm standby (minutes), or backup-restore (hours). You replicate data across regions and provide a mechanism to redirect traffic when a region fails.
A failover runbook automates promotion of the standby and traffic shift:
# Promote the standby database replica in the DR region
aws rds promote-read-replica --db-instance-identifier orders-dr
# Wait until available, then repoint the app's DNS/endpoint
aws rds wait db-instance-available --db-instance-identifier orders-dr
# Shift traffic: flip the weighted/failover DNS record to the DR LB
aws route53 change-resource-record-sets --hosted-zone-id $ZONE \
--change-batch file://failover-to-dr.jsonTest DR regularly with game days — an untested DR plan is a liability. The central tradeoff is cost vs. RPO/RTO: active-active doubles infra spend but survives a full region loss with no downtime.
48A production incident affects moderating harmful content: latency is rising, error rates are elevated, and messaging delay is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on multi-region disaster recovery and failover, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Progressive delivery reduces blast radius by rolling a change to a small slice of traffic first (canary), watching health signals, then widening only if the canary is healthy. Combined with automated rollback, a bad release affects 1-5% of users for minutes instead of everyone. The key is defining objective promotion criteria — error rate, latency, and business KPIs — that gate each step.
An Argo Rollouts canary spec encodes the steps and analysis:
apiVersion: argoproj.io/v1alpha1
kind: Rollout
spec:
strategy:
canary:
steps:
- setWeight: 5
- pause: {duration: 5m}
- analysis:
templates: [{templateName: success-rate}]
- setWeight: 25
- pause: {duration: 10m}
- setWeight: 100
# analysis fails -> automatic rollback to stable ReplicaSetPair canaries with feature flags for instant kill-switches independent of deploys. The tradeoff: progressive delivery adds pipeline complexity and slows full rollout, but that latency is cheap insurance against a full-fleet outage.
49What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying multi-region disaster recovery and failover to Meta Platforms's messaging reliability systems?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on multi-region disaster recovery and failover, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
DR design is driven by two numbers: RPO (how much data loss is acceptable) and RTO (how fast you must recover). These dictate the architecture — active-active (near-zero RPO/RTO, highest cost/complexity), active-passive with warm standby (minutes), or backup-restore (hours). You replicate data across regions and provide a mechanism to redirect traffic when a region fails.
A failover runbook automates promotion of the standby and traffic shift:
# Promote the standby database replica in the DR region
aws rds promote-read-replica --db-instance-identifier orders-dr
# Wait until available, then repoint the app's DNS/endpoint
aws rds wait db-instance-available --db-instance-identifier orders-dr
# Shift traffic: flip the weighted/failover DNS record to the DR LB
aws route53 change-resource-record-sets --hosted-zone-id $ZONE \
--change-batch file://failover-to-dr.jsonTest DR regularly with game days — an untested DR plan is a liability. The central tradeoff is cost vs. RPO/RTO: active-active doubles infra spend but survives a full region loss with no downtime.
50Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize multi-region disaster recovery and failover for messaging delivery.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on multi-region disaster recovery and failover, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
DR design is driven by two numbers: RPO (how much data loss is acceptable) and RTO (how fast you must recover). These dictate the architecture — active-active (near-zero RPO/RTO, highest cost/complexity), active-passive with warm standby (minutes), or backup-restore (hours). You replicate data across regions and provide a mechanism to redirect traffic when a region fails.
A failover runbook automates promotion of the standby and traffic shift:
# Promote the standby database replica in the DR region
aws rds promote-read-replica --db-instance-identifier orders-dr
# Wait until available, then repoint the app's DNS/endpoint
aws rds wait db-instance-available --db-instance-identifier orders-dr
# Shift traffic: flip the weighted/failover DNS record to the DR LB
aws route53 change-resource-record-sets --hosted-zone-id $ZONE \
--change-batch file://failover-to-dr.jsonTest DR regularly with game days — an untested DR plan is a liability. The central tradeoff is cost vs. RPO/RTO: active-active doubles infra spend but survives a full region loss with no downtime.
51For Meta Platforms's feed ranking requests, how would you design autoscaling, backpressure, and graceful degradation so the platform remains reliable during misinformation spread? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on autoscaling, backpressure, and graceful degradation, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Autoscaling matches capacity to demand, but scaling has lag, so systems also need backpressure (reject or queue excess load) and graceful degradation (shed non-essential work) to survive spikes faster than they can scale. The layered defense: HPA/cluster-autoscaler for sustained load, rate limiting and load shedding for instantaneous bursts, and fallbacks (cached/partial responses) when dependencies are slow.
A Kubernetes HPA on a custom QPS metric:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
scaleTargetRef: {kind: Deployment, name: api}
minReplicas: 3
maxReplicas: 40
metrics:
- type: Pods
pods:
metric: {name: http_requests_per_second}
target: {type: AverageValue, averageValue: "50"}
behavior:
scaleDown: {stabilizationWindowSeconds: 300}Combine with a concurrency limiter that returns 429 with Retry-After when saturated, so clients back off. The tradeoff: aggressive scale-up wastes money on transient spikes, while conservative scaling risks saturation — tune stabilization windows to your traffic shape.
52Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for autoscaling, backpressure, and graceful degradation in ads delivery platforms, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on autoscaling, backpressure, and graceful degradation, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Autoscaling matches capacity to demand, but scaling has lag, so systems also need backpressure (reject or queue excess load) and graceful degradation (shed non-essential work) to survive spikes faster than they can scale. The layered defense: HPA/cluster-autoscaler for sustained load, rate limiting and load shedding for instantaneous bursts, and fallbacks (cached/partial responses) when dependencies are slow.
A Kubernetes HPA on a custom QPS metric:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
scaleTargetRef: {kind: Deployment, name: api}
minReplicas: 3
maxReplicas: 40
metrics:
- type: Pods
pods:
metric: {name: http_requests_per_second}
target: {type: AverageValue, averageValue: "50"}
behavior:
scaleDown: {stabilizationWindowSeconds: 300}Combine with a concurrency limiter that returns 429 with Retry-After when saturated, so clients back off. The tradeoff: aggressive scale-up wastes money on transient spikes, while conservative scaling risks saturation — tune stabilization windows to your traffic shape.
53A production incident affects syncing a VR device: latency is rising, error rates are elevated, and ad-delivery regression is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on autoscaling, backpressure, and graceful degradation, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Start by defining Service Level Indicators (SLIs) that reflect the user's experience — typically availability (successful requests / total) and latency (proportion of requests under a threshold). Turn each SLI into a Service Level Objective (SLO), e.g. 99.9% of requests succeed over a rolling 30 days. The error budget is simply 100% minus the SLO: at 99.9% you may 'spend' 0.1% of requests failing (~43 minutes/month). When the budget is healthy you ship fast; when it's exhausted you freeze risky changes and prioritize reliability.
Measure SLIs from real telemetry, not host health. A request-based availability SLI in Prometheus looks like this:
# 30-day availability SLI (ratio of good requests)
sum(rate(http_requests_total{code!~"5.."}[30d]))
/
sum(rate(http_requests_total[30d]))
# Fast + slow burn-rate alert on the error budget (multi-window)
(
sum(rate(http_requests_total{code=~"5.."}[1h]))
/ sum(rate(http_requests_total[1h]))
) > (14.4 * 0.001) # 14.4x burn of a 0.1% budget => pageUse multi-window, multi-burn-rate alerts (fast window to catch acute outages, slow window to catch slow burns) so you page on symptoms that actually threaten the budget, not on every blip. Assign each SLO a clear owner and a runbook. The tradeoff: tighter SLOs (more nines) cost exponentially more in redundancy and toil, so set them from real user need, not vanity.
54What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying autoscaling, backpressure, and graceful degradation to Meta Platforms's content moderation pipelines?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on autoscaling, backpressure, and graceful degradation, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Autoscaling matches capacity to demand, but scaling has lag, so systems also need backpressure (reject or queue excess load) and graceful degradation (shed non-essential work) to survive spikes faster than they can scale. The layered defense: HPA/cluster-autoscaler for sustained load, rate limiting and load shedding for instantaneous bursts, and fallbacks (cached/partial responses) when dependencies are slow.
A Kubernetes HPA on a custom QPS metric:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
scaleTargetRef: {kind: Deployment, name: api}
minReplicas: 3
maxReplicas: 40
metrics:
- type: Pods
pods:
metric: {name: http_requests_per_second}
target: {type: AverageValue, averageValue: "50"}
behavior:
scaleDown: {stabilizationWindowSeconds: 300}Combine with a concurrency limiter that returns 429 with Retry-After when saturated, so clients back off. The tradeoff: aggressive scale-up wastes money on transient spikes, while conservative scaling risks saturation — tune stabilization windows to your traffic shape.
55Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize autoscaling, backpressure, and graceful degradation for social graph updates.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on autoscaling, backpressure, and graceful degradation, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Autoscaling matches capacity to demand, but scaling has lag, so systems also need backpressure (reject or queue excess load) and graceful degradation (shed non-essential work) to survive spikes faster than they can scale. The layered defense: HPA/cluster-autoscaler for sustained load, rate limiting and load shedding for instantaneous bursts, and fallbacks (cached/partial responses) when dependencies are slow.
A Kubernetes HPA on a custom QPS metric:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
scaleTargetRef: {kind: Deployment, name: api}
minReplicas: 3
maxReplicas: 40
metrics:
- type: Pods
pods:
metric: {name: http_requests_per_second}
target: {type: AverageValue, averageValue: "50"}
behavior:
scaleDown: {stabilizationWindowSeconds: 300}Combine with a concurrency limiter that returns 429 with Retry-After when saturated, so clients back off. The tradeoff: aggressive scale-up wastes money on transient spikes, while conservative scaling risks saturation — tune stabilization windows to your traffic shape.
56For Meta Platforms's messaging delivery, how would you design secrets management, identity, and zero-trust operations so the platform remains reliable during messaging delay? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on secrets management, identity, and zero-trust operations, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Secrets should never live in code or images. Store them in a dedicated secrets manager (Vault, AWS Secrets Manager), fetch them at runtime with a short-lived identity, and rotate them automatically. Zero-trust extends this: every request is authenticated and authorized regardless of network location, using workload identity (SPIFFE/IAM roles) rather than static network trust.
Fetch a rotated secret at runtime via workload identity instead of baking it in:
import boto3, json
# The pod/instance assumes an IAM role (IRSA / instance profile) - no static keys
sm = boto3.client("secretsmanager")
secret = json.loads(sm.get_secret_value(SecretId="prod/db/creds")["SecretString"])
conn = connect(user=secret["username"], password=secret["password"])
# Rotation is handled by Secrets Manager's Lambda; the app just re-fetches on auth failureEnforce least privilege (scoped IAM policies), mTLS between services, and audit every secret access. The tradeoff: short rotation windows and mTLS add operational overhead, but they shrink the blast radius of any single leaked credential to minutes.
57Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for secrets management, identity, and zero-trust operations in messaging reliability systems, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on secrets management, identity, and zero-trust operations, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Secrets should never live in code or images. Store them in a dedicated secrets manager (Vault, AWS Secrets Manager), fetch them at runtime with a short-lived identity, and rotate them automatically. Zero-trust extends this: every request is authenticated and authorized regardless of network location, using workload identity (SPIFFE/IAM roles) rather than static network trust.
Fetch a rotated secret at runtime via workload identity instead of baking it in:
import boto3, json
# The pod/instance assumes an IAM role (IRSA / instance profile) - no static keys
sm = boto3.client("secretsmanager")
secret = json.loads(sm.get_secret_value(SecretId="prod/db/creds")["SecretString"])
conn = connect(user=secret["username"], password=secret["password"])
# Rotation is handled by Secrets Manager's Lambda; the app just re-fetches on auth failureEnforce least privilege (scoped IAM policies), mTLS between services, and audit every secret access. The tradeoff: short rotation windows and mTLS add operational overhead, but they shrink the blast radius of any single leaked credential to minutes.
58A production incident affects loading an Instagram feed: latency is rising, error rates are elevated, and privacy-sensitive social graph exposure is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on secrets management, identity, and zero-trust operations, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Effective incident response separates roles: an Incident Commander (IC) owns coordination and decisions, an Operations lead drives the technical fix, and a Communications lead handles stakeholders. The goal is to restore service first (mitigate) and diagnose root cause later. Declare an incident early with a clear severity, open a dedicated channel/bridge, and keep a running timeline.
Mitigation usually means the fastest safe reversal: roll back the last deploy, shift traffic away from a bad region, or disable a feature flag. A rollback via a deployment tool:
# Roll back a Kubernetes deployment to the previous known-good revision
kubectl rollout undo deployment/checkout-api
kubectl rollout status deployment/checkout-api --timeout=120s
# Or shift traffic away from an unhealthy region at the LB/DNS layer
aws elbv2 modify-target-group-attributes \
--target-group-arn $TG --attributes Key=deregistration_delay.timeout_seconds,Value=5After recovery, run a blameless postmortem: timeline, contributing factors, what detected it, and concrete action items with owners. The tradeoff to manage is speed vs. certainty — mitigate on symptoms quickly rather than waiting for a full root cause during a customer-facing outage.
59What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying secrets management, identity, and zero-trust operations to Meta Platforms's Reality Labs device services?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on secrets management, identity, and zero-trust operations, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Secrets should never live in code or images. Store them in a dedicated secrets manager (Vault, AWS Secrets Manager), fetch them at runtime with a short-lived identity, and rotate them automatically. Zero-trust extends this: every request is authenticated and authorized regardless of network location, using workload identity (SPIFFE/IAM roles) rather than static network trust.
Fetch a rotated secret at runtime via workload identity instead of baking it in:
import boto3, json
# The pod/instance assumes an IAM role (IRSA / instance profile) - no static keys
sm = boto3.client("secretsmanager")
secret = json.loads(sm.get_secret_value(SecretId="prod/db/creds")["SecretString"])
conn = connect(user=secret["username"], password=secret["password"])
# Rotation is handled by Secrets Manager's Lambda; the app just re-fetches on auth failureEnforce least privilege (scoped IAM policies), mTLS between services, and audit every secret access. The tradeoff: short rotation windows and mTLS add operational overhead, but they shrink the blast radius of any single leaked credential to minutes.
60Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize secrets management, identity, and zero-trust operations for ad impressions.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on secrets management, identity, and zero-trust operations, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Secrets should never live in code or images. Store them in a dedicated secrets manager (Vault, AWS Secrets Manager), fetch them at runtime with a short-lived identity, and rotate them automatically. Zero-trust extends this: every request is authenticated and authorized regardless of network location, using workload identity (SPIFFE/IAM roles) rather than static network trust.
Fetch a rotated secret at runtime via workload identity instead of baking it in:
import boto3, json
# The pod/instance assumes an IAM role (IRSA / instance profile) - no static keys
sm = boto3.client("secretsmanager")
secret = json.loads(sm.get_secret_value(SecretId="prod/db/creds")["SecretString"])
conn = connect(user=secret["username"], password=secret["password"])
# Rotation is handled by Secrets Manager's Lambda; the app just re-fetches on auth failureEnforce least privilege (scoped IAM policies), mTLS between services, and audit every secret access. The tradeoff: short rotation windows and mTLS add operational overhead, but they shrink the blast radius of any single leaked credential to minutes.
61For Meta Platforms's social graph updates, how would you design compliance, auditability, and change management so the platform remains reliable during ad-delivery regression? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on compliance, auditability, and change management, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Compliance operationalizes controls: every production change is reviewed, approved, logged, and reversible, and every access to sensitive data is audited. The engineering translation is a pipeline that enforces peer review, records who deployed what and when, and ships immutable audit logs to a tamper-evident store. Change management balances control with velocity via risk-based approvals.
Emit a structured, immutable audit event on every privileged action:
{
"ts": "2026-08-31T10:04:22Z",
"actor": "deployer@ci",
"action": "deploy",
"resource": "checkout-api",
"artifact": "sha256:9f2c...",
"change_ticket": "CHG-10432",
"approver": "oncall-sre",
"result": "success"
}Ship these to an append-only, access-controlled sink (e.g. a WORM bucket or SIEM). The tradeoff: heavyweight change management slows emergency fixes, so define a break-glass process with mandatory retroactive review to keep both auditors and on-call engineers satisfied.
62Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for compliance, auditability, and change management in content moderation pipelines, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on compliance, auditability, and change management, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Compliance operationalizes controls: every production change is reviewed, approved, logged, and reversible, and every access to sensitive data is audited. The engineering translation is a pipeline that enforces peer review, records who deployed what and when, and ships immutable audit logs to a tamper-evident store. Change management balances control with velocity via risk-based approvals.
Emit a structured, immutable audit event on every privileged action:
{
"ts": "2026-08-31T10:04:22Z",
"actor": "deployer@ci",
"action": "deploy",
"resource": "checkout-api",
"artifact": "sha256:9f2c...",
"change_ticket": "CHG-10432",
"approver": "oncall-sre",
"result": "success"
}Ship these to an append-only, access-controlled sink (e.g. a WORM bucket or SIEM). The tradeoff: heavyweight change management slows emergency fixes, so define a break-glass process with mandatory retroactive review to keep both auditors and on-call engineers satisfied.
63A production incident affects sending a WhatsApp message: latency is rising, error rates are elevated, and abuse traffic spike is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on compliance, auditability, and change management, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Observability rests on three pillars: metrics (cheap, aggregate time-series for alerting and dashboards), logs (high-cardinality event detail for debugging), and traces (request flow across services to find where latency is spent). Profiling adds a fourth dimension — CPU/memory hot paths inside a process. The art is choosing the right pillar per question: alert on metrics, debug with traces, then drill into logs or profiles.
Instrument with OpenTelemetry so you emit all three from one SDK. A minimal trace + metric in code:
from opentelemetry import trace, metrics
tracer = trace.get_tracer("checkout")
meter = metrics.get_meter("checkout")
latency = meter.create_histogram("checkout.latency", unit="ms")
with tracer.start_as_current_span("process_order") as span:
span.set_attribute("order.id", order_id)
t0 = time.time()
result = charge_and_fulfill(order)
latency.record((time.time() - t0) * 1000, {"status": result.status})Control cost with sampling (tail-based sampling keeps slow/error traces) and cardinality limits on labels — an unbounded label like user_id will blow up your metrics store. Correlate the pillars with a shared trace_id so you can jump from a latency spike to the exact slow trace to its logs.
64What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying compliance, auditability, and change management to Meta Platforms's social feed ranking services?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on compliance, auditability, and change management, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Compliance operationalizes controls: every production change is reviewed, approved, logged, and reversible, and every access to sensitive data is audited. The engineering translation is a pipeline that enforces peer review, records who deployed what and when, and ships immutable audit logs to a tamper-evident store. Change management balances control with velocity via risk-based approvals.
Emit a structured, immutable audit event on every privileged action:
{
"ts": "2026-08-31T10:04:22Z",
"actor": "deployer@ci",
"action": "deploy",
"resource": "checkout-api",
"artifact": "sha256:9f2c...",
"change_ticket": "CHG-10432",
"approver": "oncall-sre",
"result": "success"
}Ship these to an append-only, access-controlled sink (e.g. a WORM bucket or SIEM). The tradeoff: heavyweight change management slows emergency fixes, so define a break-glass process with mandatory retroactive review to keep both auditors and on-call engineers satisfied.
65Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize compliance, auditability, and change management for content moderation queues.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on compliance, auditability, and change management, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Compliance operationalizes controls: every production change is reviewed, approved, logged, and reversible, and every access to sensitive data is audited. The engineering translation is a pipeline that enforces peer review, records who deployed what and when, and ships immutable audit logs to a tamper-evident store. Change management balances control with velocity via risk-based approvals.
Emit a structured, immutable audit event on every privileged action:
{
"ts": "2026-08-31T10:04:22Z",
"actor": "deployer@ci",
"action": "deploy",
"resource": "checkout-api",
"artifact": "sha256:9f2c...",
"change_ticket": "CHG-10432",
"approver": "oncall-sre",
"result": "success"
}Ship these to an append-only, access-controlled sink (e.g. a WORM bucket or SIEM). The tradeoff: heavyweight change management slows emergency fixes, so define a break-glass process with mandatory retroactive review to keep both auditors and on-call engineers satisfied.
66For Meta Platforms's ad impressions, how would you design load testing, performance engineering, and saturation analysis so the platform remains reliable during privacy-sensitive social graph exposure? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on load testing, performance engineering, and saturation analysis, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Load testing establishes how a system behaves as demand rises: you ramp traffic while watching latency percentiles and error rate to find the knee where the system saturates. The USE method (Utilization, Saturation, Errors) guides diagnosis — a resource is the bottleneck when its utilization is high AND its queue (saturation) is growing. You test to find the limit before users do.
A k6 ramping load test with latency thresholds:
import http from 'k6/http';
import { check } from 'k6';
export const options = {
stages: [
{ duration: '2m', target: 200 }, // ramp up
{ duration: '5m', target: 200 }, // hold
{ duration: '2m', target: 500 }, // push to saturation
],
thresholds: { http_req_duration: ['p(95)<400'], http_req_failed: ['rate<0.01'] },
};
export default function () {
const r = http.get('https://api.example.com/checkout');
check(r, { 'status 200': (res) => res.status === 200 });
}Profile the bottleneck (CPU, locks, DB connections) once you find saturation, and fix the constraining resource rather than blindly scaling out. The tradeoff: load testing production-like environments is expensive, so invest in it for launch-critical and peak-season paths.
67Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for load testing, performance engineering, and saturation analysis in Reality Labs device services, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on load testing, performance engineering, and saturation analysis, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Load testing establishes how a system behaves as demand rises: you ramp traffic while watching latency percentiles and error rate to find the knee where the system saturates. The USE method (Utilization, Saturation, Errors) guides diagnosis — a resource is the bottleneck when its utilization is high AND its queue (saturation) is growing. You test to find the limit before users do.
A k6 ramping load test with latency thresholds:
import http from 'k6/http';
import { check } from 'k6';
export const options = {
stages: [
{ duration: '2m', target: 200 }, // ramp up
{ duration: '5m', target: 200 }, // hold
{ duration: '2m', target: 500 }, // push to saturation
],
thresholds: { http_req_duration: ['p(95)<400'], http_req_failed: ['rate<0.01'] },
};
export default function () {
const r = http.get('https://api.example.com/checkout');
check(r, { 'status 200': (res) => res.status === 200 });
}Profile the bottleneck (CPU, locks, DB connections) once you find saturation, and fix the constraining resource rather than blindly scaling out. The tradeoff: load testing production-like environments is expensive, so invest in it for launch-critical and peak-season paths.
68A production incident affects serving a targeted ad: latency is rising, error rates are elevated, and misinformation spread is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on load testing, performance engineering, and saturation analysis, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Capacity planning turns a demand forecast into a resourcing plan with headroom. Start from historical utilization (CPU, memory, QPS, connections), model growth (organic + known events like launches or seasonal peaks), and add a safety margin so you can absorb spikes and failover load. The core equation: required capacity = peak forecast demand / target utilization, plus N+1 (or N+2) redundancy for failures.
A simple forecast fits a trend to historical peaks and projects forward:
import numpy as np
# weekly peak QPS over the last 12 weeks
peaks = np.array([120,128,131,140,138,150,159,165,171,180,188,196])
weeks = np.arange(len(peaks))
m, b = np.polyfit(weeks, peaks, 1) # linear trend
forecast_next = m * (len(peaks) + 4) + b # 4 weeks out
target_util = 0.6 # keep 40% headroom
capacity_qps = forecast_next / target_util
print(f"Provision for {capacity_qps:.0f} QPS")Validate the plan with load tests at forecast+margin, and prefer autoscaling for elastic tiers so you don't statically over-provision. The tradeoff is cost vs. risk: too much headroom wastes money, too little risks saturation during the exact peak you were planning for.
69What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying load testing, performance engineering, and saturation analysis to Meta Platforms's ads delivery platforms?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on load testing, performance engineering, and saturation analysis, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Load testing establishes how a system behaves as demand rises: you ramp traffic while watching latency percentiles and error rate to find the knee where the system saturates. The USE method (Utilization, Saturation, Errors) guides diagnosis — a resource is the bottleneck when its utilization is high AND its queue (saturation) is growing. You test to find the limit before users do.
A k6 ramping load test with latency thresholds:
import http from 'k6/http';
import { check } from 'k6';
export const options = {
stages: [
{ duration: '2m', target: 200 }, // ramp up
{ duration: '5m', target: 200 }, // hold
{ duration: '2m', target: 500 }, // push to saturation
],
thresholds: { http_req_duration: ['p(95)<400'], http_req_failed: ['rate<0.01'] },
};
export default function () {
const r = http.get('https://api.example.com/checkout');
check(r, { 'status 200': (res) => res.status === 200 });
}Profile the bottleneck (CPU, locks, DB connections) once you find saturation, and fix the constraining resource rather than blindly scaling out. The tradeoff: load testing production-like environments is expensive, so invest in it for launch-critical and peak-season paths.
70Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize load testing, performance engineering, and saturation analysis for feed ranking requests.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on load testing, performance engineering, and saturation analysis, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Load testing establishes how a system behaves as demand rises: you ramp traffic while watching latency percentiles and error rate to find the knee where the system saturates. The USE method (Utilization, Saturation, Errors) guides diagnosis — a resource is the bottleneck when its utilization is high AND its queue (saturation) is growing. You test to find the limit before users do.
A k6 ramping load test with latency thresholds:
import http from 'k6/http';
import { check } from 'k6';
export const options = {
stages: [
{ duration: '2m', target: 200 }, // ramp up
{ duration: '5m', target: 200 }, // hold
{ duration: '2m', target: 500 }, // push to saturation
],
thresholds: { http_req_duration: ['p(95)<400'], http_req_failed: ['rate<0.01'] },
};
export default function () {
const r = http.get('https://api.example.com/checkout');
check(r, { 'status 200': (res) => res.status === 200 });
}Profile the bottleneck (CPU, locks, DB connections) once you find saturation, and fix the constraining resource rather than blindly scaling out. The tradeoff: load testing production-like environments is expensive, so invest in it for launch-critical and peak-season paths.
71For Meta Platforms's content moderation queues, how would you design chaos engineering and resilience validation so the platform remains reliable during abuse traffic spike? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on chaos engineering and resilience validation, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Chaos engineering validates resilience by deliberately injecting failure — killing pods, adding latency, partitioning networks — and verifying the system degrades gracefully and recovers. You start from a hypothesis ('if one AZ fails, error rate stays under 1%'), run the experiment with a limited blast radius and an abort switch, and turn any surprise into a fix and an SLO check.
A controlled network-latency experiment with a blast-radius selector:
apiVersion: chaos-mesh.org/v1alpha1
kind: NetworkChaos
spec:
action: delay
mode: fixed-percent
value: "20" # only 20% of targets
selector:
namespaces: [payments]
labelSelectors: {app: charge-svc}
delay: {latency: "300ms", jitter: "50ms"}
duration: "5m"Run experiments in staging first, then production during business hours with on-call watching. The tradeoff: chaos in prod carries real risk, so always scope the blast radius, define steady-state metrics, and have an instant abort — the goal is to learn, not to cause the outage you were testing for.
72Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for chaos engineering and resilience validation in social feed ranking services, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on chaos engineering and resilience validation, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Chaos engineering validates resilience by deliberately injecting failure — killing pods, adding latency, partitioning networks — and verifying the system degrades gracefully and recovers. You start from a hypothesis ('if one AZ fails, error rate stays under 1%'), run the experiment with a limited blast radius and an abort switch, and turn any surprise into a fix and an SLO check.
A controlled network-latency experiment with a blast-radius selector:
apiVersion: chaos-mesh.org/v1alpha1
kind: NetworkChaos
spec:
action: delay
mode: fixed-percent
value: "20" # only 20% of targets
selector:
namespaces: [payments]
labelSelectors: {app: charge-svc}
delay: {latency: "300ms", jitter: "50ms"}
duration: "5m"Run experiments in staging first, then production during business hours with on-call watching. The tradeoff: chaos in prod carries real risk, so always scope the blast radius, define steady-state metrics, and have an instant abort — the goal is to learn, not to cause the outage you were testing for.
73A production incident affects moderating harmful content: latency is rising, error rates are elevated, and messaging delay is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on chaos engineering and resilience validation, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Progressive delivery reduces blast radius by rolling a change to a small slice of traffic first (canary), watching health signals, then widening only if the canary is healthy. Combined with automated rollback, a bad release affects 1-5% of users for minutes instead of everyone. The key is defining objective promotion criteria — error rate, latency, and business KPIs — that gate each step.
An Argo Rollouts canary spec encodes the steps and analysis:
apiVersion: argoproj.io/v1alpha1
kind: Rollout
spec:
strategy:
canary:
steps:
- setWeight: 5
- pause: {duration: 5m}
- analysis:
templates: [{templateName: success-rate}]
- setWeight: 25
- pause: {duration: 10m}
- setWeight: 100
# analysis fails -> automatic rollback to stable ReplicaSetPair canaries with feature flags for instant kill-switches independent of deploys. The tradeoff: progressive delivery adds pipeline complexity and slows full rollout, but that latency is cheap insurance against a full-fleet outage.
74What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying chaos engineering and resilience validation to Meta Platforms's messaging reliability systems?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on chaos engineering and resilience validation, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Chaos engineering validates resilience by deliberately injecting failure — killing pods, adding latency, partitioning networks — and verifying the system degrades gracefully and recovers. You start from a hypothesis ('if one AZ fails, error rate stays under 1%'), run the experiment with a limited blast radius and an abort switch, and turn any surprise into a fix and an SLO check.
A controlled network-latency experiment with a blast-radius selector:
apiVersion: chaos-mesh.org/v1alpha1
kind: NetworkChaos
spec:
action: delay
mode: fixed-percent
value: "20" # only 20% of targets
selector:
namespaces: [payments]
labelSelectors: {app: charge-svc}
delay: {latency: "300ms", jitter: "50ms"}
duration: "5m"Run experiments in staging first, then production during business hours with on-call watching. The tradeoff: chaos in prod carries real risk, so always scope the blast radius, define steady-state metrics, and have an instant abort — the goal is to learn, not to cause the outage you were testing for.
75Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize chaos engineering and resilience validation for messaging delivery.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on chaos engineering and resilience validation, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Chaos engineering validates resilience by deliberately injecting failure — killing pods, adding latency, partitioning networks — and verifying the system degrades gracefully and recovers. You start from a hypothesis ('if one AZ fails, error rate stays under 1%'), run the experiment with a limited blast radius and an abort switch, and turn any surprise into a fix and an SLO check.
A controlled network-latency experiment with a blast-radius selector:
apiVersion: chaos-mesh.org/v1alpha1
kind: NetworkChaos
spec:
action: delay
mode: fixed-percent
value: "20" # only 20% of targets
selector:
namespaces: [payments]
labelSelectors: {app: charge-svc}
delay: {latency: "300ms", jitter: "50ms"}
duration: "5m"Run experiments in staging first, then production during business hours with on-call watching. The tradeoff: chaos in prod carries real risk, so always scope the blast radius, define steady-state metrics, and have an instant abort — the goal is to learn, not to cause the outage you were testing for.
76For Meta Platforms's feed ranking requests, how would you design on-call health, runbooks, and operational maturity so the platform remains reliable during misinformation spread? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on on-call health, runbooks, and operational maturity, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Healthy on-call is sustainable: alerts are actionable (every page maps to a human decision), runbooks make responses repeatable, and toil is tracked and driven down. Measure on-call health with metrics like pages per shift, percentage of actionable alerts, and time-to-acknowledge. Non-actionable alerts are a bug — tune or delete them.
A runbook is executable documentation tied to the alert. Alert annotations should link straight to it:
- alert: HighErrorRate
expr: sum(rate(http_requests_total{code=~"5.."}[5m]))
/ sum(rate(http_requests_total[5m])) > 0.02
for: 5m
labels: {severity: page}
annotations:
summary: "5xx error rate above 2% for 5m"
runbook: "https://runbooks/svc/high-error-rate"
dashboard: "https://grafana/d/api-overview"Rotate fairly, budget for follow-the-sun where possible, and hold blameless reviews so people surface problems. The tradeoff: over-alerting causes fatigue and missed real incidents, so bias toward fewer, higher-signal, SLO-based pages.
77Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for on-call health, runbooks, and operational maturity in ads delivery platforms, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on on-call health, runbooks, and operational maturity, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Healthy on-call is sustainable: alerts are actionable (every page maps to a human decision), runbooks make responses repeatable, and toil is tracked and driven down. Measure on-call health with metrics like pages per shift, percentage of actionable alerts, and time-to-acknowledge. Non-actionable alerts are a bug — tune or delete them.
A runbook is executable documentation tied to the alert. Alert annotations should link straight to it:
- alert: HighErrorRate
expr: sum(rate(http_requests_total{code=~"5.."}[5m]))
/ sum(rate(http_requests_total[5m])) > 0.02
for: 5m
labels: {severity: page}
annotations:
summary: "5xx error rate above 2% for 5m"
runbook: "https://runbooks/svc/high-error-rate"
dashboard: "https://grafana/d/api-overview"Rotate fairly, budget for follow-the-sun where possible, and hold blameless reviews so people surface problems. The tradeoff: over-alerting causes fatigue and missed real incidents, so bias toward fewer, higher-signal, SLO-based pages.
78A production incident affects syncing a VR device: latency is rising, error rates are elevated, and ad-delivery regression is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on on-call health, runbooks, and operational maturity, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Start by defining Service Level Indicators (SLIs) that reflect the user's experience — typically availability (successful requests / total) and latency (proportion of requests under a threshold). Turn each SLI into a Service Level Objective (SLO), e.g. 99.9% of requests succeed over a rolling 30 days. The error budget is simply 100% minus the SLO: at 99.9% you may 'spend' 0.1% of requests failing (~43 minutes/month). When the budget is healthy you ship fast; when it's exhausted you freeze risky changes and prioritize reliability.
Measure SLIs from real telemetry, not host health. A request-based availability SLI in Prometheus looks like this:
# 30-day availability SLI (ratio of good requests)
sum(rate(http_requests_total{code!~"5.."}[30d]))
/
sum(rate(http_requests_total[30d]))
# Fast + slow burn-rate alert on the error budget (multi-window)
(
sum(rate(http_requests_total{code=~"5.."}[1h]))
/ sum(rate(http_requests_total[1h]))
) > (14.4 * 0.001) # 14.4x burn of a 0.1% budget => pageUse multi-window, multi-burn-rate alerts (fast window to catch acute outages, slow window to catch slow burns) so you page on symptoms that actually threaten the budget, not on every blip. Assign each SLO a clear owner and a runbook. The tradeoff: tighter SLOs (more nines) cost exponentially more in redundancy and toil, so set them from real user need, not vanity.
79What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying on-call health, runbooks, and operational maturity to Meta Platforms's content moderation pipelines?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on on-call health, runbooks, and operational maturity, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Healthy on-call is sustainable: alerts are actionable (every page maps to a human decision), runbooks make responses repeatable, and toil is tracked and driven down. Measure on-call health with metrics like pages per shift, percentage of actionable alerts, and time-to-acknowledge. Non-actionable alerts are a bug — tune or delete them.
A runbook is executable documentation tied to the alert. Alert annotations should link straight to it:
- alert: HighErrorRate
expr: sum(rate(http_requests_total{code=~"5.."}[5m]))
/ sum(rate(http_requests_total[5m])) > 0.02
for: 5m
labels: {severity: page}
annotations:
summary: "5xx error rate above 2% for 5m"
runbook: "https://runbooks/svc/high-error-rate"
dashboard: "https://grafana/d/api-overview"Rotate fairly, budget for follow-the-sun where possible, and hold blameless reviews so people surface problems. The tradeoff: over-alerting causes fatigue and missed real incidents, so bias toward fewer, higher-signal, SLO-based pages.
80Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize on-call health, runbooks, and operational maturity for social graph updates.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on on-call health, runbooks, and operational maturity, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Healthy on-call is sustainable: alerts are actionable (every page maps to a human decision), runbooks make responses repeatable, and toil is tracked and driven down. Measure on-call health with metrics like pages per shift, percentage of actionable alerts, and time-to-acknowledge. Non-actionable alerts are a bug — tune or delete them.
A runbook is executable documentation tied to the alert. Alert annotations should link straight to it:
- alert: HighErrorRate
expr: sum(rate(http_requests_total{code=~"5.."}[5m]))
/ sum(rate(http_requests_total[5m])) > 0.02
for: 5m
labels: {severity: page}
annotations:
summary: "5xx error rate above 2% for 5m"
runbook: "https://runbooks/svc/high-error-rate"
dashboard: "https://grafana/d/api-overview"Rotate fairly, budget for follow-the-sun where possible, and hold blameless reviews so people surface problems. The tradeoff: over-alerting causes fatigue and missed real incidents, so bias toward fewer, higher-signal, SLO-based pages.
81For Meta Platforms's messaging delivery, how would you design cost optimization and capacity efficiency so the platform remains reliable during messaging delay? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on cost optimization and capacity efficiency, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Cloud cost optimization follows a hierarchy: eliminate waste (idle and orphaned resources), right-size (match instance/pod size to real usage), buy commitments (savings plans/reserved capacity for steady baseload), and use spot/preemptible for fault-tolerant work. The discipline is attributing spend to teams via tagging so owners see and act on their own costs.
Right-sizing starts from observed utilization — find over-provisioned workloads:
# Containers using < 20% of requested CPU over 7 days (right-size candidates)
(
avg_over_time(rate(container_cpu_usage_seconds_total[5m])[7d:])
/
kube_pod_container_resource_requests{resource="cpu"}
) < 0.2Automate cleanup (TTLs on dev environments, deletion of unattached disks) and set budgets with alerts. The tradeoff: chasing every dollar adds engineering toil and can hurt reliability if you cut headroom too far — optimize the biggest line items first and keep enough margin for failover.
82Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for cost optimization and capacity efficiency in messaging reliability systems, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on cost optimization and capacity efficiency, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Cloud cost optimization follows a hierarchy: eliminate waste (idle and orphaned resources), right-size (match instance/pod size to real usage), buy commitments (savings plans/reserved capacity for steady baseload), and use spot/preemptible for fault-tolerant work. The discipline is attributing spend to teams via tagging so owners see and act on their own costs.
Right-sizing starts from observed utilization — find over-provisioned workloads:
# Containers using < 20% of requested CPU over 7 days (right-size candidates)
(
avg_over_time(rate(container_cpu_usage_seconds_total[5m])[7d:])
/
kube_pod_container_resource_requests{resource="cpu"}
) < 0.2Automate cleanup (TTLs on dev environments, deletion of unattached disks) and set budgets with alerts. The tradeoff: chasing every dollar adds engineering toil and can hurt reliability if you cut headroom too far — optimize the biggest line items first and keep enough margin for failover.
83A production incident affects loading an Instagram feed: latency is rising, error rates are elevated, and privacy-sensitive social graph exposure is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on cost optimization and capacity efficiency, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Effective incident response separates roles: an Incident Commander (IC) owns coordination and decisions, an Operations lead drives the technical fix, and a Communications lead handles stakeholders. The goal is to restore service first (mitigate) and diagnose root cause later. Declare an incident early with a clear severity, open a dedicated channel/bridge, and keep a running timeline.
Mitigation usually means the fastest safe reversal: roll back the last deploy, shift traffic away from a bad region, or disable a feature flag. A rollback via a deployment tool:
# Roll back a Kubernetes deployment to the previous known-good revision
kubectl rollout undo deployment/checkout-api
kubectl rollout status deployment/checkout-api --timeout=120s
# Or shift traffic away from an unhealthy region at the LB/DNS layer
aws elbv2 modify-target-group-attributes \
--target-group-arn $TG --attributes Key=deregistration_delay.timeout_seconds,Value=5After recovery, run a blameless postmortem: timeline, contributing factors, what detected it, and concrete action items with owners. The tradeoff to manage is speed vs. certainty — mitigate on symptoms quickly rather than waiting for a full root cause during a customer-facing outage.
84What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying cost optimization and capacity efficiency to Meta Platforms's Reality Labs device services?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on cost optimization and capacity efficiency, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Cloud cost optimization follows a hierarchy: eliminate waste (idle and orphaned resources), right-size (match instance/pod size to real usage), buy commitments (savings plans/reserved capacity for steady baseload), and use spot/preemptible for fault-tolerant work. The discipline is attributing spend to teams via tagging so owners see and act on their own costs.
Right-sizing starts from observed utilization — find over-provisioned workloads:
# Containers using < 20% of requested CPU over 7 days (right-size candidates)
(
avg_over_time(rate(container_cpu_usage_seconds_total[5m])[7d:])
/
kube_pod_container_resource_requests{resource="cpu"}
) < 0.2Automate cleanup (TTLs on dev environments, deletion of unattached disks) and set budgets with alerts. The tradeoff: chasing every dollar adds engineering toil and can hurt reliability if you cut headroom too far — optimize the biggest line items first and keep enough margin for failover.
85Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize cost optimization and capacity efficiency for ad impressions.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on cost optimization and capacity efficiency, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Cloud cost optimization follows a hierarchy: eliminate waste (idle and orphaned resources), right-size (match instance/pod size to real usage), buy commitments (savings plans/reserved capacity for steady baseload), and use spot/preemptible for fault-tolerant work. The discipline is attributing spend to teams via tagging so owners see and act on their own costs.
Right-sizing starts from observed utilization — find over-provisioned workloads:
# Containers using < 20% of requested CPU over 7 days (right-size candidates)
(
avg_over_time(rate(container_cpu_usage_seconds_total[5m])[7d:])
/
kube_pod_container_resource_requests{resource="cpu"}
) < 0.2Automate cleanup (TTLs on dev environments, deletion of unattached disks) and set budgets with alerts. The tradeoff: chasing every dollar adds engineering toil and can hurt reliability if you cut headroom too far — optimize the biggest line items first and keep enough margin for failover.
86For Meta Platforms's social graph updates, how would you design platform engineering and developer self-service so the platform remains reliable during ad-delivery regression? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on platform engineering and developer self-service, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Platform engineering builds an internal platform that turns common infrastructure needs into self-service, paved-road workflows — golden templates, a service catalog, and automated provisioning — so product teams ship without filing tickets or reinventing CI/CD, observability, and security. Success is measured by adoption and lead time, treating the platform as a product with internal customers.
A golden-path template scaffolds a new service with everything wired in:
# Backstage software template (excerpt)
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
spec:
steps:
- id: fetch
action: fetch:template
input: {url: ./skeleton, values: {name: '${{ parameters.name }}'}}
- id: publish
action: publish:github
input: {repoUrl: '${{ parameters.repoUrl }}'}
- id: register
action: catalog:register # auto-adds CI/CD, dashboards, on-callKeep the paved road optional-but-easy: teams can deviate, but the default gives them best practices for free. The tradeoff is standardization vs. flexibility — too rigid and teams route around the platform; too loose and you lose the consistency that makes it valuable.
87Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for platform engineering and developer self-service in content moderation pipelines, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on platform engineering and developer self-service, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Platform engineering builds an internal platform that turns common infrastructure needs into self-service, paved-road workflows — golden templates, a service catalog, and automated provisioning — so product teams ship without filing tickets or reinventing CI/CD, observability, and security. Success is measured by adoption and lead time, treating the platform as a product with internal customers.
A golden-path template scaffolds a new service with everything wired in:
# Backstage software template (excerpt)
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
spec:
steps:
- id: fetch
action: fetch:template
input: {url: ./skeleton, values: {name: '${{ parameters.name }}'}}
- id: publish
action: publish:github
input: {repoUrl: '${{ parameters.repoUrl }}'}
- id: register
action: catalog:register # auto-adds CI/CD, dashboards, on-callKeep the paved road optional-but-easy: teams can deviate, but the default gives them best practices for free. The tradeoff is standardization vs. flexibility — too rigid and teams route around the platform; too loose and you lose the consistency that makes it valuable.
88A production incident affects sending a WhatsApp message: latency is rising, error rates are elevated, and abuse traffic spike is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on platform engineering and developer self-service, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Observability rests on three pillars: metrics (cheap, aggregate time-series for alerting and dashboards), logs (high-cardinality event detail for debugging), and traces (request flow across services to find where latency is spent). Profiling adds a fourth dimension — CPU/memory hot paths inside a process. The art is choosing the right pillar per question: alert on metrics, debug with traces, then drill into logs or profiles.
Instrument with OpenTelemetry so you emit all three from one SDK. A minimal trace + metric in code:
from opentelemetry import trace, metrics
tracer = trace.get_tracer("checkout")
meter = metrics.get_meter("checkout")
latency = meter.create_histogram("checkout.latency", unit="ms")
with tracer.start_as_current_span("process_order") as span:
span.set_attribute("order.id", order_id)
t0 = time.time()
result = charge_and_fulfill(order)
latency.record((time.time() - t0) * 1000, {"status": result.status})Control cost with sampling (tail-based sampling keeps slow/error traces) and cardinality limits on labels — an unbounded label like user_id will blow up your metrics store. Correlate the pillars with a shared trace_id so you can jump from a latency spike to the exact slow trace to its logs.
89What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying platform engineering and developer self-service to Meta Platforms's social feed ranking services?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on platform engineering and developer self-service, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Platform engineering builds an internal platform that turns common infrastructure needs into self-service, paved-road workflows — golden templates, a service catalog, and automated provisioning — so product teams ship without filing tickets or reinventing CI/CD, observability, and security. Success is measured by adoption and lead time, treating the platform as a product with internal customers.
A golden-path template scaffolds a new service with everything wired in:
# Backstage software template (excerpt)
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
spec:
steps:
- id: fetch
action: fetch:template
input: {url: ./skeleton, values: {name: '${{ parameters.name }}'}}
- id: publish
action: publish:github
input: {repoUrl: '${{ parameters.repoUrl }}'}
- id: register
action: catalog:register # auto-adds CI/CD, dashboards, on-callKeep the paved road optional-but-easy: teams can deviate, but the default gives them best practices for free. The tradeoff is standardization vs. flexibility — too rigid and teams route around the platform; too loose and you lose the consistency that makes it valuable.
90Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize platform engineering and developer self-service for content moderation queues.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on platform engineering and developer self-service, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Platform engineering builds an internal platform that turns common infrastructure needs into self-service, paved-road workflows — golden templates, a service catalog, and automated provisioning — so product teams ship without filing tickets or reinventing CI/CD, observability, and security. Success is measured by adoption and lead time, treating the platform as a product with internal customers.
A golden-path template scaffolds a new service with everything wired in:
# Backstage software template (excerpt)
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
spec:
steps:
- id: fetch
action: fetch:template
input: {url: ./skeleton, values: {name: '${{ parameters.name }}'}}
- id: publish
action: publish:github
input: {repoUrl: '${{ parameters.repoUrl }}'}
- id: register
action: catalog:register # auto-adds CI/CD, dashboards, on-callKeep the paved road optional-but-easy: teams can deviate, but the default gives them best practices for free. The tradeoff is standardization vs. flexibility — too rigid and teams route around the platform; too loose and you lose the consistency that makes it valuable.
91For Meta Platforms's ad impressions, how would you design data pipeline reliability and batch/stream recovery so the platform remains reliable during privacy-sensitive social graph exposure? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on data pipeline reliability and batch/stream recovery, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Reliable data pipelines are idempotent, checkpointed, and observable. Idempotency (via dedup keys or upserts) means a retried or replayed batch produces the same result. Checkpointing lets a stream job resume from the last committed offset after a crash instead of reprocessing everything or losing data. You monitor freshness (lag) and completeness, not just job success.
A streaming consumer that commits offsets only after successful, idempotent writes:
for msg in consumer: # Kafka consumer, auto-commit disabled
event = parse(msg.value)
# idempotent upsert keyed by event id => safe to reprocess
db.upsert("events", key=event.id, values=event.to_row())
consumer.commit({msg.partition: msg.offset + 1}) # commit AFTER write
# On restart, consumption resumes from the last committed offset (at-least-once)For batch, make jobs re-runnable for a given partition/day and backfill by replaying that window. The tradeoff is exactly-once (complex, via transactional sinks) vs. at-least-once + idempotency (simpler, usually sufficient) — most pipelines choose the latter.
92Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for data pipeline reliability and batch/stream recovery in Reality Labs device services, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on data pipeline reliability and batch/stream recovery, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Reliable data pipelines are idempotent, checkpointed, and observable. Idempotency (via dedup keys or upserts) means a retried or replayed batch produces the same result. Checkpointing lets a stream job resume from the last committed offset after a crash instead of reprocessing everything or losing data. You monitor freshness (lag) and completeness, not just job success.
A streaming consumer that commits offsets only after successful, idempotent writes:
for msg in consumer: # Kafka consumer, auto-commit disabled
event = parse(msg.value)
# idempotent upsert keyed by event id => safe to reprocess
db.upsert("events", key=event.id, values=event.to_row())
consumer.commit({msg.partition: msg.offset + 1}) # commit AFTER write
# On restart, consumption resumes from the last committed offset (at-least-once)For batch, make jobs re-runnable for a given partition/day and backfill by replaying that window. The tradeoff is exactly-once (complex, via transactional sinks) vs. at-least-once + idempotency (simpler, usually sufficient) — most pipelines choose the latter.
93A production incident affects serving a targeted ad: latency is rising, error rates are elevated, and misinformation spread is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on data pipeline reliability and batch/stream recovery, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Capacity planning turns a demand forecast into a resourcing plan with headroom. Start from historical utilization (CPU, memory, QPS, connections), model growth (organic + known events like launches or seasonal peaks), and add a safety margin so you can absorb spikes and failover load. The core equation: required capacity = peak forecast demand / target utilization, plus N+1 (or N+2) redundancy for failures.
A simple forecast fits a trend to historical peaks and projects forward:
import numpy as np
# weekly peak QPS over the last 12 weeks
peaks = np.array([120,128,131,140,138,150,159,165,171,180,188,196])
weeks = np.arange(len(peaks))
m, b = np.polyfit(weeks, peaks, 1) # linear trend
forecast_next = m * (len(peaks) + 4) + b # 4 weeks out
target_util = 0.6 # keep 40% headroom
capacity_qps = forecast_next / target_util
print(f"Provision for {capacity_qps:.0f} QPS")Validate the plan with load tests at forecast+margin, and prefer autoscaling for elastic tiers so you don't statically over-provision. The tradeoff is cost vs. risk: too much headroom wastes money, too little risks saturation during the exact peak you were planning for.
94What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying data pipeline reliability and batch/stream recovery to Meta Platforms's ads delivery platforms?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on data pipeline reliability and batch/stream recovery, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Reliable data pipelines are idempotent, checkpointed, and observable. Idempotency (via dedup keys or upserts) means a retried or replayed batch produces the same result. Checkpointing lets a stream job resume from the last committed offset after a crash instead of reprocessing everything or losing data. You monitor freshness (lag) and completeness, not just job success.
A streaming consumer that commits offsets only after successful, idempotent writes:
for msg in consumer: # Kafka consumer, auto-commit disabled
event = parse(msg.value)
# idempotent upsert keyed by event id => safe to reprocess
db.upsert("events", key=event.id, values=event.to_row())
consumer.commit({msg.partition: msg.offset + 1}) # commit AFTER write
# On restart, consumption resumes from the last committed offset (at-least-once)For batch, make jobs re-runnable for a given partition/day and backfill by replaying that window. The tradeoff is exactly-once (complex, via transactional sinks) vs. at-least-once + idempotency (simpler, usually sufficient) — most pipelines choose the latter.
95Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize data pipeline reliability and batch/stream recovery for feed ranking requests.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on data pipeline reliability and batch/stream recovery, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Reliable data pipelines are idempotent, checkpointed, and observable. Idempotency (via dedup keys or upserts) means a retried or replayed batch produces the same result. Checkpointing lets a stream job resume from the last committed offset after a crash instead of reprocessing everything or losing data. You monitor freshness (lag) and completeness, not just job success.
A streaming consumer that commits offsets only after successful, idempotent writes:
for msg in consumer: # Kafka consumer, auto-commit disabled
event = parse(msg.value)
# idempotent upsert keyed by event id => safe to reprocess
db.upsert("events", key=event.id, values=event.to_row())
consumer.commit({msg.partition: msg.offset + 1}) # commit AFTER write
# On restart, consumption resumes from the last committed offset (at-least-once)For batch, make jobs re-runnable for a given partition/day and backfill by replaying that window. The tradeoff is exactly-once (complex, via transactional sinks) vs. at-least-once + idempotency (simpler, usually sufficient) — most pipelines choose the latter.
96For Meta Platforms's content moderation queues, how would you design ML platform reliability, model serving, and GPU/accelerator operations so the platform remains reliable during abuse traffic spike? Cover architecture, ownership, SLOs, runbooks, and failure modes.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on ML platform reliability, model serving, and GPU/accelerator operations, measurable outcomes, failure modes, and trade-offs. 1) Clarify the critical user journey, dependencies, traffic pattern, blast radius, and SLO targets. 2) Design for isolation, redundancy, capacity headroom, progressive rollout, rollback, and graceful degradation. 3) Define ownership: service owner, incident commander, escalation path, runbook owner, and postmortem owner. 4) Operationalize the SLO with burn-rate alerts, dashboards, release gates, and error-budget policy. 5) Validate with load tests, failover drills, chaos tests, and game days. I would close by saying reliability should be tied to customer impact, not uptime vanity metrics.
Serving ML reliably means treating the model as a versioned artifact behind a stable API, with autoscaling tuned for accelerator economics. GPUs are expensive and don't fractionally autoscale like CPU, so you batch requests to raise utilization, use separate pools per model size, and scale on queue depth / GPU utilization rather than CPU. Health checks must verify the model actually loaded, not just that the process is up.
A KServe/Kubernetes serving spec with GPU requests and a model-ready probe:
spec:
containers:
- name: model
image: registry/llm-serve:v2
resources:
limits: {nvidia.com/gpu: 1}
readinessProbe:
httpGet: {path: /v2/models/llm/ready, port: 8080}
initialDelaySeconds: 60 # model load takes time
env:
- {name: MAX_BATCH_SIZE, value: "16"}
- {name: BATCH_TIMEOUT_MS, value: "20"}Watch for GPU OOM (cap batch size / sequence length), cold-start latency (keep warm replicas), and drift in model quality. The tradeoff: larger batches raise throughput and GPU efficiency but add per-request latency — tune batch size and timeout to your latency SLO.
97Which SLIs, alerts, dashboards, logs, traces, and profiling signals would you define for ML platform reliability, model serving, and GPU/accelerator operations in social feed ranking services, and how would you prevent noisy paging?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on ML platform reliability, model serving, and GPU/accelerator operations, measurable outcomes, failure modes, and trade-offs. I would build metrics in layers. Symptom metrics: availability, p50/p95/p99 latency, error rate, saturation, queue depth, data freshness, and user-visible failure rate. Cause metrics: CPU, memory, disk, network, dependency latency, deploy version, quota pressure, and config changes. I would page only on sustained customer impact or multi-window burn-rate alerts, not every resource spike. Dashboards should connect golden signals to deploy history, dependency maps, and top regressions. Logs should be structured and correlated with traces; traces should expose tail latency and fan-out. Good observability reduces diagnosis time and alert noise.
Serving ML reliably means treating the model as a versioned artifact behind a stable API, with autoscaling tuned for accelerator economics. GPUs are expensive and don't fractionally autoscale like CPU, so you batch requests to raise utilization, use separate pools per model size, and scale on queue depth / GPU utilization rather than CPU. Health checks must verify the model actually loaded, not just that the process is up.
A KServe/Kubernetes serving spec with GPU requests and a model-ready probe:
spec:
containers:
- name: model
image: registry/llm-serve:v2
resources:
limits: {nvidia.com/gpu: 1}
readinessProbe:
httpGet: {path: /v2/models/llm/ready, port: 8080}
initialDelaySeconds: 60 # model load takes time
env:
- {name: MAX_BATCH_SIZE, value: "16"}
- {name: BATCH_TIMEOUT_MS, value: "20"}Watch for GPU OOM (cap batch size / sequence length), cold-start latency (keep warm replicas), and drift in model quality. The tradeoff: larger batches raise throughput and GPU efficiency but add per-request latency — tune batch size and timeout to your latency SLO.
98A production incident affects moderating harmful content: latency is rising, error rates are elevated, and messaging delay is suspected. Walk through triage, mitigation, rollback, and postmortem actions.Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on ML platform reliability, model serving, and GPU/accelerator operations, measurable outcomes, failure modes, and trade-offs. I would run this as an incident. First, declare severity, assign an incident commander, freeze risky changes, and confirm impact through SLIs. Then segment by region, version, dependency, tenant, traffic class, and recent deploys to narrow blast radius. Mitigation comes before root cause: rollback, shed noncritical load, fail over if safe, add capacity, or enable fallback. In parallel, capture evidence from metrics, logs, traces, config diffs, and deployment history. After recovery, I would write a blameless postmortem with timeline, contributing factors, detection gaps, owners, and deadlines.
Progressive delivery reduces blast radius by rolling a change to a small slice of traffic first (canary), watching health signals, then widening only if the canary is healthy. Combined with automated rollback, a bad release affects 1-5% of users for minutes instead of everyone. The key is defining objective promotion criteria — error rate, latency, and business KPIs — that gate each step.
An Argo Rollouts canary spec encodes the steps and analysis:
apiVersion: argoproj.io/v1alpha1
kind: Rollout
spec:
strategy:
canary:
steps:
- setWeight: 5
- pause: {duration: 5m}
- analysis:
templates: [{templateName: success-rate}]
- setWeight: 25
- pause: {duration: 10m}
- setWeight: 100
# analysis fails -> automatic rollback to stable ReplicaSetPair canaries with feature flags for instant kill-switches independent of deploys. The tradeoff: progressive delivery adds pipeline complexity and slows full rollout, but that latency is cheap insurance against a full-fleet outage.
99What trade-offs would you make between reliability, release velocity, cost, and operational complexity when applying ML platform reliability, model serving, and GPU/accelerator operations to Meta Platforms's messaging reliability systems?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on ML platform reliability, model serving, and GPU/accelerator operations, measurable outcomes, failure modes, and trade-offs. I would compare reliability, release velocity, cost, and operational complexity. For customer-critical paths, I would spend more on redundancy, headroom, release gates, rollback automation, and on-call readiness. For lower-risk internal paths, I would tolerate simpler controls and lower SLOs. Error budgets should drive the decision: healthy services can move faster; burned budgets shift attention to reliability. I would also discuss hidden costs such as pager load, noisy alerts, manual runbooks, and brittle automation. The right answer is not maximum reliability everywhere; it is reliability proportional to business impact and reversibility.
Serving ML reliably means treating the model as a versioned artifact behind a stable API, with autoscaling tuned for accelerator economics. GPUs are expensive and don't fractionally autoscale like CPU, so you batch requests to raise utilization, use separate pools per model size, and scale on queue depth / GPU utilization rather than CPU. Health checks must verify the model actually loaded, not just that the process is up.
A KServe/Kubernetes serving spec with GPU requests and a model-ready probe:
spec:
containers:
- name: model
image: registry/llm-serve:v2
resources:
limits: {nvidia.com/gpu: 1}
readinessProbe:
httpGet: {path: /v2/models/llm/ready, port: 8080}
initialDelaySeconds: 60 # model load takes time
env:
- {name: MAX_BATCH_SIZE, value: "16"}
- {name: BATCH_TIMEOUT_MS, value: "20"}Watch for GPU OOM (cap batch size / sequence length), cold-start latency (keep warm replicas), and drift in model quality. The tradeoff: larger batches raise throughput and GPU efficiency but add per-request latency — tune batch size and timeout to your latency SLO.
100Describe the automation, IaC, deployment checks, test gates, and runbooks you would build to institutionalize ML platform reliability, model serving, and GPU/accelerator operations for messaging delivery.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For DevOps/SRE, I would keep the answer focused on ML platform reliability, model serving, and GPU/accelerator operations, measurable outcomes, failure modes, and trade-offs. I would implement this as a reusable platform capability: IaC modules, policy-as-code, CI/CD gates, canary analysis, automatic rollback, standardized dashboards, and runbooks generated from service metadata. I would require preproduction load tests, dependency checks, and clear ownership before production launch. Operationally, I would add SLO definitions, burn-rate alerts, incident templates, capacity reviews, and DR drills. Governance should trace every change to a ticket, artifact, config version, and approver. Success is lower MTTR, fewer repeat incidents, higher deployment confidence, and less toil.
Serving ML reliably means treating the model as a versioned artifact behind a stable API, with autoscaling tuned for accelerator economics. GPUs are expensive and don't fractionally autoscale like CPU, so you batch requests to raise utilization, use separate pools per model size, and scale on queue depth / GPU utilization rather than CPU. Health checks must verify the model actually loaded, not just that the process is up.
A KServe/Kubernetes serving spec with GPU requests and a model-ready probe:
spec:
containers:
- name: model
image: registry/llm-serve:v2
resources:
limits: {nvidia.com/gpu: 1}
readinessProbe:
httpGet: {path: /v2/models/llm/ready, port: 8080}
initialDelaySeconds: 60 # model load takes time
env:
- {name: MAX_BATCH_SIZE, value: "16"}
- {name: BATCH_TIMEOUT_MS, value: "20"}Watch for GPU OOM (cap batch size / sequence length), cold-start latency (keep warm replicas), and drift in model quality. The tradeoff: larger batches raise throughput and GPU efficiency but add per-request latency — tune batch size and timeout to your latency SLO.
More Meta interview prep
Practice other Meta tracks: AI / ML · Data Science · Software Developer / Engineer · Database Engineer. Or browse 1,000+ general interview questions and role quizzes.