How do you alert on something that should be happening but is not (absence of data)? [Intermediate]
Answer
To alert on something that should be happening but is not, I use absence checks, stale timestamp gauges, or expected-rate checks. The safest pattern is often a last_success_timestamp metric for jobs and pipelines.
Technical explanation
absent() is useful when an entire metric or target disappears, but it must be scoped carefully.
For cron jobs, export last success time and alert when it is too old.
For event streams, alert on rate below expected baseline only when the business schedule says traffic should exist.
Hands-on example
PromQL examples: absent(up{job='checkout'} == 1) detects missing targets. For a cron: time() - batch_last_success_timestamp_seconds{job='billing-reconcile'} > 26*3600. For Kafka: rate(events_consumed_total[15m]) == 0 during business hours.
Check how well your resume matches the role with our free resume checker— match score, ATS check, and the skills you're missing.
More Observability interview questions
- What is observability, and how is it different from traditional monitoring? [Basic]
- What are the three pillars of observability (metrics, logs, traces)? [Basic]
- What is the difference between monitoring and observability in practice? [Basic]
- What are the four golden signals of monitoring? [Basic]
- What is the difference between the USE method and the RED method? [Basic]
- When would you use the USE method versus the RED method? [Basic]
- What is an SLI, an SLO, and an SLA, and how do they relate? [Basic]
- How do you choose good SLIs for a service? [Basic]