Companies › Meta › Data Science
Meta Data Science interview questions
100 real Meta Data Science interview questions with model answers, key talking points, and common pitfalls — free prep for your Meta interview.
Paste the job description and your resume into SkillFitly's free resume checker to see your match score and missing skills, then practice with timed interview quizzes.
1Define the North Star metric and supporting guardrail metrics for feed ranking requests at Meta Platforms. How would North Star metrics and KPI decomposition influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on North Star metrics and KPI decomposition, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
A North Star metric is the single measure that best captures the value your product delivers to users (e.g. weekly active teams, nights booked). You decompose it into an input tree of drivers you can actually influence, so teams see how their work ladders up. Good decomposition is multiplicative or additive and MECE (mutually exclusive, collectively exhaustive) so no driver is double-counted.
Decompose revenue into levers you can move, then size each:
-- Revenue = Users x Conversion x Orders/converter x AOV
SELECT
COUNT(DISTINCT user_id) AS users,
COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END)*1.0
/ COUNT(DISTINCT user_id) AS conversion,
SUM(orders)*1.0
/ NULLIF(COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END),0) AS orders_per_converter,
SUM(revenue)*1.0 / NULLIF(SUM(orders),0) AS aov
FROM user_month WHERE month = '2026-08-01';Pick a North Star that leads revenue rather than lagging it, so it's actionable. The tradeoff: a single metric can be gamed or miss quality, so pair it with guardrail metrics (e.g. retention, complaints) that prevent optimizing the number at the users' expense.
2Using media embeddings and engagement signals, describe an analytical plan to determine whether serving a targeted ad improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on North Star metrics and KPI decomposition, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
A North Star metric is the single measure that best captures the value your product delivers to users (e.g. weekly active teams, nights booked). You decompose it into an input tree of drivers you can actually influence, so teams see how their work ladders up. Good decomposition is multiplicative or additive and MECE (mutually exclusive, collectively exhaustive) so no driver is double-counted.
Decompose revenue into levers you can move, then size each:
-- Revenue = Users x Conversion x Orders/converter x AOV
SELECT
COUNT(DISTINCT user_id) AS users,
COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END)*1.0
/ COUNT(DISTINCT user_id) AS conversion,
SUM(orders)*1.0
/ NULLIF(COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END),0) AS orders_per_converter,
SUM(revenue)*1.0 / NULLIF(SUM(orders),0) AS aov
FROM user_month WHERE month = '2026-08-01';Pick a North Star that leads revenue rather than lagging it, so it's actionable. The tradeoff: a single metric can be gamed or miss quality, so pair it with guardrail metrics (e.g. retention, complaints) that prevent optimizing the number at the users' expense.
3How would you design an experiment or causal study for North Star metrics and KPI decomposition at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on North Star metrics and KPI decomposition, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
A North Star metric is the single measure that best captures the value your product delivers to users (e.g. weekly active teams, nights booked). You decompose it into an input tree of drivers you can actually influence, so teams see how their work ladders up. Good decomposition is multiplicative or additive and MECE (mutually exclusive, collectively exhaustive) so no driver is double-counted.
Decompose revenue into levers you can move, then size each:
-- Revenue = Users x Conversion x Orders/converter x AOV
SELECT
COUNT(DISTINCT user_id) AS users,
COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END)*1.0
/ COUNT(DISTINCT user_id) AS conversion,
SUM(orders)*1.0
/ NULLIF(COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END),0) AS orders_per_converter,
SUM(revenue)*1.0 / NULLIF(SUM(orders),0) AS aov
FROM user_month WHERE month = '2026-08-01';Pick a North Star that leads revenue rather than lagging it, so it's actionable. The tradeoff: a single metric can be gamed or miss quality, so pair it with guardrail metrics (e.g. retention, complaints) that prevent optimizing the number at the users' expense.
4What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about content moderation queues to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on North Star metrics and KPI decomposition, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
A North Star metric is the single measure that best captures the value your product delivers to users (e.g. weekly active teams, nights booked). You decompose it into an input tree of drivers you can actually influence, so teams see how their work ladders up. Good decomposition is multiplicative or additive and MECE (mutually exclusive, collectively exhaustive) so no driver is double-counted.
Decompose revenue into levers you can move, then size each:
-- Revenue = Users x Conversion x Orders/converter x AOV
SELECT
COUNT(DISTINCT user_id) AS users,
COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END)*1.0
/ COUNT(DISTINCT user_id) AS conversion,
SUM(orders)*1.0
/ NULLIF(COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END),0) AS orders_per_converter,
SUM(revenue)*1.0 / NULLIF(SUM(orders),0) AS aov
FROM user_month WHERE month = '2026-08-01';Pick a North Star that leads revenue rather than lagging it, so it's actionable. The tradeoff: a single metric can be gamed or miss quality, so pair it with guardrail metrics (e.g. retention, complaints) that prevent optimizing the number at the users' expense.
5Turn an analysis of North Star metrics and KPI decomposition into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on North Star metrics and KPI decomposition, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
A North Star metric is the single measure that best captures the value your product delivers to users (e.g. weekly active teams, nights booked). You decompose it into an input tree of drivers you can actually influence, so teams see how their work ladders up. Good decomposition is multiplicative or additive and MECE (mutually exclusive, collectively exhaustive) so no driver is double-counted.
Decompose revenue into levers you can move, then size each:
-- Revenue = Users x Conversion x Orders/converter x AOV
SELECT
COUNT(DISTINCT user_id) AS users,
COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END)*1.0
/ COUNT(DISTINCT user_id) AS conversion,
SUM(orders)*1.0
/ NULLIF(COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END),0) AS orders_per_converter,
SUM(revenue)*1.0 / NULLIF(SUM(orders),0) AS aov
FROM user_month WHERE month = '2026-08-01';Pick a North Star that leads revenue rather than lagging it, so it's actionable. The tradeoff: a single metric can be gamed or miss quality, so pair it with guardrail metrics (e.g. retention, complaints) that prevent optimizing the number at the users' expense.
6Define the North Star metric and supporting guardrail metrics for messaging delivery at Meta Platforms. How would SQL analytics and dimensional modeling influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on SQL analytics and dimensional modeling, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Analytical SQL sits on a dimensional model: fact tables hold events/measures at a defined grain, dimension tables hold descriptive attributes, joined on keys (a star schema). Getting the grain right — one row per what — is the foundation; mixing grains causes double-counting. Window functions handle the analytics that GROUP BY can't: running totals, rankings, and period-over-period comparisons without self-joins.
A window-function query for per-user order recency and running spend:
SELECT
user_id, order_id, order_ts, amount,
ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY order_ts) AS order_seq,
SUM(amount) OVER (PARTITION BY user_id ORDER BY order_ts) AS running_spend,
order_ts - LAG(order_ts) OVER (PARTITION BY user_id
ORDER BY order_ts) AS gap_since_prev
FROM fct_orders;Pre-aggregate heavy queries into summary tables for dashboards, and document the grain of every fact. The tradeoff: a fully normalized model saves storage but requires many joins; a denormalized/star model is faster to query and easier for analysts, which is why warehouses favor it.
7Using privacy-governed user activity data, describe an analytical plan to determine whether moderating harmful content improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on SQL analytics and dimensional modeling, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Analytical SQL sits on a dimensional model: fact tables hold events/measures at a defined grain, dimension tables hold descriptive attributes, joined on keys (a star schema). Getting the grain right — one row per what — is the foundation; mixing grains causes double-counting. Window functions handle the analytics that GROUP BY can't: running totals, rankings, and period-over-period comparisons without self-joins.
A window-function query for per-user order recency and running spend:
SELECT
user_id, order_id, order_ts, amount,
ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY order_ts) AS order_seq,
SUM(amount) OVER (PARTITION BY user_id ORDER BY order_ts) AS running_spend,
order_ts - LAG(order_ts) OVER (PARTITION BY user_id
ORDER BY order_ts) AS gap_since_prev
FROM fct_orders;Pre-aggregate heavy queries into summary tables for dashboards, and document the grain of every fact. The tradeoff: a fully normalized model saves storage but requires many joins; a denormalized/star model is faster to query and easier for analysts, which is why warehouses favor it.
8How would you design an experiment or causal study for SQL analytics and dimensional modeling at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on SQL analytics and dimensional modeling, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Analytical SQL sits on a dimensional model: fact tables hold events/measures at a defined grain, dimension tables hold descriptive attributes, joined on keys (a star schema). Getting the grain right — one row per what — is the foundation; mixing grains causes double-counting. Window functions handle the analytics that GROUP BY can't: running totals, rankings, and period-over-period comparisons without self-joins.
A window-function query for per-user order recency and running spend:
SELECT
user_id, order_id, order_ts, amount,
ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY order_ts) AS order_seq,
SUM(amount) OVER (PARTITION BY user_id ORDER BY order_ts) AS running_spend,
order_ts - LAG(order_ts) OVER (PARTITION BY user_id
ORDER BY order_ts) AS gap_since_prev
FROM fct_orders;Pre-aggregate heavy queries into summary tables for dashboards, and document the grain of every fact. The tradeoff: a fully normalized model saves storage but requires many joins; a denormalized/star model is faster to query and easier for analysts, which is why warehouses favor it.
9What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about feed ranking requests to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on SQL analytics and dimensional modeling, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
Analytical SQL sits on a dimensional model: fact tables hold events/measures at a defined grain, dimension tables hold descriptive attributes, joined on keys (a star schema). Getting the grain right — one row per what — is the foundation; mixing grains causes double-counting. Window functions handle the analytics that GROUP BY can't: running totals, rankings, and period-over-period comparisons without self-joins.
A window-function query for per-user order recency and running spend:
SELECT
user_id, order_id, order_ts, amount,
ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY order_ts) AS order_seq,
SUM(amount) OVER (PARTITION BY user_id ORDER BY order_ts) AS running_spend,
order_ts - LAG(order_ts) OVER (PARTITION BY user_id
ORDER BY order_ts) AS gap_since_prev
FROM fct_orders;Pre-aggregate heavy queries into summary tables for dashboards, and document the grain of every fact. The tradeoff: a fully normalized model saves storage but requires many joins; a denormalized/star model is faster to query and easier for analysts, which is why warehouses favor it.
10Turn an analysis of SQL analytics and dimensional modeling into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on SQL analytics and dimensional modeling, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Analytical SQL sits on a dimensional model: fact tables hold events/measures at a defined grain, dimension tables hold descriptive attributes, joined on keys (a star schema). Getting the grain right — one row per what — is the foundation; mixing grains causes double-counting. Window functions handle the analytics that GROUP BY can't: running totals, rankings, and period-over-period comparisons without self-joins.
A window-function query for per-user order recency and running spend:
SELECT
user_id, order_id, order_ts, amount,
ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY order_ts) AS order_seq,
SUM(amount) OVER (PARTITION BY user_id ORDER BY order_ts) AS running_spend,
order_ts - LAG(order_ts) OVER (PARTITION BY user_id
ORDER BY order_ts) AS gap_since_prev
FROM fct_orders;Pre-aggregate heavy queries into summary tables for dashboards, and document the grain of every fact. The tradeoff: a fully normalized model saves storage but requires many joins; a denormalized/star model is faster to query and easier for analysts, which is why warehouses favor it.
11Define the North Star metric and supporting guardrail metrics for social graph updates at Meta Platforms. How would A/B testing and experimentation design influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on A/B testing and experimentation design, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
An A/B test isolates causal impact by randomly assigning users to control and treatment, so the only systematic difference is the change. Sound design means randomizing at the right unit (usually the user, to avoid within-user contamination), choosing one primary metric plus guardrails in advance, and computing the required sample size for adequate power before launch — underpowered tests waste traffic and mislead.
Sample-size calculation before you launch:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
baseline, mde = 0.10, 0.005 # detect a 0.5pp absolute lift
effect = proportion_effectsize(baseline, baseline + mde)
n = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8)
print(f"Need ~{n:,.0f} users per arm")Check for sample-ratio mismatch (a broken split invalidates results) and don't peek-and-stop without a sequential correction. The tradeoff: bigger effects need less traffic, so for small expected lifts you need large samples or longer runtime — decide if the change is worth the experimental cost.
12Using social graph events, feed interactions, ad logs, messaging telemetry, and moderation labels, describe an analytical plan to determine whether syncing a VR device improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on A/B testing and experimentation design, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
An A/B test isolates causal impact by randomly assigning users to control and treatment, so the only systematic difference is the change. Sound design means randomizing at the right unit (usually the user, to avoid within-user contamination), choosing one primary metric plus guardrails in advance, and computing the required sample size for adequate power before launch — underpowered tests waste traffic and mislead.
Sample-size calculation before you launch:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
baseline, mde = 0.10, 0.005 # detect a 0.5pp absolute lift
effect = proportion_effectsize(baseline, baseline + mde)
n = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8)
print(f"Need ~{n:,.0f} users per arm")Check for sample-ratio mismatch (a broken split invalidates results) and don't peek-and-stop without a sequential correction. The tradeoff: bigger effects need less traffic, so for small expected lifts you need large samples or longer runtime — decide if the change is worth the experimental cost.
13How would you design an experiment or causal study for A/B testing and experimentation design at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on A/B testing and experimentation design, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
An A/B test isolates causal impact by randomly assigning users to control and treatment, so the only systematic difference is the change. Sound design means randomizing at the right unit (usually the user, to avoid within-user contamination), choosing one primary metric plus guardrails in advance, and computing the required sample size for adequate power before launch — underpowered tests waste traffic and mislead.
Sample-size calculation before you launch:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
baseline, mde = 0.10, 0.005 # detect a 0.5pp absolute lift
effect = proportion_effectsize(baseline, baseline + mde)
n = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8)
print(f"Need ~{n:,.0f} users per arm")Check for sample-ratio mismatch (a broken split invalidates results) and don't peek-and-stop without a sequential correction. The tradeoff: bigger effects need less traffic, so for small expected lifts you need large samples or longer runtime — decide if the change is worth the experimental cost.
14What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about messaging delivery to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on A/B testing and experimentation design, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
An A/B test isolates causal impact by randomly assigning users to control and treatment, so the only systematic difference is the change. Sound design means randomizing at the right unit (usually the user, to avoid within-user contamination), choosing one primary metric plus guardrails in advance, and computing the required sample size for adequate power before launch — underpowered tests waste traffic and mislead.
Sample-size calculation before you launch:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
baseline, mde = 0.10, 0.005 # detect a 0.5pp absolute lift
effect = proportion_effectsize(baseline, baseline + mde)
n = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8)
print(f"Need ~{n:,.0f} users per arm")Check for sample-ratio mismatch (a broken split invalidates results) and don't peek-and-stop without a sequential correction. The tradeoff: bigger effects need less traffic, so for small expected lifts you need large samples or longer runtime — decide if the change is worth the experimental cost.
15Turn an analysis of A/B testing and experimentation design into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on A/B testing and experimentation design, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
An A/B test isolates causal impact by randomly assigning users to control and treatment, so the only systematic difference is the change. Sound design means randomizing at the right unit (usually the user, to avoid within-user contamination), choosing one primary metric plus guardrails in advance, and computing the required sample size for adequate power before launch — underpowered tests waste traffic and mislead.
Sample-size calculation before you launch:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
baseline, mde = 0.10, 0.005 # detect a 0.5pp absolute lift
effect = proportion_effectsize(baseline, baseline + mde)
n = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8)
print(f"Need ~{n:,.0f} users per arm")Check for sample-ratio mismatch (a broken split invalidates results) and don't peek-and-stop without a sequential correction. The tradeoff: bigger effects need less traffic, so for small expected lifts you need large samples or longer runtime — decide if the change is worth the experimental cost.
16Define the North Star metric and supporting guardrail metrics for ad impressions at Meta Platforms. How would causal inference and quasi-experimental methods influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on causal inference and quasi-experimental methods, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
When randomization isn't possible, quasi-experimental methods estimate causal effects from observational data by approximating a control group. Difference-in-differences compares the change over time in a treated group against an untreated group; propensity-score matching pairs treated and control units with similar covariates; instrumental variables and regression discontinuity exploit natural experiments. All rely on assumptions (e.g. parallel trends) that you must test, not assume.
A difference-in-differences estimate via regression:
import statsmodels.formula.api as smf
# treated x post interaction = the causal DiD effect
model = smf.ols('outcome ~ treated + post + treated:post', data=df).fit()
print(model.params['treated:post']) # estimated treatment effect
# validate: pre-period trends of treated vs control should be parallelAlways probe robustness with placebo tests and sensitivity to unobserved confounders. The tradeoff: quasi-experiments are the only option when A/B testing is infeasible, but their validity hinges on untestable assumptions, so present effects with appropriate caveats rather than as clean causal truth.
17Using media embeddings and engagement signals, describe an analytical plan to determine whether loading an Instagram feed improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on causal inference and quasi-experimental methods, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
When randomization isn't possible, quasi-experimental methods estimate causal effects from observational data by approximating a control group. Difference-in-differences compares the change over time in a treated group against an untreated group; propensity-score matching pairs treated and control units with similar covariates; instrumental variables and regression discontinuity exploit natural experiments. All rely on assumptions (e.g. parallel trends) that you must test, not assume.
A difference-in-differences estimate via regression:
import statsmodels.formula.api as smf
# treated x post interaction = the causal DiD effect
model = smf.ols('outcome ~ treated + post + treated:post', data=df).fit()
print(model.params['treated:post']) # estimated treatment effect
# validate: pre-period trends of treated vs control should be parallelAlways probe robustness with placebo tests and sensitivity to unobserved confounders. The tradeoff: quasi-experiments are the only option when A/B testing is infeasible, but their validity hinges on untestable assumptions, so present effects with appropriate caveats rather than as clean causal truth.
18How would you design an experiment or causal study for causal inference and quasi-experimental methods at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on causal inference and quasi-experimental methods, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
When randomization isn't possible, quasi-experimental methods estimate causal effects from observational data by approximating a control group. Difference-in-differences compares the change over time in a treated group against an untreated group; propensity-score matching pairs treated and control units with similar covariates; instrumental variables and regression discontinuity exploit natural experiments. All rely on assumptions (e.g. parallel trends) that you must test, not assume.
A difference-in-differences estimate via regression:
import statsmodels.formula.api as smf
# treated x post interaction = the causal DiD effect
model = smf.ols('outcome ~ treated + post + treated:post', data=df).fit()
print(model.params['treated:post']) # estimated treatment effect
# validate: pre-period trends of treated vs control should be parallelAlways probe robustness with placebo tests and sensitivity to unobserved confounders. The tradeoff: quasi-experiments are the only option when A/B testing is infeasible, but their validity hinges on untestable assumptions, so present effects with appropriate caveats rather than as clean causal truth.
19What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about social graph updates to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on causal inference and quasi-experimental methods, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
When randomization isn't possible, quasi-experimental methods estimate causal effects from observational data by approximating a control group. Difference-in-differences compares the change over time in a treated group against an untreated group; propensity-score matching pairs treated and control units with similar covariates; instrumental variables and regression discontinuity exploit natural experiments. All rely on assumptions (e.g. parallel trends) that you must test, not assume.
A difference-in-differences estimate via regression:
import statsmodels.formula.api as smf
# treated x post interaction = the causal DiD effect
model = smf.ols('outcome ~ treated + post + treated:post', data=df).fit()
print(model.params['treated:post']) # estimated treatment effect
# validate: pre-period trends of treated vs control should be parallelAlways probe robustness with placebo tests and sensitivity to unobserved confounders. The tradeoff: quasi-experiments are the only option when A/B testing is infeasible, but their validity hinges on untestable assumptions, so present effects with appropriate caveats rather than as clean causal truth.
20Turn an analysis of causal inference and quasi-experimental methods into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on causal inference and quasi-experimental methods, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
When randomization isn't possible, quasi-experimental methods estimate causal effects from observational data by approximating a control group. Difference-in-differences compares the change over time in a treated group against an untreated group; propensity-score matching pairs treated and control units with similar covariates; instrumental variables and regression discontinuity exploit natural experiments. All rely on assumptions (e.g. parallel trends) that you must test, not assume.
A difference-in-differences estimate via regression:
import statsmodels.formula.api as smf
# treated x post interaction = the causal DiD effect
model = smf.ols('outcome ~ treated + post + treated:post', data=df).fit()
print(model.params['treated:post']) # estimated treatment effect
# validate: pre-period trends of treated vs control should be parallelAlways probe robustness with placebo tests and sensitivity to unobserved confounders. The tradeoff: quasi-experiments are the only option when A/B testing is infeasible, but their validity hinges on untestable assumptions, so present effects with appropriate caveats rather than as clean causal truth.
21Define the North Star metric and supporting guardrail metrics for content moderation queues at Meta Platforms. How would customer, user, or workload segmentation influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on customer, user, or workload segmentation, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Segmentation groups entities into meaningful cohorts to tailor decisions. It can be rule-based (RFM: recency, frequency, monetary), or unsupervised (k-means, hierarchical clustering on standardized features). The goal isn't mathematically tight clusters but actionable, stable, and interpretable segments that a team can actually target differently — a segmentation nobody acts on is wasted analysis.
K-means segmentation on standardized behavioral features:
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
X = StandardScaler().fit_transform(df[['recency','frequency','monetary']])
# choose k via elbow/silhouette, not arbitrarily
km = KMeans(n_clusters=4, n_init=10, random_state=0).fit(X)
df['segment'] = km.labels_
df.groupby('segment')[['recency','frequency','monetary']].mean() # profile themProfile and name each segment, and validate stability over time before building programs on it. The tradeoff: more segments enable finer targeting but add operational complexity and shrink sample sizes per segment — keep the count to what the business can genuinely act on.
22Using privacy-governed user activity data, describe an analytical plan to determine whether sending a WhatsApp message improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on customer, user, or workload segmentation, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Segmentation groups entities into meaningful cohorts to tailor decisions. It can be rule-based (RFM: recency, frequency, monetary), or unsupervised (k-means, hierarchical clustering on standardized features). The goal isn't mathematically tight clusters but actionable, stable, and interpretable segments that a team can actually target differently — a segmentation nobody acts on is wasted analysis.
K-means segmentation on standardized behavioral features:
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
X = StandardScaler().fit_transform(df[['recency','frequency','monetary']])
# choose k via elbow/silhouette, not arbitrarily
km = KMeans(n_clusters=4, n_init=10, random_state=0).fit(X)
df['segment'] = km.labels_
df.groupby('segment')[['recency','frequency','monetary']].mean() # profile themProfile and name each segment, and validate stability over time before building programs on it. The tradeoff: more segments enable finer targeting but add operational complexity and shrink sample sizes per segment — keep the count to what the business can genuinely act on.
23How would you design an experiment or causal study for customer, user, or workload segmentation at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on customer, user, or workload segmentation, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Segmentation groups entities into meaningful cohorts to tailor decisions. It can be rule-based (RFM: recency, frequency, monetary), or unsupervised (k-means, hierarchical clustering on standardized features). The goal isn't mathematically tight clusters but actionable, stable, and interpretable segments that a team can actually target differently — a segmentation nobody acts on is wasted analysis.
K-means segmentation on standardized behavioral features:
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
X = StandardScaler().fit_transform(df[['recency','frequency','monetary']])
# choose k via elbow/silhouette, not arbitrarily
km = KMeans(n_clusters=4, n_init=10, random_state=0).fit(X)
df['segment'] = km.labels_
df.groupby('segment')[['recency','frequency','monetary']].mean() # profile themProfile and name each segment, and validate stability over time before building programs on it. The tradeoff: more segments enable finer targeting but add operational complexity and shrink sample sizes per segment — keep the count to what the business can genuinely act on.
24What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about ad impressions to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on customer, user, or workload segmentation, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
Segmentation groups entities into meaningful cohorts to tailor decisions. It can be rule-based (RFM: recency, frequency, monetary), or unsupervised (k-means, hierarchical clustering on standardized features). The goal isn't mathematically tight clusters but actionable, stable, and interpretable segments that a team can actually target differently — a segmentation nobody acts on is wasted analysis.
K-means segmentation on standardized behavioral features:
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
X = StandardScaler().fit_transform(df[['recency','frequency','monetary']])
# choose k via elbow/silhouette, not arbitrarily
km = KMeans(n_clusters=4, n_init=10, random_state=0).fit(X)
df['segment'] = km.labels_
df.groupby('segment')[['recency','frequency','monetary']].mean() # profile themProfile and name each segment, and validate stability over time before building programs on it. The tradeoff: more segments enable finer targeting but add operational complexity and shrink sample sizes per segment — keep the count to what the business can genuinely act on.
25Turn an analysis of customer, user, or workload segmentation into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on customer, user, or workload segmentation, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Segmentation groups entities into meaningful cohorts to tailor decisions. It can be rule-based (RFM: recency, frequency, monetary), or unsupervised (k-means, hierarchical clustering on standardized features). The goal isn't mathematically tight clusters but actionable, stable, and interpretable segments that a team can actually target differently — a segmentation nobody acts on is wasted analysis.
K-means segmentation on standardized behavioral features:
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
X = StandardScaler().fit_transform(df[['recency','frequency','monetary']])
# choose k via elbow/silhouette, not arbitrarily
km = KMeans(n_clusters=4, n_init=10, random_state=0).fit(X)
df['segment'] = km.labels_
df.groupby('segment')[['recency','frequency','monetary']].mean() # profile themProfile and name each segment, and validate stability over time before building programs on it. The tradeoff: more segments enable finer targeting but add operational complexity and shrink sample sizes per segment — keep the count to what the business can genuinely act on.
26Define the North Star metric and supporting guardrail metrics for feed ranking requests at Meta Platforms. How would retention, churn, activation, and lifecycle analysis influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on retention, churn, activation, and lifecycle analysis, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Lifecycle analysis tracks users from activation (reaching first value) through retention and churn. Retention is best measured with cohort curves — the fraction of each signup cohort still active over time — which reveal whether the product has a leaky bucket or a stabilizing plateau. The activation metric (the 'aha' action correlated with long-term retention) is the highest-leverage early lever.
A cohort retention query by signup month:
SELECT
DATE_TRUNC('month', u.signup_ts) AS cohort,
DATE_DIFF('month', u.signup_ts, a.activity_month) AS months_since,
COUNT(DISTINCT a.user_id)*1.0
/ COUNT(DISTINCT u.user_id) OVER (PARTITION BY DATE_TRUNC('month', u.signup_ts)) AS retained
FROM users u JOIN monthly_activity a USING (user_id)
GROUP BY 1,2 ORDER BY 1,2;Distinguish churn types (voluntary vs. involuntary/payment failure) since they need different fixes. The tradeoff: chasing at-risk users with incentives can be costly and train bad behavior, so target retention efforts where predicted churn and user value are both high.
27Using social graph events, feed interactions, ad logs, messaging telemetry, and moderation labels, describe an analytical plan to determine whether serving a targeted ad improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on retention, churn, activation, and lifecycle analysis, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Lifecycle analysis tracks users from activation (reaching first value) through retention and churn. Retention is best measured with cohort curves — the fraction of each signup cohort still active over time — which reveal whether the product has a leaky bucket or a stabilizing plateau. The activation metric (the 'aha' action correlated with long-term retention) is the highest-leverage early lever.
A cohort retention query by signup month:
SELECT
DATE_TRUNC('month', u.signup_ts) AS cohort,
DATE_DIFF('month', u.signup_ts, a.activity_month) AS months_since,
COUNT(DISTINCT a.user_id)*1.0
/ COUNT(DISTINCT u.user_id) OVER (PARTITION BY DATE_TRUNC('month', u.signup_ts)) AS retained
FROM users u JOIN monthly_activity a USING (user_id)
GROUP BY 1,2 ORDER BY 1,2;Distinguish churn types (voluntary vs. involuntary/payment failure) since they need different fixes. The tradeoff: chasing at-risk users with incentives can be costly and train bad behavior, so target retention efforts where predicted churn and user value are both high.
28How would you design an experiment or causal study for retention, churn, activation, and lifecycle analysis at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on retention, churn, activation, and lifecycle analysis, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Lifecycle analysis tracks users from activation (reaching first value) through retention and churn. Retention is best measured with cohort curves — the fraction of each signup cohort still active over time — which reveal whether the product has a leaky bucket or a stabilizing plateau. The activation metric (the 'aha' action correlated with long-term retention) is the highest-leverage early lever.
A cohort retention query by signup month:
SELECT
DATE_TRUNC('month', u.signup_ts) AS cohort,
DATE_DIFF('month', u.signup_ts, a.activity_month) AS months_since,
COUNT(DISTINCT a.user_id)*1.0
/ COUNT(DISTINCT u.user_id) OVER (PARTITION BY DATE_TRUNC('month', u.signup_ts)) AS retained
FROM users u JOIN monthly_activity a USING (user_id)
GROUP BY 1,2 ORDER BY 1,2;Distinguish churn types (voluntary vs. involuntary/payment failure) since they need different fixes. The tradeoff: chasing at-risk users with incentives can be costly and train bad behavior, so target retention efforts where predicted churn and user value are both high.
29What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about content moderation queues to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on retention, churn, activation, and lifecycle analysis, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
A North Star metric is the single measure that best captures the value your product delivers to users (e.g. weekly active teams, nights booked). You decompose it into an input tree of drivers you can actually influence, so teams see how their work ladders up. Good decomposition is multiplicative or additive and MECE (mutually exclusive, collectively exhaustive) so no driver is double-counted.
Decompose revenue into levers you can move, then size each:
-- Revenue = Users x Conversion x Orders/converter x AOV
SELECT
COUNT(DISTINCT user_id) AS users,
COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END)*1.0
/ COUNT(DISTINCT user_id) AS conversion,
SUM(orders)*1.0
/ NULLIF(COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END),0) AS orders_per_converter,
SUM(revenue)*1.0 / NULLIF(SUM(orders),0) AS aov
FROM user_month WHERE month = '2026-08-01';Pick a North Star that leads revenue rather than lagging it, so it's actionable. The tradeoff: a single metric can be gamed or miss quality, so pair it with guardrail metrics (e.g. retention, complaints) that prevent optimizing the number at the users' expense.
30Turn an analysis of retention, churn, activation, and lifecycle analysis into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on retention, churn, activation, and lifecycle analysis, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Lifecycle analysis tracks users from activation (reaching first value) through retention and churn. Retention is best measured with cohort curves — the fraction of each signup cohort still active over time — which reveal whether the product has a leaky bucket or a stabilizing plateau. The activation metric (the 'aha' action correlated with long-term retention) is the highest-leverage early lever.
A cohort retention query by signup month:
SELECT
DATE_TRUNC('month', u.signup_ts) AS cohort,
DATE_DIFF('month', u.signup_ts, a.activity_month) AS months_since,
COUNT(DISTINCT a.user_id)*1.0
/ COUNT(DISTINCT u.user_id) OVER (PARTITION BY DATE_TRUNC('month', u.signup_ts)) AS retained
FROM users u JOIN monthly_activity a USING (user_id)
GROUP BY 1,2 ORDER BY 1,2;Distinguish churn types (voluntary vs. involuntary/payment failure) since they need different fixes. The tradeoff: chasing at-risk users with incentives can be costly and train bad behavior, so target retention efforts where predicted churn and user value are both high.
31Define the North Star metric and supporting guardrail metrics for messaging delivery at Meta Platforms. How would forecasting demand, traffic, capacity, or revenue influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on forecasting demand, traffic, capacity, or revenue, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Business forecasting projects demand/traffic/revenue to inform staffing, inventory, and capacity. Decompose the series into trend, seasonality (weekly, yearly), and holiday/event effects, then choose a method matched to the data: exponential smoothing or Prophet for strong seasonality, gradient boosting on calendar/lag features when you have many drivers. Always backtest on held-out future periods.
A seasonal forecast with Prophet including holidays:
from prophet import Prophet
m = Prophet(yearly_seasonality=True, weekly_seasonality=True)
m.add_country_holidays(country_name='US')
m.fit(df.rename(columns={'date':'ds','demand':'y'}))
future = m.make_future_dataframe(periods=90)
fcst = m.predict(future) # yhat + uncertainty interval
# backtest: compare yhat to actuals on a rolling holdout, report MAPEReport uncertainty intervals, not just a point estimate, so planners can size buffers. The tradeoff: complex models can overfit noise and mishandle regime changes (a launch, a shock), so keep a simple seasonal-naive baseline and override the model with human judgment around known one-off events.
32Using media embeddings and engagement signals, describe an analytical plan to determine whether moderating harmful content improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on forecasting demand, traffic, capacity, or revenue, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Business forecasting projects demand/traffic/revenue to inform staffing, inventory, and capacity. Decompose the series into trend, seasonality (weekly, yearly), and holiday/event effects, then choose a method matched to the data: exponential smoothing or Prophet for strong seasonality, gradient boosting on calendar/lag features when you have many drivers. Always backtest on held-out future periods.
A seasonal forecast with Prophet including holidays:
from prophet import Prophet
m = Prophet(yearly_seasonality=True, weekly_seasonality=True)
m.add_country_holidays(country_name='US')
m.fit(df.rename(columns={'date':'ds','demand':'y'}))
future = m.make_future_dataframe(periods=90)
fcst = m.predict(future) # yhat + uncertainty interval
# backtest: compare yhat to actuals on a rolling holdout, report MAPEReport uncertainty intervals, not just a point estimate, so planners can size buffers. The tradeoff: complex models can overfit noise and mishandle regime changes (a launch, a shock), so keep a simple seasonal-naive baseline and override the model with human judgment around known one-off events.
33How would you design an experiment or causal study for forecasting demand, traffic, capacity, or revenue at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on forecasting demand, traffic, capacity, or revenue, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Business forecasting projects demand/traffic/revenue to inform staffing, inventory, and capacity. Decompose the series into trend, seasonality (weekly, yearly), and holiday/event effects, then choose a method matched to the data: exponential smoothing or Prophet for strong seasonality, gradient boosting on calendar/lag features when you have many drivers. Always backtest on held-out future periods.
A seasonal forecast with Prophet including holidays:
from prophet import Prophet
m = Prophet(yearly_seasonality=True, weekly_seasonality=True)
m.add_country_holidays(country_name='US')
m.fit(df.rename(columns={'date':'ds','demand':'y'}))
future = m.make_future_dataframe(periods=90)
fcst = m.predict(future) # yhat + uncertainty interval
# backtest: compare yhat to actuals on a rolling holdout, report MAPEReport uncertainty intervals, not just a point estimate, so planners can size buffers. The tradeoff: complex models can overfit noise and mishandle regime changes (a launch, a shock), so keep a simple seasonal-naive baseline and override the model with human judgment around known one-off events.
34What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about feed ranking requests to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on forecasting demand, traffic, capacity, or revenue, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
Analytical SQL sits on a dimensional model: fact tables hold events/measures at a defined grain, dimension tables hold descriptive attributes, joined on keys (a star schema). Getting the grain right — one row per what — is the foundation; mixing grains causes double-counting. Window functions handle the analytics that GROUP BY can't: running totals, rankings, and period-over-period comparisons without self-joins.
A window-function query for per-user order recency and running spend:
SELECT
user_id, order_id, order_ts, amount,
ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY order_ts) AS order_seq,
SUM(amount) OVER (PARTITION BY user_id ORDER BY order_ts) AS running_spend,
order_ts - LAG(order_ts) OVER (PARTITION BY user_id
ORDER BY order_ts) AS gap_since_prev
FROM fct_orders;Pre-aggregate heavy queries into summary tables for dashboards, and document the grain of every fact. The tradeoff: a fully normalized model saves storage but requires many joins; a denormalized/star model is faster to query and easier for analysts, which is why warehouses favor it.
35Turn an analysis of forecasting demand, traffic, capacity, or revenue into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on forecasting demand, traffic, capacity, or revenue, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Business forecasting projects demand/traffic/revenue to inform staffing, inventory, and capacity. Decompose the series into trend, seasonality (weekly, yearly), and holiday/event effects, then choose a method matched to the data: exponential smoothing or Prophet for strong seasonality, gradient boosting on calendar/lag features when you have many drivers. Always backtest on held-out future periods.
A seasonal forecast with Prophet including holidays:
from prophet import Prophet
m = Prophet(yearly_seasonality=True, weekly_seasonality=True)
m.add_country_holidays(country_name='US')
m.fit(df.rename(columns={'date':'ds','demand':'y'}))
future = m.make_future_dataframe(periods=90)
fcst = m.predict(future) # yhat + uncertainty interval
# backtest: compare yhat to actuals on a rolling holdout, report MAPEReport uncertainty intervals, not just a point estimate, so planners can size buffers. The tradeoff: complex models can overfit noise and mishandle regime changes (a launch, a shock), so keep a simple seasonal-naive baseline and override the model with human judgment around known one-off events.
36Define the North Star metric and supporting guardrail metrics for social graph updates at Meta Platforms. How would anomaly detection and root-cause analysis influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on anomaly detection and root-cause analysis, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Anomaly detection flags points that deviate from expected behavior; the method depends on the signal. For metrics with seasonality, model the expected value (e.g. rolling median or a forecast) and alert on residuals beyond a robust threshold (MAD-based, not mean/std which outliers distort). Once flagged, root-cause analysis segments the anomaly by dimensions (region, version, device) to localize the driver.
Robust residual-based anomaly detection:
import numpy as np
resid = actual - expected # expected from forecast/rolling median
med = np.median(resid)
mad = np.median(np.abs(resid - med)) + 1e-9
robust_z = 0.6745 * (resid - med) / mad # robust to outliers
anomalies = np.where(np.abs(robust_z) > 3.5)[0]For root cause, drill down by contribution: which segment moved most and explains the aggregate change. The tradeoff: sensitive thresholds catch real issues but flood on-call with false positives, so tune to the cost of a miss vs. a false alarm and add a minimum-duration filter.
37Using privacy-governed user activity data, describe an analytical plan to determine whether syncing a VR device improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on anomaly detection and root-cause analysis, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Anomaly detection flags points that deviate from expected behavior; the method depends on the signal. For metrics with seasonality, model the expected value (e.g. rolling median or a forecast) and alert on residuals beyond a robust threshold (MAD-based, not mean/std which outliers distort). Once flagged, root-cause analysis segments the anomaly by dimensions (region, version, device) to localize the driver.
Robust residual-based anomaly detection:
import numpy as np
resid = actual - expected # expected from forecast/rolling median
med = np.median(resid)
mad = np.median(np.abs(resid - med)) + 1e-9
robust_z = 0.6745 * (resid - med) / mad # robust to outliers
anomalies = np.where(np.abs(robust_z) > 3.5)[0]For root cause, drill down by contribution: which segment moved most and explains the aggregate change. The tradeoff: sensitive thresholds catch real issues but flood on-call with false positives, so tune to the cost of a miss vs. a false alarm and add a minimum-duration filter.
38How would you design an experiment or causal study for anomaly detection and root-cause analysis at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on anomaly detection and root-cause analysis, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Anomaly detection flags points that deviate from expected behavior; the method depends on the signal. For metrics with seasonality, model the expected value (e.g. rolling median or a forecast) and alert on residuals beyond a robust threshold (MAD-based, not mean/std which outliers distort). Once flagged, root-cause analysis segments the anomaly by dimensions (region, version, device) to localize the driver.
Robust residual-based anomaly detection:
import numpy as np
resid = actual - expected # expected from forecast/rolling median
med = np.median(resid)
mad = np.median(np.abs(resid - med)) + 1e-9
robust_z = 0.6745 * (resid - med) / mad # robust to outliers
anomalies = np.where(np.abs(robust_z) > 3.5)[0]For root cause, drill down by contribution: which segment moved most and explains the aggregate change. The tradeoff: sensitive thresholds catch real issues but flood on-call with false positives, so tune to the cost of a miss vs. a false alarm and add a minimum-duration filter.
39What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about messaging delivery to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on anomaly detection and root-cause analysis, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
An A/B test isolates causal impact by randomly assigning users to control and treatment, so the only systematic difference is the change. Sound design means randomizing at the right unit (usually the user, to avoid within-user contamination), choosing one primary metric plus guardrails in advance, and computing the required sample size for adequate power before launch — underpowered tests waste traffic and mislead.
Sample-size calculation before you launch:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
baseline, mde = 0.10, 0.005 # detect a 0.5pp absolute lift
effect = proportion_effectsize(baseline, baseline + mde)
n = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8)
print(f"Need ~{n:,.0f} users per arm")Check for sample-ratio mismatch (a broken split invalidates results) and don't peek-and-stop without a sequential correction. The tradeoff: bigger effects need less traffic, so for small expected lifts you need large samples or longer runtime — decide if the change is worth the experimental cost.
40Turn an analysis of anomaly detection and root-cause analysis into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on anomaly detection and root-cause analysis, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Anomaly detection flags points that deviate from expected behavior; the method depends on the signal. For metrics with seasonality, model the expected value (e.g. rolling median or a forecast) and alert on residuals beyond a robust threshold (MAD-based, not mean/std which outliers distort). Once flagged, root-cause analysis segments the anomaly by dimensions (region, version, device) to localize the driver.
Robust residual-based anomaly detection:
import numpy as np
resid = actual - expected # expected from forecast/rolling median
med = np.median(resid)
mad = np.median(np.abs(resid - med)) + 1e-9
robust_z = 0.6745 * (resid - med) / mad # robust to outliers
anomalies = np.where(np.abs(robust_z) > 3.5)[0]For root cause, drill down by contribution: which segment moved most and explains the aggregate change. The tradeoff: sensitive thresholds catch real issues but flood on-call with false positives, so tune to the cost of a miss vs. a false alarm and add a minimum-duration filter.
41Define the North Star metric and supporting guardrail metrics for ad impressions at Meta Platforms. How would attribution, funnel analysis, and conversion measurement influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on attribution, funnel analysis, and conversion measurement, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Funnel analysis measures drop-off at each step from acquisition to conversion, revealing the biggest leak to fix. Attribution assigns credit for conversions across touchpoints — last-touch is simple but overcredits the final step, while data-driven or multi-touch models distribute credit more fairly. The measurement must define the conversion window and de-duplicate users to avoid inflating rates.
A step-by-step funnel with per-step conversion:
WITH steps AS (
SELECT user_id,
MAX(step='view') AS s1,
MAX(step='add_cart') AS s2,
MAX(step='checkout') AS s3,
MAX(step='purchase') AS s4
FROM events GROUP BY user_id)
SELECT
SUM(s1) AS viewed,
SUM(s2) AS carted, ROUND(SUM(s2)*100.0/NULLIF(SUM(s1),0),1) AS view_to_cart_pct,
SUM(s3) AS checkout, SUM(s4) AS purchased
FROM steps;Segment the funnel (by channel, device) since the leak often differs by cohort. The tradeoff: last-touch attribution is transparent but biased, while data-driven attribution is fairer but harder to explain and validate — pick based on how decisions will use it.
42Using social graph events, feed interactions, ad logs, messaging telemetry, and moderation labels, describe an analytical plan to determine whether loading an Instagram feed improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on attribution, funnel analysis, and conversion measurement, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Funnel analysis measures drop-off at each step from acquisition to conversion, revealing the biggest leak to fix. Attribution assigns credit for conversions across touchpoints — last-touch is simple but overcredits the final step, while data-driven or multi-touch models distribute credit more fairly. The measurement must define the conversion window and de-duplicate users to avoid inflating rates.
A step-by-step funnel with per-step conversion:
WITH steps AS (
SELECT user_id,
MAX(step='view') AS s1,
MAX(step='add_cart') AS s2,
MAX(step='checkout') AS s3,
MAX(step='purchase') AS s4
FROM events GROUP BY user_id)
SELECT
SUM(s1) AS viewed,
SUM(s2) AS carted, ROUND(SUM(s2)*100.0/NULLIF(SUM(s1),0),1) AS view_to_cart_pct,
SUM(s3) AS checkout, SUM(s4) AS purchased
FROM steps;Segment the funnel (by channel, device) since the leak often differs by cohort. The tradeoff: last-touch attribution is transparent but biased, while data-driven attribution is fairer but harder to explain and validate — pick based on how decisions will use it.
43How would you design an experiment or causal study for attribution, funnel analysis, and conversion measurement at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on attribution, funnel analysis, and conversion measurement, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Funnel analysis measures drop-off at each step from acquisition to conversion, revealing the biggest leak to fix. Attribution assigns credit for conversions across touchpoints — last-touch is simple but overcredits the final step, while data-driven or multi-touch models distribute credit more fairly. The measurement must define the conversion window and de-duplicate users to avoid inflating rates.
A step-by-step funnel with per-step conversion:
WITH steps AS (
SELECT user_id,
MAX(step='view') AS s1,
MAX(step='add_cart') AS s2,
MAX(step='checkout') AS s3,
MAX(step='purchase') AS s4
FROM events GROUP BY user_id)
SELECT
SUM(s1) AS viewed,
SUM(s2) AS carted, ROUND(SUM(s2)*100.0/NULLIF(SUM(s1),0),1) AS view_to_cart_pct,
SUM(s3) AS checkout, SUM(s4) AS purchased
FROM steps;Segment the funnel (by channel, device) since the leak often differs by cohort. The tradeoff: last-touch attribution is transparent but biased, while data-driven attribution is fairer but harder to explain and validate — pick based on how decisions will use it.
44What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about social graph updates to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on attribution, funnel analysis, and conversion measurement, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
When randomization isn't possible, quasi-experimental methods estimate causal effects from observational data by approximating a control group. Difference-in-differences compares the change over time in a treated group against an untreated group; propensity-score matching pairs treated and control units with similar covariates; instrumental variables and regression discontinuity exploit natural experiments. All rely on assumptions (e.g. parallel trends) that you must test, not assume.
A difference-in-differences estimate via regression:
import statsmodels.formula.api as smf
# treated x post interaction = the causal DiD effect
model = smf.ols('outcome ~ treated + post + treated:post', data=df).fit()
print(model.params['treated:post']) # estimated treatment effect
# validate: pre-period trends of treated vs control should be parallelAlways probe robustness with placebo tests and sensitivity to unobserved confounders. The tradeoff: quasi-experiments are the only option when A/B testing is infeasible, but their validity hinges on untestable assumptions, so present effects with appropriate caveats rather than as clean causal truth.
45Turn an analysis of attribution, funnel analysis, and conversion measurement into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on attribution, funnel analysis, and conversion measurement, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Funnel analysis measures drop-off at each step from acquisition to conversion, revealing the biggest leak to fix. Attribution assigns credit for conversions across touchpoints — last-touch is simple but overcredits the final step, while data-driven or multi-touch models distribute credit more fairly. The measurement must define the conversion window and de-duplicate users to avoid inflating rates.
A step-by-step funnel with per-step conversion:
WITH steps AS (
SELECT user_id,
MAX(step='view') AS s1,
MAX(step='add_cart') AS s2,
MAX(step='checkout') AS s3,
MAX(step='purchase') AS s4
FROM events GROUP BY user_id)
SELECT
SUM(s1) AS viewed,
SUM(s2) AS carted, ROUND(SUM(s2)*100.0/NULLIF(SUM(s1),0),1) AS view_to_cart_pct,
SUM(s3) AS checkout, SUM(s4) AS purchased
FROM steps;Segment the funnel (by channel, device) since the leak often differs by cohort. The tradeoff: last-touch attribution is transparent but biased, while data-driven attribution is fairer but harder to explain and validate — pick based on how decisions will use it.
46Define the North Star metric and supporting guardrail metrics for content moderation queues at Meta Platforms. How would pricing, promotion, auctions, or marketplace analytics influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on pricing, promotion, auctions, or marketplace analytics, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Pricing analytics estimates how demand responds to price (elasticity) to set prices that optimize revenue or margin. In marketplaces you also balance both sides (buyers and sellers) and design incentives/auctions that are efficient and hard to game. Because price and demand are confounded by many factors, causal methods or experiments are needed to isolate true elasticity rather than correlation.
Estimate price elasticity from a log-log demand model:
import statsmodels.formula.api as smf
# log(quantity) ~ log(price): the price coefficient IS the elasticity
m = smf.ols('np.log(qty) ~ np.log(price) + C(week) + C(region)', data=df).fit()
elasticity = m.params['np.log(price)']
print(f"Elasticity: {elasticity:.2f}") # e.g. -1.5 => 10% price up -> 15% qty down
# revenue-optimal only if |elasticity| considered vs marginValidate with a pricing experiment before rolling out, and watch cross-effects (promotions cannibalizing full-price sales). The tradeoff: aggressive price optimization can maximize short-term revenue but erode trust and long-term demand, so include fairness and retention guardrails.
47Using media embeddings and engagement signals, describe an analytical plan to determine whether sending a WhatsApp message improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on pricing, promotion, auctions, or marketplace analytics, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Pricing analytics estimates how demand responds to price (elasticity) to set prices that optimize revenue or margin. In marketplaces you also balance both sides (buyers and sellers) and design incentives/auctions that are efficient and hard to game. Because price and demand are confounded by many factors, causal methods or experiments are needed to isolate true elasticity rather than correlation.
Estimate price elasticity from a log-log demand model:
import statsmodels.formula.api as smf
# log(quantity) ~ log(price): the price coefficient IS the elasticity
m = smf.ols('np.log(qty) ~ np.log(price) + C(week) + C(region)', data=df).fit()
elasticity = m.params['np.log(price)']
print(f"Elasticity: {elasticity:.2f}") # e.g. -1.5 => 10% price up -> 15% qty down
# revenue-optimal only if |elasticity| considered vs marginValidate with a pricing experiment before rolling out, and watch cross-effects (promotions cannibalizing full-price sales). The tradeoff: aggressive price optimization can maximize short-term revenue but erode trust and long-term demand, so include fairness and retention guardrails.
48How would you design an experiment or causal study for pricing, promotion, auctions, or marketplace analytics at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on pricing, promotion, auctions, or marketplace analytics, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Pricing analytics estimates how demand responds to price (elasticity) to set prices that optimize revenue or margin. In marketplaces you also balance both sides (buyers and sellers) and design incentives/auctions that are efficient and hard to game. Because price and demand are confounded by many factors, causal methods or experiments are needed to isolate true elasticity rather than correlation.
Estimate price elasticity from a log-log demand model:
import statsmodels.formula.api as smf
# log(quantity) ~ log(price): the price coefficient IS the elasticity
m = smf.ols('np.log(qty) ~ np.log(price) + C(week) + C(region)', data=df).fit()
elasticity = m.params['np.log(price)']
print(f"Elasticity: {elasticity:.2f}") # e.g. -1.5 => 10% price up -> 15% qty down
# revenue-optimal only if |elasticity| considered vs marginValidate with a pricing experiment before rolling out, and watch cross-effects (promotions cannibalizing full-price sales). The tradeoff: aggressive price optimization can maximize short-term revenue but erode trust and long-term demand, so include fairness and retention guardrails.
49What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about ad impressions to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on pricing, promotion, auctions, or marketplace analytics, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
Segmentation groups entities into meaningful cohorts to tailor decisions. It can be rule-based (RFM: recency, frequency, monetary), or unsupervised (k-means, hierarchical clustering on standardized features). The goal isn't mathematically tight clusters but actionable, stable, and interpretable segments that a team can actually target differently — a segmentation nobody acts on is wasted analysis.
K-means segmentation on standardized behavioral features:
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
X = StandardScaler().fit_transform(df[['recency','frequency','monetary']])
# choose k via elbow/silhouette, not arbitrarily
km = KMeans(n_clusters=4, n_init=10, random_state=0).fit(X)
df['segment'] = km.labels_
df.groupby('segment')[['recency','frequency','monetary']].mean() # profile themProfile and name each segment, and validate stability over time before building programs on it. The tradeoff: more segments enable finer targeting but add operational complexity and shrink sample sizes per segment — keep the count to what the business can genuinely act on.
50Turn an analysis of pricing, promotion, auctions, or marketplace analytics into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on pricing, promotion, auctions, or marketplace analytics, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Pricing analytics estimates how demand responds to price (elasticity) to set prices that optimize revenue or margin. In marketplaces you also balance both sides (buyers and sellers) and design incentives/auctions that are efficient and hard to game. Because price and demand are confounded by many factors, causal methods or experiments are needed to isolate true elasticity rather than correlation.
Estimate price elasticity from a log-log demand model:
import statsmodels.formula.api as smf
# log(quantity) ~ log(price): the price coefficient IS the elasticity
m = smf.ols('np.log(qty) ~ np.log(price) + C(week) + C(region)', data=df).fit()
elasticity = m.params['np.log(price)']
print(f"Elasticity: {elasticity:.2f}") # e.g. -1.5 => 10% price up -> 15% qty down
# revenue-optimal only if |elasticity| considered vs marginValidate with a pricing experiment before rolling out, and watch cross-effects (promotions cannibalizing full-price sales). The tradeoff: aggressive price optimization can maximize short-term revenue but erode trust and long-term demand, so include fairness and retention guardrails.
51Define the North Star metric and supporting guardrail metrics for feed ranking requests at Meta Platforms. How would supply-demand optimization and operations analytics influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on supply-demand optimization and operations analytics, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Operations analytics matches supply to demand under constraints — staffing, inventory, dispatch, routing. Many reduce to optimization problems: minimize cost or maximize fill rate subject to capacity and service-level constraints. You forecast demand, then solve an assignment/allocation problem, often with linear or integer programming, and re-solve as conditions change.
A small allocation LP with a solver:
import pulp
prob = pulp.LpProblem('alloc', pulp.LpMinimize)
x = pulp.LpVariable.dicts('ship', [(w,s) for w in warehouses for s in stores], lowBound=0)
prob += pulp.lpSum(cost[w][s]*x[(w,s)] for w in warehouses for s in stores) # minimize cost
for s in stores: # meet each store's demand
prob += pulp.lpSum(x[(w,s)] for w in warehouses) >= demand[s]
for w in warehouses: # respect warehouse capacity
prob += pulp.lpSum(x[(w,s)] for s in stores) <= supply[w]
prob.solve()Add buffers for forecast error so a service target is met with high probability, not just on average. The tradeoff: a cost-minimal plan can be fragile to demand spikes, so optimize with safety stock/slack tuned to the cost of a stockout vs. holding cost.
52Using privacy-governed user activity data, describe an analytical plan to determine whether serving a targeted ad improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on supply-demand optimization and operations analytics, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Operations analytics matches supply to demand under constraints — staffing, inventory, dispatch, routing. Many reduce to optimization problems: minimize cost or maximize fill rate subject to capacity and service-level constraints. You forecast demand, then solve an assignment/allocation problem, often with linear or integer programming, and re-solve as conditions change.
A small allocation LP with a solver:
import pulp
prob = pulp.LpProblem('alloc', pulp.LpMinimize)
x = pulp.LpVariable.dicts('ship', [(w,s) for w in warehouses for s in stores], lowBound=0)
prob += pulp.lpSum(cost[w][s]*x[(w,s)] for w in warehouses for s in stores) # minimize cost
for s in stores: # meet each store's demand
prob += pulp.lpSum(x[(w,s)] for w in warehouses) >= demand[s]
for w in warehouses: # respect warehouse capacity
prob += pulp.lpSum(x[(w,s)] for s in stores) <= supply[w]
prob.solve()Add buffers for forecast error so a service target is met with high probability, not just on average. The tradeoff: a cost-minimal plan can be fragile to demand spikes, so optimize with safety stock/slack tuned to the cost of a stockout vs. holding cost.
53How would you design an experiment or causal study for supply-demand optimization and operations analytics at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on supply-demand optimization and operations analytics, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Operations analytics matches supply to demand under constraints — staffing, inventory, dispatch, routing. Many reduce to optimization problems: minimize cost or maximize fill rate subject to capacity and service-level constraints. You forecast demand, then solve an assignment/allocation problem, often with linear or integer programming, and re-solve as conditions change.
A small allocation LP with a solver:
import pulp
prob = pulp.LpProblem('alloc', pulp.LpMinimize)
x = pulp.LpVariable.dicts('ship', [(w,s) for w in warehouses for s in stores], lowBound=0)
prob += pulp.lpSum(cost[w][s]*x[(w,s)] for w in warehouses for s in stores) # minimize cost
for s in stores: # meet each store's demand
prob += pulp.lpSum(x[(w,s)] for w in warehouses) >= demand[s]
for w in warehouses: # respect warehouse capacity
prob += pulp.lpSum(x[(w,s)] for s in stores) <= supply[w]
prob.solve()Add buffers for forecast error so a service target is met with high probability, not just on average. The tradeoff: a cost-minimal plan can be fragile to demand spikes, so optimize with safety stock/slack tuned to the cost of a stockout vs. holding cost.
54What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about content moderation queues to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on supply-demand optimization and operations analytics, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
A North Star metric is the single measure that best captures the value your product delivers to users (e.g. weekly active teams, nights booked). You decompose it into an input tree of drivers you can actually influence, so teams see how their work ladders up. Good decomposition is multiplicative or additive and MECE (mutually exclusive, collectively exhaustive) so no driver is double-counted.
Decompose revenue into levers you can move, then size each:
-- Revenue = Users x Conversion x Orders/converter x AOV
SELECT
COUNT(DISTINCT user_id) AS users,
COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END)*1.0
/ COUNT(DISTINCT user_id) AS conversion,
SUM(orders)*1.0
/ NULLIF(COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END),0) AS orders_per_converter,
SUM(revenue)*1.0 / NULLIF(SUM(orders),0) AS aov
FROM user_month WHERE month = '2026-08-01';Pick a North Star that leads revenue rather than lagging it, so it's actionable. The tradeoff: a single metric can be gamed or miss quality, so pair it with guardrail metrics (e.g. retention, complaints) that prevent optimizing the number at the users' expense.
55Turn an analysis of supply-demand optimization and operations analytics into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on supply-demand optimization and operations analytics, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Operations analytics matches supply to demand under constraints — staffing, inventory, dispatch, routing. Many reduce to optimization problems: minimize cost or maximize fill rate subject to capacity and service-level constraints. You forecast demand, then solve an assignment/allocation problem, often with linear or integer programming, and re-solve as conditions change.
A small allocation LP with a solver:
import pulp
prob = pulp.LpProblem('alloc', pulp.LpMinimize)
x = pulp.LpVariable.dicts('ship', [(w,s) for w in warehouses for s in stores], lowBound=0)
prob += pulp.lpSum(cost[w][s]*x[(w,s)] for w in warehouses for s in stores) # minimize cost
for s in stores: # meet each store's demand
prob += pulp.lpSum(x[(w,s)] for w in warehouses) >= demand[s]
for w in warehouses: # respect warehouse capacity
prob += pulp.lpSum(x[(w,s)] for s in stores) <= supply[w]
prob.solve()Add buffers for forecast error so a service target is met with high probability, not just on average. The tradeoff: a cost-minimal plan can be fragile to demand spikes, so optimize with safety stock/slack tuned to the cost of a stockout vs. holding cost.
56Define the North Star metric and supporting guardrail metrics for messaging delivery at Meta Platforms. How would statistical modeling and uncertainty quantification influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on statistical modeling and uncertainty quantification, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Sound analysis reports not just point estimates but the uncertainty around them, so decisions account for how much the data actually supports a conclusion. Confidence intervals, bootstrapping, and Bayesian credible intervals all quantify this. Bootstrapping is especially handy: resample the data many times to get an empirical distribution of any statistic without assuming a formula.
A bootstrap confidence interval for a metric:
import numpy as np
def bootstrap_ci(data, stat=np.mean, n=10000, alpha=0.05):
boots = [stat(np.random.choice(data, len(data), replace=True)) for _ in range(n)]
lo, hi = np.percentile(boots, [100*alpha/2, 100*(1-alpha/2)])
return stat(data), (lo, hi)
est, (lo, hi) = bootstrap_ci(conversion_flags)
print(f"conversion={est:.3f} 95% CI=({lo:.3f}, {hi:.3f})")Present intervals alongside estimates so stakeholders don't overreact to noise. The tradeoff: quantifying uncertainty rigorously can slow analysis and complicate the narrative, but omitting it invites overconfident decisions on thin evidence.
57Using social graph events, feed interactions, ad logs, messaging telemetry, and moderation labels, describe an analytical plan to determine whether moderating harmful content improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on statistical modeling and uncertainty quantification, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Sound analysis reports not just point estimates but the uncertainty around them, so decisions account for how much the data actually supports a conclusion. Confidence intervals, bootstrapping, and Bayesian credible intervals all quantify this. Bootstrapping is especially handy: resample the data many times to get an empirical distribution of any statistic without assuming a formula.
A bootstrap confidence interval for a metric:
import numpy as np
def bootstrap_ci(data, stat=np.mean, n=10000, alpha=0.05):
boots = [stat(np.random.choice(data, len(data), replace=True)) for _ in range(n)]
lo, hi = np.percentile(boots, [100*alpha/2, 100*(1-alpha/2)])
return stat(data), (lo, hi)
est, (lo, hi) = bootstrap_ci(conversion_flags)
print(f"conversion={est:.3f} 95% CI=({lo:.3f}, {hi:.3f})")Present intervals alongside estimates so stakeholders don't overreact to noise. The tradeoff: quantifying uncertainty rigorously can slow analysis and complicate the narrative, but omitting it invites overconfident decisions on thin evidence.
58How would you design an experiment or causal study for statistical modeling and uncertainty quantification at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on statistical modeling and uncertainty quantification, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Sound analysis reports not just point estimates but the uncertainty around them, so decisions account for how much the data actually supports a conclusion. Confidence intervals, bootstrapping, and Bayesian credible intervals all quantify this. Bootstrapping is especially handy: resample the data many times to get an empirical distribution of any statistic without assuming a formula.
A bootstrap confidence interval for a metric:
import numpy as np
def bootstrap_ci(data, stat=np.mean, n=10000, alpha=0.05):
boots = [stat(np.random.choice(data, len(data), replace=True)) for _ in range(n)]
lo, hi = np.percentile(boots, [100*alpha/2, 100*(1-alpha/2)])
return stat(data), (lo, hi)
est, (lo, hi) = bootstrap_ci(conversion_flags)
print(f"conversion={est:.3f} 95% CI=({lo:.3f}, {hi:.3f})")Present intervals alongside estimates so stakeholders don't overreact to noise. The tradeoff: quantifying uncertainty rigorously can slow analysis and complicate the narrative, but omitting it invites overconfident decisions on thin evidence.
59What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about feed ranking requests to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on statistical modeling and uncertainty quantification, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
Analytical SQL sits on a dimensional model: fact tables hold events/measures at a defined grain, dimension tables hold descriptive attributes, joined on keys (a star schema). Getting the grain right — one row per what — is the foundation; mixing grains causes double-counting. Window functions handle the analytics that GROUP BY can't: running totals, rankings, and period-over-period comparisons without self-joins.
A window-function query for per-user order recency and running spend:
SELECT
user_id, order_id, order_ts, amount,
ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY order_ts) AS order_seq,
SUM(amount) OVER (PARTITION BY user_id ORDER BY order_ts) AS running_spend,
order_ts - LAG(order_ts) OVER (PARTITION BY user_id
ORDER BY order_ts) AS gap_since_prev
FROM fct_orders;Pre-aggregate heavy queries into summary tables for dashboards, and document the grain of every fact. The tradeoff: a fully normalized model saves storage but requires many joins; a denormalized/star model is faster to query and easier for analysts, which is why warehouses favor it.
60Turn an analysis of statistical modeling and uncertainty quantification into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on statistical modeling and uncertainty quantification, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Sound analysis reports not just point estimates but the uncertainty around them, so decisions account for how much the data actually supports a conclusion. Confidence intervals, bootstrapping, and Bayesian credible intervals all quantify this. Bootstrapping is especially handy: resample the data many times to get an empirical distribution of any statistic without assuming a formula.
A bootstrap confidence interval for a metric:
import numpy as np
def bootstrap_ci(data, stat=np.mean, n=10000, alpha=0.05):
boots = [stat(np.random.choice(data, len(data), replace=True)) for _ in range(n)]
lo, hi = np.percentile(boots, [100*alpha/2, 100*(1-alpha/2)])
return stat(data), (lo, hi)
est, (lo, hi) = bootstrap_ci(conversion_flags)
print(f"conversion={est:.3f} 95% CI=({lo:.3f}, {hi:.3f})")Present intervals alongside estimates so stakeholders don't overreact to noise. The tradeoff: quantifying uncertainty rigorously can slow analysis and complicate the narrative, but omitting it invites overconfident decisions on thin evidence.
61Define the North Star metric and supporting guardrail metrics for social graph updates at Meta Platforms. How would sample-size, power, and sequential testing influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on sample-size, power, and sequential testing, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Power analysis determines how much data an experiment needs to reliably detect a given effect (the minimum detectable effect, MDE). It ties together significance level (alpha), power (1 - beta), baseline rate, and MDE — fix three and solve the fourth. Sequential testing lets you monitor and stop early when results are decisive, but only with a correction (alpha spending) that preserves the false-positive rate against repeated peeking.
MDE for a fixed runtime, and why naive peeking inflates error:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
# given fixed n per arm, what effect can we detect at 80% power?
n, baseline = 50000, 0.10
def power_at(mde):
es = proportion_effectsize(baseline, baseline+mde)
return NormalIndPower().power(es, nobs1=n, alpha=0.05)
mde = next(m for m in [i/1000 for i in range(1,50)] if power_at(m) >= 0.8)
print(f"MDE at n={n}: {mde:.3f}") # sequential/peeking needs alpha-spendingPre-commit to a runtime or a proper sequential method. The tradeoff: fixed-horizon tests are simple and robust but slower; sequential tests end faster but require careful statistics to avoid the inflated false positives that casual peeking causes.
62Using media embeddings and engagement signals, describe an analytical plan to determine whether syncing a VR device improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on sample-size, power, and sequential testing, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Power analysis determines how much data an experiment needs to reliably detect a given effect (the minimum detectable effect, MDE). It ties together significance level (alpha), power (1 - beta), baseline rate, and MDE — fix three and solve the fourth. Sequential testing lets you monitor and stop early when results are decisive, but only with a correction (alpha spending) that preserves the false-positive rate against repeated peeking.
MDE for a fixed runtime, and why naive peeking inflates error:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
# given fixed n per arm, what effect can we detect at 80% power?
n, baseline = 50000, 0.10
def power_at(mde):
es = proportion_effectsize(baseline, baseline+mde)
return NormalIndPower().power(es, nobs1=n, alpha=0.05)
mde = next(m for m in [i/1000 for i in range(1,50)] if power_at(m) >= 0.8)
print(f"MDE at n={n}: {mde:.3f}") # sequential/peeking needs alpha-spendingPre-commit to a runtime or a proper sequential method. The tradeoff: fixed-horizon tests are simple and robust but slower; sequential tests end faster but require careful statistics to avoid the inflated false positives that casual peeking causes.
63How would you design an experiment or causal study for sample-size, power, and sequential testing at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on sample-size, power, and sequential testing, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Power analysis determines how much data an experiment needs to reliably detect a given effect (the minimum detectable effect, MDE). It ties together significance level (alpha), power (1 - beta), baseline rate, and MDE — fix three and solve the fourth. Sequential testing lets you monitor and stop early when results are decisive, but only with a correction (alpha spending) that preserves the false-positive rate against repeated peeking.
MDE for a fixed runtime, and why naive peeking inflates error:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
# given fixed n per arm, what effect can we detect at 80% power?
n, baseline = 50000, 0.10
def power_at(mde):
es = proportion_effectsize(baseline, baseline+mde)
return NormalIndPower().power(es, nobs1=n, alpha=0.05)
mde = next(m for m in [i/1000 for i in range(1,50)] if power_at(m) >= 0.8)
print(f"MDE at n={n}: {mde:.3f}") # sequential/peeking needs alpha-spendingPre-commit to a runtime or a proper sequential method. The tradeoff: fixed-horizon tests are simple and robust but slower; sequential tests end faster but require careful statistics to avoid the inflated false positives that casual peeking causes.
64What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about messaging delivery to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on sample-size, power, and sequential testing, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
An A/B test isolates causal impact by randomly assigning users to control and treatment, so the only systematic difference is the change. Sound design means randomizing at the right unit (usually the user, to avoid within-user contamination), choosing one primary metric plus guardrails in advance, and computing the required sample size for adequate power before launch — underpowered tests waste traffic and mislead.
Sample-size calculation before you launch:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
baseline, mde = 0.10, 0.005 # detect a 0.5pp absolute lift
effect = proportion_effectsize(baseline, baseline + mde)
n = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8)
print(f"Need ~{n:,.0f} users per arm")Check for sample-ratio mismatch (a broken split invalidates results) and don't peek-and-stop without a sequential correction. The tradeoff: bigger effects need less traffic, so for small expected lifts you need large samples or longer runtime — decide if the change is worth the experimental cost.
65Turn an analysis of sample-size, power, and sequential testing into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on sample-size, power, and sequential testing, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Power analysis determines how much data an experiment needs to reliably detect a given effect (the minimum detectable effect, MDE). It ties together significance level (alpha), power (1 - beta), baseline rate, and MDE — fix three and solve the fourth. Sequential testing lets you monitor and stop early when results are decisive, but only with a correction (alpha spending) that preserves the false-positive rate against repeated peeking.
MDE for a fixed runtime, and why naive peeking inflates error:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
# given fixed n per arm, what effect can we detect at 80% power?
n, baseline = 50000, 0.10
def power_at(mde):
es = proportion_effectsize(baseline, baseline+mde)
return NormalIndPower().power(es, nobs1=n, alpha=0.05)
mde = next(m for m in [i/1000 for i in range(1,50)] if power_at(m) >= 0.8)
print(f"MDE at n={n}: {mde:.3f}") # sequential/peeking needs alpha-spendingPre-commit to a runtime or a proper sequential method. The tradeoff: fixed-horizon tests are simple and robust but slower; sequential tests end faster but require careful statistics to avoid the inflated false positives that casual peeking causes.
66Define the North Star metric and supporting guardrail metrics for ad impressions at Meta Platforms. How would dashboard design and executive storytelling influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on dashboard design and executive storytelling, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
An effective dashboard answers a specific question at a glance, leading with the headline metric and its trend, then supporting breakdowns. Executive storytelling inverts the analyst's process: state the conclusion and 'so what' first, then the evidence. Reduce cognitive load — few, well-chosen charts, clear labels, consistent definitions — so the audience acts rather than deciphers.
Structure the underlying query so the dashboard shows metric + trend + delta:
SELECT
metric_date,
daily_active,
AVG(daily_active) OVER (ORDER BY metric_date
ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS trailing_7d,
daily_active - LAG(daily_active, 7) OVER (ORDER BY metric_date) AS wow_delta
FROM daily_metrics ORDER BY metric_date DESC;Annotate anomalies and launches directly on the chart so viewers understand causes. The tradeoff: dense dashboards feel thorough but bury the signal — resist adding every metric, and design for the one decision the dashboard exists to support.
67Using privacy-governed user activity data, describe an analytical plan to determine whether loading an Instagram feed improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on dashboard design and executive storytelling, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
An effective dashboard answers a specific question at a glance, leading with the headline metric and its trend, then supporting breakdowns. Executive storytelling inverts the analyst's process: state the conclusion and 'so what' first, then the evidence. Reduce cognitive load — few, well-chosen charts, clear labels, consistent definitions — so the audience acts rather than deciphers.
Structure the underlying query so the dashboard shows metric + trend + delta:
SELECT
metric_date,
daily_active,
AVG(daily_active) OVER (ORDER BY metric_date
ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS trailing_7d,
daily_active - LAG(daily_active, 7) OVER (ORDER BY metric_date) AS wow_delta
FROM daily_metrics ORDER BY metric_date DESC;Annotate anomalies and launches directly on the chart so viewers understand causes. The tradeoff: dense dashboards feel thorough but bury the signal — resist adding every metric, and design for the one decision the dashboard exists to support.
68How would you design an experiment or causal study for dashboard design and executive storytelling at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on dashboard design and executive storytelling, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
An effective dashboard answers a specific question at a glance, leading with the headline metric and its trend, then supporting breakdowns. Executive storytelling inverts the analyst's process: state the conclusion and 'so what' first, then the evidence. Reduce cognitive load — few, well-chosen charts, clear labels, consistent definitions — so the audience acts rather than deciphers.
Structure the underlying query so the dashboard shows metric + trend + delta:
SELECT
metric_date,
daily_active,
AVG(daily_active) OVER (ORDER BY metric_date
ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS trailing_7d,
daily_active - LAG(daily_active, 7) OVER (ORDER BY metric_date) AS wow_delta
FROM daily_metrics ORDER BY metric_date DESC;Annotate anomalies and launches directly on the chart so viewers understand causes. The tradeoff: dense dashboards feel thorough but bury the signal — resist adding every metric, and design for the one decision the dashboard exists to support.
69What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about social graph updates to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on dashboard design and executive storytelling, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
When randomization isn't possible, quasi-experimental methods estimate causal effects from observational data by approximating a control group. Difference-in-differences compares the change over time in a treated group against an untreated group; propensity-score matching pairs treated and control units with similar covariates; instrumental variables and regression discontinuity exploit natural experiments. All rely on assumptions (e.g. parallel trends) that you must test, not assume.
A difference-in-differences estimate via regression:
import statsmodels.formula.api as smf
# treated x post interaction = the causal DiD effect
model = smf.ols('outcome ~ treated + post + treated:post', data=df).fit()
print(model.params['treated:post']) # estimated treatment effect
# validate: pre-period trends of treated vs control should be parallelAlways probe robustness with placebo tests and sensitivity to unobserved confounders. The tradeoff: quasi-experiments are the only option when A/B testing is infeasible, but their validity hinges on untestable assumptions, so present effects with appropriate caveats rather than as clean causal truth.
70Turn an analysis of dashboard design and executive storytelling into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on dashboard design and executive storytelling, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
An effective dashboard answers a specific question at a glance, leading with the headline metric and its trend, then supporting breakdowns. Executive storytelling inverts the analyst's process: state the conclusion and 'so what' first, then the evidence. Reduce cognitive load — few, well-chosen charts, clear labels, consistent definitions — so the audience acts rather than deciphers.
Structure the underlying query so the dashboard shows metric + trend + delta:
SELECT
metric_date,
daily_active,
AVG(daily_active) OVER (ORDER BY metric_date
ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS trailing_7d,
daily_active - LAG(daily_active, 7) OVER (ORDER BY metric_date) AS wow_delta
FROM daily_metrics ORDER BY metric_date DESC;Annotate anomalies and launches directly on the chart so viewers understand causes. The tradeoff: dense dashboards feel thorough but bury the signal — resist adding every metric, and design for the one decision the dashboard exists to support.
71Define the North Star metric and supporting guardrail metrics for content moderation queues at Meta Platforms. How would data quality, missingness, and instrumentation validation influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on data quality, missingness, and instrumentation validation, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Trustworthy analysis depends on trustworthy data. You validate instrumentation (are events firing correctly and completely?), profile missingness (is it random, or systematically tied to the outcome?), and enforce contracts on schema, volume, and freshness. Missing-not-at-random is dangerous — dropping such rows biases results, so understand the mechanism before imputing or excluding.
Automated data-quality checks that fail the pipeline on regressions:
-- Daily data-quality assertions (run in dbt tests / Great Expectations)
SELECT
COUNT(*) AS row_count, -- volume
SUM(CASE WHEN user_id IS NULL THEN 1 END) AS null_user_ids, -- completeness
COUNT(DISTINCT event_id) AS distinct_events, -- uniqueness
MAX(event_ts) AS freshness
FROM events
WHERE event_date = CURRENT_DATE - 1
HAVING SUM(CASE WHEN user_id IS NULL THEN 1 END) = 0; -- fail if any nullsAlert on volume anomalies (a 50% drop usually means broken tracking, not real behavior). The tradeoff: strict quality gates can block pipelines on minor issues, so tier checks — hard-fail on correctness, warn on cosmetic — and document known data caveats for analysts.
72Using social graph events, feed interactions, ad logs, messaging telemetry, and moderation labels, describe an analytical plan to determine whether sending a WhatsApp message improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on data quality, missingness, and instrumentation validation, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Trustworthy analysis depends on trustworthy data. You validate instrumentation (are events firing correctly and completely?), profile missingness (is it random, or systematically tied to the outcome?), and enforce contracts on schema, volume, and freshness. Missing-not-at-random is dangerous — dropping such rows biases results, so understand the mechanism before imputing or excluding.
Automated data-quality checks that fail the pipeline on regressions:
-- Daily data-quality assertions (run in dbt tests / Great Expectations)
SELECT
COUNT(*) AS row_count, -- volume
SUM(CASE WHEN user_id IS NULL THEN 1 END) AS null_user_ids, -- completeness
COUNT(DISTINCT event_id) AS distinct_events, -- uniqueness
MAX(event_ts) AS freshness
FROM events
WHERE event_date = CURRENT_DATE - 1
HAVING SUM(CASE WHEN user_id IS NULL THEN 1 END) = 0; -- fail if any nullsAlert on volume anomalies (a 50% drop usually means broken tracking, not real behavior). The tradeoff: strict quality gates can block pipelines on minor issues, so tier checks — hard-fail on correctness, warn on cosmetic — and document known data caveats for analysts.
73How would you design an experiment or causal study for data quality, missingness, and instrumentation validation at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on data quality, missingness, and instrumentation validation, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Trustworthy analysis depends on trustworthy data. You validate instrumentation (are events firing correctly and completely?), profile missingness (is it random, or systematically tied to the outcome?), and enforce contracts on schema, volume, and freshness. Missing-not-at-random is dangerous — dropping such rows biases results, so understand the mechanism before imputing or excluding.
Automated data-quality checks that fail the pipeline on regressions:
-- Daily data-quality assertions (run in dbt tests / Great Expectations)
SELECT
COUNT(*) AS row_count, -- volume
SUM(CASE WHEN user_id IS NULL THEN 1 END) AS null_user_ids, -- completeness
COUNT(DISTINCT event_id) AS distinct_events, -- uniqueness
MAX(event_ts) AS freshness
FROM events
WHERE event_date = CURRENT_DATE - 1
HAVING SUM(CASE WHEN user_id IS NULL THEN 1 END) = 0; -- fail if any nullsAlert on volume anomalies (a 50% drop usually means broken tracking, not real behavior). The tradeoff: strict quality gates can block pipelines on minor issues, so tier checks — hard-fail on correctness, warn on cosmetic — and document known data caveats for analysts.
74What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about ad impressions to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on data quality, missingness, and instrumentation validation, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
Segmentation groups entities into meaningful cohorts to tailor decisions. It can be rule-based (RFM: recency, frequency, monetary), or unsupervised (k-means, hierarchical clustering on standardized features). The goal isn't mathematically tight clusters but actionable, stable, and interpretable segments that a team can actually target differently — a segmentation nobody acts on is wasted analysis.
K-means segmentation on standardized behavioral features:
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
X = StandardScaler().fit_transform(df[['recency','frequency','monetary']])
# choose k via elbow/silhouette, not arbitrarily
km = KMeans(n_clusters=4, n_init=10, random_state=0).fit(X)
df['segment'] = km.labels_
df.groupby('segment')[['recency','frequency','monetary']].mean() # profile themProfile and name each segment, and validate stability over time before building programs on it. The tradeoff: more segments enable finer targeting but add operational complexity and shrink sample sizes per segment — keep the count to what the business can genuinely act on.
75Turn an analysis of data quality, missingness, and instrumentation validation into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on data quality, missingness, and instrumentation validation, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Trustworthy analysis depends on trustworthy data. You validate instrumentation (are events firing correctly and completely?), profile missingness (is it random, or systematically tied to the outcome?), and enforce contracts on schema, volume, and freshness. Missing-not-at-random is dangerous — dropping such rows biases results, so understand the mechanism before imputing or excluding.
Automated data-quality checks that fail the pipeline on regressions:
-- Daily data-quality assertions (run in dbt tests / Great Expectations)
SELECT
COUNT(*) AS row_count, -- volume
SUM(CASE WHEN user_id IS NULL THEN 1 END) AS null_user_ids, -- completeness
COUNT(DISTINCT event_id) AS distinct_events, -- uniqueness
MAX(event_ts) AS freshness
FROM events
WHERE event_date = CURRENT_DATE - 1
HAVING SUM(CASE WHEN user_id IS NULL THEN 1 END) = 0; -- fail if any nullsAlert on volume anomalies (a 50% drop usually means broken tracking, not real behavior). The tradeoff: strict quality gates can block pipelines on minor issues, so tier checks — hard-fail on correctness, warn on cosmetic — and document known data caveats for analysts.
76Define the North Star metric and supporting guardrail metrics for feed ranking requests at Meta Platforms. How would privacy, aggregation, and measurement under constraints influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on privacy, aggregation, and measurement under constraints, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Measuring under privacy constraints means deriving insight without exposing individuals. Techniques include aggregation with minimum cohort sizes (k-anonymity — suppress cells below a threshold), differential privacy (calibrated noise with a formal budget), and working from privacy-safe aggregates rather than row-level PII. The analyst's job is to answer the business question within these limits, not around them.
Enforce a k-anonymity threshold on a reporting query:
SELECT region, age_band, COUNT(*) AS users, AVG(spend) AS avg_spend
FROM customers
GROUP BY region, age_band
HAVING COUNT(*) >= 25; -- suppress small cells that could re-identify individualsAvoid joining datasets in ways that re-identify people, and log access to sensitive aggregates. The tradeoff is precision vs. privacy — noise and suppression reduce granularity, so design metrics that remain useful at the aggregation level your privacy policy allows.
77Using media embeddings and engagement signals, describe an analytical plan to determine whether serving a targeted ad improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on privacy, aggregation, and measurement under constraints, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
A North Star metric is the single measure that best captures the value your product delivers to users (e.g. weekly active teams, nights booked). You decompose it into an input tree of drivers you can actually influence, so teams see how their work ladders up. Good decomposition is multiplicative or additive and MECE (mutually exclusive, collectively exhaustive) so no driver is double-counted.
Decompose revenue into levers you can move, then size each:
-- Revenue = Users x Conversion x Orders/converter x AOV
SELECT
COUNT(DISTINCT user_id) AS users,
COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END)*1.0
/ COUNT(DISTINCT user_id) AS conversion,
SUM(orders)*1.0
/ NULLIF(COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END),0) AS orders_per_converter,
SUM(revenue)*1.0 / NULLIF(SUM(orders),0) AS aov
FROM user_month WHERE month = '2026-08-01';Pick a North Star that leads revenue rather than lagging it, so it's actionable. The tradeoff: a single metric can be gamed or miss quality, so pair it with guardrail metrics (e.g. retention, complaints) that prevent optimizing the number at the users' expense.
78How would you design an experiment or causal study for privacy, aggregation, and measurement under constraints at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on privacy, aggregation, and measurement under constraints, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Measuring under privacy constraints means deriving insight without exposing individuals. Techniques include aggregation with minimum cohort sizes (k-anonymity — suppress cells below a threshold), differential privacy (calibrated noise with a formal budget), and working from privacy-safe aggregates rather than row-level PII. The analyst's job is to answer the business question within these limits, not around them.
Enforce a k-anonymity threshold on a reporting query:
SELECT region, age_band, COUNT(*) AS users, AVG(spend) AS avg_spend
FROM customers
GROUP BY region, age_band
HAVING COUNT(*) >= 25; -- suppress small cells that could re-identify individualsAvoid joining datasets in ways that re-identify people, and log access to sensitive aggregates. The tradeoff is precision vs. privacy — noise and suppression reduce granularity, so design metrics that remain useful at the aggregation level your privacy policy allows.
79What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about content moderation queues to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on privacy, aggregation, and measurement under constraints, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
A North Star metric is the single measure that best captures the value your product delivers to users (e.g. weekly active teams, nights booked). You decompose it into an input tree of drivers you can actually influence, so teams see how their work ladders up. Good decomposition is multiplicative or additive and MECE (mutually exclusive, collectively exhaustive) so no driver is double-counted.
Decompose revenue into levers you can move, then size each:
-- Revenue = Users x Conversion x Orders/converter x AOV
SELECT
COUNT(DISTINCT user_id) AS users,
COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END)*1.0
/ COUNT(DISTINCT user_id) AS conversion,
SUM(orders)*1.0
/ NULLIF(COUNT(DISTINCT CASE WHEN orders>0 THEN user_id END),0) AS orders_per_converter,
SUM(revenue)*1.0 / NULLIF(SUM(orders),0) AS aov
FROM user_month WHERE month = '2026-08-01';Pick a North Star that leads revenue rather than lagging it, so it's actionable. The tradeoff: a single metric can be gamed or miss quality, so pair it with guardrail metrics (e.g. retention, complaints) that prevent optimizing the number at the users' expense.
80Turn an analysis of privacy, aggregation, and measurement under constraints into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on privacy, aggregation, and measurement under constraints, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Measuring under privacy constraints means deriving insight without exposing individuals. Techniques include aggregation with minimum cohort sizes (k-anonymity — suppress cells below a threshold), differential privacy (calibrated noise with a formal budget), and working from privacy-safe aggregates rather than row-level PII. The analyst's job is to answer the business question within these limits, not around them.
Enforce a k-anonymity threshold on a reporting query:
SELECT region, age_band, COUNT(*) AS users, AVG(spend) AS avg_spend
FROM customers
GROUP BY region, age_band
HAVING COUNT(*) >= 25; -- suppress small cells that could re-identify individualsAvoid joining datasets in ways that re-identify people, and log access to sensitive aggregates. The tradeoff is precision vs. privacy — noise and suppression reduce granularity, so design metrics that remain useful at the aggregation level your privacy policy allows.
81Define the North Star metric and supporting guardrail metrics for messaging delivery at Meta Platforms. How would cohort analysis and long-term impact measurement influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on cohort analysis and long-term impact measurement, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Cohort analysis groups users by a shared start event (signup month, first-purchase week) and tracks them over time, separating the effect of a change from mix shifts in who's arriving. It's essential for measuring long-term impact — a feature might lift day-1 engagement but you only see its true value in how cohorts retain and monetize over months.
Compare LTV curves across acquisition cohorts:
SELECT
DATE_TRUNC('month', signup_ts) AS cohort,
DATE_DIFF('month', signup_ts, txn_month) AS month_n,
SUM(revenue)*1.0 / COUNT(DISTINCT user_id) AS cumulative_ltv
FROM users JOIN transactions USING (user_id)
GROUP BY 1, 2 ORDER BY 1, 2; -- read down a cohort to see LTV maturationWait for cohorts to mature before declaring long-term wins, and use early proxies cautiously. The tradeoff: long-term measurement is the most honest but the slowest, so pair it with validated leading indicators to make decisions before full maturity while confirming later.
82Using privacy-governed user activity data, describe an analytical plan to determine whether moderating harmful content improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on cohort analysis and long-term impact measurement, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Analytical SQL sits on a dimensional model: fact tables hold events/measures at a defined grain, dimension tables hold descriptive attributes, joined on keys (a star schema). Getting the grain right — one row per what — is the foundation; mixing grains causes double-counting. Window functions handle the analytics that GROUP BY can't: running totals, rankings, and period-over-period comparisons without self-joins.
A window-function query for per-user order recency and running spend:
SELECT
user_id, order_id, order_ts, amount,
ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY order_ts) AS order_seq,
SUM(amount) OVER (PARTITION BY user_id ORDER BY order_ts) AS running_spend,
order_ts - LAG(order_ts) OVER (PARTITION BY user_id
ORDER BY order_ts) AS gap_since_prev
FROM fct_orders;Pre-aggregate heavy queries into summary tables for dashboards, and document the grain of every fact. The tradeoff: a fully normalized model saves storage but requires many joins; a denormalized/star model is faster to query and easier for analysts, which is why warehouses favor it.
83How would you design an experiment or causal study for cohort analysis and long-term impact measurement at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on cohort analysis and long-term impact measurement, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Cohort analysis groups users by a shared start event (signup month, first-purchase week) and tracks them over time, separating the effect of a change from mix shifts in who's arriving. It's essential for measuring long-term impact — a feature might lift day-1 engagement but you only see its true value in how cohorts retain and monetize over months.
Compare LTV curves across acquisition cohorts:
SELECT
DATE_TRUNC('month', signup_ts) AS cohort,
DATE_DIFF('month', signup_ts, txn_month) AS month_n,
SUM(revenue)*1.0 / COUNT(DISTINCT user_id) AS cumulative_ltv
FROM users JOIN transactions USING (user_id)
GROUP BY 1, 2 ORDER BY 1, 2; -- read down a cohort to see LTV maturationWait for cohorts to mature before declaring long-term wins, and use early proxies cautiously. The tradeoff: long-term measurement is the most honest but the slowest, so pair it with validated leading indicators to make decisions before full maturity while confirming later.
84What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about feed ranking requests to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on cohort analysis and long-term impact measurement, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
Analytical SQL sits on a dimensional model: fact tables hold events/measures at a defined grain, dimension tables hold descriptive attributes, joined on keys (a star schema). Getting the grain right — one row per what — is the foundation; mixing grains causes double-counting. Window functions handle the analytics that GROUP BY can't: running totals, rankings, and period-over-period comparisons without self-joins.
A window-function query for per-user order recency and running spend:
SELECT
user_id, order_id, order_ts, amount,
ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY order_ts) AS order_seq,
SUM(amount) OVER (PARTITION BY user_id ORDER BY order_ts) AS running_spend,
order_ts - LAG(order_ts) OVER (PARTITION BY user_id
ORDER BY order_ts) AS gap_since_prev
FROM fct_orders;Pre-aggregate heavy queries into summary tables for dashboards, and document the grain of every fact. The tradeoff: a fully normalized model saves storage but requires many joins; a denormalized/star model is faster to query and easier for analysts, which is why warehouses favor it.
85Turn an analysis of cohort analysis and long-term impact measurement into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on cohort analysis and long-term impact measurement, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Cohort analysis groups users by a shared start event (signup month, first-purchase week) and tracks them over time, separating the effect of a change from mix shifts in who's arriving. It's essential for measuring long-term impact — a feature might lift day-1 engagement but you only see its true value in how cohorts retain and monetize over months.
Compare LTV curves across acquisition cohorts:
SELECT
DATE_TRUNC('month', signup_ts) AS cohort,
DATE_DIFF('month', signup_ts, txn_month) AS month_n,
SUM(revenue)*1.0 / COUNT(DISTINCT user_id) AS cumulative_ltv
FROM users JOIN transactions USING (user_id)
GROUP BY 1, 2 ORDER BY 1, 2; -- read down a cohort to see LTV maturationWait for cohorts to mature before declaring long-term wins, and use early proxies cautiously. The tradeoff: long-term measurement is the most honest but the slowest, so pair it with validated leading indicators to make decisions before full maturity while confirming later.
86Define the North Star metric and supporting guardrail metrics for social graph updates at Meta Platforms. How would launch readiness, guardrails, and post-launch readouts influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on launch readiness, guardrails, and post-launch readouts, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Launch analytics de-risks releases with a pre-launch checklist (instrumentation verified, primary and guardrail metrics defined, rollback plan), a monitored rollout (staged exposure with automated guardrail alerts), and a post-launch readout that states whether the launch met its goal against the pre-registered metric. Guardrails catch harm the primary metric misses — latency, errors, complaints, revenue per session.
A guardrail check that would halt a rollout:
-- Compare treatment vs control on a guardrail during staged rollout
SELECT variant,
AVG(latency_ms) AS avg_latency,
SUM(is_error)*1.0 / COUNT(*) AS error_rate
FROM exposures
WHERE exposure_date >= CURRENT_DATE - 1
GROUP BY variant;
-- if treatment error_rate or latency regresses beyond threshold -> auto-holdWrite the readout around the decision (ship / iterate / roll back), not just numbers. The tradeoff: thorough launch analytics slows shipping, so scale rigor to risk — a copy tweak needs light monitoring, a checkout change needs full guardrails and staged exposure.
87Using social graph events, feed interactions, ad logs, messaging telemetry, and moderation labels, describe an analytical plan to determine whether syncing a VR device improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on launch readiness, guardrails, and post-launch readouts, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
An A/B test isolates causal impact by randomly assigning users to control and treatment, so the only systematic difference is the change. Sound design means randomizing at the right unit (usually the user, to avoid within-user contamination), choosing one primary metric plus guardrails in advance, and computing the required sample size for adequate power before launch — underpowered tests waste traffic and mislead.
Sample-size calculation before you launch:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
baseline, mde = 0.10, 0.005 # detect a 0.5pp absolute lift
effect = proportion_effectsize(baseline, baseline + mde)
n = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8)
print(f"Need ~{n:,.0f} users per arm")Check for sample-ratio mismatch (a broken split invalidates results) and don't peek-and-stop without a sequential correction. The tradeoff: bigger effects need less traffic, so for small expected lifts you need large samples or longer runtime — decide if the change is worth the experimental cost.
88How would you design an experiment or causal study for launch readiness, guardrails, and post-launch readouts at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on launch readiness, guardrails, and post-launch readouts, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Launch analytics de-risks releases with a pre-launch checklist (instrumentation verified, primary and guardrail metrics defined, rollback plan), a monitored rollout (staged exposure with automated guardrail alerts), and a post-launch readout that states whether the launch met its goal against the pre-registered metric. Guardrails catch harm the primary metric misses — latency, errors, complaints, revenue per session.
A guardrail check that would halt a rollout:
-- Compare treatment vs control on a guardrail during staged rollout
SELECT variant,
AVG(latency_ms) AS avg_latency,
SUM(is_error)*1.0 / COUNT(*) AS error_rate
FROM exposures
WHERE exposure_date >= CURRENT_DATE - 1
GROUP BY variant;
-- if treatment error_rate or latency regresses beyond threshold -> auto-holdWrite the readout around the decision (ship / iterate / roll back), not just numbers. The tradeoff: thorough launch analytics slows shipping, so scale rigor to risk — a copy tweak needs light monitoring, a checkout change needs full guardrails and staged exposure.
89What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about messaging delivery to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on launch readiness, guardrails, and post-launch readouts, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
An A/B test isolates causal impact by randomly assigning users to control and treatment, so the only systematic difference is the change. Sound design means randomizing at the right unit (usually the user, to avoid within-user contamination), choosing one primary metric plus guardrails in advance, and computing the required sample size for adequate power before launch — underpowered tests waste traffic and mislead.
Sample-size calculation before you launch:
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
baseline, mde = 0.10, 0.005 # detect a 0.5pp absolute lift
effect = proportion_effectsize(baseline, baseline + mde)
n = NormalIndPower().solve_power(effect_size=effect, alpha=0.05, power=0.8)
print(f"Need ~{n:,.0f} users per arm")Check for sample-ratio mismatch (a broken split invalidates results) and don't peek-and-stop without a sequential correction. The tradeoff: bigger effects need less traffic, so for small expected lifts you need large samples or longer runtime — decide if the change is worth the experimental cost.
90Turn an analysis of launch readiness, guardrails, and post-launch readouts into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on launch readiness, guardrails, and post-launch readouts, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Launch analytics de-risks releases with a pre-launch checklist (instrumentation verified, primary and guardrail metrics defined, rollback plan), a monitored rollout (staged exposure with automated guardrail alerts), and a post-launch readout that states whether the launch met its goal against the pre-registered metric. Guardrails catch harm the primary metric misses — latency, errors, complaints, revenue per session.
A guardrail check that would halt a rollout:
-- Compare treatment vs control on a guardrail during staged rollout
SELECT variant,
AVG(latency_ms) AS avg_latency,
SUM(is_error)*1.0 / COUNT(*) AS error_rate
FROM exposures
WHERE exposure_date >= CURRENT_DATE - 1
GROUP BY variant;
-- if treatment error_rate or latency regresses beyond threshold -> auto-holdWrite the readout around the decision (ship / iterate / roll back), not just numbers. The tradeoff: thorough launch analytics slows shipping, so scale rigor to risk — a copy tweak needs light monitoring, a checkout change needs full guardrails and staged exposure.
91Define the North Star metric and supporting guardrail metrics for ad impressions at Meta Platforms. How would fraud, abuse, risk, or safety analytics influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on fraud, abuse, risk, or safety analytics, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
Fraud and abuse analytics detect bad actors amid overwhelming legitimate activity — a rare-event, adversarial problem. Because fraudsters adapt, static rules decay, so you combine rules (fast, explainable) with ML (catches novel patterns) and graph analysis (rings share devices, cards, addresses). You optimize for the business cost tradeoff between blocking fraud and false-positive friction on good users.
A velocity/graph signal that flags shared-device rings:
-- Devices linked to many distinct accounts in a short window = ring signal
SELECT device_id,
COUNT(DISTINCT account_id) AS linked_accounts,
COUNT(DISTINCT card_hash) AS distinct_cards
FROM sessions
WHERE session_ts >= NOW() - INTERVAL '24 hours'
GROUP BY device_id
HAVING COUNT(DISTINCT account_id) >= 5 -- unusual fan-out
ORDER BY linked_accounts DESC;Evaluate with precision/recall at the operating point and the dollar impact, not accuracy. The tradeoff is catch rate vs. friction: aggressive blocking stops fraud but frustrates real users and can cost more than the fraud, so tune thresholds to net business value and add step-up verification instead of hard blocks.
92Using media embeddings and engagement signals, describe an analytical plan to determine whether loading an Instagram feed improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on fraud, abuse, risk, or safety analytics, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
When randomization isn't possible, quasi-experimental methods estimate causal effects from observational data by approximating a control group. Difference-in-differences compares the change over time in a treated group against an untreated group; propensity-score matching pairs treated and control units with similar covariates; instrumental variables and regression discontinuity exploit natural experiments. All rely on assumptions (e.g. parallel trends) that you must test, not assume.
A difference-in-differences estimate via regression:
import statsmodels.formula.api as smf
# treated x post interaction = the causal DiD effect
model = smf.ols('outcome ~ treated + post + treated:post', data=df).fit()
print(model.params['treated:post']) # estimated treatment effect
# validate: pre-period trends of treated vs control should be parallelAlways probe robustness with placebo tests and sensitivity to unobserved confounders. The tradeoff: quasi-experiments are the only option when A/B testing is infeasible, but their validity hinges on untestable assumptions, so present effects with appropriate caveats rather than as clean causal truth.
93How would you design an experiment or causal study for fraud, abuse, risk, or safety analytics at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on fraud, abuse, risk, or safety analytics, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
Fraud and abuse analytics detect bad actors amid overwhelming legitimate activity — a rare-event, adversarial problem. Because fraudsters adapt, static rules decay, so you combine rules (fast, explainable) with ML (catches novel patterns) and graph analysis (rings share devices, cards, addresses). You optimize for the business cost tradeoff between blocking fraud and false-positive friction on good users.
A velocity/graph signal that flags shared-device rings:
-- Devices linked to many distinct accounts in a short window = ring signal
SELECT device_id,
COUNT(DISTINCT account_id) AS linked_accounts,
COUNT(DISTINCT card_hash) AS distinct_cards
FROM sessions
WHERE session_ts >= NOW() - INTERVAL '24 hours'
GROUP BY device_id
HAVING COUNT(DISTINCT account_id) >= 5 -- unusual fan-out
ORDER BY linked_accounts DESC;Evaluate with precision/recall at the operating point and the dollar impact, not accuracy. The tradeoff is catch rate vs. friction: aggressive blocking stops fraud but frustrates real users and can cost more than the fraud, so tune thresholds to net business value and add step-up verification instead of hard blocks.
94What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about social graph updates to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on fraud, abuse, risk, or safety analytics, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
When randomization isn't possible, quasi-experimental methods estimate causal effects from observational data by approximating a control group. Difference-in-differences compares the change over time in a treated group against an untreated group; propensity-score matching pairs treated and control units with similar covariates; instrumental variables and regression discontinuity exploit natural experiments. All rely on assumptions (e.g. parallel trends) that you must test, not assume.
A difference-in-differences estimate via regression:
import statsmodels.formula.api as smf
# treated x post interaction = the causal DiD effect
model = smf.ols('outcome ~ treated + post + treated:post', data=df).fit()
print(model.params['treated:post']) # estimated treatment effect
# validate: pre-period trends of treated vs control should be parallelAlways probe robustness with placebo tests and sensitivity to unobserved confounders. The tradeoff: quasi-experiments are the only option when A/B testing is infeasible, but their validity hinges on untestable assumptions, so present effects with appropriate caveats rather than as clean causal truth.
95Turn an analysis of fraud, abuse, risk, or safety analytics into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on fraud, abuse, risk, or safety analytics, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
Fraud and abuse analytics detect bad actors amid overwhelming legitimate activity — a rare-event, adversarial problem. Because fraudsters adapt, static rules decay, so you combine rules (fast, explainable) with ML (catches novel patterns) and graph analysis (rings share devices, cards, addresses). You optimize for the business cost tradeoff between blocking fraud and false-positive friction on good users.
A velocity/graph signal that flags shared-device rings:
-- Devices linked to many distinct accounts in a short window = ring signal
SELECT device_id,
COUNT(DISTINCT account_id) AS linked_accounts,
COUNT(DISTINCT card_hash) AS distinct_cards
FROM sessions
WHERE session_ts >= NOW() - INTERVAL '24 hours'
GROUP BY device_id
HAVING COUNT(DISTINCT account_id) >= 5 -- unusual fan-out
ORDER BY linked_accounts DESC;Evaluate with precision/recall at the operating point and the dollar impact, not accuracy. The tradeoff is catch rate vs. friction: aggressive blocking stops fraud but frustrates real users and can cost more than the fraud, so tune thresholds to net business value and add step-up verification instead of hard blocks.
96Define the North Star metric and supporting guardrail metrics for content moderation queues at Meta Platforms. How would stakeholder communication and decision recommendations influence the metric tree?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on stakeholder communication and decision recommendations, measurable outcomes, failure modes, and trade-offs. I would start with the decision the metric is meant to improve. Then I would define a North Star metric that captures durable user or business value and decompose it into input metrics, diagnostic metrics, and guardrails. The tree should separate volume, conversion, retention, latency, quality, cost, and risk so leaders know which lever moved. I would check that the metric is measurable, sensitive, hard to game, and attributable to the team. I would also define counter-metrics so we do not improve one number while damaging trust, quality, or reliability.
An analysis only creates value if it changes a decision. Effective communication leads with a clear recommendation and confidence level, tailors depth to the audience (executives want the 'so what', peers want the method), and pre-empts the obvious objections. Frame findings around the decision at hand and quantify the expected impact and risk of each option.
Structure a recommendation like a decision memo:
Recommendation: Launch variant B to 100%.
Why: +2.1% conversion (95% CI +0.9% to +3.3%, p=0.004), guardrails neutral.
Impact: ~$1.2M/yr incremental at current traffic.
Risks / caveats: effect concentrated in mobile; monitor desktop for 2 weeks.
Alternative considered: hold for more data -> rejected, result already decisive.Show uncertainty honestly so trust compounds over time, and separate what the data says from your judgment. The tradeoff: more caveats are more rigorous but can muddy the call, so state the recommendation crisply first, then let the nuance support rather than obscure it.
97Using privacy-governed user activity data, describe an analytical plan to determine whether sending a WhatsApp message improved after a product or platform launch.Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on stakeholder communication and decision recommendations, measurable outcomes, failure modes, and trade-offs. My analysis plan would start with instrumentation validation: event definitions, joins, deduplication, time zones, missingness, bot filtering, and cohort assignment. Then I would define the population, pre/post windows, segments, baseline, and guardrails. I would compare the launch group to an appropriate control or historical baseline while accounting for seasonality, traffic mix, and concurrent changes. I would quantify uncertainty with confidence intervals and practical significance, then slice by important cohorts. The final deliverable should be a recommendation: scale, iterate, rollback, or investigate.
Segmentation groups entities into meaningful cohorts to tailor decisions. It can be rule-based (RFM: recency, frequency, monetary), or unsupervised (k-means, hierarchical clustering on standardized features). The goal isn't mathematically tight clusters but actionable, stable, and interpretable segments that a team can actually target differently — a segmentation nobody acts on is wasted analysis.
K-means segmentation on standardized behavioral features:
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
X = StandardScaler().fit_transform(df[['recency','frequency','monetary']])
# choose k via elbow/silhouette, not arbitrarily
km = KMeans(n_clusters=4, n_init=10, random_state=0).fit(X)
df['segment'] = km.labels_
df.groupby('segment')[['recency','frequency','monetary']].mean() # profile themProfile and name each segment, and validate stability over time before building programs on it. The tradeoff: more segments enable finer targeting but add operational complexity and shrink sample sizes per segment — keep the count to what the business can genuinely act on.
98How would you design an experiment or causal study for stakeholder communication and decision recommendations at Meta Platforms, including sample size, randomization, guardrails, and interpretation?Advanced
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on stakeholder communication and decision recommendations, measurable outcomes, failure modes, and trade-offs. I would design the experiment around decision risk. I would define hypothesis, primary metric, guardrails, minimum detectable effect, power, randomization unit, eligibility, and duration. Then I would check for interference, sample-ratio mismatch, novelty effects, seasonality, and compliance constraints. If randomization is not possible, I would consider difference-in-differences, synthetic control, matching, regression discontinuity, or instrumental variables and state the assumptions. I would predefine stopping and readout criteria to avoid cherry-picking. Experimentation is product judgment plus causal identification.
An analysis only creates value if it changes a decision. Effective communication leads with a clear recommendation and confidence level, tailors depth to the audience (executives want the 'so what', peers want the method), and pre-empts the obvious objections. Frame findings around the decision at hand and quantify the expected impact and risk of each option.
Structure a recommendation like a decision memo:
Recommendation: Launch variant B to 100%.
Why: +2.1% conversion (95% CI +0.9% to +3.3%, p=0.004), guardrails neutral.
Impact: ~$1.2M/yr incremental at current traffic.
Risks / caveats: effect concentrated in mobile; monitor desktop for 2 weeks.
Alternative considered: hold for more data -> rejected, result already decisive.Show uncertainty honestly so trust compounds over time, and separate what the data says from your judgment. The tradeoff: more caveats are more rigorous but can muddy the call, so state the recommendation crisply first, then let the nuance support rather than obscure it.
99What statistical pitfalls, instrumentation gaps, confounders, or data quality risks would you check before presenting results about ad impressions to leadership?Senior
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on stakeholder communication and decision recommendations, measurable outcomes, failure modes, and trade-offs. Before presenting results, I would pressure-test the analysis. I would check instrumentation changes, missing data, join duplication, survivorship bias, selection bias, Simpson's paradox, seasonality, bot traffic, outliers, and concurrent launches. I would verify that numerator and denominator definitions match the business question. I would compare aggregate and segment-level results to catch offsetting effects. I would distinguish statistical from practical significance and include uncertainty. If the result is ambiguous, I would state what decision is safe and what data is still needed.
Segmentation groups entities into meaningful cohorts to tailor decisions. It can be rule-based (RFM: recency, frequency, monetary), or unsupervised (k-means, hierarchical clustering on standardized features). The goal isn't mathematically tight clusters but actionable, stable, and interpretable segments that a team can actually target differently — a segmentation nobody acts on is wasted analysis.
K-means segmentation on standardized behavioral features:
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
X = StandardScaler().fit_transform(df[['recency','frequency','monetary']])
# choose k via elbow/silhouette, not arbitrarily
km = KMeans(n_clusters=4, n_init=10, random_state=0).fit(X)
df['segment'] = km.labels_
df.groupby('segment')[['recency','frequency','monetary']].mean() # profile themProfile and name each segment, and validate stability over time before building programs on it. The tradeoff: more segments enable finer targeting but add operational complexity and shrink sample sizes per segment — keep the count to what the business can genuinely act on.
100Turn an analysis of stakeholder communication and decision recommendations into a decision recommendation for Meta Platforms. What dashboard, narrative, and next action would you deliver?Intermediate
I would frame this for Meta Platforms's context: Facebook, Instagram, WhatsApp, Threads, advertising, social graph, Reality Labs, AI ranking. For Data Science, I would keep the answer focused on stakeholder communication and decision recommendations, measurable outcomes, failure modes, and trade-offs. I would turn the work into a decision memo: question, context, recommendation, evidence, caveats, expected impact, and next steps. The dashboard should show the North Star metric, guardrails, segment cuts, trend lines, and anomaly notes, while the narrative should focus on what changed and what to do. I would separate facts from assumptions and quantify impact in business terms such as revenue, retention, reliability, cost, or risk reduction. I would name an action owner and a measurement plan. Data science creates value when it changes a decision.
An analysis only creates value if it changes a decision. Effective communication leads with a clear recommendation and confidence level, tailors depth to the audience (executives want the 'so what', peers want the method), and pre-empts the obvious objections. Frame findings around the decision at hand and quantify the expected impact and risk of each option.
Structure a recommendation like a decision memo:
Recommendation: Launch variant B to 100%.
Why: +2.1% conversion (95% CI +0.9% to +3.3%, p=0.004), guardrails neutral.
Impact: ~$1.2M/yr incremental at current traffic.
Risks / caveats: effect concentrated in mobile; monitor desktop for 2 weeks.
Alternative considered: hold for more data -> rejected, result already decisive.Show uncertainty honestly so trust compounds over time, and separate what the data says from your judgment. The tradeoff: more caveats are more rigorous but can muddy the call, so state the recommendation crisply first, then let the nuance support rather than obscure it.
More Meta interview prep
Practice other Meta tracks: DevOps / SRE · AI / ML · Software Developer / Engineer · Database Engineer. Or browse 1,000+ general interview questions and role quizzes.