Director, Media Experimentation & Engagement

The Director, Media Experimentation & Engagement leads NBCU’s incrementality and engagement measurement program for Performance and Device media. Experimentation is the causal calibration source for Growth Media Analytics’ measurement triangle: the function that keeps Marketing Mix Modeling (MMM) and Multi-Touch Attribution (MTA) decision-grade. You own the learning agenda, set the standard for rigor, and translate test results into clear budget guidance leaders can act on with speed and confidence. You define how NBCU measures media’s effect on subscriber value and engagement, a critical causal measurement practice for the business. You manage a team to high-quality execution and train the next set of analytic leaders, and you are the face of Growth Media Analytics to Performance Media, Device Media, Growth Strategy, and Decision Sciences leadership on experimentation methodology. 

Key Responsibilities 

• Own and scale the experimentation roadmap. Grow structured testing from roughly 15 to 75 panels per year while raising rigor on budget-grade decisions. Define the learning agenda, decision questions, hypotheses, success metrics, guardrails, and power thresholds, and approve or reject vendor designs against those standards. 

• Define and lead engagement measurement. Build NBCU’s forward approach to measuring media’s effect on subscriber engagement and retention, in partnership with Growth Strategy and Decision Sciences. This includes variance reduction techniques for low-signal continuous outcomes, and the translation from measured lift to revenue and retention value. 

• Own always-on holdouts. Lead holdout design and execution across Performance and Device media, including persistent engagement holdouts. 

• Collaborate with Media Partners. Direct how NBCU uses conversion-lift (CLS) programs and clean room environments (Meta Advanced Analytics, Google ADH, Amazon AMC) for exposure mapping and lift measurement and set the standard for what partners can and cannot answer. 

• Own measurement calibration. Serve as the source of truth for calibrating MMM and MTA, keeping media measurement consistent and decision-grade across the portfolio. 

• Lead experimentation vendor relationships. Manage relationships across incrementality vendors, including new vendor onboarding, data transfer, validating methodology and result, and ensuring critical deliverables are shared to the business.  

• Represent the team at the executive level. Own measurement-methodology relationships with Performance Media and Device Media leadership, Growth Strategy, Finance, and Decision Sciences. Challenge entrenched measurement beliefs and drive adoption of decision-grade testing. 

• Build and develop the pod. Directly manage and develop a Manager and an Analyst, and uplevel cross-functional partner teams on test design, statistical power, and common pitfalls. 

• Drive AI-enabled scale. Use agentic AI and automated analytics workflows to expand testing throughput, insight generation, and decisioning.