Netflix develops model to measure personalized recommendation impact on viewing behavior

Introduction to Personalization Measurement

Personalization is a core component of internet services, influencing content discovery across various platforms. At Netflix, personalized recommendations have been integral to the member experience for decades, helping users navigate a large and dynamic content catalog.

Measuring the value created by personalization is complex. It requires distinguishing between a title's inherent appeal and the value added by recommending it to the right member at the right time.

Addressing the Measurement Challenge

A straightforward approach to valuing recommendations, such as tracking viewing from recommended titles, does not establish causality. Recommendations are not randomly assigned; they are based on predictions of user interest. Therefore, watching a recommended title could be due to the recommendation itself or the member's pre-existing preference.

This targeting of recommendations to demand makes it difficult to isolate the effect of personalization from a member's underlying preferences.

Netflix's Hybrid Approach

Running a large-scale experiment by removing personalization would provide clear data but would negatively impact the user experience, as personalization is crucial for content discovery. Netflix opted against this method.

Instead, Netflix developed a behavioral model that separates members' underlying preferences from the impact of personalized recommendations. This model estimates how recommendations influence both the decision to watch and the specific titles chosen.

Research Publication

The findings from this research are detailed in a new article published in the International Journal of Industrial Organization. The work is also being presented at the ACM Conference on Economics & Computation (EC '26).