Netflix implements multimodal embeddings for improved asset personalization at scale

Addressing the Cold-Start Problem in Personalization

Netflix has implemented a new system called MAPS (Multimodal Asset Personalization at Scale) to enhance how it recommends visual assets like artwork and video previews. Previously, personalization models struggled with new titles because there was insufficient interaction data to determine which assets a member would prefer. This led to a 'cold-start' problem where initial recommendations relied on popularity heuristics rather than individual taste.

Integrating Multimodal Embeddings

The core of the new approach involves using multimodal embeddings, which allow Netflix's models to process and understand the actual content of visual and audio assets. By encoding assets with tools like CLIP for images, the models can interpret visual cues directly. This means that when a new asset is introduced, its embedding carries taste signals from similar, existing assets, enabling immediate personalization without extensive interaction history.

Impact on Personalization

This change allows personalization to begin much earlier in a title's lifecycle, closer to its launch. The models require significantly less interaction data before they can make tailored recommendations. This improvement applies to various aspects of the Netflix experience, including artwork personalization, query-aware artwork ranking, and video preview personalization.