● Covered by 2 sources · 2 reportsMedium impact2 neutral
Netflix Observability Team Discusses Ontology-Driven Knowledge Graph for E2E Monitoring
Key points
- Netflix handles 65M concurrent streams during peak events.
- Processes over 2 billion requests and 38M logging events daily.
- Uses an ontology-driven knowledge graph for observability.
- Focuses on end-to-end user experience monitoring.
- The knowledge graph correlates data from metrics, events, logs, and traces (MELT).
- The system aims to reduce investigation time for complex regressions from several hours.
- Jake Paul and Mike Tyson fight in November 2024 was delivered to 65 million concurrent streams.
- Netflix infrastructure handles over 38 million real-time logging events per second.
- A regression caused the home screen to take 1.5 seconds longer to render.
Observability Challenges at Netflix Scale
Prasanna Vijayanathan and Renzo Sanchez-Silva from Netflix's observability team discussed the challenges of monitoring their platform. Netflix experiences significant traffic, including 65 million concurrent streams during major live events and over 2 billion requests daily from applications to servers. Their systems also process upwards of 38 million real-time logging events per second.
This scale presents a substantial data engineering problem, requiring sophisticated solutions to maintain performance and quality of experience across numerous client platforms and thousands of microservices.
Ontology-Driven Approach
To tackle these challenges, Netflix is implementing an ontology-driven approach to build an end-to-end knowledge graph. This system is designed to provide comprehensive insights into the user experience, from the moment a user clicks on the Netflix logo until they begin watching content.
The knowledge graph helps in identifying relevant metrics and tools, and in generating insights for understanding user preferences and system behavior.
Impact on Monitoring and AIOps
Renzo Sanchez-Silva, with a background in monitoring and alerting, noted that this initiative is part of Netflix's move into the AIOps space. The convergence of AI with traditional monitoring infrastructure is central to managing the complexity of their distributed systems.
The knowledge graph serves as a foundational element for advanced alerting and intelligent operations, allowing Netflix to proactively address issues and optimize user experience.
Updates
- The knowledge graph correlates data from metrics, events, logs, and traces (MELT).
- The system aims to reduce investigation time for complex regressions from several hours.
- Jake Paul and Mike Tyson fight in November 2024 was delivered to 65 million concurrent streams.
- Netflix infrastructure handles over 38 million real-time logging events per second.
- A regression caused the home screen to take 1.5 seconds longer to render.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Primary sources
GitHub Netflix/atlasHow outlets covered it
Netflix developed an ontology-driven knowledge graph to improve observability across its massive infrastructure. This system helps identify root causes of performance issues faster by correlating data from metrics, events, logs, and traces (MELT) across various services and applications. The approach aims to reduce the time spent on investigations, which can currently take several hours for complex regressions.
Netflix's observability team presented on building an ontology-driven end-to-end knowledge graph to manage observability at their scale. This approach addresses the data engineering challenges of monitoring 65 million concurrent streams, 2 billion daily requests, and 38 million logging events per second. The knowledge graph aims to provide insights into user experience from application click to content playback.