● Covered by 1 source · 1 reportMedium impact1 neutral
LinkedIn Implements OpenTelemetry for End-to-End Member Journey Tracing
Key points
- LinkedIn built an OpenTelemetry platform for end-to-end tracing.
- Connects client-side actions with backend service interactions.
- Introduced "pageForestId" to group related requests from a single page.
- Uses Apache Kafka and ClickHouse for telemetry transport and storage.
Bridging Client and Backend Observability
LinkedIn has implemented a new telemetry platform utilizing OpenTelemetry to provide a unified view of member interactions. This system links activities performed by users on iOS, Android, and web applications directly to the backend services that process these actions. The primary objective is to resolve the challenge of correlating client-side analytics, which show user actions like button taps or errors, with backend traces that detail service operations.
Tracing the Full Member Journey
The new approach enables engineers to follow a member's interaction from the initial client-side action through frontend and backend services, down to the underlying data stores. To address the complexity of a single page generating multiple requests, each potentially traversing numerous services, LinkedIn introduced a "pageForestId". This identifier provides a contextual layer above individual distributed traces, allowing related requests from the same page context to be associated. While each request retains its OpenTelemetry traceId, the pageForestId allows for investigation of both individual requests and the broader activity contributing to a member's experience.
Technology Stack and Data Flow
LinkedIn's diverse technology stack, which includes mobile and web clients alongside backend services in Java, Python, Go, and Rust, benefits from OpenTelemetry's common model for instrumentation and trace context propagation. LinkedIn developed custom wrappers and extensions for standard OpenTelemetry SDKs to ensure consistent instrumentation across product teams, handling aspects like mobile lifecycle and LinkedIn-specific metadata. Telemetry data is transported via Apache Kafka and stored in ClickHouse, with pageForestId, traceId, and timestamps serving as correlation keys. This setup allows engineers to start with a user-facing interaction and drill down into its corresponding backend execution.
Sampling for Scalability
Given the vast amount of telemetry that would be generated by capturing every span, sampling is a critical component of the architecture. LinkedIn employs coordinated head sampling to manage the volume of data, ensuring that related client and backend traces are sampled together to maintain a complete view of the member journey without overwhelming the system.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Reporting from
LinkedIn developed an OpenTelemetry-based platform to connect user interactions in its mobile and web applications with backend service processing. This system allows engineers to trace a member's activity from the client through frontend and backend services, closing a common observability gap between client analytics and backend tracing. The platform uses a "pageForestId" to group related requests within a single user page view, providing a broader context than individual distributed traces.