Sundara Raman Ramachandran, a tech lead at LinkedIn, presented on scaling LLM-based ranking systems for latency-critical search and recommendation workloads. His work focuses on LLM inference and its application in ranking, including contributions to the SGLang scoring API.
LinkedIn's search operates at a large scale, serving 1.3 billion members across 200 countries. The platform experiences high queries per second (QPS) for its ranking and recommendation systems. The evolution of search from keyword-based queries to natural language inputs, such as "find me design roles in the entertainment industry," necessitates advanced understanding of user intent.
LLMs are being used to interpret complex natural language queries, like "who in my network can help me file a patent for my business?" This capability helps in understanding user intent and providing more accurate search results. Integrating LLMs into existing search and ranking infrastructure, which is already complex, presents engineering challenges.
The high-level architecture for LinkedIn's search begins with a user's search query. This query first passes through a query understanding layer, which identifies the user's intent. Following this, a retrieval layer processes the understood intent to fetch relevant results.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's briefNew every morning, and the back catalogue is archived by date.
LinkedIn is implementing LLMs to improve search and recommendation systems, addressing the challenge of integrating these models into high-QPS, latency-sensitive environments. The company is focusing on understanding natural language user intent to deliver more relevant results across its 1.3 billion members.