← All stories
● Covered by 1 source · 1 reportMedium impact1 neutral

Airbnb Introduces CASTLE for Training Natural Language Search Models with Synthetic Data

🔄 Updated 5d ago
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • Airbnb introduced CASTLE for natural language search model training.
  • CASTLE uses LLMs and de-identified booking data to create synthetic queries and labels.
  • It addresses the cold-start problem for new search systems.
  • The framework achieved a KL divergence of 1.01 against real query distributions.

Addressing the Cold-Start Problem in Natural Language Search

Airbnb researchers have released CASTLE, a new framework designed to overcome the cold-start problem in natural language search. This issue arises when there is insufficient real search traffic to train models effectively, particularly for new features or platforms. CASTLE uses large language models (LLMs) and de-identified booking session data to generate synthetic search queries and corresponding relevance labels.

Joint Generation of Queries and Relevance Labels

Unlike prior approaches such as InPars and Promptagator, which treat query generation and relevance labeling as separate steps, CASTLE integrates these tasks. This joint generation process aims to reduce errors and minimize discrepancies between synthetic and real-user query distributions. The framework constructs relevance labels during the generation process itself, rather than through a separate evaluation step.

Leveraging Booking Sessions for Data Synthesis

CASTLE utilizes contrastive listing pairs derived from de-identified booking sessions. When a user books a property after viewing multiple options, the chosen listing becomes a positive example, and an unbooked but viewed property serves as a negative example. An LLM is then prompted to generate a query that specifically matches the positive listing while excluding the negative one. This method inherently establishes relevance labels.

Query Generation Strategies and Evaluation

The framework incorporates three query-generation strategies: template-based prompts, few-shot prompts using approximately 500 seed queries from user research, and attribute-grounded prompts that focus on distinguishing listing characteristics. A variety mechanism is also included to encourage diverse query lengths and phrasing. For listings without sufficient booking history, CASTLE employs a Virtual Judge, an LLM-based evaluator, to assign relevance labels. A variant, CASTLE-R, is intended for production environments with existing search traffic.

In evaluations, CASTLE demonstrated a Kullback–Leibler (KL) divergence of 1.01 when compared to real-user query distributions. This figure is significantly lower than the 9.33 achieved by the best alternative method, indicating a closer resemblance to actual user behavior.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~4 min · 3 stories · Nov 21

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

Primary sources

arXiv 2605.21812

Reporting from

Airbnb researchers developed CASTLE, a framework that uses de-identified booking sessions and large language models (LLMs) to generate synthetic search queries and relevance labels. This system addresses the cold-start problem in natural language search by enabling model training before sufficient real user data is available. CASTLE jointly generates queries and relevance labels, differing from previous methods that separate these tasks, and achieved a KL divergence of 1.01 against real-user query distributions.