● Covered by 1 source · 1 reportLow impact1 neutral
SearXNG-style metadata search engine implemented in Rust
Key points
- Rust-based metadata search engine released.
- Queries DuckDuckGo, Brave, Startpage, and Yahoo concurrently.
- Deduplicates results by normalized URL.
- Ranks results using Reciprocal Rank Fusion (RRF).
New Rust-based Search Engine
A new metadata search engine, inspired by SearXNG, has been developed using Rust. This project allows users to fan out search queries to multiple search engines simultaneously, including DuckDuckGo, Brave, Startpage, and Yahoo.
How it Works
Upon receiving a search request, the engine sends the query to the configured search providers concurrently. It then scrapes the HTML responses from each engine using the `scraper` library, which utilizes CSS selectors over Mozilla's html5ever parser. Results are deduplicated by normalizing URLs, which involves stripping tracking parameters, removing locale prefixes, and sorting query parameters.
Result Aggregation and Ranking
Duplicate URLs are merged, and the results are scored using Reciprocal Rank Fusion (RRF). This method assigns higher scores to pages returned by multiple engines, indicating broader consensus on relevance. The top results are then returned as JSON.
Implementation Details
The project requires Rust 1.75+ and can be added as a dependency via Cargo. It provides examples for querying individual engines or aggregating results from multiple engines, demonstrating its capabilities for concurrent search and result processing.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Primary sources
GitHub cmc_internal/api GitHub github/collect GitHub MikeLuu99/searxng-rust GitHub _private/browser GitHub get-started/accessibility GitHub open-source/sponsorsReporting from
A new open-source project provides a metadata search engine written in Rust, replicating the functionality of SearXNG. This tool concurrently queries multiple search engines, processes their HTML results, and ranks them using Reciprocal Rank Fusion.