● Covered by 2 sources · 2 reportsMedium impact2 neutral
Respan Launches Open-Source Rust AI Gateway with Plugin Architecture
Key points
- Respan launched an open-source AI gateway written in Rust.
- The gateway uses a small core and a plugin-based architecture.
- Plugins support various providers, protocols, modalities, and gateway features.
- The new system replaces one built on LiteLLM after an upgrade caused failures.
- Developers can build custom features as plugins without altering the core.
Respan Introduces New AI Gateway
Respan, a Y Combinator W24 company, has released an open-source AI gateway developed in Rust. This new gateway is designed with a decoupled architecture where all functionalities, including support for different providers, protocols (like MCP and voice), modalities (images, videos), and gateway features (such as fallbacks, PII redaction, and smart routing), are implemented as plugins.
Addressing Prior Challenges with LiteLLM
The development of this new gateway follows a period where Respan had built extensively on LiteLLM for two years, accumulating over 100,000 lines of wrapper code. A significant issue arose in January 2026 when a routine LiteLLM upgrade led to intermittent connection failures. Although Respan rolled back the update, the root cause of the problem remained unidentified, leading engineers to manually pull in upstream fixes rather than directly upgrading LiteLLM.
This experience highlighted the difficulties in adding new features to existing LLM gateways, often forcing developers to build entirely new gateways for specific problems.
Plugin-Based Architecture for Flexibility
To prevent future maintenance challenges, Respan adopted a design philosophy inspired by Nginx, focusing on a minimal core. This approach moves features like routing, model providers, and security into separate plugins. The small core and public interfaces allow teams to assemble the specific AI infrastructure their applications require.
Developers can create their own features as additional plugins using the same interfaces, eliminating the need to modify the gateway's core code. This design aims to provide greater flexibility and maintainability for AI infrastructure.
Impact on AI Infrastructure Development
The plugin-centric design of Respan's gateway offers a solution to the common problem of integrating diverse AI providers and functionalities. By providing a framework where new capabilities can be added without altering the core system, Respan aims to simplify the process of building and maintaining complex AI applications. This approach could reduce the need for developers to create bespoke gateway solutions for every new requirement.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
How outlets covered it
Respan released an open-source AI gateway written in Rust, replacing a system built on LiteLLM after an upgrade caused intermittent connection failures. The new gateway uses a small core with a plugin architecture for features like model providers, routing, and security, aiming to improve maintainability and flexibility.
Respan, a Y Combinator W24 startup, launched an open-source AI gateway built in Rust, featuring a plugin-based architecture. This design allows for custom extensions for providers, protocols, modalities, and gateway features, addressing difficulties in adding new functionalities to existing LLM gateways.