Speech-to-text inference is a computationally demanding process, involving multiple deep learning models (acoustic, language, and post-processing) that require NVIDIA GPUs to meet latency targets. For AssemblyAI, this translates to operating thousands of GPU instances continuously. Running such a large-scale operation solely on On-Demand Instances would be financially prohibitive.
AssemblyAI adopted Amazon EC2 Spot Instances to achieve substantial cost savings. Spot Instances can be reclaimed by AWS with a two-minute warning, necessitating an architecture capable of handling interruptions without compromising service level agreement (SLA) commitments. This approach allows AssemblyAI to manage GPU-intensive workloads efficiently.
Additionally, AssemblyAI serves customers in regulated sectors like healthcare and finance, which require strict data residency. The implemented solution ensures audio data remains within specified geographic boundaries, particularly in European AWS Regions.
AssemblyAI's optimized inference platform operates on Amazon Elastic Container Service (Amazon ECS), with compute capacity managed by ECS capacity providers. The compute fleet integrates both Amazon EC2 Spot Instances and On-Demand Instances across various NVIDIA GPU instance families. This setup runs a single inference model per GPU across thousands of instances.
The architecture is built on four core pillars, including Amazon ECS capacity providers that mix Spot and On-Demand instances, designed to deliver cost-optimized and resilient inference.
Available via AWS Marketplace, AssemblyAI provides an API that offers transcription, speaker detection, sentiment analysis, chapter detection, and personally identifiable information (PII) redaction. These models are used for transcribing and analyzing high-volume speech workloads across diverse industries, including technology, media, healthcare, and financial services.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's briefNew every morning, and the back catalogue is archived by date.
AssemblyAI implemented a cost-optimization strategy for its speech-to-text inference platform on AWS, utilizing Amazon EC2 Spot Instances, Amazon ECS, and Amazon SQS. This architecture allows AssemblyAI to run GPU-intensive deep learning models at scale with significant cost savings while maintaining service availability and latency requirements. The solution addresses the need for cost-effective processing of high-volume audio data, including compliance with data residency requirements in European AWS Regions.