← All stories
● Covered by 1 source · 1 reportLow impact1 neutral

AssemblyAI Optimizes Speech-to-Text Inference Costs on AWS Using EC2 Spot Instances

🔄 Updated 8d ago
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • AssemblyAI uses EC2 Spot Instances for GPU-intensive speech-to-text inference.
  • Architecture combines Amazon ECS, EC2 Spot, and Amazon SQS for resilience.
  • Achieves cost savings while meeting SLA commitments and data residency needs.
  • Supports transcription, speaker detection, sentiment analysis, and PII redaction.

Challenge of Speech-to-Text Inference at Scale

Speech-to-text inference is a computationally demanding process, involving multiple deep learning models (acoustic, language, and post-processing) that require NVIDIA GPUs to meet latency targets. For AssemblyAI, this translates to operating thousands of GPU instances continuously. Running such a large-scale operation solely on On-Demand Instances would be financially prohibitive.

Leveraging EC2 Spot Instances for Cost Efficiency

AssemblyAI adopted Amazon EC2 Spot Instances to achieve substantial cost savings. Spot Instances can be reclaimed by AWS with a two-minute warning, necessitating an architecture capable of handling interruptions without compromising service level agreement (SLA) commitments. This approach allows AssemblyAI to manage GPU-intensive workloads efficiently.

Additionally, AssemblyAI serves customers in regulated sectors like healthcare and finance, which require strict data residency. The implemented solution ensures audio data remains within specified geographic boundaries, particularly in European AWS Regions.

Architectural Solution Overview

AssemblyAI's optimized inference platform operates on Amazon Elastic Container Service (Amazon ECS), with compute capacity managed by ECS capacity providers. The compute fleet integrates both Amazon EC2 Spot Instances and On-Demand Instances across various NVIDIA GPU instance families. This setup runs a single inference model per GPU across thousands of instances.

The architecture is built on four core pillars, including Amazon ECS capacity providers that mix Spot and On-Demand instances, designed to deliver cost-optimized and resilient inference.

AssemblyAI's Offerings on AWS Marketplace

Available via AWS Marketplace, AssemblyAI provides an API that offers transcription, speaker detection, sentiment analysis, chapter detection, and personally identifiable information (PII) redaction. These models are used for transcribing and analyzing high-volume speech workloads across diverse industries, including technology, media, healthcare, and financial services.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~4 min · 3 stories · Nov 21

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

Reporting from

AssemblyAI implemented a cost-optimization strategy for its speech-to-text inference platform on AWS, utilizing Amazon EC2 Spot Instances, Amazon ECS, and Amazon SQS. This architecture allows AssemblyAI to run GPU-intensive deep learning models at scale with significant cost savings while maintaining service availability and latency requirements. The solution addresses the need for cost-effective processing of high-volume audio data, including compliance with data residency requirements in European AWS Regions.