● Covered by 1 source · 1 reportMedium impact1 neutral
Deepgram optimizes Nova-3 voice AI model for AWS Trainium2, improving efficiency
Key points
- Deepgram's Nova-3 model ported to AWS Trainium2.
- Achieved 11-70% more audio processed per dollar.
- Optimization targets offline batch workloads like transcription.
- Benefits extend to self-hosted models on Amazon SageMaker AI.
Deepgram's Voice AI Scale
Deepgram processes quadrillions of audio tokens annually for over 200,000 developers and 1,400 organizations, offering speech-to-text, text-to-speech, and voice agent models. Customers access these models via the Deepgram API or by running them in their own environments, including Amazon SageMaker AI.
Efficiency Goals with AWS Trainium2
Deepgram focused on inference efficiency to manage high-volume batch workloads within budget constraints. The company partnered with AWS to optimize its Nova-3 speech-to-text model for AWS Trainium2, specifically for offline batch tasks such as contact-center analytics and medical transcription.
Porting and Optimization Process
The collaboration involved Deepgram providing Nova-3 as standard PyTorch code. The AWS Forward Deployed Engineering (FDE) team then worked with Deepgram over several months to port Nova-3 to AWS Trainium2 using PyTorch reconstruction and tune it for the new hardware. This included using TorchNeuron, the native PyTorch integration of the Neuron Compiler.
Performance and Cost Improvements
The optimized configuration runs eight model replicas on a single Amazon EC2 trn2.3xlarge instance. This setup processed 11–70 percent more audio per dollar on Deepgram’s offline, throughput-oriented batch workloads compared to previous compute options. These gains enable Deepgram to scale efficiently and potentially reduce costs for customers.
Broader Impact and Future Work
The benefits of this optimization can also extend to customers who self-host Deepgram models in their own virtual private cloud (VPC) on Amazon SageMaker AI. Deepgram and AWS are continuing to improve the ease of setup, reliability, and observability for these self-hosted deployments.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Primary sources
GitHub aws-neuron/nki-samplesReporting from
Deepgram collaborated with AWS to port its Nova-3 speech-to-text model to AWS Trainium2, achieving 11-70% more audio processing per dollar for batch workloads. This optimization allows Deepgram to scale its voice AI services more efficiently and potentially offer lower prices to customers.