From AWS Machine Learning Blog · 40 stories
AWS SageMaker AI now supports WhisperX for speaker-labeled transcription
AWS SageMaker AI now supports WhisperX, an extension of OpenAI's Whisper model, for speaker-labeled transcription. This integration provides precise per-word timestamps and speaker diarization, addressing limitations of generic speech-to-text solutions for various enterprise workloads.
Amazon Bedrock Introduces Prompt Caching to Reduce Costs and Latency
Amazon Bedrock now offers prompt caching, which can reduce input token costs by up to 90% and lower time-to-first-token (TTFT) when repeatedly sending the same context to foundation models. This feature allows Bedrock to store and reuse partially processed input, avoiding redundant computation for subsequent requests with matching prefixes.
TwelveLabs Marengo Embed 3.0 now available in Amazon Bedrock Knowledge Bases
TwelveLabs Marengo Embed 3.0 is now generally available as an embedding model within Amazon Bedrock Knowledge Bases. This integration allows for natural language search across video, audio, and image content, simplifying the process of building semantic search capabilities for multimodal assets.
AWS introduces Ray Serve Deep Learning Containers for TorchServe inference workloads
AWS has launched Ray Serve Deep Learning Containers (DLCs) to support model inference workloads, particularly for users migrating from the unmaintained TorchServe. These DLCs provide pre-built, optimized Docker images with a complete inference stack, including PyTorch and Ray Serve, maintained by AWS. This offering addresses the operational burden of managing dependencies and security for deep learning model serving.
NVIDIA MPS and Triton on Amazon EC2 Reduce ASR Inference Costs by 75%
AWS, NVIDIA, and Heidi Health demonstrated a 75% reduction in ASR inference costs on Amazon EC2 by using NVIDIA CUDA Multi-Process Service (MPS) with NVIDIA Triton Inference Server. This setup allows for more efficient GPU utilization, reducing the number of required GPU instances from 16 to 4 for Heidi Health's clinical consultation processing. The improved efficiency addresses the challenge of low GPU utilization during ASR inference while maintaining sub-second latency.
Amazon SageMaker HyperPod introduces tiered KV cache with Curvine for LLM inference
Amazon SageMaker HyperPod now supports a tiered KV cache architecture using Curvine, a distributed cache filesystem, to extend KV cache beyond GPU and CPU memory into a shared NVMe pool. This development allows for KV cache reuse across replicas, improving time-to-first-token (TTFT) and reducing infrastructure costs for large language model (LLM) inference.
Amazon Bedrock AgentCore Identity Adds Private Key JWT Authentication
Amazon Bedrock AgentCore Identity now supports Private Key JWT client authentication for agents, allowing them to authenticate to identity providers using signed JSON Web Tokens instead of shared secrets. This enhancement improves security by keeping private keys within AWS Key Management Service (KMS) and enabling more secure machine-to-machine authentication flows for AI agents.
Amazon Nova introduces Self-Distilled Reasoning for enhanced model fine-tuning
Amazon's Nova 2 models now utilize Self-Distilled Reasoning (SDR) to improve fine-tuning. SDR improves model performance on datasets lacking reasoning traces, addressing issues like catastrophic forgetting during training.
Custom OS Installation Enabled for AWS DeepRacer Devices
AWS has launched a developer bootloader for AWS DeepRacer devices allowing for custom OS installations. This update enables developers to utilize modern Linux distributions and tailor the devices for enhanced functionality and educational use.
Amazon Quick adds Mobile Layout for Free Form dashboards
Amazon Quick now offers a Mobile Layout feature for Free Form dashboards, allowing automatic rendering in a single-column view for mobile devices. This change facilitates easier access to data on phones and tablets, enhancing user experience without requiring any modifications from dashboard authors.
Smartsheet Develops Remote MCP Server on AWS for AI Integration
Smartsheet has built a remote Model Context Protocol (MCP) server on AWS to enable direct AI access to its data. This development allows AI agents to interact with Smartsheet's capabilities, enhancing workflow efficiency for enterprise users.
Built Technologies launches AI document intelligence solution on AWS for real estate finance
Built Technologies has launched an AI-powered document intelligence solution on AWS to enhance processing in real estate finance. This solution aims to automate complex and manual document processing, significantly reducing workflow times and supporting various document types throughout the real estate lifecycle.
Amazon introduces custom CloudWatch dashboards for SageMaker Pipelines monitoring
Amazon has released a new solution for monitoring SageMaker Pipelines across multiple AWS accounts and Regions using customized CloudWatch dashboards. This centralization helps organizations manage their machine learning operations more efficiently and reduces the operational overhead associated with monitoring distributed environments.
Amazon Nova Act Introduces Scalable UX Testing Approach for User Flow Analysis
Amazon launched Nova Act, a multimodal foundation model designed to enhance User Experience (UX) testing by utilizing visual understanding for intelligent navigation. This approach allows for more comprehensive testing of user workflows while adapting to interface changes, addressing the limitations of traditional automated testing methods.
Flo Health Implements AI Content Review System with Amazon Bedrock
Flo Health's engineering team has developed a production-grade medical content review system using Amazon Bedrock, resulting in a 60% reduction in review time and tripled content throughput. This advancement addresses the challenges of scaling medical content production while maintaining rigorous accuracy standards without the need for expanding the medical team.
ScienceSoft launches HIPAA-compliant AI voice scheduler using AWS
ScienceSoft has developed a HIPAA-compliant AI voice scheduler leveraging Amazon technologies to streamline healthcare appointment scheduling. This solution addresses inefficiencies in traditional manual workflows, potentially transforming patient scheduling in healthcare settings.
Henry Schein One implements AI for real-time dental image verification
Henry Schein One launched Image Verify, an AI system using Amazon SageMaker, to assess dental X-ray quality in real-time. This innovation addresses a significant bottleneck in dental claims processing, minimizing delays and improving patient care as the system scales to 40,000 locations globally.
Amazon Quick Automate Introduces Native Case Management for Workflows
Amazon Quick Automate now includes native case management, enhancing the tracking and management of workflows across AI agents. This feature enables organizations to monitor work item progress, handle exceptions, and allow human intervention when necessary, facilitating efficient enterprise-scale automation.
Amazon SageMaker HyperPod Enhances LLM Inference with New Features
Amazon SageMaker HyperPod has introduced new features to enhance large language model inference, including Disaggregated Prefill and Decode, support for Hugging Face, NVMe integration, and model quantization with Unsloth. These updates aim to improve performance, observability, and resource efficiency for enterprise users.
BYOKG and GraphRAG initiatives enhance pharmaceutical drug discovery
Pharmaceutical researchers are adopting BYOKG and GraphRAG to improve access to fragmented knowledge systems. These solutions aim to integrate data sources, enhancing efficiency in drug discovery.
Amazon Bedrock Offers AI-Powered Email Management for Public Sector
Amazon Bedrock introduces an AI-driven email management solution for public sector organizations to automatically classify and prioritize incoming communications. This system enhances responsiveness and optimizes resource allocation in local government settings where email volume is high.
Amazon QuickSight Introduces Multi-Dataset Topics and Relationships for Enhanced Analytics
Amazon QuickSight has launched Multi-Dataset Topics and Multi-Dataset Relationships, allowing analytical questions across multiple datasets without the need for pre-joining them into a single table. This new capability enables AI-generated SQL queries and supports seven distinct data modeling patterns, facilitating more efficient data analysis and management. The updates are part of efforts to streamline dataset management and enhance data governance.
Amazon Nova introduces selective unlearning for customizable content moderation
Amazon Nova has launched Customizable Content Moderation Settings (CCMS), allowing clients to selectively adjust model safeguards. This development is significant as it enables organizations to generate needed content while adhering to essential AI safety policies.
AWS Introduces Multi-Turn Reinforcement Learning in SageMaker AI for Enterprise Agents
AWS has introduced new capabilities for multi-turn reinforcement learning in Amazon SageMaker AI and Amazon Nova. These developments facilitate the training of AI agents that can handle complex workflows and multi-step tasks, improving their reliability and performance. This advancement marks a significant step in enhancing enterprise AI solutions such as support ticket resolution and content moderation.
Amazon Nova automates PII redaction in images for compliance
Amazon has introduced Nova, a foundation model that automates the redaction of PII in images. This tool addresses compliance challenges by ensuring sensitive information is accurately identified and redacted, reducing the risk of regulatory penalties for organizations sharing images containing potential PII.
Amazon Bedrock introduces tools to combat AI-generated phishing risks
Amazon Bedrock offers capabilities to detect and address AI-generated phishing, adapting to sophisticated attacks. This response is crucial as traditional phishing filters fail against today's contextually accurate threats.
HippoRAG Framework Integrates Amazon Services for Enhanced Knowledge Retrieval
HippoRAG is a new Retrieval Augmented Generation framework inspired by human memory systems. It utilizes Amazon Bedrock, Neptune, and personalized PageRank for efficient multi-hop reasoning and knowledge integration across documents.
BoltzGen Deploys on Amazon SageMaker AI for Protein Design
BoltzGen is now available on Amazon SageMaker AI, optimizing protein binder design processes by managing GPU infrastructure automatically. This integration allows researchers and developers to focus on design rather than infrastructure, significantly reducing operational overhead and costs in protein design projects.
Amazon Bedrock enhances resilience for LLM inference in production environments
Amazon Bedrock introduces resilience patterns to improve large language model (LLM) inference. These patterns focus on maintaining availability, response time, and cost-effectiveness, critical as generative AI scales in production.
Outpost VFX Achieves 8x Faster AI Training Using AWS
Outpost VFX improved their AI model training speeds by 8x by leveraging AWS infrastructure. This enhancement optimizes face replacement workflows, reducing project delays and improving overall efficiency in visual effects production.
IBS Software Develops Bilingual NER with Amazon Bedrock for Cargo Logistics
IBS Software created a bilingual Named Entity Recognition (NER) system utilizing Amazon Bedrock to process cargo logistics emails in English and Japanese. This solution achieves over 95% accuracy while reducing operational costs significantly, addressing challenges in manual interventions and accuracy trade-offs.
Amazon Nova models fine-tuned for better email data extraction accuracy
Parcel Perform has fine-tuned Amazon Nova models to improve data extraction accuracy from emails, achieving 94.77% accuracy. The optimization reduced inference latency and costs, enhancing e-commerce logistics operations.
Amazon Nova 2 Lite and Anthropic Claude streamline document digitization
Amazon Nova 2 Lite and Anthropic's Claude Sonnet 4.6 have been paired to create a cost-effective solution for digitizing scanned documents. This two-model pipeline reduces costs by two-thirds compared to single-model alternatives, providing high confidence name-to-face associations from yearbook scans.
PAR Technology Develops Secure Multi-Tenant LLM Analytics on AWS
PAR Technology has built a secure multi-tenant analytics system on AWS that uses row-level security. This solution allows business users to perform self-serve analytics without exposing sensitive data across tenants, addressing the complex challenge of data access and correctness.
Hugging Face Models Now One-Click Deployable to Amazon SageMaker Studio
AWS and Hugging Face have integrated deep-linking, allowing developers to move a model from Hugging Face directly into Amazon SageMaker Studio in a single click. This eliminates prior multi-step processes, enabling quicker model experimentation and deployment. The update is significant for faster AI development and deployment in enterprise environments.
Amazon Connect Talent Uses AI for Structured Behavioral Interviews at Scale
Amazon Connect Talent integrates AI into structured behavioral interviews to standardize candidate evaluation. The service uses consistent, job-related questions and adaptive follow-ups to gather evidence, aiming to improve hiring consistency and scale. This approach separates the interview process from scoring, with AI focusing solely on evidence collection.
Social Department Develops QuickSearch for Culture-Aware Video Discovery in Entertainment Marketing
Social Department, an AI operating system for entertainment marketing, developed QuickSearch, a tool that uses AI to identify culturally relevant video moments like quotable quotes or memeable scenes. This system addresses the challenge of finding specific, impactful video clips for marketing campaigns that traditional semantic search cannot identify, allowing marketers to quickly find content that resonates with audiences.
Stretto Deploys AI Voice Agent 'Tara' for Bankruptcy Intake Using Amazon Nova 2 Sonic
Stretto launched an AI voice agent named Tara, powered by Amazon Nova 2 Sonic, to conduct bankruptcy intake interviews by phone. This agent replaces web forms, gathering detailed financial data conversationally and sorting leads for attorneys, addressing the cost of human intake teams.
AWS Agent Registry Adds Custom Metadata Schemas for Resource Governance and Discovery
AWS Agent Registry now supports custom metadata schemas, allowing organizations to define and enforce specific key-value pairs for agents, Model Context Protocol (MCP) servers, and skills. This feature provides a standardized way to describe, govern, and discover resources within the registry, ensuring consistent tracking of organizational attributes.
Novo Nordisk uses Amazon Bedrock AgentCore to modernize manufacturing document management
Novo Nordisk is developing an intelligent support system with Amazon Bedrock AgentCore to manage its manufacturing process documents. This initiative aims to reduce lead times, improve documentation visibility, and make manufacturing knowledge more accessible by automating document updates and tracking.