Circle K reduces incident triage time to under 15 minutes using Amazon Bedrock

Automating Incident Triage

Circle K, a global convenience and fuel retailer, faced increasing challenges with incident triage as its cloud-native infrastructure expanded. Manual investigation of alerts, involving log review and correlation across multiple tools, typically took 45-60 minutes before remediation could begin. This overhead grew with the company's digital operations, which support point-of-sale, loyalty, and EV charging applications on AWS.

Amazon Bedrock Implementation

To address this, Circle K developed an automated triage system utilizing Amazon Bedrock, a managed service for foundation models. The system employs an agentic AI architecture to autonomously investigate incidents. It then delivers a structured summary of the incident to Slack within minutes of an alert firing. This provides on-call engineers with actionable context, enabling immediate action rather than prolonged manual investigation.

Significant Time Reduction

Since deploying this automated system, Circle K has reduced its mean triage time to under 15 minutes. This represents a 67-75 percent reduction compared to previous manual processes, based on the team's internal incident-response metrics. The improvement allows engineers to focus on resolving issues rather than spending significant time on initial diagnosis.

System Architecture and Evolution

The system monitors infrastructure across multiple AWS accounts using Grafana, handling alerts for various issues like container restarts, error rates, and latency spikes. The architecture involved an initial Amazon Bedrock Flows implementation, which was later migrated to Amazon Bedrock AgentCore. This evolution helps Circle K manage the scale of its digital operations and the volume of alerts generated.