Netflix Titus Adds Single-Tenant VM Isolation for Containers for Performance-Critical Workloads

Introducing Single-Tenant Titus

Netflix has implemented a single-tenant mode within its Titus container orchestration platform. This new mode allows individual tasks to operate in isolation on their own dedicated and appropriately sized EC2 instances. The primary goal is to mitigate performance interference that can occur in multi-tenant environments, particularly for sensitive workloads.

Addressing Multi-Tenant Limitations

Titus typically operates in a multi-tenant configuration, launching millions of containers weekly across tens of thousands of EC2 instances. While existing isolation mechanisms like cgroups, seccomp, iocost, and traffic shaping are used, Netflix found they could not fully isolate shared hardware resources such as last-level cache and memory bandwidth. Issues like kernel CPU starvation from packet floods also persisted, leading to the conclusion that shared-kernel tuning had inherent limits.

Rationale for Single-Tenant Approach

Netflix evaluated alternatives like MicroVMs (Firecracker, Kata) and gVisor but found them unsuitable due to requirements for bare-metal or nested virtualization, added overhead, lack of GPU support, or maintenance concerns. Directly running containers on EC2 was also rejected as it would shift host management responsibilities back to application teams. The single-tenant Titus approach provides performance isolation without requiring defense against hostile tenants.

Implementation Details

The Titus federation layer manages the single-tenant setup. An allowlist controls task eligibility and instance family usage. The autoscaler selects the smallest EC2 instance matching the task's resource requests. Isolation is enforced using pod anti-affinity, a non-preemptible priority class, and node taints to ensure one task per node. Defragmentation is disabled to prevent task relocation. Performance isolation is measured by comparing a reference application's throughput and P99 latency when run alone versus alongside a 'havoc' container.

Costs and Mitigations

The main drawback of this approach is increased launch latency, as a new instance requires tens of seconds to boot before image pull. While warm pools help, their effectiveness is reduced due to the variety of instance shapes needed for right-sizing. Netflix maintains a small warm pool for the most latency-sensitive starts and restricts the available instance sizes to manage this overhead.