Runway ML turned an AI video generation bug into a feature for its Characters model

Bug Becomes Feature in AI Video

Runway ML faced a persistent bug in its AI video generation model where AI-generated avatars would drift off-center during real-time video. Instead of a back-end patch, the company developed a new front-end feature that circumvented the problem, effectively turning the bug into a functional aspect of the system.

Ryan Phillips, head of enterprise product at Runway ML, discussed this development at VB Transform 2026. He presented it as an example of how companies, even those not building foundation models, can learn from Runway's approach to building, evaluating, and deploying AI models.

Runway Characters and Real-Time Generation

Runway ML is an applied AI research company focused on creating general world models for generative tools. Phillips showcased Runway Characters, a real-time video model that enables zero-latency, interactive experiences with AI-generated avatars. He noted that creating similar video content previously took artists hundreds of hours, whereas Runway's models now generate interactive video on the fly.

Demystifying AI Model Evaluation

Phillips emphasized the importance of a high-quality evaluation set for building robust AI products. He stated that creating this set requires cross-functional alignment across product, design, research, and sales to define what constitutes "quality." Runway conducts internal workshops where teams review generated examples to align on specific failure modes and establish a unified definition of successful generation.

The evaluation set must cover broad customer use cases and extreme edge cases. For instance, Runway used a non-human character named "Tooth," which lacked a nose and had unusual teeth, to ensure the model's predictable behavior when pushed beyond standard human facial structures. The team grades generations by looking for subtle artifacts.