PhiloLabs AI Agents Reconstruct Virtual Union Square for $33, Identify Visual Errors

Virtual Union Square Construction

PhiloLabs conducted an experiment where Claude Fable 5.1 AI agents reconstructed a 3D virtual model of San Francisco's Union Square. The project, built from real-world geographic data and reference images, cost approximately $33 in API calls and used about 8 million tokens.

Within two hours, the agents produced a working Three.js version of Union Square in a browser. This included 453 building footprints, 75 custom façades, 129 named storefronts, 220 pedestrians, and 109 vehicles, including cable cars.

Integrating Playwright for Visual Verification

Beyond functional code, PhiloLabs aimed for the agents to detect visual discrepancies. Playwright was integrated into the development loop to capture screenshots from 34 predetermined camera positions. These screenshots were then compared against photographs of the actual Union Square.

This process generated 147 comparison sheets, which helped in identifying issues where elements were technically correct in code but visually inaccurate, such as incorrect building proportions or misplaced storefronts.

Agent-Based Review Process

Specialist AI agents were then tasked with reviewing the generated material. Some agents focused on architectural and geographical accuracy, while others concentrated on technical art and interactions. This review process resulted in nine reports on the Union Square build.

These reports served as a punch list, allowing the development agents to address the identified findings and rerun the scene for corrections. This division of labor is useful for catching mistakes that are not easily translated into conventional automated tests.