OpenAI develops Pando to optimize Git monorepo performance

OpenAI has developed a custom Git backend named Pando to address scaling challenges within its monorepo. The system was built in approximately 10 weeks, with significant assistance from Codex, the company’s AI coding agents.

According to OpenAI, Pando has reduced median push latency by nearly fivefold, representing an 80% improvement. The backend is designed to handle a massive surge in activity, as quarterly commits to the monorepo are projected to increase by over 2,400% compared to the first quarter of 2024.

Pando manages two primary issues: high-volume concurrent writes to different branches and heavy read traffic from fetches and clones. The system utilizes causal consistency per branch, allowing parallel updates to unrelated branches while maintaining the correct sequence of changes within each individual branch.

Despite the custom backend, OpenAI maintained full compatibility with the existing Git protocol and clients. The system currently serves nearly one petabyte of Git data per day, reaching peak throughputs of over 57 TB of Git pack data in a single hour.