Git at Any Scale
Git hosting is becoming AI infrastructure: agents create more repositories, pushes, clones, and CI pressure, so version-control storage needs database-like durability and scaling properties.
Logged at IST: 2026-08-19 09:51 IST
What it is: Vicent Martí's Cursor post on the storage system behind Origin, Cursor's Git hosting product.
Gist: The useful framing is that Git hosting is hard because Git was designed so every repository copy is equivalent. That makes the local developer workflow great, but it makes server-side scaling awkward: packfiles are optimized for local disk and Git clients still expect packfiles over the network.
The post walks through prior approaches. Object-level distributed stores map nicely to content-addressed Git objects, but DAG walks become round-trip-heavy and clone still needs packfiles. Distributed filesystems preserve the normal Git layout but run into filesystem semantics and performance issues. GitHub's Spokes-style design works better by keeping real Git repos on local NVMe and replicating packfiles consistently, but its 3PC shape struggles when repositories need many replicas or when agents create many small, mostly idle repositories.
Cursor's Continuity keeps the good part, normal Git repositories on fast local disks, but moves truth into an S3-backed write-ahead log. Pushes are persisted as WAL entries before acknowledgement, visibility is controlled through a WAL index, and any node can materialize a repository from the WAL. Local repositories become warm cache rather than pets. S3 compare-and-swap gives linearizable pushes, while fully consistent replicas scale read-heavy Git operations, API reads, UI interactions, and agent workflows.
Newsletter angle: Strong systems design read on treating source control like a database: WAL as truth, local disk as cache, linearizable writes, compaction as part of replication, and AI-agent traffic changing both the floor and ceiling of Git infrastructure.
Update, 2026-08-24: Vicent Martí pointed out that Tobi Lütke and Scott Chacon both spent a weekend implementing the Continuity shape. Tobi's walgit is a Rust Git server in front of S3/GCS with no database, no leader, and disposable local cache nodes; it adds practical pieces like bundle-uri clones, Git LFS, push policy, webhooks, remote readers for repositories larger than the machine, and history packs. Chacon's gitwal is a live accountless Git host where S3 holds the WAL, local disk is cache, names are claimed by first valid push, and the architecture page walks through the compare-and-swap commit point, routing, fetch, and rebuild path.
The closest “original paper” in this cluster is still Martí's Cursor design essay itself, not an academic paper. The academic paper explicitly linked from the design is Dean and Barroso's The Tail at Scale (PDF), which supplies the latency-tail framing behind the “more replicas can make tail behavior worse” point. A useful older precursor is Shawn Pearce's 2011 JGit on DHT update: he describes abandoning object-level DHT storage for Git after clone/read traffic proved too expensive, and moving toward native pack/index files over an S3-like storage layer.