Why Google Stores Billions of Lines of Code in a Single Repository
Why Hacker News recommends it
The go-to reference in monorepo and trunk-based development discussions, recommended as a firsthand account of how Google manages source control.
What it is
Rachel Potvin describes the scale of Google's codebase and its custom monolithic source repository. She explains the reasons for the monorepo model, the tooling that makes it workable, and its tradeoffs.
Recommended by 2 HN commenters across 3 different months, 2016 to 2022.
What Hacker News said
Recommends twotwotwo · 2016-07 · on “Go Packaging Proposal Process”
It's common practice for folks building Go binaries (but not libraries) to check in the code for their dependencies. So whenever you build Docker, you get the same thing unless Docker made a change (either by changing Docker itself or explicitly checking in an upgrade to a dep). There are a few nice things about it; vendoring means you won't have a dependency on other repos or archives being available, and if you're a binary maintainer you can see source changes in your own code and that of your dependencies side-by-side if you're, say, investigating a bug that came up between certain revisions. The not-so-nice things are pretty much what you'd think. There isn't a command to say "give me up
Recommends kyrra · 2017-01 · on “Trunk-Based Development”
Google doesn't use perforce anymore. They have a custom in-house SCM that has various interfaces wrapped around it. I'd recommend this video that describes it. [this video]
Recommends kyrra · 2022-05 · on “Google is 2B lines of code and it's all in one place (2015)”
Googler opinions are my own. I also highly recommend this ACM article talking about our source control from 2016. [LINK] There's also a YouTube video which is linked from the wired article that is good. [this video]
Cites valarauca1 · 2016-06 · on “Why Google Stores Billions of Lines of Code in a Single Repository”
Google gave a talk at the @scale conference in 2015 about this very topic. You can watch it here (30 minutes): [this video]
Cites weinzierl · 2019-01 · on “Software Engineering at Google (2017)”
What I don't understand is how they accomplish larger collaborative changes. The paper says: "Almost all development occurs at the 'head' of the repository, not on branches." Googler Rachel Potvin made an even stronger statement in her presentation about "The Motivation for a Monolithic Codebase" [1]: "Branching for development at Google is exceedingly rare [..]" In the related ACM paper she published with Josh Levenberg there is the statement that: "Development on branches is unusual and not well supported at Google, though branches are typically used for releases." I my world when we have to make a bigger change we create a branch and only merge it into the trunk when it is good enough to
Cites svat · 2019-03 · on “Case Study: Monolithic Repository at Google (2018) [pdf]”
This paper from last year surveys Google “engineers who have experience with both monolithic repos and multiple, per-project repos” to see what exactly they like and dislike about monorepos. The results match my experience, and I find the paper well-written (especially like the “Threats to Validity” section that says what biases may have affected the results; more papers and arguments should have them IMO). Older reading on monorepos that I'm aware of: • Dan Luu, Advantages of monorepos (2015) [LINK] • Why Google Stores Billions of Lines of Code in a Single Repository (2015): [LINK] (HTML) / [LINK] (PDF) / [this video] (video) This paper is from the “Software Engineering in Practice” track of the