Transitioning Apples Downstream Repositories To The Monorepo Alex - - PowerPoint PPT Presentation

▶

Feb 15, 2024 151 likes •337 views

Transitioning Apples Downstream Repositories To The Monorepo Alex Lorenz Apple Lets recreate our downstream repos Lets improve our workflows Monorepo Tooling Downstream monorepo generation tools $ git

SLIDE 1

Transitioning Apple’s Downstream Repositories To The Monorepo

Alex Lorenz Apple

SLIDE 2

SLIDE 3

🤩

SLIDE 4

😲

SLIDE 5

💢

🤡

Let’s improve our workflows Let’s recreate our downstream repos

SLIDE 6

Monorepo Tooling

Downstream monorepo generation tools Auto merger infrastructure and tools GitHub PR & CI tools $ git apple-llvm … $ git apple-llvm am … $ git apple-llvm pr …

👊 github.com/apple/apple-llvm-infrastructure-tools 👊 LLVM Project License

SLIDE 7

Generating Downstream Monorepos

SLIDE 8

Downstreams @ Apple

3+ levels of forks / branches Each level had llvm, clang, lldb, etc. Git merges used between levels https://git.llvm.org/git/{llvm,clang,lldb,...}.git github.com/apple/swift-{llvm,clang,lldb,...} Internal master branch for llvm, clang, lldb, … Additional branches/forks of llvm, clang, …

SLIDE 9

Are all downstream branches preserved?

github.com/apple/swift-{llvm,clang,lldb,...} Internal master branch for llvm, clang, lldb, … Additional branches/forks of llvm, clang, …

Can we preserve downstream history? Can historic checkouts build clang and lldb?

🤕

Monorepo Concerns

SLIDE 10

Are all downstream branches preserved?

Yes…

Can we preserve downstream history? Can historic checkouts build clang and lldb?

Monorepo Concerns

Let’s generate the Git history ourselves

SLIDE 11

llvm

downstream downstream llvm.org llvm.org

clang

Creating New History

To generate a non-llvm.org commit: Move Git tree to a subdirectory Rewrite parents to new commit hashes To generate a branch: Interleave first-parent generated commits Order the disjoint commits by date Prioritize merges based on directory Yes… Let’s generate the Git history ourselves

SLIDE 12

llvm

downstream downstream llvm.org llvm.org

clang Generated monorepo

SLIDE 13

Generated monorepo

😏

Upstream monorepo hashes are used A checkout of any downstream commit contains full downstream monorepo

Generated Monorepo

Downstream commit history preserved Downstream tags preserved

SLIDE 14

Workflow Improvements

SLIDE 15

Auto Merger

Merges upstream branches downstream Configuration in target branch Builds and tests before merging Merge conflicts create PRs

llvm.org/master apple/master swift/master-next apple/stable/20190619 swift/master

🤗

SLIDE 16

PR & CI Tools

Create, merge and list monorepo PRs Test PRs using custom Jenkins test plans Test PRs using swift-ci

$ git apple-llvm pr test #42 Triggering pull request testing for pr #42 by hyp: Support git-cat-file as read_file function ✅ you commented "@swift-ci please test" on the pull request.

SLIDE 17

The End

Thank you!