How to Properly Mirror CyanogenMod

Background and Goal

We are using Gerrit as a Git server. It's a decent Git host solution has some nice access control features. We have built a set of tools that can automatically merge a given set of experimental branches to our baseline branch, and generates incremental OTA updates that we can push to our participants.

Starting from summer 2016, we are using the Nexus 6 device (code name shamu). We have been using the stock AOSP mirrors since the very beginning. But this year we decided to give CyanogenMod a try (particularly because it is a huge pain to even get a working ROM for Nexus 6 using stock AOSP). But we still want the automated process of merging experimental changes and do OTA updates.

The goals of our mirror are:

• A simple repo init/repo sync using our manifest should give you a working code-base, meaning you can build a working ROM for Nexus 6 with it. No any special tweaks needed on the experimenter's side.
• Each repo should have a common baseline branch (called phonelab/cm-13.0/develop that somebody can fork from and start making experimental changes.

The Overall Picture

• Produce a local clone that are suitable to server as a mirror
• Push this local clone to our Gerrit server
• Compose a proper repo manifest to point things to the right place.

Get a Working Mirror Repo

We have chosen the latest stable CyanogenMod release for Nexus 6 (stable/cm-13.0-ZNH2K). The first trap is that: the default manifest does not work if you want to create a mirror. In particular, CM has used shallow clones (clone-depth="1") for certain repos. This is OK if you do not intent to ever push the repo, but most likely Gerrit will complain about this and eventually claim that these repos are corrupted because the history is not complete.

So the first step is to fork the CyanogenMod manifest (mine is here: https://github.com/jhshi/android, check the stable/cm-13.0-ZNH2K branch) and remove any shallow clones. This can be done via this VIM command:

1

%s:/clone-depth="1" //g

Also, since we are using a different manfiest repos, we also set the default fetch URL to be an absolute path:

1
2
3

<remote name="github"
    fetch="https://github.com/"
    review="review.cyanogenmod.org" />

Now, do a repo init using this manifest.

1
2

$ repo init -u https://github.com/jhshi/android -b stable/cm-13.0-ZNH2K
$ repo sync

This will download all repositories properly. After this finished, we need to also grab the repos for our specific device.

1
2

$ source build/envsetup.sh
$ breakfast shamu

This will grab two extra repos:

• device/moto/shamu
• kernel/moto/shamu

The next step is to create a common baseline branch based on the current tip.

1

$ repo forall -ec 'echo $REPO_PATH; git checkout -b phonelab/cm-13.0/develop'

Then, we create the corresponding repositories on the Gerrit server. Here is the second trap. In the CM manifest, there are several these projects:

1
2
3
4
5
6
7
8

<project path="hardware/qcom/audio-caf/apq8084" name="CyanogenMod/android_hardware_qcom_audio" groups="qcom,qcom_audio" revision="stable/cm-13.0-caf-8084-ZNH2K" />
<project path="hardware/qcom/audio-caf/msm8916" name="CyanogenMod/android_hardware_qcom_audio" groups="qcom,qcom_audio" revision="stable/cm-13.0-caf-8916-ZNH2K" />
<project path="hardware/qcom/audio-caf/msm8937" name="CyanogenMod/android_hardware_qcom_audio" groups="qcom,qcom_audio" revision="stable/cm-13.0-caf-8937-ZNH2K" />
<project path="hardware/qcom/audio-caf/msm8952" name="CyanogenMod/android_hardware_qcom_audio" groups="qcom,qcom_audio" revision="stable/cm-13.0-caf-8952-ZNH2K" />
<project path="hardware/qcom/audio-caf/msm8960" name="CyanogenMod/android_hardware_qcom_audio" groups="qcom,qcom_audio" revision="stable/cm-13.0-caf-8960-ZNH2K" />
<project path="hardware/qcom/audio-caf/msm8974" name="CyanogenMod/android_hardware_qcom_audio" groups="qcom,qcom_audio" revision="stable/cm-13.0-caf-8974-ZNH2K" />
<project path="hardware/qcom/audio-caf/msm8994" name="CyanogenMod/android_hardware_qcom_audio" groups="qcom,qcom_audio" revision="stable/cm-13.0-caf-8994-ZNH2K" />
<project path="hardware/qcom/audio-caf/msm8996" name="CyanogenMod/android_hardware_qcom_audio" groups="qcom,qcom_audio" revision="stable/cm-13.0-caf-8996-ZNH2K" />

As you can see, they are in fact from the same remove repositories, just with different revision name, and they are expected in different folders. Since we want a common branch name for each repository, we have to create multiple repositories on our server, so that we can let the same name phonelab/cm-13.0/develop to point to different commits. The key point here is: name is no longer a unique key to identify a project, but path are. So we will name the repos by their path, not by name.

1

$ repo forall -ec 'echo $REPO_PATH && ssh -p 29418 user@server gerrit create-project cm-shamu/$REPO_PATH'

Note that I am using $REPO_PATH, which is the local filesystem folder name, rather than $REPO_NAME. Also, all such repos are under the cm-shamu/ name space on our server.

Next, upload all those repos:

1

$ repo forall -ec 'echo $REPO_PATH && git push user@server:29418/cm-shamu/$REPO_PATH refs/heads/*'

This shall take a while.

Get a Working Manifest

Now all repos are in our Gerrit server, we need to compose a proper manifest for repo init.

Start with the default manifest at https://github.com/CyanogenMod/android. We made these changes:

• There should be only one remote called phonelab, which points to our Gerrit server.
• The default revision of every project should be phonelab/cm-13.0/develop.
• Remove any individual revision or remote project attributes. This can be done by this VIM command: :%s/revision=".\{-}" //g and :%s/remote=".\{-}" //g, where .\{-} is VIM's non-greedy regex syntax.
• Remove all name attribute, since the name will be the path: :%s/name=".\{-}" //g.
• Change path=.. to be name=..: :%s/path=/name=/g.
• Change default fetch URL to be . since the manifest and all other repos are now in the same level.

I ended with this manifest: https://github.com/jhshi/cm.manifest.shamu/blob/phonelab/cm-13.0/develop/default.xml

Create a Gerrit project with the path cm-shamu/manifest.git and push the modified manifest to it.

At this point, anybody should be able to do a single repo init using the above manifest, and all repos will be pulled from our Gerrit server.