/ DevOps

DevOps Tools - Repo

Nowadays, the engineer became a full stack and the software project became more and more complex. In my running project, we have multiple git repositories and in each git repository have own version to control it. But when the projects are more than 5 or more, the total version control will be totally messy.

When we are still using Subversion as the version control tool, we use svn:externals to handle subproject. It works well, but you know the branch of the svn is not easy to use and it's very hard to switch and also cost a lot of storage to do the branch. Therefore, we came to Git. But the problems came, how to handle multiple Git projects?

Repo

Repo is the official tool from Android Open Source Project (AOSP) to manage all the git repositories.

Repo has the main project which called manifest project. This manifest project including one default.XML file. In this default.XML file, that described all used git projects individual. e.g. AOSP itself contain a very large one which is use for compiling for an Android ROM and it's a very large one over than 200 projects inside.

The main project itself has the version control by git and for each project itself, also use git to handle the project with version control.

The documentation for Repo assumes you will clone AOSP and building it. I will introduce how to use Repo into your own project from scratch.

Install Repo

  • If you're using Mac OS X, please us brew install repo to install tool.
  • If you're using Linux or don't have HomeBrew, use the following command to install Repo.
$ mkdir ~/bin
$ PATH=~/bin:$PATH

$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo

This is the official steps from Android to 'Downloading the Source'.

And please check your downloaded file SHA-1 checksum is right. link

Manifest

By using repo init -u <mainfest_url> command, repo can initial one default.XML including all projects. This default.xml is for describing the project, and the relationships between the various components. The manifest MUST BE called default.xml, and must live in the root directory of a repository.

The Android own manifest is too large to understand. Here is a small demo for our project.

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  <remote  name="origin" fetch=".." />
  <default revision="master" remote="origin" sync-s="true" sync-j="4" sync-c="true" />
  <project path="imateuro_php" name="imateuro_php.git" revision="master"/>
  <project path="imateuro_python" name="imateuro_python.git" revision="develop" />
  <project path="imateuro_forum" name="imateuro_forum" />
</manifest>

The full definition of the manifest XML format please see details Repo Manifest Format. This manifest is hosted in https://vardestiny@bitbucket.org/31mins/imateuro_manifest.git, therefore the project imateuro_php is fetch from https://vardestiny@bitbucket.org/31mins/imateuro_manifest.git/../imateuro_php.git which is https://vardestiny@bitbucket.org/31mins/imateuro_php.git

Initialization

Please prepare the repository for your manifest project. e.g. I use 'imateuro_manifest' as the main project.

$mkdir src
$cd src
$repo init -u https://vardestiny@bitbucket.org/31mins/imateuro_manifest.git

Repo init will create .repo folder to store all project information. Also if you have special the

Note: BitBucket also can use SSH for git repository, you just need to add your public RSA key to your account setting.

Repo Sync

Before sync with all your git projects. Please make sure you have created the related git repository in the server. And if you have set property 'revision' for your element 'project' or 'default', please make sure you have the related branch in the git server. Otherwise, the repo sync will fail.

$repo sync

After sync, you will get all the source code with the ${revision} branch. e.g. Here in our case, the imateuro_python project has the develop branch.

Repo Basic Workflow

For git, the benefit is easy to use a branch and merge. Therefore, we choose git-flow as the basic git workflow, each version or feature will start a new feature for development.

Repo Basic Workflow is same as git-flow, after your repo sync all the source code and before start a new version, please run following command.

$repo start NEW_VERSION_AND_FEATURE --all
$repo forall -c git push origin NEW_VERSION_AND_FEATURE

The first command will create all the project with a new branch NEW_VERSION_AND_FEATURE. Then use repo forall command to push all branch to the git repository.

Then when you are in developing, please use the basic git command to commit and push as usual. e.g. git commit, git add.

After finish the development, here we use git merge to the master branch or which branch you want. From the official website of repo tool, they use Gerrit Code Review system to handle all the merge via command repo upload.

Note: For Gerrit I will write another blog for desirable the detail.

basic-android-workflow

Other commands please refer Using Repo.

Benefits

Repo can help us to handle multiple projects easy for continuous integration and deployment. The Repo tool shares a maintainer team, and it’s home repository with Gerrit, and there’s a Google Group for both with lots of activity and a proactive and helpful community.

References