Process for syncing branches between git remotes

I am working on my own copy of a git repo called buildroot. I created a copy of the repo on the free bitbucket server. I am trying to periodically sync my private repo from the mainline buildroot repo.

I pulled down the local copy to my PC using

git clone https://[email protected]/my_user/buildroot.git

Then I added the mainline buildroot repo as a remote

git remote add mainline https://git.buildroot.net/buildroot

git remote -v now looks like this

mainline    https://git.buildroot.net/buildroot (fetch)
mainline    https://git.buildroot.net/buildroot (push)
origin      https://[email protected]/my_user/buildroot.git (fetch)
origin      https://[email protected]/my_user/buildroot.git (push)

I can pull the changes from as single branch (such as master) on the mainline remote and these changes show up in my origin repo after a git push

get pull mainline master
git push

However, new branches never show up on my origin repo. Do I have to explicitly checkout each one of these from mainline and then push to origin or is there a better way to do this?

As seen below, my private repo is missing branches 2018.02.x and 2018.05.x

git branch -a

* master
remotes/mainline/2017.02.x
remotes/mainline/2017.05.x
remotes/mainline/2017.08.x
remotes/mainline/2017.11.x
remotes/mainline/2018.02.x
remotes/mainline/2018.05.x
remotes/mainline/master
remotes/mainline/next
remotes/origin/2017.02.x
remotes/origin/2017.05.x
remotes/origin/2017.08.x

Solution

TL;DR

Do I have to explicitly ... push to origin

Yes, but not quite the way you are thinking. Still, that may be the most convenient.

Long

Every Git repository is an independent entity. Remotes (as in your own repository) or fork linkages (as in a GitHub or Bitbucket fork) record the identity of another Git repository, but the two repositories remain separate entities. As such, each has its own branches, not shared with any other Git repository.

What is shared with other Git repositories, but only at "sharing time" (git fetch and git push), are the commits. These are shared using their raw hash IDs. This is also where branch names really enter the picture, though, because a Git repository finds its commits by its branch names (and other references, but we'll concentrate only on branch names here).

Remember that the role of a branch name in a Git repository is to contain the hash ID of the last commit that is to be considered "on" that branch, i.e., the tip commit of the branch. When you use git checkout branch followed by an eventual git commit, your Git:

builds the new commit
while setting the new commit's parent hash ID to the current commit hash ID
then writes the new commit's hash ID into the current name

so that the branch name once again points to the latest (tip) commit. That tip commit points backwards to the previous tip, which points backwards to another earlier commit, and so on.

Hence, what git push does is:

hand over some commit(s) that, ideally, simply extend some existing branch(es), then
requests that the other Git set its branch name(s) to remember the tipmost commits of these added commits.

(Extending a branch like this is a fast-forward operation.)

If the new commit(s) don't just extend some existing branch(es), those particular name-update(s) must be "forced": the other Git will just say no to those update requests that are not fast-forward operations and are not forced.

There is, however, no requirement that these branch-name value changes correspond to any actual branch name(s) in your own repository. For instance, consider the common (for me at least) case where I am working on some feature and I write a half dozen or more commits. Out of these, I determine that one is definitely-for-sure a good idea, so I run git rebase -i feature and put that one up front. Then I may run:

git push origin <hash>:for-review

and make a pull request based on for-review. Pictorially, what I have in my own repository at this point looks like this:

...--o--o   <-- origin/feature
         \
          *   <-- origin/for-review
           \
            o--o--o--o--o   <-- feature

Five of the six commits remain unpushed; only one, marked * here, has been pushed, to the name for-review on the repository at origin. Yet I can continue working in my own repository on my feature branch, which is six commits ahead of origin/feature and five commits ahead of origin/for-review. I just need to be careful to remember that commit * is now shared (which is easy enough: it has an origin/ name that can reach it).

This brings us to what you have to do in your own repository, in order to transfer commits from the Git repository at mainline over to the repository at origin. First you must obtain the commits (and their tip-pointing names) from mainline:

git fetch mainline

Now you have all the commits; now you can update origin using, e.g.:

git push origin mainline/2018.05.x:2018.05.x

which will send any commits that origin does not have, but does need, to make this work, followed by creating the name 2018.05.x on origin, pointing to the same commit as your own mainline/2018.05.x.

This is admittedly all a little messy—but the point is that you don't have to have a branch name for the commits, as long as you have some name by which you (and your Git) can find the commits in your own repository. To send those commits to another Git, you can git push them by hash ID, or origin/whatever name, or you can create a branch name, or do whatever you like as long as it identifies the appropriate tip commit. On the right-hand side of the lhs:rhs name-pair, however, you must provide a name that the Git at origin will be OK with creating or updating.

Convenience

With git push (but not git fetch!), if you run:

git push origin abc

that "means" the same thing as:

git push origin abc:abc

so it's definitely more convenient in many ways to have your own local names for each branch.