Search code examples
gitgit-mergegit-rebase

How to rebase my local commits on top of new commits in base branch in remote

I'm trying the bread-and-butter usecase of git rebase and it's not working as expected.

According to a guide, when there are new commits to your feature's base branch in remote: "You want to get the latest updates to the master branch in your feature branch, but you want to keep your branch's history clean so it appears as if you've been working off the latest master branch."

Expectation is that after rebasing (git rebase master), the commits in feature branch are re-written. But reality is local-feature branch ends up 1 behind and 2 ahead than remote-feature branch ([feature/123 ↓·1↑·2|✔]). See bash-git-prompt for format.

I also tried git rebase --onto <SHA of master's head>. Similar result [feature/123 ↓·1↑·1|✔].

So obviously both pull and push are rejected (see #4 below). What do I do next?

Notes:

enter image description here

A working example:

1. Create first commit in master

git clone ssh://git@bitbucket.company.com/~kash/rebase-test.git
cd rebase-test/
date > only-changed-in-master.txt
git add only-changed-in-master.txt 
git commit -m 'creation'
git push

2. Create a feature branch from master and add a commit to feature

git checkout -b feature/123
date > only-changed-in-feature.txt
git add only-changed-in-feature.txt 
git commit -m 'creation'
git push --set-upstream origin feature/123

3. Add new changes to master

git checkout master
date > new-file-in-master.txt
git add new-file-in-master.txt 
git commit -m 'adding new file in master'
git push

4. Rebase feature onto master to bring latest changes from master in feature

✔ ~/workspaces/rebase-test [feature/123|✔]
$ git rebase master
Successfully rebased and updated refs/heads/feature/123.
✔ ~/workspaces/rebase-test [feature/123 ↓·1↑·2|✔]
$ git push
To ssh://bitbucket.company.com/~kash/rebase-test.git
 ! [rejected]        feature/123 -> feature/123 (non-fast-forward)
error: failed to push some refs to 'ssh://bitbucket.company.com/~kash/rebase-test.git'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
✘-1 ~/workspaces/rebase-test [feature/123 ↓·1↑·2|✔]
$ git pull
hint: You have divergent branches and need to specify how to reconcile them.
hint: You can do so by running one of the following commands sometime before
hint: your next pull:
hint: 
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint: 
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
fatal: Need to specify how to reconcile divergent branches.
✘-128 ~/workspaces/rebase-test [feature/123 ↓·1↑·2|✔]
$

5. Rebase feature onto latest commit SHA from master's git log

$ git checkout feature/123
$ git reset --hard origin/feature/123
HEAD is now at c3a8f2c creation
✔ ~/workspaces/rebase-test [feature/123|✔]

git checkout master
git log 
git checkout feature/123
...
$ git rebase --onto 5e3a2ed1f3657e7cde741d014a2c86afd99f1d92
Successfully rebased and updated refs/heads/feature/123.
✔ ~/workspaces/rebase-test [feature/123 ↓·1↑·1|✔]
$
Solution

  • As already pointed out, your push is failing because it is diverging from origin/feature/123 and the remedy is to force push (use --force-with-lease). But I want to start by explaining in detail what the git prompt actually means since I think there is some misconception.

    Replaying the steps to step 4 (including) gives the following history, displayed with gitk --all (pink overlays added with Gimp):

    gitk screenshot

    which clearly shows the meaning of [feature/123 ↓·1↑·2|✔]: your local feature/123 branch is

    • one (remote) commit behind origin/feature/123
    • two (local) commits ahead of origin/feature/123.

    And I want to stress that this has nothing to do with where the main or master branch is. Yes, that branch was incidentally used to arrive at this point but the same could also happen even if feature/123 was the only branch existing. Replaying up to (including) step 3 and then running

    git checkout feature/123
date >> only-changed-in-feature.txt
git add only-changed-in-feature.txt
git commit --amend -m 'creation modified'
date >> only-changed-in-feature.txt
git add only-changed-in-feature.txt
git commit -m 'new commit'

    also results in [feature/123 ↓·1↑·2|✔] and gives the following history:

    gitk screenshot 2

    Where main is (or if it even exists) in this scenario is irrelevant.

    So your rebase operation was 100% successful and resulted in no problems by itself. The only issue you have is the divergingness of the feature/123 branch. This now needs a force push to resolve, and there are very good reasons for cautioning against doing this willy nilly without a careful thought about what the consequences will be. But it is absolutely not a you should never, never ever do this thing.

    There are two potential problems with force push

    • you might silently discard/loose commits pushed by other collaborators
    • other collaborators are required to perform "recovery" that is not necessarily trivial

    So right of the bat, if you are the only person creating commits on branch feature/123 then there really is no issues with force pushing it.

    To go a bit deeper on the two issues, imagine Alice and Bob working together on branch somebranch. At the beginning of the day both starts by pulling, and the branch contains commits C1 -> C2 -> C3.

    Alice works on completing a feature, and creates two commits A1 and A2 and pushes those so that the origin repository now contains C1 -> C2 -> C3 -> A1 -> A2.

    Bob on the other hand has just updated the compiler and discovers that commit C3 does not compile with the new version of the compiler. He spends some time figuring out what the issue is. When arriving at a solution he amends the commit so it does compile, replacing C3 with a new B1 commit.

    Bob has not run git fetch since he started (after all he has been busy figuring out why the code does not compile!) so when he runs

    git push origin --force somebranch

    this is virtually identical to

    git push origin --delete somebranch
git push origin somebranch

    and the origin repo then ends up containing commits C1 -> C2 -> B1 and by that Alice's two A1 and A2 commits were discarded and lost by this force push.

    This is the big bad danger of force push that people rightfully are worried about.

    There is a much safer alternative to --force called --force-with-lease that would prevent Bob from discarding Alice's two commits in the above scenario. However depending on when git fetch is run (and notice that tools and editors (like vscode) might run git fetch in the background behind your back!) you still might accidentally override other peoples commits so there is an additional --force-if-includes option you ought to use together with --force-with-lease.

    Alternatively you can supply your expectancy of the remote head directly as an value to the --force-with-lease option like

    git push --force-with-lease=refs/heads/somebranch:<expected-remote-sha> origin somebranch

    I agree with Devin Rhode that this probably is the safest way to force push, and would recommend doing this for "unusual"/important cases (for instance force pushing the main branch).

    For just updating after rebases on a feature branch that only you work on, creating and using an alias

    [alias]
    forcepush = push --force-with-lease --force-if-includes

    is perfectly fine.

    The second issue is recovery. Say Bob used --force-with-lease, detected that Alice had done some work and then incorporated those commits (hurray, no loss of commits!). So Bob pushes his updates, and the remote repo now contains C1 -> C2 -> B1 -> B2 -> B3, where B1 is the fixed C3 commit, and B2 and B3 corresponds to A1 and A2 just with different parents.

    Meanwhile Alice has started on a new feature and created a new A3 commit locally, so her repo looks like C1 -> C2 -> C3 -> A1 -> A2 -> A3. She gets to know that Bob fixed something related to a new compiler version and wants to get that fix in. To do that she has to

    git commit -am "==== save ===="    # A4
    • Retrieve Bob's changes and inspect the history to see what she needs to do to recover
    git fetch origin
gitk --all &
    • Update her somebranch branch to be on top of Bob's latest origin/somebranch commits.

    Just running the two argument version of rebase1

    git rebase origin/somebranch somebranch  # Will almost guaranteed trigger conflict
git reset HEAD^  # Undo A4 back to uncommitted changes

    might work, however this will trigger a git conflict due to the difference between C3 and B1 which is unrelated to what Alice worked on but suddenly she gets to resolve that conflict anyhow... And even when successfully resolving conflicts there is a high chance of ending up with phantom dummy commit that maybe just contains a small (duplicated) part of conflicting commits, e.g. Alice starts with

    C1 -> C2 -> C3 -> A1 -> A2 -> A3 -> A4

    and should end up with

    C1 -> C2 -> B1 -> B2 -> B3 -> A3' -> A4'

    but actually ends up with

    C1 -> C2 -> B1 -> B2 -> B3 -> C3' -> A3' -> A4'

    where C3'2 is a commit with some artifact of some automatic (or incorrect manual) conflict resolvement which then needs to be removed with interactive rebase after the rebase is done.

    So the much better way to run the rebase here is to use the three argument version of rebase3:

    git rebase --onto origin/somebranch A2 somebranch
git reset HEAD^  # Undo A4 back to uncommitted changes

    because this then skips the commits whose conflicts are already resolved. Depending on what changes are between C3 and B1 this might cause conflicts for the new A3' and A4' commits, but this would be unavoidable under any circumstances. The three argument rebase command is the most conflictless way to do this.

    As you can see, this does require some skill level on Alice's part beyond the very basic git commands, although it is quite manageable once you learn it. But some people will struggle with this so this is something to watch out for.

    1 You should never just use the one argument version of rebase.

    2 It is common in mathematics (and elsewhere) to add ' to a variable to indicate it is transformed in some way, e.g. X -> X' for some kind of modification. I am using this here to not let different commits use the same name even though they have identical content, their parents are different and hence have different hash ids.

    3 So git rebase --onto origin/somebranch A2 somebranch means rebase branch somebranch from (not including) commit A2 on top of origin/somebranch.