Looks like it is a bug, but Galaxy S5 and Note 2 add an extra space in front of home drawable when setDisplayHomeAsUpEnabled() is set to false
We finished by enabling home as up indicator as the following:
getSupportActionBar().setDisplayHomeAsUpEnabled(true);
but then hiding it inside styles.xml:
<item name="android:homeAsUpIndicator">@null</item>