Why does including a gem in Gemfile resolve a railtie issue, even though this same gem is already included in Gemfile.lock?

I'm trying to wrap my head around why a problem I was struggling with is now magically resolved.

I am building a Rails app that uses Spotify OAuth (via the rspotify gem) and became stuck on the exact issue described here. After spinning my wheels, I finally came across this comment, recommending that I explicitly add the omniauth gem to my Gemfile.

Now, this omniauth gem was already a dependency in Gemfile.lock for omniauth-oauth2 specifically. As the linked comment recommends, I included omniauth in my Gemfile and now my problem is seemingly resolved, but I don't really know why.

Why does including a gem in your Gemfile resolve a railtie issue in this case?
If a gem is already installed as a dependency (according to Gemfile.lock) isn't that evidence that a given gem was installed? If, say, gem_foo is listed as a dependency in Gemfile.lock and I add gem_foo in Gemfile and then run Bundler, how does Rails interpret this change?

Solution

This is related to how gems are loaded by bundler. Bundler.require requires gems listed in Gemfile but does not require its dependecy. Its upto the library to require/load its dependency.

The mentioned issue happens when omniauth is not added explicitly to Gemfile, hence bundler does not require it.

But since omniauth-rails_csrf_protection assumes the ominauth is already required, it errors out when user only adds omniauth-rails_csrf_protection but does not add omniauth to Gemfile.

I have created a possible fix for the issue https://github.com/cookpad/omniauth-rails_csrf_protection/pull/13

UPDATE: The is fix has been merged in the gem repo.