Devise authenticate_admin_user! does not work after upgrading to rails 5

I have an old, rather big Rails app I need to upgrade to a current version. It is currently running on Rails 4.2.11. I've managed to upgrade all my gems now, so it runs Rails version 5.0.7. And I am in a state where the app starts again and mostly works. While doing so, I've upgraded the devise gem from version 3.4.0 to 4.0.0, but I've also tried 4.7.3. It does not make a difference to my problem.

The only thing which does not work correctly is authentication. I can load the login screen and login with a user. The login is successful, but then I get redirected back to the main application page, instead of the protected resource.

From what I could found out, the Devise session is not persisted in the session, but I don't understand why it does not work. I don't get any error in the log. The log displays the initial 401 error when I request the protected resource, and we are redirected to the login form (as expected). After a successful login (I see the sign_in_count increase in the database), a redirect to the home page happens, instead of the protected resource.

I've added the following code into the index method of the main page controller (to which I get redirected):

class MainController < ApplicationController
  def index
    puts "Current Admin User: #{current_admin_user} nil: #{current_admin_user.nil?} signedIn: #{admin_user_signed_in?}"

   # rest of the code omitted for simplicity
  end
end

The output is as follows:

web_1 | [pid: 1] [c48b7285-3f9e-4cb7-94ba-64b6c9d9bd0e] Processing by MainController#index as HTML
web_1 | Current User:  is nil: true signed_in: false

The (simplified) routes.rb file looks like this:

root 'main#index'
devise_for :admin_users

namespace :admin do
  constraints(CheckIp.new) do
    devise_scope :admin_user do # a
      root to: '/admin/main#index' # b
      
      resources :main_admin, path: :main do
        ... # contains sub resources
      end
    end
  end
end

I've added the lines a and b after the upgrade in the hope it fixes my issues, but I could not see any difference. My understanding is that the devise 4 should redirect to the root (line b) inside my scope, but this is not happening. I also tried to move the line a before the constraints check and again before the admin namespace. The results are the same in all cases.

Solution

I've finally found the reason for my issues. I've made some modification to the middleware stack for log tagging like this:

Rails.configuration.middleware.delete(ActionDispatch::Cookies)
Rails.configuration.middleware.delete(ActionDispatch::Session::CookieStore)
Rails.configuration.middleware.insert_before(Rails::Rack::Logger, ActionDispatch::Session::CookieStore)   
Rails.configuration.middleware.insert_before(ActionDispatch::Session::CookieStore, ActionDispatch::Cookies)

This does not longer work. So for the time being I remove the log tagging, as authentication is more important.