js requests to Rails 6.0 app render html template instead of js

Question

I'm in the process of migrating a legacy Rails 5.0 app to (hopefully) 7.x. I got to 5.2.x without too much trouble. I'm currently trying to upgrade to 6.0 and I have a problem with controller actions that are set up to render either js or html depending on the type of request. For example, in a standard 'new' action of the general form:

def new
  @post = Post.new
  respond_to do |format|
    format.js
    format.html
  end
end

if a jQuery ajax request is sent with dataType: 'script', the server renders the views/posts/new.html.erb template instead of views/posts/new.js.coffee. The console confirms that the request is received as a js request but that html is being rendered, e.g.

Started GET "/posts/new" for 127.0.0.1
Processing by PostsController#new as JS
  Rendering posts/new.html.erb
  Rendered posts/new.html.erb

However, if I remove new.html.erb from the view directory, then the server renders the js template as required.

Started GET "/posts/new" for 127.0.0.1
Processing by PostsController#new as JS
  Rendering posts/new.js.coffee
  Rendered posts/new.js.coffee

I'm seeing the same behavior with a number of different controllers/actions.

Any suggestions on what I'm missing? FWIW, I am not using webpacker, just regular sprockets. And ruby 2.6.6. The only thing I've changed from 5.2.x related to javascript is to add an assets/config/manifest.js file with the following:

//= link_tree ../images
//= link_tree ../javascripts .js
//= link_directory ../stylesheets .css

Edit: Also FWIW, I added a block with a test statement to the #js and #html methods -

  respond_to do |format|
    format.js {puts "calling js"}
    format.html {puts "calling html"}
  end

and confirmed that the #js method is the one that's being called, even though it is rendering the html template.

Answer 1

Without going too deep, this bit here is an issue:
https://github.com/rails/rails/blob/v6.0.0/actionview/lib/action_view/path_set.rb#L48

def find(*args)
  find_all(*args).first || raise(MissingTemplate.new(self, *args))
  #              ^^^^^^
end
# where `find_all` returns
# =>
# [
#   #<ActionView::Template app/views/posts/new.html.erb locals=[]>,
#   #<ActionView::Template app/views/posts/new.js.coffee locals=[]>
# ]

html format is added along the way as a fallback for js:
https://github.com/rails/rails/blob/v6.0.0/actionview/lib/action_view/lookup_context.rb#L291-L294

if values == [:js]
  values << :html
  @html_fallback_for_js = true
end

This is definitely a rails bug. I'm not exactly sure when it was fixed but everything works fine in rails v7.

---

If you're planning on upgrading, try some later rails 6 versions, see if they fixed it back then. For now I can think of a couple of solutions:

respond_to do |format|
  format.js { render formats: :js } # explicitly render js format only
  format.html
end

Other than that, you could override find_all method and fix the order of templates to correspond to the order of formats - [:js, :html]:

# config/initializers/render_coffee_fix.rb

module RenderCoffeeFix
  def find_all(path, prefixes = [], *args)
    templates = super
    _, options = args

    ordered_templates = options[:formats].map do |format|
      templates.detect { |template| template.format == format }
    end.compact

    ordered_templates
  end
end

ActionView::PathSet.prepend(RenderCoffeeFix)