Testing Rake in Rails: Multiple Error Raises Silenced In Test

Question

I have a rake task that guards against dangerous Rails rake rasks, based on the environment. It works fine. When I test each individual dangerous method in RSpec, the test passes. When I test multiple in a row, for multiple environments, the test fails after the first one. Even if I run the test multiple times for the same dangerous action, rake db:setup for example, it will only pass the first time. If I run the tests as individual it statements, one for each dangerous action, only the first two will pass (there are 4).

How can I get RSpec to behave correctly here, and pass all the tests when run in a suite?

The rake task

# guard_dangerous_tasks.rake
class InvalidTaskError < StandardError; end
task :guard_dangerous_tasks => :environment do
  unless Rails.env == 'development'
    raise InvalidTaskError
  end
end

%w[ db:setup db:reset ].each do |task|
  Rake::Task[task].enhance ['guard_dangerous_tasks']
end

The RSpec test

require 'spec_helper'
require 'rake'
load 'Rakefile'

describe 'dangerous_tasks' do
  context 'given a production environment' do
    it 'prevents dangerous tasks' do
      allow(Rails).to receive(:env).and_return('production')

      %w[ db:setup db:reset ].each do |task_name|
        expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
      end
    end
  end

  context 'given a test environment' do
    it 'prevents dangerous tasks' do
      allow(Rails).to receive(:env).and_return('test')

      %w[ db:setup db:reset ].each do |task_name|
        expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
      end
    end
  end
end

RSpec Output

# we know the guard task did its job,
# because the rake task didn't actually run.
Failure/Error: expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
   expected InvalidTaskError but nothing was raised

Answer 1

I can think about two solution of your problem.

But first we need to find out where is the root of the problem.

Root of the problem

Let's start with a line from your code:

Rake::Task[task].enhance ['guard_dangerous_tasks']

Comparing it with source code of Rake::Task

# File rake/task.rb, line 96
def enhance(deps=nil, &block)
  @prerequisites |= deps if deps
  @actions << block if block_given?
  self
end

you can see, that guard_dangerous_tasks should be added to @prerequisites array. It can be easily checked:

p Rake::Task['db:reset'].prerequisites # => ["environment", "load_config", "guard_dangerous_tasks"]

Continuing with you source code.

You use invoke to execute tasks. If we pay close attention to invoke's' documentation, it states:

Invoke the task if it is needed.

Once the task is executed, it could not be invoked again (unless we reenable it).

But why should this to be a problem? We are running different tasks, aren't we? But actually we don't!

We run guard_dangerous_tasks before all tasks in our tasks array! And it's being executed only once.

Solution #1 not the best one

As soon as we know where is our problem we can think about one (not the best solution).

Let's reenable guard_dangerous_tasks after each iteration:

dangerous_task = Rake::Task['guard_dangerous_tasks']
%w[ db:setup db:reset ].each do |task_name|
  expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
  dangerous_task.reenable
end

Solution #2 guard_dangerous_tasks is not a prerequisite

We get better solution of our problem if we realize, that guard_dangerous_tasks should not be a prerequisite! Prerequisite are supposed to "prepare" stage and be executed only once. But we should never blind our eyes to dangers!

This is why we should extend with guard_dangerous_tasks as an action, which will be executed each time the parent task is run.

According to the source code of Rake::Task (see above) we should pass our logic in a block if we want it to be added as an action.

%w[ db:setup db:reset ].each do |task|
  Rake::Task[task].enhance do
    Rake::Task['guard_dangerous_tasks'].execute
  end
end

We can leave our test unchanged now and it passes:

%w[ db:setup db:reset ].each do |task_name|
  expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
end

But leaving invoke is a ticket for new problems. It's better to be replaced with execute:

%w[ db:setup db:reset ].each do |task_name|
  expect { Rake::Task[task_name].execute }.to raise_error(InvalidTaskError)
end

Be careful with invoke!

We said above, that using invoke is a ticket for new problems. What kind of problems?

Let's try to test our code for both test and production environments. If we wrap our tests inside this loop:

['production','test'].each do |env_name|
  env = ActiveSupport::StringInquirer.new(env_name)
  allow(Rails).to receive(:env).and_return(env)

  %w[ db:setup db:reset ].each do |task_name|
    expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
  end
end

our test will fail with original reason. You can easily fix this by replacing the line

expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)

with

expect { Rake::Task[task_name].execute }.to raise_error(InvalidTaskError)

So what was the reason? You probably already guess it.

In failing test we invoked the same two tasks twice. First time they were executed. The second time they should be reenabled before invokation to execute. When we use execute, action is reenable automatically.

Note You can find working example of this project here: https://github.com/dimakura/stackoverflow-projects/tree/master/31821220-testing-rake