What exactly is the 'Saff Squeeze' method of finding a bug?

Question

I have read Kent Beck's original blog post on the Saff Squeeze method. I have also read this InfoQ post that elaborates a bit more on the topic but does not provide any examples.

I know that it is essentially a way of homing in on a bug without relying on the debugger. However I find Kent's example to be not that clear.

Can someone more enlightened educate me on how to use this approach with a clear, concrete example? It'll hopefully serve as a learning resource for anyone else researching the method too.

Answer 1

The Saff Squeeze is a systematic technique for deleting both test code and non-test code from a failing test until the test and code are small enough to understand.

I agree that Kent's original description of the Saff Squeeze is a little difficult, partly because the software he's testing, JUnit, is highly abstracted, and partly because he doesn't give enough examples of step 2, "Place a (failing) assertion earlier in the test than the existing assertions."

In his first round he just moves the assertion higher in the test, and his summary of later steps might lead you to think that the only thing you can do in step 2 is move existing assertions, but by his final step he's come up with a new, simpler failing assertion. The assertion in step 2 can just be an existing one moved higher in the test, which is common, but it can also be a new one that you come up with as your understanding of the code and the bug evolves.

Here's an example. It's too simple to need the Saff Squeeze, but it illustrates the technique.

I just wrote this mission-critical class:

class Autopilot

  def self.fly_to(city)
    runways_in_city = runways_in city
    runway = closest_available runways_in_city
    flight_plan = flight_plan_to runway
    carry_out flight_plan
  end

  def self.runways_in(city)
    Airport.where(city: city).map(&:runways).flatten
  end

  def self.closest_available(runways)
    runways.select { |r| r.available? }
      .sort_by { |r| distance_between current_position, r.position }.last
  end

  def self.flight_plan_to(runway)
    FlightPlan.new runway.latitude, runway.longitude
  end

  # other methods left to the imagination

end

Here's the first rspec example I wrote to test it:

describe Autopilot
  describe ".fly_to" do
    it "flies to the only available runway" do
      Autopilot.stub(:current_position) { Position.new 0, 0 }
      nearby_runway = create :runway, latitude: 1, longitude: 1
      create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
      flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
      # Think of the following line as being at the end of the example, since that's when it takes effect
      Autopilot.should_receive(:carry_out).with flight_plan
      Autopilot.fly_to nearby_runway.airport.city
    end
  end
end

Oh no -- the last line fails with this message: "Expectation failed: Expected Autopilot.carry_out to be called with FlightPlan(latitude: 1, longitude: 1), but it was called with FlightPlan(latitude: 2, longitude: 2)". I have no idea how that happened. We'd better use the Saff Squeeze.

Inline the method (renaming a local to avoid name collision):

it "flies to the only available runway" do
  Autopilot.stub(:current_position) { Position.new 0, 0 }
  nearby_runway = create :runway, latitude: 1, longitude: 1
  create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
  flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
  Autopilot.should_receive(:carry_out).with flight_plan
  runways_in_city = runways_in city
  runway = closest_available runways_in_city
  actual_flight_plan = flight_plan_to runway
  Autopilot.carry_out actual_flight_plan
end

I don't see how that last line could fail to meet the expectation, as long as it's getting the right FlightPlan. Let's see if we can write a failing assertion higher up in the test:

it "flies to the only available runway" do
  Autopilot.stub(:current_position) { Position.new 0, 0 }
  nearby_runway = create :runway, latitude: 1, longitude: 1
  create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
  flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
  Autopilot.should_receive(:carry_out).with flight_plan
  runways_in_city = runways_in city
  runway = closest_available runways_in_city
  actual_flight_plan = flight_plan_to runway
  actual_flight_plan.should == flight_plan
  Autopilot.carry_out actual_flight_plan
end

Ah, the new assertion fails too, with "expected FlightPlan(latitude: 1, longitude: 1), but got FlightPlan(latitude: 2, longitude: 2)". OK, let's simplify the test:

it "flies to the only available runway" do
  Autopilot.stub(:current_position) { Position.new 0, 0 }
  nearby_runway = create :runway, latitude: 1, longitude: 1
  create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
  flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
  runways_in_city = runways_in city
  runway = closest_available runways_in_city
  actual_flight_plan = flight_plan_to runway
  actual_flight_plan.should == flight_plan
end

We're getting somewhere, but I still don't see what's wrong. Better Saff Squeeze again, inlining flight_plan_to:

it "flies to the only available runway" do
  Autopilot.stub(:current_position) { Position.new 0, 0 }
  nearby_runway = create :runway, latitude: 1, longitude: 1
  create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
  flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
  runways_in_city = runways_in city
  runway = closest_available runways_in_city
  actual_flight_plan = FlightPlan.new runway.latitude, runway.longitude
  actual_flight_plan.should == flight_plan
end

Well, obviously that's going to pass as long as flight_plan_to gets the right Runway. Let's assert that:

it "flies to the only available runway" do
  Autopilot.stub(:current_position) { Position.new 0, 0 }
  nearby_runway = create :runway, latitude: 1, longitude: 1
  create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
  flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
  runways_in_city = runways_in city
  runway = closest_available runways_in_city
  runway.should == nearby_runway
  actual_flight_plan = FlightPlan.new runway.latitude, runway.longitude
  actual_flight_plan.should == flight_plan
end

Good, the new assertion fails, with "expected Runway(id: 1) but got Runway(id: 2)". Simplify the test again:

it "flies to the only available runway" do
  Autopilot.stub(:current_position) { Position.new 0, 0 }
  nearby_runway = create :runway, latitude: 1, longitude: 1
  create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
  runways_in_city = runways_in city
  runway = closest_available runways_in_city
  runway.should == nearby_runway
end

We've pruned our original test and code to the point where it's obvious that the bug is in closest_available -- it should use first instead of last.

But what if it's still not obvious, you say? Well, let's try to Saff Squeeze again, inlining closest_available:

it "flies to the only available runway" do
  Autopilot.stub(:current_position) { Position.new 0, 0 }
  nearby_runway = create :runway, latitude: 1, longitude: 1
  create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
  runways_in_city = runways_in city
  runway = runways_in_city.select { |r| r.available? }
    .sort_by { |r| Autopilot.distance_between Autopilot.current_position, r.position }.last
  runway.should == nearby_runway
end

Now, where am I going to place a failing assertion higher in the test? I can't -- the bug is in the very last line of the test. Eventually I'll be forced to realize that it was in closest_available before I inlined it.