What are people's strategies for maximizing test coverage while minimizing test duplication and overlap, particularly between unit tests and functional or integration tests? The issue is not specific to any particular language or framework, but just as an example, say you have a Rails app that allows users to post comments. You might have a User model that looks something like this:
class User < ActiveRecord::Base
def post_comment(attributes)
comment = self.comments.create(attributes)
notify_friends('created', comment)
share_on_facebook('created', comment)
share_on_twitter('created', comment)
award_badge('first_comment') unless self.comments.size > 1
end
def notify_friends(action, object)
friends.each do |f|
f.notifications.create(subject: self, action: action, object: object)
end
end
def share_on_facebook(action, object)
FacebookClient.new.share(subject: self, action: action, object: object)
end
def share_on_twitter(action, object)
TwitterClient.new.share(subject: self, action: action, object: object)
end
def award_badge(badge_name)
self.badges.create(name: badge_name)
end
end
As an aside, I would actually use service objects rather put this type of application logic in models, but I wrote the example this way just to keep it simple.
Anyway, unit testing the post_comment method is pretty straightforward. You would write tests to assert that:
But then how do you write your functional and/or integration tests to ensure the controller actually invokes this logic and produces the desired results in all of the different scenarios?
One approach is just to reproduce all of the unit test cases in the functional and integration tests. This achieves good test coverage but makes the tests extremely burdensome to write and maintain, especially when you have more complex logic. This does not seem like a viable approach for even a moderately complex application.
Another approach is to just test that the controller invokes the post_comment method on the user with the expected params. Then you can rely on the unit test of post_comment to cover all of the relevant test cases and verify the results. This seems like an easier way to achieve the desired coverage, but now your tests are coupled with the specific implementation of the underlying code. Say you find that your models have gotten bloated and difficult to maintain and you refactor all of this logic into a service object like this:
class PostCommentService
attr_accessor :user, :comment_attributes
attr_reader :comment
def initialize(user, comment_attributes)
@user = user
@comment_attributes = comment_attributes
end
def post
@comment = self.user.comments.create(self.comment_attributes)
notify_friends('created', comment)
share_on_facebook('created', comment)
share_on_twitter('created', comment)
award_badge('first_comment') unless self.comments.size > 1
end
private
def notify_friends(action, object)
self.user.friends.each do |f|
f.notifications.create(subject: self.user, action: action, object: object)
end
end
def share_on_facebook(action, object)
FacebookClient.new.share(subject: self.user, action: action, object: object)
end
def share_on_twitter(action, object)
TwitterClient.new.share(subject: self.user, action: action, object: object)
end
def award_badge(badge_name)
self.user.badges.create(name: badge_name)
end
end
Maybe the actions like notifying friends, sharing on twitter, etc. would logically be refactored into their own service objects too. Regardless of how or why you refactor, your functional or integration test would now need to be rewritten if it was previously expecting the controller to call post_comment on the User object. Also, these types of assertions can get pretty unwieldy. In the case of this particular refactoring, you would now have to assert that the PostCommentService constructor is invoked with the appropriate User object and comment attributes, and then assert that the post method is invoked on the returned object. This gets messy.
Also, your test output is a lot less useful as documentation if the functional and integration tests describe implementation rather than behavior. For example, the following test (using Rspec) is not that helpful:
it "creates a PostCommentService object and executes the post method on it" do
...
end
I would much rather have tests like this:
it "creates a comment with the given attributes" do
...
end
it "creates notifications for the user's friends" do
...
end
How do people solve this problem? Is there another approach that I'm not considering? Am I going overboard in trying to achieve complete code coverage?
I'm talking from a .Net/C# perspective here, but I think its generally applicable.
To me a unit test just tests the object under test, and not any of the dependencies. The class is tested to make sure it communicates correctly with any dependencies using Mock objects to verify the appropriate calls are made, and it handles returned objects in the correct manner (in other words the class under test is isolated). In the example above, this would mean mocking the facebook/twitter interfaces, and checking communication with the interface, not the actual api calls themselves.
It looks like in your original unit test example above you are talking about testing all of the logic (i.e. Post to facebook, twitter etc...) in the same tests. I would say if these tests are written in this way, its actually a functional test already. Now if you absolutely cannot modify the class under test at all, writing a unit test at this point would be unnecessary duplication. But if you can modify the class under test, refactoring out so dependencies are behind interfaces, you could have a set of unit tests for each individual object, and a smaller set of functional tests that test the whole system together appears to behave correctly.
I know you said regardless of how they are refactored above, but to me, refactoring and TDD go hand in hand together. To try to do TDD or unit testing without refactoring is an unnecessarily painful experience, and leads to a more difficult design to change, and maintain.