Imagine the following contrived example:
public class LoginController {
private readonly IValidate _validator;
private readonly IAuthenticate _authenticator;
public LoginController(IValidate validator, IAuthenticate authenticator) {
_validator = validator;
_authenticator = authenticator;
}
public HttpStatusCode Login(LoginRequest request) {
if (!_validator.IsValid(request)) {
return HttpStatusCode.BadRequest;
}
if (!_authenticator.IsAuthenticated(request.Email, request.Password)) {
return HttpStatusCode.Unauthorized;
}
return HttpStatusCode.OK;
}
}
public class LoginRequest {
public string Email {get; set;}
public string Password {get; set;}
}
public interface IValidate {
bool IsValid(LoginRequest request);
}
public interface IAuthenticate {
bool IsAuthenticated(string email, string password);
}
Typically I would write tests like the following:
[TestFixture]
public class InvalidRequest
{
private LoginRequest _invalidRequest;
private IValidate _validator;
private HttpStatusCode _response;
void GivenARequest()
{
_invalidRequest = new LoginRequest();
}
void AndGivenThatRequestIsInvalid() {
_validator = Substitute.For<IValidate>();
_validator.IsValid(_invalidRequest).Returns(false);
}
void WhenAttemptingLogin()
{
_response = new LoginController(_validator, null)
.Login(_invalidRequest);
}
void ThenShouldRespondWithBadRequest()
{
Assert.AreEqual(HttpStatusCode.BadRequest, _response);
}
[Test]
public void Execute()
{
this.BDDfy();
}
}
public class LoginUnsuccessful
{
private LoginRequest _request;
private IValidate _validator;
private IAuthenticate _authenticate;
private HttpStatusCode _response;
void GivenARequest()
{
_request = new LoginRequest();
}
void AndGivenThatRequestIsValid() {
_validator = Substitute.For<IValidate>();
_validator.IsValid(_request).Returns(true);
}
void ButGivenTheLoginCredentialsDoNotExist() {
_authenticate = Substitute.For<IAuthenticate>();
_authenticate.IsAuthenticated(
_request.Email,
_request.Password
).Returns(false);
}
void WhenAttemptingLogin()
{
_response = new LoginController(_validator, _authenticate)
.Login(_request);
}
void ThenShouldRespondWithUnauthorized()
{
Assert.AreEqual(HttpStatusCode.Unauthorized, _response);
}
[Test]
public void Execute()
{
this.BDDfy();
}
}
However after watching the following video Ian Cooper: TDD, where did it all go wrong and doing some more reading, I'm starting to think that my tests are too closely tied to the implementation of the code. For instance, the behavior I'm trying to test in the first instance is that if we try to login with an invalid request we response with the http status code of bad request. The issue is that I'm testing this by stubbing the IValidate dependency. If the implementer decides the IValidate abstraction is no longer useful and decides to validate the request inline in the Login method, then the behaviour of the system hasn't changed however my tests now break.
But then, the only other alternate is an integration test where I launch the web server and hit the login endpoint and assert on the response. The issue is that this is brittle and complicated, as we would ultimately need to have a valid user in the third party credential store for testing the user login successful scenario.
So my question is, is my understanding incorrect, or is there a middle ground between testing against the implementation and full-blown integration testing?
Like most other aspects of our trade, there are trade-offs involved.
Lots of people have declared unit testing and Test-Driven Development (TDD) dead, and see Behaviour-Driven Development (BDD) as the new silver bullet. Obviously, neither of them are silver bullets.
In your question, you've already outlined one type of problem with unit tests, so although I'd like to get back to those, let's start by looking at BDD.
The problem with Integration Tests
In his seminal talk Integration Tests Are a Scam, J.B. Rainsberger explains why Integration Tests (including most BDD-style tests) are problematic. You really should see the recording, but the essence of it is that Integration Testing involves a combinatorial explosion of test cases.
Consider your own trivial example. The Login method of the LoginController has a Cyclomatic Complexity of 3, since there are 3 ways through it. If you want to test only the behaviour, you'll need to integrate it with the appropriate implementation of its dependencies.
Just by looking at the method signatures, we can see that since both _validator.IsValid and _authenticator.IsAuthenticated return bool, there must be at least 2 ways through each of them.
Thus, with these optimistic numbers, the upper bound on the number of permutations of integrating these three objects is 3 * 2 * 2 = 12. The actual number is less than that, because you are returning early in some branches, but the order of magnitude is about right. The problem is that if e.g. the validator has a higher degree of complexity, and particularly if it has dependencies of its own, the number of possible combinations explode, and quickly reach five- or six-digit numbers.
There's no way you can write all those test cases.
The problem with unit tests
When you write unit tests, you can keep the number of combinations down. Instead of having to multiply all possible combinations of code paths, you can add them together in order to get an idea about the number of test cases you have to write. This enables you to keep the number of tests down, and you can get better coverage. In fact, you can get perfect coverage with unit tests.
The problem, then, is exactly as you describe. In a sense, you test what feels like the implementation. It is, but it's only part of the implementation, and that's the whole point. Still, it means that when things change, unit tests are affected, which Integration Tests should be to a far lesser degree.
Adopting an Append-Only strategy for tests help a bit, but it still can feel like overhead.
The test pyramid
All of this explains why Mike Cohn recommends the Test Pyramid: