I am trying to write my very first xunit test with Database, instead of mocking the DbContext I used the inMemoryDatabase as I read in articles, so I did like following
public class GetCustomersTest { DbContextOptions _context;
public GetCustomersTest()
{
if (_context==null)
_context = CreateContextForCustomer();
}
[Theory]
[InlineData(1)]
[InlineData(2)]
public void GetCustomerById_ShouldReturnCorrectObject(int id)
{
using (var context = new DataBaseContext(_context))
{
var customerByIdService = new GetCustomerByIdService(context);
var customer = customerByIdService.Execute(id);
var customerActual = context.Customers.Where(x => x.Id == id).SingleOrDefault();
var customerTmp = new Customer()
{
Id = id,
FirstName = customer.Data.FirstName,
LastName = customer.Data.LastName,
Phone = customer.Data.Phone,
ClientNote = customer.Data.ClientNote
};
Assert.Equal(customerTmp.FirstName, customerActual.FirstName);
Assert.Equal(customerTmp.LastName, customerActual.LastName);
Assert.Equal(customerTmp.Phone, customerActual.Phone);
Assert.Equal(customerTmp.ClientNote, customerActual.ClientNote);
}
}
private DbContextOptions<DataBaseContext> CreateContextForCustomer() {
var options = new DbContextOptionsBuilder<DataBaseContext>()
.UseInMemoryDatabase(databaseName: "SalonDatabase")
.Options;
using (var context = new DataBaseContext(options))
{
context.Customers.Add(new Customer
{
Id = 1,
FirstName = "User1",
LastName = "Surname1",
Phone = "123",
ClientNote = ""
});
context.Customers.Add(new Customer
{
Id = 2,
FirstName = "User2",
LastName = "Surname2",
Phone = "4567",
ClientNote = "The best"
});
context.SaveChanges();
}
return options;
}
}
it works find on [InlineData(1)] but when it comes to [InlineData(2)], it seems that it starts to run the constructor again , so as it wants to add the customerdata to table, it says that the record with that Id key exists.what is the best way for doing that?
When building your DB context options, add a GUID to the database name to make it unique:
var options = new DbContextOptionsBuilder<DataBaseContext>()
.UseInMemoryDatabase(databaseName: "SalonDatabase" + Guid.NewGuid().ToString())
.Options;
Or if you're using a new enough language version you can use string interpolation instead of concatenation:
var options = new DbContextOptionsBuilder<DataBaseContext>()
.UseInMemoryDatabase(databaseName: $"SalonDatabase{Guid.NewGuid()}")
.Options;
If you do this then every test uses a brand new database, unaffected by any previous tests.
As @MicheleMassari says, it's good practice to follow the Arrange Act Assert pattern, so that it's clear which lines are setting things up ready for the test and which are performing the action that you want to test the outcome of.
- Arrange inputs and targets. Arrange steps should set up the test case. Does the test require any objects or special settings? Does it need to prep a database? Does it need to log into a web app? Handle all of these operations at the start of the test.
- Act on the target behavior. Act steps should cover the main thing to be tested. This could be calling a function or method, calling a REST API, or interacting with a web page. Keep actions focused on the target behavior.
- Assert expected outcomes. Act steps should elicit some sort of response. Assert steps verify the goodness or badness of that response. Sometimes, assertions are as simple as checking numeric or string values. Other times, they may require checking multiple facets of a system. Assertions will ultimately determine if the test passes or fails.
The code samples in that page are written in Python rather than C#, but the pattern is valid for unit tests in any language. In the case of your test, structuring the test in this way makes it clear whether you're testing the behaviour of GetCustomerByIdService.Execute or of Entity Framework's Where method.
Simone's answer got me thinking... Creating and populating a new database for each and every test does result in more work being done than really needs to be done, and so will slow down the test execution time, and the more tests there are and the more work is involved in populating the database, the slower it will be. But I was wondering, how much slower would it be? So I thought I'd try to measure it for a few different scenarios.
I wrote a couple of unit test classes which are functionally identical, except that one sets up the database in the constructor of the test class, therefore performing this test setup once per test, and the other sets up the database in a class fixture, therefore once per test class. I then wrote a PowerShell script to run each of these test classes 5 times and to capture 2 metrics (both in milliseconds):
All the code is below, but for now, let's skip to the results.
First, populating the database with 10 records and running 4 tests.
ClassName ElapsedTime ReportedDuration
--------- ----------- ----------------
TestUsingClassFixture 9441.295 215.777
TestUsingClassFixture 5003.6518 198.6189
TestUsingClassFixture 5295.3667 217.8755
TestUsingClassFixture 4960.2444 209.483
TestUsingClassFixture 5270.8495 204.8843
TestUsingConstructor 5258.3407 755.6575
TestUsingConstructor 5432.9548 758.5794
TestUsingConstructor 5095.9814 733.1778
TestUsingConstructor 5069.6216 734.8657
TestUsingConstructor 5063.8004 693.8751
My first observation is that the first result appears to have a "warm up" effect, making it much slower than the rest - this happens consistently in the other results, so I think we can consider it an outlier.
My second observation is that although xUnit reports a significant difference in execution times (~200ms vs ~700ms), there is very little difference in the elapsed time to run the tests (all around 5000ms).
For the next scenario, populate the database with 100 records and run 40 tests.
ClassName ElapsedTime ReportedDuration
--------- ----------- ----------------
TestUsingClassFixture 9810.6109 217.3786
TestUsingClassFixture 4941.647 225.9348
TestUsingClassFixture 4950.5457 230.5555
TestUsingClassFixture 5172.1967 233.9416
TestUsingClassFixture 5326.0593 224.4598
TestUsingConstructor 5548.1943 839.2111
TestUsingConstructor 5128.0491 910.9815
TestUsingConstructor 5299.1093 849.8785
TestUsingConstructor 5277.194 837.469
TestUsingConstructor 5251.0753 822.47
Again, there is a significant difference between the xUnit reported execution times, but the difference between the elapsed times is still so small that I doubt it would be noticeable.
Third scenario - populate the database with 1000 records and run 400 tests.
ClassName ElapsedTime ReportedDuration
--------- ----------- ----------------
TestUsingClassFixture 10361.8977 584.4645
TestUsingClassFixture 5587.361 512.0562
TestUsingClassFixture 6263.8155 733.1085
TestUsingClassFixture 5703.3829 639.1928
TestUsingClassFixture 5429.4198 513.8941
TestUsingConstructor 9724.1059 5031.9186
TestUsingConstructor 9590.1758 5043.6101
TestUsingConstructor 9665.5815 5123.7166
TestUsingConstructor 9634.3568 5083.3954
TestUsingConstructor 9743.8075 5188.206
Now there is a noticeable difference in elapsed times (~6 seconds when performing the setup in a class fixture vs ~10 seconds when performing the setup in a constructor).
However, the thought of 400 unit tests which are all dependent on a database rings alarm bells for me. It suggests that maybe database access hasn't been abstracted away from the rest of the application as well as it could be. In any case, at this point I'd be taking a good look at the application for design issues like this.
Of course, all these timings are dependent on a variety of factors, such as how powerful the machine running the tests is, whether they're run from a dotnet command line, some other command line tool, Visual Studio test explorer, some other IDE or some completely different means. Anyone who's interested enough can use the code below to measure it for their environment.
Customer.cs
namespace Domain
{
public class Customer
{
public int Id { get; set; }
public string? Name { get; set; }
}
}
CustomerRepository.cs
namespace Domain
{
public class CustomerRepository
{
private readonly DatabaseContext context;
public CustomerRepository(DatabaseContext context)
{
this.context = context;
}
public Customer? GetById(int id)
{
return this.context.Customers.SingleOrDefault(c => c.Id == id);
}
}
}
DatabaseContext.cs
namespace Domain
{
using Microsoft.EntityFrameworkCore;
public class DatabaseContext : DbContext
{
public DatabaseContext(DbContextOptions options)
: base(options)
{
}
public DbSet<Customer> Customers { get; set; }
}
}
TestData.cs (this is where you can change the number of records in the database and the number of tests to run)
namespace Domain.Test
{
using System.Collections;
/// <summary>
/// This class can be used in a ClassData attribute on a unit test method
/// to supply it with test data.
/// <see href="https://andrewlock.net/creating-parameterised-tests-in-xunit-with-inlinedata-classdata-and-memberdata/#using-a-dedicated-data-class-with-classdata-"/>.
/// </summary>
public class TestData : IEnumerable<object[]>
{
// The following two properties are not part of the ClassData pattern,
// Instead you can edit them to see the impact on test execution time
// of seeding the in-memory database with different numbers of records,
// and of running different numbers of unit tests.
/// <summary>
/// Gets the number of records to seed the in-memory database with.
/// </summary>
public static int NumberOfRecordsInDatabase => 1000;
/// <summary>
/// Gets the number of records of test data to pass to the test method.
/// This will determine the number of tests to be run for each test class.
/// </summary>
public static int NumberOfTestsToRun => 400;
/// <inheritdoc/>
public IEnumerator<object[]> GetEnumerator()
{
var random = new Random();
var numberOfRecordsInDatabase = NumberOfRecordsInDatabase;
var numberOfTestsToRun = NumberOfTestsToRun;
if (numberOfTestsToRun > numberOfRecordsInDatabase)
{
var msg = $"Number of tests to run ({numberOfTestsToRun}) must be <= number of records in database ({numberOfRecordsInDatabase})";
throw new InvalidOperationException(msg);
}
var customerIds = new List<int>();
for (var i = 0; i < numberOfTestsToRun; i++)
{
var customerId = random.Next(1, numberOfRecordsInDatabase + 1);
// The number of unique customer IDs determines the number of tests to run, so
// ensure that each customer ID is unique.
// This is important because we want to run the same number of tests using each
// test class in order to compare their execution times.
if (customerIds.Contains(customerId))
{
i--;
}
else
{
customerIds.Add(customerId);
}
}
foreach (var customerId in customerIds)
{
yield return new object[] { customerId };
}
}
/// <inheritdoc/>
IEnumerator IEnumerable.GetEnumerator() => this.GetEnumerator();
}
}
TestHelper.cs (the actual implementation of the unit test, called by both test classes)
namespace Domain.Test
{
using Microsoft.EntityFrameworkCore;
using Xunit;
/// <summary>
/// This class contains logic which is common to both test classes, to ensure
/// that when comparing their execution times we are comparing like with like.
/// </summary>
public static class TestHelper
{
/// <summary>
/// Creates an in-memory database using the DatabaseContext schema, and
/// populates its Customers table with some records.
/// </summary>
/// <returns>DB context options for the created database.</returns>
public static DbContextOptions<DatabaseContext> CreateContextOptions()
{
var numberOfCustomers = TestData.NumberOfRecordsInDatabase;
var databaseName = "MyDatabase" + Guid.NewGuid().ToString();
var options = new DbContextOptionsBuilder<DatabaseContext>()
.UseInMemoryDatabase(databaseName: databaseName)
.EnableSensitiveDataLogging(true)
.Options;
using (var context = new DatabaseContext(options))
{
for (var i = 1; i <= numberOfCustomers; i++)
{
var customer = new Customer { Id = i, Name = Guid.NewGuid().ToString() };
context.Customers.Add(customer);
}
context.SaveChanges();
}
return options;
}
/// <summary>
/// The unit test methods delegate the actual test steps to this method.
/// </summary>
/// <param name="customerId">The customer ID to test with.</param>
/// <param name="contextOptions">DB context options for the in-memory database.</param>
public static void RunTheTest(int customerId, DbContextOptions<DatabaseContext> contextOptions)
{
using (var context = new DatabaseContext(contextOptions))
{
// Arrange
var repository = new CustomerRepository(context);
var expectedCustomer = context.Customers.Where(c => c.Id == customerId).SingleOrDefault();
// Act
var actualCustomer = repository.GetById(customerId);
// Assert
Assert.NotNull(actualCustomer);
Assert.Equal(expectedCustomer!.Id, actualCustomer.Id);
Assert.Equal(expectedCustomer.Name, actualCustomer.Name);
}
}
}
}
TestUsingClassFixture.cs
namespace Domain.Test
{
using Domain;
using Microsoft.EntityFrameworkCore;
using Xunit;
/// <summary>
/// This test class will seed the database only once, saving execution time
/// over seeding the database once per test.
/// </summary>
public class TestUsingClassFixture : IClassFixture<DatabaseFixture>
{
private readonly DatabaseFixture fixture;
public TestUsingClassFixture(DatabaseFixture fixture)
{
this.fixture = fixture;
}
[Theory]
[ClassData(typeof(TestData))]
public void GetById_ReturnsCorrectObject(int id)
{
TestHelper.RunTheTest(id, this.fixture.ContextOptions);
}
}
/// <summary>
/// Contains test setup logic to be performed once per test class.
/// </summary>
public class DatabaseFixture
{
public DatabaseFixture()
{
this.ContextOptions = TestHelper.CreateContextOptions();
}
public DbContextOptions<DatabaseContext> ContextOptions { get; }
}
}
TestUsingConstructor.cs
namespace Domain.Test
{
using Microsoft.EntityFrameworkCore;
using Xunit;
public class TestUsingConstructor
{
private readonly DbContextOptions<DatabaseContext> options;
public TestUsingConstructor()
{
this.options = TestHelper.CreateContextOptions();
}
[Theory]
[ClassData(typeof(TestData))]
public void GetById_ReturnsCorrectObject(int id)
{
TestHelper.RunTheTest(id, this.options);
}
}
}
Run-Tests.ps1 (If you want to run more or fewer than 5 iterations of each test run, change it at the bottom of this file)
function Log([string]$Message) {
Write-Host "Run-Tests: ${Message}"
}
function Run-TestClass([string[]]$ClassNames, [int]$Iterations) {
Log "Building test project before we start timing anything"
$Discard = dotnet build -target:Rebuild
Log "Finished building test project"
$AnalysisTable = New-Object 'System.Collections.Generic.List[object]'
$ClassNames | Foreach-Object {
$ClassName = $_
for ($i = 0; $i -lt $Iterations; $i++) {
Log "Running tests from ${ClassName} - iteration ${i}"
# Initialise an object to hold the results for this iteration of the test
$Analysis = [PSCustomObject]@{
ClassName = $ClassName
ElapsedTime = 0
ReportedDuration = 0
}
# Run the test and save results to a .trx file
$ResultsFileName = "${ClassName}.trx"
$StartTime = [System.DateTime]::Now
$Discard = dotnet test --no-build --filter $ClassName --logger "trx;LogFileName=${ResultsFileName}"
$EndTime = [System.DateTime]::Now
# Get the actual elapsed time to run this test
$ElapsedTime = ($EndTime - $StartTime).TotalMilliseconds
$Analysis.ElapsedTime = $ElapsedTime
# Get the total test duration as reported in the .trx file
$QualifiedResultsFileName = [System.IO.Path]::Combine("TestResults", $ResultsFileName)
$TrxContent = [xml](Get-Content $QualifiedResultsFileName)
$Results = $TrxContent.TestRun.Results.UnitTestResult
$TotalDuration = [System.TimeSpan]::FromMilliseconds(0)
$Results | ForEach-Object {
$TestDuration = [System.TimeSpan]::Parse($_.duration)
$TotalDuration += $TestDuration
}
$Analysis.ReportedDuration = $TotalDuration.TotalMilliseconds
# Add the results for this run to a table of all results
$AnalysisTable.Add($Analysis)
}
}
return $AnalysisTable
}
$ClassNames = "TestUsingClassFixture","TestUsingConstructor"
$AnalysisTable = Run-TestClass -ClassNames $ClassNames -Iterations 5
$AnalysisTable