We Keep Coding

How to test API that calls a service and that service calls a repository?

I have four projects that need to be tested (my design patter is similar to this https://exceptionnotfound.net/the-repository-service-pattern-with-dependency-injection-and-asp-net-core/). The first project is a data access layer project that fetches data from the database. The second one is a repository project that just makes CRUD operations on the entities in the database through the data access layer project.
Here's an example of how the generic interface in the repository project looks like which is implemented for all the entities in the database,
public interface IEntityBaseRepository<T> where T : class, IEntityBase, new()
{
IEnumerable<T> AllIncluding(
params Expression<Func<T, object>>[] includeProperties
);
IEnumerable<T> GetAll();
int Count();
T GetSingle(int id);
T GetSingle(Expression<Func<T, bool>> predicate);
T GetSingle(
Expression<Func<T, bool>> predicate,
params Expression<Func<T, object>>[] includeProperties
);
IEnumerable<T> FindBy(Expression<Func<T, bool>> predicate);
T Add(T entity);
void Update(T entity);
void Delete(T entity);
void DeleteWhere(Expression<Func<T, bool>> predicate);
void Commit();
}
The third project is a service project that cater for the logic required by the application. It deals with the repository project and creates objects and functions based on the need of the system. Here's an extract of one of the service in the project,
public class EmployeeService : IEmployeeService
{
private IEntityBaseRepository<Employee> _employeeRepository;
private IEntityBaseRepository<User> _userRepository;
private IEntityBaseRepository<UserRole> _userRoleRepository;
private IEntityBaseRepository<Role> _roleRepository;
public EmployeeService(IEntityBaseRepository<Employee> employeeRepository, IEntityBaseRepository<User> userRepository, IEntityBaseRepository<UserRole> userRoleRepository, IEntityBaseRepository<Role> roleRepository)
{
_employeeRepository = employeeRepository;
_userRepository = userRepository;
_userRoleRepository = userRoleRepository;
_roleRepository = roleRepository;
}
.
.
.
public List<EmployeeServiceModel> GetAllEmployeesForUser(string username)
{
var user = _userRepository.GetAll().FirstOrDefault(u => u.UserName == username);
var userRoles = _userRoleRepository.GetAll().Where(ur => ur.UserId == user.Id);
var roles = _roleRepository.GetAll();
var currentRoles = new List<Role>();
if (userRoles != null && userRoles.Count() > 0)
{
currentRoles = roles.Where(r => userRoles.Any(ur => ur.RoleId == r.Id)).ToList();
}
else
{
throw new UnauthorizedAccessException();
}
.
.
.
As you can see from the above, it depends on the repository project.
Finally, API project that exposes the services to the world
public class EmployeesController : ControllerBase
{
private IEmployeeService _employeeService;
public EmployeesController(IEmployeeService employeeService)
{
_employeeService = employeeService;
}
// GET: api/<EmployeesController>
[HttpGet]
public async Task<IActionResult> Get(string username)
{
var employees = _employeeService.GetAllEmployeesForUser(username);
return Ok(employees);
}
The problem is, how can unit test the above solution? Should I just test the API's? If yes, how can do that? The project depends on the service interface which in turn depends on the repositories. I cannot just create a fake list of employees and make the "GetAllEmployeesForUser" function in the service return them. How about if I want to test when a user is unauthorised, or doesn't have employees?
Should I just test the service project? And no need to test the API's?

If you want to test the whole web API (integration tests) you can find all info here:
https://learn.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-7.0
If you want to do unit tests, you can mock parts that you don't need in specific test case with Moq or Substitute libraries and perform execution in parts where you want to test. For instance, if you want to test only EmployeeService logic:
var employeesRepMock = new Mock<IEntityBaseRepository<Employee>>();
var usersRepMock = new Mock<IEntityBaseRepository<User>>();
var userRolesRepMock = new Mock<IEntityBaseRepository<UserRole>>();
var rolesRepMock = new Mock<IEntityBaseRepository<Role>>();
rolesRepMock.Setup(m => m.GetAll().Returns(new List<Role> {
// test roles here
}));
usersRepMock.Setup(m => m.GetAll().Returns(new List<User> {
// test users here
}));
employeesRepMock.Setup(m => m.GetAll().Returns(new List<Employee> {
// test employees here
}));
userRolesRepMock.Setup(m => m.GetAll().Returns(new List<UserRoles> {
// test user roles here
}));
var service = new EmployeeService(employeesRepMock.Object,
usersRepMock.Object, userRolesRepMock.Object, rolesRepMock.Object);
var result = service.GetAllEmployeesForUser("test-username");
// your checks ..
Assert.Greater(result.Length, 0);
To test the EF Core logic, you can use InMemory database.

Mock with 'dynamic' generic and anonymous object 'new {}' fails

Visual Studio 2019 Enterprise 16.9.4; Moq 4.16.1; xunit 2.4.1; net5.0
I'm trying to unit test my AlbumData.GetAlbumsAsync() method. I mock the SqlDataAccess layer which is making a call to the DB using Dapper in a generic method.
This is my setup. The mock is not working. In the AlbumData.GetAlbumsAsync() method the call to the mocked object (_sql.LoadDataAsync) returns null and output is set to null.
Can anyone tell me what I'm dong wrong?
SqlDataAccess.cs
public async Task<List<T>> LoadDataAsync<T, U>(string storedProcedure,
U parameters, string connectionStringName)
{
string connectionString = GetConnectionString(connectionStringName);
using (IDbConnection connection = new SqlConnection(connectionString))
{
IEnumerable<T> result = await connection.QueryAsync<T>(storedProcedure, parameters,
commandType: CommandType.StoredProcedure);
List<T> rows = result.ToList();
return rows;
}
}
AlbumData.cs
public class AlbumData : IAlbumData
{
private readonly ISqlDataAccess _sql;
public AlbumData(ISqlDataAccess sql)
{
_sql = sql;
}
public async Task<List<AlbumModel>> GetAlbumsAsync()
{
var output = await _sql.LoadDataAsync<AlbumModel, dynamic>
("dbo.spAlbum_GetAll", new { }, "AlbumConnection");
return output;
}
...
}
AlbumDataTest.cs
public class AlbumDataTest
{
private readonly List<AlbumModel> _albums = new()
{
new AlbumModel { Title = "Album1", AlbumId = 1 },
new AlbumModel { Title = "Album2", AlbumId = 2 },
new AlbumModel { Title = "Album3", AlbumId = 3 }
};
[Fact]
public async Task getAlbums_returns_multiple_records_test()
{
Mock<ISqlDataAccess> sqlDataAccessMock = new();
sqlDataAccessMock.Setup(d => d.LoadDataAsync<AlbumModel, dynamic>
(It.IsAny<string>(), new { }, It.IsAny<string>()))
.Returns(Task.FromResult(_albums));
AlbumData albumData = new AlbumData(sqlDataAccessMock.Object);
List<AlbumModel> actual = await albumData.GetAlbumsAsync();
Assert.True(actual.Count == 3);
}
...
}
UPDATE1:
Following #freeAll and #brent.reynolds suggestions I updated the test to use It.IsAny<string>()
Also updated #brent.reynolds fiddle to actually implement a unit test:
https://dotnetfiddle.net/nquthR
It all works in the fiddle but when I paste the exact same test into my AlbumDataTest it still returns null. Assert.Null(actual); passes, Assert.True(actual.Count == 3); fails.
UPDATE2:
I've posted a project with the failing test to https://github.com/PerProjBackup/Failing-Mock.
If you run the API.Library.ConsoleTests project the Mock works. If you run the tests in the API.Library.Tests project with the Test Explorer the Mock fails.
#brent.reynolds was able to get the Mock to work by changing the dynamic generic to object. Now trying to debug the dynamic issue.
UPDATE3:
If I move the AlbumData class into the same project as the AllbumDataTest class the mock works (using dynamic) returning the list with three objects. But when the AlbumData class is in a separate project (as it would be in the real world) the mock returns null.
I've updated the https://github.com/PerProjBackup/Failing-Mock repository. I deleted the console app and created a Failing and Passing folder with the two scenarios.
Why would the class that the mock is being passed to being in a different project make the mock fail?
UPDATE4:
See brent.reynolds accepted answer and my comment there. Issue was the use of an anonymous object in the the mock setup. I've deleted the Failing-Mock repository and the dotnetfiddle.

It might be related to the async method. According to the documentation for async methods, You can either do
sqlDataAccessMock
.Setup(d => d.LoadDataAsync<AlbumModel, dynamic>(
It.IsAny<string>(),
new {},
It.IsAny<string>())
.Result)
.Returns(_albums);
or
sqlDataAccessMock
.Setup(d => d.LoadDataAsync<AlbumModel, dynamic>(
It.IsAny<string>(),
new {},
It.IsAny<string>()))
.ReturnsAsync(_albums);
Edited:
Try adding It.IsAny<object>() to the setup:
sqlDataAccessMock
.Setup(d => d.LoadDataAsync<AlbumModel, object>(
It.IsAny<string>(),
It.IsAny<object>(),
It.IsAny<string>()))
.Returns(Task.FromResult(_albums));
and changing the type parameter in GetAlbumsAsync() to:
var output = await _sql.LoadDataAsync<AlbumModel, object>(
"dbo.spAlbum_GetAll",
new { },
"AlbumConnection");
OP Note/Summary:
The use of an anonymous object new {} in the mock setup is the central issue. It works when the test class and class being tested are in the same project but not when they are in separate projects since it cannot then be reused. It.IsAny<dynamic>() will not work because the compiler forbids dynamic inside LINQ expression trees. brent.reynolds use of object resolves the issue.

Use It.IsAny<string>() instead of the empty strings
sqlDataAccessMock.Setup(d => d.LoadDataAsync<AlbumModel, dynamic>
(It.IsAny<string>(), new { }, It.IsAny<string>())).Returns(Task.FromResult(_albums));
Note: you can't use It.IsAny<T>() on dynamic objects

C# Unit Testing Async Error

Just to set the question, when I have come to write a unit test I hit the following error:
Error: The source IQueryable doesn't implement
IDbAsyncEnumerable
The problem happens when testing a method which calls ToListAsync(), the unit test is as follows:
[TestMethod]
public async Task Method1CallsCount()
{
//arrange
MockContainer container = new MockContainer();
IQueryable<Entity1DTO> querableentity1DTO = new List<Entity1DTO>().AsQueryable();
container.DefaultQueryFactory.Setup(p => p.Load(It.IsAny<ContextEnums>(), It.IsAny<Expression<Func<Entity1, Entity1DTO>>>(),
It.IsAny<Expression<Func<Entity1, bool>>>(), It.IsAny<int>(), It.IsAny<bool>())).Returns(querableentity1DTO);
var manager = new Manager1(container.DefaultQueryFactory.Object);
//act
var result = await manager.Method1();
//assert
//container.repo1.Verify(x => x.repoMethod(It.IsAny<Expression<Func<Entity1,bool>>>()), Times.Once);
}
And here is the method I am testing:
public async Task<List<Entity1DTO>> Method1()
{
Expression<Func<Entity1, Entity1DTO>> select = (x => new Entity1DTO()
{
...
});
Expression<Func<Entity1, bool>> where = (x => x.Property == "Test");
return await _defaultQueryFactory.Load(ContextEnums.Enum1, select, where).ToListAsync();
}
To help a bit, I've tried mocking up the method that loads the data in the query factory and the error appears because the DTO model doesn't implement IDbAsyncEnumerable, now the method that is getting tested sends off a select statement and a where statement and an entity type which the query factory then uses to generate a query this is then executed with ToListAsync() when it returns from the load Method. The error message shows that the DTO model is the one that doesn't implement the IDbAsync not the DB entity itself.
I understand that there are a few other questions out there that are simular but my difference is that I use a DTO model and the method in question does not use the context itself as it is injected into the load method and not in the place in which ToListAsync() is called.
anyone any ideas?

Error happens because Entity Framework async extension methods does not work with any IQueryable - it should also implement IDbAsyncEnumerable interface. Consider this:
var query = new List<EntityDTO>().AsQueryable();
var result = query.ToListAsync().Result;
This will throw the same exception you observe in your code. EnumerableQuery returned by AsQueryable does not implement required interface, so we need to use some other implementation. We can find one in this article (or just create ourselves since it's not hard):
static class TestExtensions {
public static IQueryable<T> AsAsyncQueryable<T>(this IEnumerable<T> source) {
return new TestDbAsyncEnumerable<T>(source);
}
}
internal class TestDbAsyncEnumerable<T> : EnumerableQuery<T>, IDbAsyncEnumerable<T>
{
public TestDbAsyncEnumerable(IEnumerable<T> enumerable)
: base(enumerable)
{ }
public TestDbAsyncEnumerable(Expression expression)
: base(expression)
{ }
public IDbAsyncEnumerator<T> GetAsyncEnumerator()
{
return new TestDbAsyncEnumerator<T>(this.AsEnumerable().GetEnumerator());
}
IDbAsyncEnumerator IDbAsyncEnumerable.GetAsyncEnumerator()
{
return GetAsyncEnumerator();
}
}
internal class TestDbAsyncEnumerator<T> : IDbAsyncEnumerator<T> {
private readonly IEnumerator<T> _inner;
public TestDbAsyncEnumerator(IEnumerator<T> inner) {
_inner = inner;
}
public void Dispose() {
_inner.Dispose();
}
public Task<bool> MoveNextAsync(CancellationToken cancellationToken) {
return Task.FromResult(_inner.MoveNext());
}
public T Current => _inner.Current;
object IDbAsyncEnumerator.Current => Current;
}
Now we can do:
IQueryable<Entity1DTO> querableentity1DTO = new List<Entity1DTO>().AsAsyncQueryable();
And EF async methods will execute correctly on it.

There are actually two problems here, firstly you do need to use the solution in the other answer and change the return type to TestDbAsyncEnumerator and the other problem is to do with Moq, the setup needs to have the same parameters called, so in your case you have
return await _defaultQueryFactory.Load(ContextEnums.Enum1, select, where).ToListAsync();
and
container.DefaultQueryFactory.Setup(p => p.Load(It.IsAny<ContextEnums>(), It.IsAny<Expression<Func<Entity1, Entity1DTO>>>(),
It.IsAny<Expression<Func<Entity1, bool>>>(), It.IsAny<int>(), It.IsAny<bool>())).Returns(querableentity1DTO);
notice how you have two extra parameters on the end, try leaving them as null. That should sort out the problem, just to make sure try adding an item to the return list as well.

EF repository created from mock db context returns the dbset only once

I am writing a unit test for my controller.
The test uses an actual Repository class that uses a mock dbcontext inside. I am creating the dbset as in this helper article, and it works all fine - except that it only works one time per unit test.
http://codethug.com/2015/03/20/mocking-dbset/
This means that:
IQueryable<User> membersQuery = this.unitOfWork.UserRepository.Get();
List<User> members = membersQuery.ToList(); //here I get 6 members
membersQuery = this.unitOfWork.UserRepository.Get();
members = membersQuery.ToList(); //when its called again, I get 0 members
Why does it behave like that, I would expect that it shoud return the same collection of members both times.
Here is how I create the repository
var enumerable = new List<User>();
// ... populate the mocks list
var mockedContext = new Mock<MyDbContext>();
mockedContext.Setup(c => c.Users).ReturnsDbSet(enumerable);
mockedContext.Setup(c => c.Set<User>()).ReturnsDbSet(enumerable);
var repo = new Mock<IRepository<User>>();
return new Repository<User>(mockedContext.Object, this.logger);
For clarity, some most important bits of the repository code:
public class Repository<TEntity> : IRepository<TEntity> where TEntity : class
{
protected internal MyDbContext Context;
protected internal DbSet<TEntity> DbSet;
public Repository(MyDbContext context)
{
this.Context = context;
this.DbSet = context.Set<TEntity>();
}
public virtual IQueryable<TEntity> Get(Expression<Func<TEntity, bool>> filter = null)
{
IQueryable<TEntity> query = this.DbSet;
if (filter != null)
{
query = query.Where(filter);
}
return query;
}
Any idea why does it work like that?

Based on the linked example. The described problem experienced is usually because of this line.
mockSet.As<IQueryable<T>>()
.Setup(m => m.GetEnumerator())
.Returns(queryableData.GetEnumerator()); //<-- this here
return queryableData.GetEnumerator() here alony allows for a one time forward only enumeration.
To allow for multiple enumerations return a function.
mockSet.As<IQueryable<T>>()
.Setup(m => m.GetEnumerator())
.Returns(() => queryableData.GetEnumerator()); //<-- note the change () =>
the function will be called every time an enumeration is needed, allowing for multiple enumerations of the collection.

Manipulating objects with DbSet<T> and IQueryable<T> with NSubstitute returns error

I'd like to use NSubstitute to unit test Entity Framework 6.x by mocking DbSet. Fortunately, Scott Xu provides a good unit testing library, EntityFramework.Testing.Moq using Moq. So, I modified his code to be suitable for NSubstitute and it's been looking good so far, until I wanted to test DbSet<T>.Add(), DbSet<T>.Remove() methods. Here's my code bits:
public static class NSubstituteDbSetExtensions
{
public static DbSet<TEntity> SetupData<TEntity>(this DbSet<TEntity> dbset, ICollection<TEntity> data = null, Func<object[], TEntity> find = null) where TEntity : class
{
data = data ?? new List<TEntity>();
find = find ?? (o => null);
var query = new InMemoryAsyncQueryable<TEntity>(data.AsQueryable());
((IQueryable<TEntity>)dbset).Provider.Returns(query.Provider);
((IQueryable<TEntity>)dbset).Expression.Returns(query.Expression);
((IQueryable<TEntity>)dbset).ElementType.Returns(query.ElementType);
((IQueryable<TEntity>)dbset).GetEnumerator().Returns(query.GetEnumerator());
#if !NET40
((IDbAsyncEnumerable<TEntity>)dbset).GetAsyncEnumerator().Returns(new InMemoryDbAsyncEnumerator<TEntity>(query.GetEnumerator()));
((IQueryable<TEntity>)dbset).Provider.Returns(query.Provider);
#endif
...
dbset.Remove(Arg.Do<TEntity>(entity =>
{
data.Remove(entity);
dbset.SetupData(data, find);
}));
...
dbset.Add(Arg.Do<TEntity>(entity =>
{
data.Add(entity);
dbset.SetupData(data, find);
});
...
return dbset;
}
}
And I created a test method like:
[TestClass]
public class ManipulationTests
{
[TestMethod]
public void Can_remove_set()
{
var blog = new Blog();
var data = new List<Blog> { blog };
var set = Substitute.For<DbSet<Blog>, IQueryable<Blog>, IDbAsyncEnumerable<Blog>>()
.SetupData(data);
set.Remove(blog);
var result = set.ToList();
Assert.AreEqual(0, result.Count);
}
}
public class Blog
{
...
}
The issue arises when the test method calls set.Remove(blog). It throws an InvalidOperationException with error message of
Collection was modified; enumeration operation may not execute.
This is because the fake data object has been modified when the set.Remove(blog) method is called. However, the original Scott's way using Moq doesn't result in the issue.
Therefore, I wrapped the set.Remove(blog) method with a try ... catch (InvalidOperationException ex) block and let the catch block do nothing, then the test doesn't throw an exception (of course) and does get passed as expected.
I know this is not the solution, but how can I achieve my goal to unit test DbSet<T>.Add() and DbSet<T>.Remove() methods?

What's happening here?
set.Remove(blog); - this calls the previously configured lambda.
data.Remove(entity); - The item is removed from the list.
dbset.SetupData(data, find); - We call SetupData again, to reconfigure the Substitute with the new list.
SetupData runs...
In there, dbSetup.Remove is being called, in order to reconfigure what happens when Remove is called next time.
Okay, we have a problem here. dtSetup.Remove(Arg.Do<T.... doesn't reconfigure anything, it rather adds a behavior to the Substitute's internal list of things that should happen when you call Remove. So we're currently running the previously configured Remove action (1) and at the same time, down the stack, we're adding an action to the list (5). When the stack returns and the iterator looks for the next action to call, the underlying list of mocked actions has changed. Iterators don't like changes.
This leads to the conclusion: We can't modify what a Substitute does while one of its mocked actions is running. If you think about it, nobody who reads your test would assume this to happen, so you shouldn't do this at all.
How can we fix it?
public static DbSet<TEntity> SetupData<TEntity>(
this DbSet<TEntity> dbset,
ICollection<TEntity> data = null,
Func<object[], TEntity> find = null) where TEntity : class
{
data = data ?? new List<TEntity>();
find = find ?? (o => null);
Func<IQueryable<TEntity>> getQuery = () => new InMemoryAsyncQueryable<TEntity>(data.AsQueryable());
((IQueryable<TEntity>) dbset).Provider.Returns(info => getQuery().Provider);
((IQueryable<TEntity>) dbset).Expression.Returns(info => getQuery().Expression);
((IQueryable<TEntity>) dbset).ElementType.Returns(info => getQuery().ElementType);
((IQueryable<TEntity>) dbset).GetEnumerator().Returns(info => getQuery().GetEnumerator());
#if !NET40
((IDbAsyncEnumerable<TEntity>) dbset).GetAsyncEnumerator()
.Returns(info => new InMemoryDbAsyncEnumerator<TEntity>(getQuery().GetEnumerator()));
((IQueryable<TEntity>) dbset).Provider.Returns(info => getQuery().Provider);
#endif
dbset.Remove(Arg.Do<TEntity>(entity => data.Remove(entity)));
dbset.Add(Arg.Do<TEntity>(entity => data.Add(entity)));
return dbset;
}
The getQuery lambda creates a new query. It always uses the captured list data.
All .Returns configuration calls use a lambda. In there, we create a new query instance and delegate our call there.
Remove and Add only modify our captured list. We don't have to reconfigure our Substitute, because every call reevaluates the query using the lambda expressions.
While I really like NSubstitute, I would strongly recommend looking into Effort, the Entity Framework Unit Testing Tool.
You would use it like this:
// DbContext needs additional constructor:
public class MyDbContext : DbContext
{
public MyDbContext(DbConnection connection)
: base(connection, true)
{
}
}
// Usage:
DbConnection connection = Effort.DbConnectionFactory.CreateTransient();
MyDbContext context = new MyDbContext(connection);
And there you have an actual DbContext that you can use with everything that Entity Framework gives you, including migrations, using a fast in-memory-database.