I use a code generator (CodeSmith with .NetTiers template) to generate all the DAL code. I write unit tests for my code (business layer), and these tests are becoming pretty slow to run. The problem is that for each test, I reset the database to have a clean state. Also, as I do a lot of tests, it seems that the latency of the database operations sum up to a pretty bit delay.
All DB operations are performed through a DataRepository class that is generated by .NetTiers. Do you know if there is a way to generate (or code myself) a mock-DataRepository that would use in-memory storage instead of using the database ?
This way, I would be able to use this mock repository in my unit tests, speeding them a lot, without actually changing anything to my current code !
Take a look at Dependency injection (DI) and Inversion of Control containers (IOC). Essentially, you will create an interface that that a new mock DB object can implement, and then the DI framework will inject your mock DB when running tests, and the real DB when running you app.
There are numerous free and open source libraries that you could use to help you out. Since you are in C#, one of the new and up and coming DI libraries is Ninject. There are many others too. Check out this Wikipedia article for others and a high level description.
From the description of the issue, I think you are performing the Integration test because your test is making use of the Business and the DAL and live database.
For unit testing, you deal with one layer of code with all other dependencies either mocked or stubbed. With this approach, you unit tests will be really fast to execute on every incremental code changes.
There are various mocking frameworks that you can use like Rhino Mock, Moq, typemock to name a few. (In my project, I use Rhino mock to mock the DAL layer and unit test Business Layer in Isolation)
Harsha
Some of our unit tests use data fetched from XML's which were generated from a database to mock db access. DAL classes are replaced by mock ones because they are all stored in a DI container.
The generation of the xml's is custom code, if you find an open source solution for this then I'm happy to hear it.
Edit after Stefan's answer: I recall another team using SQL CE for their test database
Related
I know my question looks like I am searching for some shortcut but actually I have searched a lot and couldn't find a resource to learn unit testing specially for business layer which involves data update in database.
I did get resources to learn writing unit test cases in nunit but they all were testing static data. I don't want to learn unit test writing as a beginner, I have the idea of writing unit tests in nunit, my main question is how do we test methods which involves data insert/update/delete in database? Do we actually insert data while testing? Do mocking play a role in it?
Please help me with any resource that covers specially this case.
Thanks!
If your business layer is written in C#, as your tag suggests, then mocking the database is the best practice. You use a mocking framework that allows you to control the data layer and write asserts that your business layer does the correct thing under controlled (mocked) responses. If you use this approach then your test will be isolated and execute very fast without any connection to the database (you should be able to execute thousands of unit test within a second.)
If your database contains a lot of business logic itself then you can unit test it in isolation, but I would not use nunit for that.
If you decide to include the database in you nunit C# business logic test then you will have to deal with connections to the database, and shared state between test. You will be in a much more complex setup and deal with more complex errors.
Here is a good resource to start with: https://www.amazon.com/Pragmatic-Unit-Testing-Nunit-Programmers/dp/0974514020
I am trying to do unit testing with a recent project I have been working on.
It involves a SQL Server 2008 R2 database and a WCF Service using C#, .NET 4.5 and Visual Studio 2013 Premium. We use Entity Framework (EF) 6.0.1.
I am trying to test the WCF in isolation using Microsoft Fakes so that it doesn't require the database. We aim to do this using an "in-memory database". My difficulty is "stubbing" out the dbcontext for EF so I know it is in a known state that can be queried, altered and monitored.
I have read around that this can be bad idea because of the different Linq providers between linq-to-objects vs linq-to-sql. The functionality could pass at compile time but fail at runtime. To combat this we have integration tests as well (connecting the WCF to a real database) once deployed to our DEV servers via TFS.
I have also read around that the dbcontext could be shimmed using MS FAKES but just feels wrong.
Also adding a repository pattern (dependancy injection (DI)) is not causing our code coverage to increase which is one of our desired outcomes.
I then found this article http://msdn.microsoft.com/en-gb/data/dn314429.aspx and this article http://frankdecaire.blogspot.co.uk/2013/11/entity-framework-6-mocking-and-unit.html?showComment=1392224065716
This achieves what I want to do but uses Moq. Can this code be converted from Moq to MS FAKES? Is FAKES capable of doing everything that Moq does or shall I just learn Moq as well to add to my limited knowledge of FAKES?
var mockSet = new Mock<DbSet<account>>();
mockSet.As<IQueryable<account>>().Setup(m => m.Provider)
.Returns(data.Provider);
mockSet.As<IQueryable<account>>().Setup(m => m.Expression)
.Returns(data.Expression);
mockSet.As<IQueryable<account>>().Setup(m => m.ElementType)
.Returns(data.ElementType);
mockSet.As<IQueryable<account>>().Setup(m => m.GetEnumerator())
.Returns(data.GetEnumerator());
Any questions feel free to ask
Cheers
Kyle
IMO, you need to start decoupling your code by moving towards a more layered approach. I am not quite sure what you need to achieve by testing WCF in isolation.
I suggest having three layers and their tests as below -
Data access - uses EF and its DB context and implements a data access interface. You should unit test this without mocking EF's db context. Tests for this layer would be "state" dependent. By that I mean, your tests will work with real data and database for the CRUD operations. You just need to make sure you do not persist the changes to the db after the test run. One can use Spring.Net's testing library to achieve this or simply run your tests under a transaction scope and roll back the transaction after every test run (in the clean up).
Business logic - contains the business logic and works with the data access interface. Use any DI framework like spring.net or ms unity to inject the data access layer. You should unit test this by trying to avoid actual database calls. This is where something like a NMock, Rhinomock or MOQ comes into picture. Set up boundary and exception conditions using mocks and make sure your business logic addresses all the concerns.
WCF service layer - works with you operation and data contract. Ideally only forwards calls to business logic and translates response to data contract. I prefer having two types of tests at this level: a) unit tests for testing the translation and proper call forwarding. b) few basic integration tests using a proxy and some test data that go through the entire stack without any mocking.
The only problem I have with MS fakes is that it ships with VS2012 ultimate version and hence has a much lesser user base than something like a MOQ.
I have the following method:
Void UpdateUser(User user){}
I need to check this method whether will work properly.
I've used a separate db to check this in unit testing. But many experienced people said if I use this method that won't be unit testing; that's integration testing.
But I don't know how to mock for unit testing.
The code written in the UpdateUser method, will try to update data using Entity framework.
If I mock (Actually I don't how to do this either), how this will work with entity framework?
Mocking means that you develop your software components (classes) in a way that any class with behaviour is used/consumed/called-upon as an interface (or abstract class). You program to an abstraction. Run-time you use something (service locator, DI container, factory, ...) to retrieve/create those instances.
The most common way is to use construction injection. Here is an excellent explanation of why one would use DI, and examples of how to do it.
In your case, your component that uses the Entity Framework (your repository for instance) must implement a repository-interface, and any class that uses your repository should use it as an interface.
This way, you can mock the repository in your unittests. Which means that you create a unit-test-repository class (which has nothing to do with any database or EF), and use that when you create the instance of the class that you want to unit-test.
Hopefully this helps. There are many source to be found. Personally I just read this book and I found it to be very good. This is the authors blog.
You can use transaction and rollback or create a test user try its update. Assert and then in the finally block delete the test user.
You can use mocking framework like moq, rhino etc. moq is quite easy and you can find many example that demonstrate moq with DI like unity framework.
If your class is like this
public class UserRepository()
{
Sqlcontext _context;
void UpdateUser(User user)
{
_context.Users.Add(user);
}
}
then this is not unit testable.
Although this is not a unit test, if you insist on connecting on database and testing it, you could change your function to
User UpdateUser(User user)
{
_context.Users.Add(user);
return user;
}
and test if
user.Id > 0
Here, you are basically just testing entity framework.
"I've used a separate db to check this in unit testing. But many
experienced people said if I use this method that won't be unit
testing; that's integration testing."
Those people are mistaken, despite their supposed experience. For some reason, the incorrect notion that unit tests are all about testing parts of your code in isolation has grown in popularity in recent years. In reality, unit testing is all about writing tests that act as a unit, in other words they exist in isolation and the result of one unit test cannot influence another test.
If your UpdateUser method directly accesses EF, then as long as you ensure the database is guaranteed to be rolled back to its starting state at the end of each test, then you have unit tests. However, setting the database up for each test and ensuring it can be reliably rolled back can be a lot of work. That is why mocks are often used. Other answers have covered mcoking EF, so I won't go over that.
To greatly simplify your tests, you could have an extrapolation layer between UpdateUser and EF. In other words, the UpdateUser class is supplied with an instance of an interface, which is its gateway into EF. It doesn't talk to EF directly. To then mock EF, you simply supply a mocked implementation of the interface. This then pushes the need to test against EF down into a more basic layer, with more basic CRUD-like behaviours.
A non official trick, not-best-practice can be to use an in-memory database (context) for the time of testing.
Use transaction and rollback your transaction at the end of test
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Unit testing on code that uses the Database
I am just starting with unit testing and wondering how to unit test methods that are making actual changes to my database. Would the best way be to put them into transactions and then rollback, or are there better approaches to this?
If you want proper test coverage, you need two types of tests:
Unit-tests which mock all your actual data access. These tests will not acually write to the database, but test the behaviour of the class that does (which methods it calls on other dependencies, etc.)
System tests (or integration tests) which check that your database can be accessed and modified. I would considered two types of tests here: simple plain CRUD tests (create / read / update / delete) for each one of your model objects, and more complex system tests for your actual methods, and everything you deem interesting or valuable to test. Good practices here are to have each test starting from an empty (or "ready for the test") database, do its stuff, then check state of the database. Transactions / rollbacks are one good way to achieve this.
For unit testing you need to mock or stub the data access code, mostly you have repository interface and you can stub it by creating a concrete repository which stores data in memory, or you could mock it using dynamic mocking framework ..
For system or integration testing, you need to re-create the entire database before each test method in order to maintain a stable state before each test.
As per some of the previous answers if you want to test your data access code then you might want to think about mocks and a system/integration test strategy.
But, if you want to unit test your SQL objects (eg sprocs, views, constraints in tables etc) - then there are a number of database unit testing frameworks out there that might be of interest (including one that I have written).
Some implement tests within SQL, others within your code and use mbUnit/NUnit etc.
I have written a number of articles with examples on how I approach this - see http://dbtestunit.wordpress.com/
Other resources that might be of use:
http://www.simple-talk.com/sql/t-sql-programming/close-those-loopholes---testing-stored-procedures--/
http://tsqlt.org/articles/
The general approach is to have a way to mock you database actions. So that your unit tests are not reliant on the database being available or in a certain state. That said it also implies design that facilitates the isolation required to mock away your data layer. Unit test and how to do it well is a huge topic. Take a look on the googley for Mock Frameworks, and Dependency injection for a start.
If you are not developing an O/R mapper, there's no need to test database code. You don't want to test ADO.NET methods, right? Instead you want to verify that the ADO.NET methods are called with the right values.
Search Google for repository pattern. You will create an implementation of IRepository interface with CRUD methods and test/mock this.
If you want to test against a real database, this would be more of an integration then a unit test. Wrapping your tests in transaction could be an idea to keep your database in a consistent state.
We've done this in a base class and used the TestInitialize and TestCleanup functions to make sure this always happens.
But testing against a real database will certainly bring you into performance problems. So make sure from the beginning that you can swap your database access code with something that runs in memory. I don't now which database access code your targeting but design patterns like UnitOfWork and Repository can help you to isolate your database code and replace it with an in memory solution.
In my ASP.Net MVC application I am using IoC to facilitate unit testing. The structure of my application is a Controller -> Service Class -> Repository type of structure. In order to do unit testing, I have I have an InMemoryRepository class that inherits my IRepository, which instead of going out to a database, it uses an internal List<T> member. When I construct my unit tests, I just pass an instance of an internal repository instead of my EF repository.
My service classes retrieve objects from the repository through an AsQueryable interface that my repository classes implement, thus allowing me to use Linq in my service classes without the service class while still abstracting the data access layer out. In practice this seems to work well.
The problem that I am seeing is that every time I see Unit Testing talked about, they are using mock objects instead of the internal method that I see. On the face value it makes sense, because if my InMemoryRepository fails, not only will my InMemoryRepository unit tests fail, but that failure will cascade down into my service classes and controllers as well. More realistically I am more concerned about failures in my service classes affecting controller unit tests.
My method also requires me to do more setup for each unit test, and as things become more complicated (e.g. I implement authorization into the service classes) the setup becomes much more complicated, because I then have to make sure each unit test authorizes it with the service classes correctly so the main aspect of that unit test doesn't fail. I can clearly see how mock objects would help out in that regard.
However, I can't see how to solve this completely with mocks and still have valid tests. For example, one of my unit tests is that if I call _service.GetDocumentById(5), It gets the correct document from the repository. The only way this is a valid unit test (as far as I understand it) is if I have 2 or 3 documents stored, and my GetdocumentById() method correctly retrieves the one with an Id of 5.
How would I have a mocked repository with an AsQueryable call, and how would I make sure I don't mask any issues I make with my Linq statements by hardcoding the return statements when setting up the mocked repository? Is it better to keep my service class unit test using the InMemoryRepository but change my controller unit tests to use mocked service objects?
Edit:
After going over my structure again I remembered a complication that is preventing mocking in controller unit tests, as I forgot my structure is a bit more complicated than I originally said.
A Repository is a data store for one type of object, so if my document service class needs document entities, it creates a IRepository<Document>.
Controllers are passed an IRepositoryFactory. The IRepositoryFactory is a class which is supposed to make it easy to create repositories without having to repositories directly into the controller, or having the controller worry about what service classes require which repositories. I have an InMemoryRepositoryFactory, which gives the service classes InMemoryRepository<Entity> instantiations, and the same idea goes for my EFRepositoryFactory.
In the controller's constructors, private service class objects are instantiated by passing in the IRepositoryFactory object that is passed into that controller.
So for example
public class DocumentController : Controller
{
private DocumentService _documentService;
public DocumentController(IRepositoryFactory factory)
{
_documentService = new DocumentService(factory);
}
...
}
I can't see how to mock my service layer with this architecture so that my controllers are unit tested and not integration tested. I could have a bad architecture for unit testing, but I'm not sure how to better solve the issues that made me want to make a repository factory in the first place.
One solution to your problem is to change your controllers to demand IDocumentService instances instead of constructing the services themselves:
public class DocumentController : Controller
{
private IDocumentService _documentService;
// The controller doesn't construct the service itself
public DocumentController(IDocumentService documentService)
{
_documentService = documentService;
}
...
}
In your real application, let your IoC container inject IRepositoryFactory instances into your services. In your controller unit tests, just mock the services as needed.
(And see Misko Hevry's article about constructors doing real work for an extended discussion of the benefits of restructuring your code like this.)
Personally, I would design the system around the Unit of Work pattern that references repositories. This could make things much simpler and allows you to have more complex operations running atomically. You would typically have a IUnitOfWorkFactory that is supplied as dependency in the Service classes. A service class would create a new unit of work and that unit of work references repositories. You can see an example of this here.
If I understand correctly you are concerned about errors in one piece of (low level) code failing a lot of tests, making it harder to see the actual problem. You take InMemoryRepository as a concrete example.
While your concern is valid, I personally wouldn't worry about a failing InMemoryRepository. It is a test objects, and you should keep those tests objects as simple as possible. This prevents you from having to write tests for your test objects. Most of the time I assume they are correct (however, I sometimes use self checks in such a class by writing Assert statements). A test will fail when such an object misbehaves. It's not optimal, but you would normally find out quick enough what the problem is in my experience. To be productive, you will have to draw a line somewhere.
Errors in the controller caused by the service are another cup of tea IMO. While you could mock the service, this would make testing more difficult and less trustworthy. It would be better to NOT test the service at all. Only test the controller! The controller will call into the service and if your service doens't behave well, your controller tests would find out. This way you only test the top level objects in your application. Code coverage will help you spot parts of your code you don't test. Of course this isn't possible in all scenario's, but this often works well. When the service works with a mocked repository (or unit of work) this would work very well.
Your second concern was that those depedencies make you have much test setup. I've got two things to say about this.
First of all, I try to minimize my dependency inversion to only what I need to be able to run my unit tests. Calls to the system clock, database, Smtp server and file system should be faked to make unit tests fast and reliable. Other things I try not to invert, because the more you mock, the less reliable the tests become. You are testing less. Minimizing the dependency inversion (to what you need to have good RTM unit tests) helps making test setup easier.
But (second point) you also need to write your unit tests in a way that they are readable and maintainable (the hard part about unit testing, or in fact making software in general). Having big test setups makes them hard to understand and makes test code hard to change when a class gets a new dependency. I found out that one of the best ways to make tests more readable and maintainable is to use simple factory methods in your test class to centralize the creation of types that you need in the test (I never use mocking frameworks). There are two patterns that I use. One is a simple factory method, such as one that creates a valid type:
FakeDocumentService CreateValidService()
{
return CreateValidService(CreateInitializedContext());
}
FakeDocumentService CreateValidService(InMemoryUnitOfWork context)
{
return new FakeDocumentSerice(context);
}
This way tests can simply call these methods and when they need a valid object they simply call one of the factory methods. Of course when one of these methods accidentally creates an invalid object, many tests will fail. It's hard to prevent this, but easily fixed. And easily fixed means that the tests are maintainable.
The other pattern I use is the use of a container type that holds the arguments/properties of the actual object you want to create. This gets especially useful when an object has many different properties and/or constructor arguments. Mix this with a factory for the container and a builder method for the object to create and you get very readable test code:
[TestMethod]
public void Operation_WithValidArguments_Succeeds()
{
// Arrange
var validArgs = CreateValidArgs();
var service = BuildNewService(validArgs);
// Act
service.Operation();
}
[TestMethod]
[ExpectedException(typeof(InvalidOperationException))]
public void Operation_NegativeAge_ThrowsException()
{
// Arrange
var invalidArgs = CreateValidArgs();
invalidArgs.Age = -1;
var service = BuildNewService(invalidArgs);
// Act
service.Operation();
}
This allows you to let the test only specify what matters! This is very important to make tests readable! The CreateValidArgs() method could create an container with over 100 arguments that would make a valid SUT (system under test). You now centralized in one place the default valid configuration. I hope this makes sense.
Your third concern was about not being able to test if LINQ queries behave expectedly with the given LINQ provider. This is a valid problem, because it is quite easy to write LINQ (to Expression tree) queries that run perfectly when used over in-memory objects, but fail when querying the database. Sometimes it is impossible to translate a query (because you call an .NET method that has no counterpart in the database) or the LINQ provider has limitations (or bugs). Especially the LINQ provider of Entity Framework 3.5 sucks hard.
However, this is a problem you cannot solve with unit tests per definition. Because when you call the database in your tests, it's not a unit test anymore. Unit tests however never totally replace manual testing :-)
Still, it's a valid concern. In addition to unit testing you can do integration testing. In this case you run your code with the real provider and a (dedicated) test database. Run each test within a database transaction and rollback the transaction at the end of the test (TransactionScope works great with this!). Note however that writing maintainable integration tests is even harder than writing maintainable unit tests. You have to make sure that the model of your test database is in sync. Each integration test should insert the data it needs for that test in the database, which is often a lot of work to write and maintain. Best is to keep the amount of integration tests to a minimum. Have enough integration tests to make you feel confident about making changes to the system. For instance, having to call a service method with a complicated LINQ statement in a single test will often be enough to test if your LINQ provider is able to build valid SQL out of it. Most of the time I just assume the LINQ provider will have the same behavior as the LINQ to Objects (.AsQueryable()) provider. Again, you will have to draw the line somewhere.
I hope this helps.
I think your approach is sound for testing the service layer itself, but, as you suggested, it would be better if the service layer is mocked out completely for your business logic and other high-level testing. This makes your higher-level tests easier to implement/maintain, as there's no need to exercise the service layer again if it's already been tested.