How to unittest a fairly simple application?
By "fairly simple application" understand:
An application that queries a database (that can be complex) through an ORM and displays the result without any further processing. That's, the Linq query returns a simple IEnumerable that will be displayed on the screen.
The kind of tests I'm writing so far tends to be slow and I delay more and more their execution. If I keep this process, these tests will only be executed when I commit the code and thus I'll loose all the benefits of unit test: highlight bugs or errors within a minute.
I tried to make an object graph in memory but as the application grows this graph become very complex and it's taking more and more time to be maintained. And when I plug the ORM instead of object graph, I see bugs my unit tests didn't foresee...
In my case, I need a generic solution as the ORM should be either Entity Framework or nHibernate.
EDIT:
I rephrase my question as it is maybe a bit unclear: test this kind of application means test the queries. What is the best manner to test Linq queries?
The simple answer to your question is not to unit test data queries. As you found out the value of unit testing this part of the application is very limited and most of the bugs don't show up. The ORM is a good place for integration testing.
To enable this your data access code must be put in one place. Depending on your application this may be a data access layer, a Repository or any other structure that enables you to split your business logic from your data access. In the business logic part you use unit tests and mock or replace the data access with an in memory solution.
The data access part you test with integration tests. But here you only have to test if you can insert, update, select or delete data and if your queries work. That should be doable with way less tests than if you would try to do this all within your unit tests.
Entity Framework and nHibernate both have tutorials on how you can use those frameworks for this kind of testing.
Johnny Graber's answer is similar, but I'll add mine anyway for sake of several opinions.
To answer specifically on your issue with testing Linq queries (and still, this is my opinion, like in the comment above): you don't test them.
As nice and simple Linq is, it is basically as bad having a lot of Linq queries in a project as it is to have hardcoded SQL statements.
Hence, the Linq queries should be part of a data abstraction layer and that layer is what you test. That layer provides basic access methods, mostly CRUD operations. These operations might be implemented using Linq against whatever (SQL, SQLite.Net, XML, you name it).
You should then have another implementation of your DAL which returns/generates mock data. No need to use Linq here.
You can then test, if you're application works against the implementation that contains mock data, which is fast, as it's memory based.
You worry about the outcome of these methods.
You can from time to time switch over to the productive version that's based on Linq and run the slower tests.
Linq itself works. Microsoft did a pretty good job. You need to worry about your access methods returning proper data.
Related
I need to write integration tests for my layer that exposes methods of service. But I need my database to be in a certain state for the tests to pass
For example for testing the GetStoreByID method, I need to have store 1 in my database but not store 2 (for the ko test)
The database is developed and deployed by a another team using a sql project (dacpac)
I use Entity Framework 6.1.3 with an Edmx
What is the best way, in this case, to setup the data in database before tests ?
This link https://msdn.microsoft.com/en-us/library/dn314431(v=vs.113).aspx gives the Microsoft MSDN article on how to write tests using EF6 by faking out the database.
In summary the article covers the subject 'When writing tests for your application it is often desirable to avoid hitting the database. Entity Framework allows you to achieve this by creating a context – with behavior defined by your tests – that makes use of in-memory data.'
If your data access is separated by an interface (i.e. using something like IDBSet for each of the types), personally I'd avoid depending on the database completely where possible, and use either a fakedbsets (if you need to test repository/DAL code) or just mocking with moq or nsubstitute if you don't.
I know this doesn't specifically answer your question, but the best way to setup test data is do it in memory in my experience with as few external dependencies as possible. The database adds extra moving parts that you don't really want to have to depend on in unit/integration tests. This also adds complexities if you have a CI server etc... that you generally want to avoid.
Short Question
Is there something you can run sql commands that have JOINS and WHEREs that is not a DB
Long Question
I am putting in unit tests for a brown field win forms app.
I have complete freedom of choice on what kinda unit test framework I have
The problem I have is there is masses of SQL string statement in the code.
Think something like this
SELECT *
FROM Sale
INNER JOIN SaleItem ON Sale.ID = SaleItem.SaleID
WHERE ID = 5
It is parameterized, and has IF statements to build up the where, so it might be where CustomerId = 5 or DispatchDate was in last year.
The query is a lot bigger that this, and I kinda want to check that all the joins work and all the possible wheres work. Do think this might be me looking at the detail to much
I dont want to have to manage a database of data, which if the data changes it will break tests, and I'm scared that will root and people will just kill of the tests.
I want to run this sql against some object or a thing, that is NOT a DB and get a item.
It has to be smart enough to actually filter So it the Sale object was like the following table, it would only return the one with the ID 5.
ID DateDispatched CustomerID
1 1/1/1 5
2 2/2/2 6
5 3/3/3 7
I have thought of running sql command on datasets and XML, and relised that wont work.
I guess LINQ has spoiled me over that last few years cus I cant work out how to do this. And im afraid there is so much logic building up massive SQL statement, I have to put some tests on them.
Would be more than willing to hear about other options like moving the SQL to stored procedure in the DB, if you can recommend a good unit testing framework.
Now I don't like SQL being built in the app and would love to change it to entity framework, but its a 10 year old application and that's just not a option.
Okay some quick edits
The database is on SQL Server 2012, so stored procedures are a option, as in some places they use stored procedures.
Let me try to understand your problem.
You have got an winform application and you are writing unit tests for
this. But if you run the the test, you afraid it will going to hit the
DB and spoil the data. So you want some mechanism which allows to run
your unit tests but will not hit the actual database. Correct?
If I got your problem right, I suggest to separate out your db interaction logic and make it interface driven. Then you can create mock objects, wherein you define the expectation of your db interaction interfaces. So for example, if some GetSales() method is called, what should be returned from this method? and so on. Sharing some links on details about unit testing and mocking.
https://msdn.microsoft.com/en-us/library/ff650441.aspx
https://github.com/Moq/moq4
http://www.developerhandbook.com/unit-testing/writing-unit-tests-with-nunit-and-moq/
Testing a MVC Controller fails with NULL reference exception
A SQLite database is designed for exactly this kind of requirement.
The database is a simple file. The database driver reads from and writes to this file. You can run all the SQL queries and so on that you are used to against the database. (However, you might have problems if your SQL uses language or syntax specific to SQL Server.)
It is true that you have to manage a 'database' full of data. But you should be able to:
Write a script which quickly sets up all the tables. You might already have a piece of code which create a database with the same schema, or you might be able to dump one automatically.
Keep some test data around in CSV's, SQL files, or similar. This isn't easy, but it is very very useful. You should add the minimum and only build it up as the testing demands.
Check the whole SQLite database file into source control if you like.
Thanks for the question - I have been thinking a lot about testable database code recently, and I hadn't figured out a solution to this type of problem until your question made me realize I already knew it.
In my opinion, there are three approaches to make this kind of testing as painless as possible in the long-run:
Use an ORM or another wrapper layer, such as Entity Framework as you mentioned. This means that when testing you don't need a 'real database' at all - just a test double of your wrapper.
Only use standard portable SQL such as JOIN, SELECT, etc, with nothing which can't be run on a SQLite database. This can be very restrictive, as types vary so much between DBMS.
Use SPs exclusively as an interface to your database. This means that your test double only has to recognize which SP is being called, and respond correctly to that. I personally don't like this approach as I think lots of untested, unversioned business logic ends up in the SPs.
Here is an option that we use for our unit testing with MS test
Use the TransactionScope from the System.Transactions namespace
in your TestInitialize method Create an instance of the transaction and in the TestCleanup method dispose of it. You can do the insert into the db in the test initialize method or in the individual test methods
[TestCleanup]
public void testClean()
{
_Trans.Dispose();
}
[TestInitialize]
public void testInit()
{
_Trans = new TransactionScope();
}
[TestMethod]
public void TestQuery()
{
// arrange
//' insert data
// act
Obj Target = Obj.New();
// Assert
Assert.AreEqual("someValue",Obj.SomeProperty);
}
Okay I took the old adage of if you cant change your employer, change your employer.
I think tsqlt http://tsqlt.org/ would of been the best fit for this exact problem.
As Entity framework would not be allowed and they had so many dependency tables. Which would let me mock tables move all logic to stored procedures and mock the tables. Kinda mix of Yogi and JWG answer.
I have written a .Net application which has many components, some of those components are database access layers which abstract from the rest of the components where the data comes from.
I have unit tested the rest of the components by mocking the database access layer. One way I have of testing the database access layers is to use create new empty databases on the test servers. This can be slow, and most would argue that it is not a unit tests as I depend on the database server.
What I think I want, is a mocked database which I can use to test my database access layer. The mocked database can be given a schema and process SQL commands as if it were a remote database, but in fact it is all in-memory. Does this exists? Or how else can I test my SQL and database <-> data model code.
To solve my problem you may want to know I am using SQL Server, versions 2008 and later, and my code is written in C#, running with .Net 4.5 and using Visual Studio 2013
Note: I do not want to use Linq2SQL/EntityFramework to replace my database access layer, as in my experience it results difficult to debug issues and performance problems.
I tried to phrase my question carefully to avoid people lecturing me on their beliefs in what should be tested and how, but perhaps to be a little more blunt:
I want to unit test my SQL, small changes to that have a big impact on the outcome of the program. I do have integration tests, but it takes much longer to create a release for the test environment than it does to tweak code and run the unit tests.
I appreciate people taking the time to read my question and respond anyhow.
I don't know if it's going to be the best answer, but. The way we're doing is that we're using SQLite, which is an in-memory database. There are a number of different ways to set it up, we use NHibernate as an ORM, and for that it is fairly easy to set up using FluentNHibernate, but I don't think it's much harder using any other framework either:
Fluently.Configure()
.Database(SQLiteConfiguration.Standard.InMemory())
.Mappings(m => ... )
.BuildConfiguration()
.BuildSessionFactory();
I think you can run queries against a SQLite database without any ORMs as well, e.g. using the base SqlConnection class.
This database can accept migrations, schemas, etc. It behaves very close to a proper MsSql database, we can run any DDL an DML statements against it, and it works fine. Since it's all in-memory, it's also pretty fast.
I have a time consuming operation that needs tested. The operation makes changes to a database. Current, the majority of my tests are very similar, a query is performed on the database to determine the expected result after the operation. Then, the operation under test is ran. Finally, the database is again queried to determine if the expected condition has been met. Since I am only querying the database in each test to determine the expected and actual results, and the function under test is always performing the same operations on the database. Is there a way that I could run the operation once and still test each condition in its own test?
The standard strategy for running unit tests on code that has side effects such as connecting to databases is to provide mock objects instead of a real database connection or other service, but it depending on how your code is structured this may be harder or easier to adopt.
Frameworks for inversion of control and dependency injection can be very helpful for this kind of purpose. The dependancy injection pattern makes sure that objects can have dependant implementations passed to them at runtime, rather than constructing them directly, for testing purposes or just to create more modular code. StructureMap is such a framework for C#, but there are others, such as Spring.NET.
Sounds like you want to be using Unit Testing Frameworks and/or Mocking Frameworks.
As a general rule, you shouldn't be writing additional queries to test the query has worked as expected on the database as these themselves will require development and testing.
Depending on the database you're writing to, you may be able to get the number of rows affected on the database (take a look at ##rowcount for SQL Server). But again, you're adding additional code just to do testing which should really be a separate activity.
I am still having a issue getting over a small issue when it comes to TDD.
I need a method that will get a certain record set of filtered data from the data layer (linq2SQL). Please note that i am using the linq generated classes from that are generated from the DBML. Now the problem is that i want to write a test for this.
do i:
a) first insert the records in the test and then execute the method and test the results
b) use data that might be in the database. Not to keen on this logic cause it could cause things to break.
c) what ever you suggest?
You should choose option a).
A unit test should be repeatable and has to be fully under your control. So for the test to be meaningful it is absolutely necessary that the test itself prepares the data for its execution - only this way you can rely on the test outcome.
Use a testdatabase and clean it each time you run the tests. Or you might try to create a mock object.
When I run tests using a database, I usually use an in-memory SQLite database.
Using an in memory db generally makes the tests quicker.
Also it is easy to maintain, because the database is "gone" after you close the connection to it.
In the test setup, I set up the db connection and I create the database schema.
In the test, I insert the data needed by the test. (your option a))
In the test teardown, I close the connection to the db.
I used this approach successfully for my NHibernate applications (howto 1 | howto 2 + nice summary), but I'm not that familiar with Linq2SQL.
Some pointers on running SQLite and Linq2SQL are on SO (link 1 | link 2).
Some people argue that a test using a database isn't a unit test. Regardless, I belief that there are situations where you want automated testing using a database:
You can have an architecture / design, where the database is hard to mock out, for instance when using an ActiveRecord pattern, or when you're using Linq2SQL (although there is an interesting solution in one of the comments to Peter's answer)
You want to run integration tests, with the complete system of application and database
What I have done in the past:
Start a transaction
Delete all data from all the tables in the database
Setup the reference data all your tests need
Setup the test data you need in database tables
Run your test
Abort the transaction
This works well provided your database does not have much data in it, otherwise it is slow. So you will wish to use a test database. If you have a test database that is well controlled, you could just run the test in the transaction without the need to delete all data first.
Try to design your system, so you get mock the data access layer for most of your tests. It is valid (and often useful) to unit test database code, however the unit tests for your other code should not need to touch the database.
You should consider if you would get more benefits from “end to end” system tests, with unit tests only for your “logic” code. This depend to an large extent on other factors within the project.