We Keep Coding

How to 'Unit' Test a database access layer

I have written a .Net application which has many components, some of those components are database access layers which abstract from the rest of the components where the data comes from.
I have unit tested the rest of the components by mocking the database access layer. One way I have of testing the database access layers is to use create new empty databases on the test servers. This can be slow, and most would argue that it is not a unit tests as I depend on the database server.
What I think I want, is a mocked database which I can use to test my database access layer. The mocked database can be given a schema and process SQL commands as if it were a remote database, but in fact it is all in-memory. Does this exists? Or how else can I test my SQL and database <-> data model code.
To solve my problem you may want to know I am using SQL Server, versions 2008 and later, and my code is written in C#, running with .Net 4.5 and using Visual Studio 2013
Note: I do not want to use Linq2SQL/EntityFramework to replace my database access layer, as in my experience it results difficult to debug issues and performance problems.
I tried to phrase my question carefully to avoid people lecturing me on their beliefs in what should be tested and how, but perhaps to be a little more blunt:
I want to unit test my SQL, small changes to that have a big impact on the outcome of the program. I do have integration tests, but it takes much longer to create a release for the test environment than it does to tweak code and run the unit tests.
I appreciate people taking the time to read my question and respond anyhow.

I don't know if it's going to be the best answer, but. The way we're doing is that we're using SQLite, which is an in-memory database. There are a number of different ways to set it up, we use NHibernate as an ORM, and for that it is fairly easy to set up using FluentNHibernate, but I don't think it's much harder using any other framework either:
Fluently.Configure()
.Database(SQLiteConfiguration.Standard.InMemory())
.Mappings(m => ... )
.BuildConfiguration()
.BuildSessionFactory();
I think you can run queries against a SQLite database without any ORMs as well, e.g. using the base SqlConnection class.
This database can accept migrations, schemas, etc. It behaves very close to a proper MsSql database, we can run any DDL an DML statements against it, and it works fine. Since it's all in-memory, it's also pretty fast.

Is there any way to get all data retrieved between two points in C# code (Linq-to-sql)

My cenario is the follow:
I'm working in one system developed in C# Asp.Net (a big, huge and definetly grown anyway system). And I'm trying to begin create some unit tests to start refactor (believe, it's need refactor (there some controllers with 10k, 12k lines).
The problem is that a lot of things in this system is related to database (and system is tightly coupled to database). Database context is instantiated in a lot of pieces of code, and not injected.
So, my point now is to Mock some data into local MDB file to refactor code and create unit tests that will have their own MDB (with all database structure, but with only the data that he will use).
In what I though? Something like that:
[TestMethod()]
public void AnyTest()
{
var dbLogger = new DbLogger(); //this class is not created, it's just an example.
dbLogger.Start();
WorstWrittenMethodEver(); //this method will call any other methods insides,
//a couple of then
//and I really don't know the order and the
//complexity (much times very high), and this will probably
//instantiate the DataContext a lot of times and do
//a lot of data retrieval.
db.StopLog();
Console.WriteLine(db.DataRetrieved); //And in this line I will print all the tables
//and data retrieved between this two points.
}
After that, I will get that data, mock into one MDB file, and refactor the unit test above to get a really Unit Test.
Is there anyway to do that?

Looks that it is not very easy task. And I think you should use some popular and tested library or tool. My recommendation is to use the MiniProfiler. It allows to capture all SQL queries (also includes a support of LinqToSql). It has a good UI and API to interact within your code. Actually to get all SQL queries data you can use the following method:
MiniProfiler.Current.GetSqlTimings();

Unit Testing Legacy Code without DI

We are trying to add unit testing into our Business layer. The technology stack is asp.net web forms, WCF, ADO.Net calling stored procedures). The business layer is calling static methods on data classes, so it makes it difficult to introduce DI without making a lot of changes.
It may not be a conventional way to do it, but I'm thinking of keeping the DB in the unit test (dependency), but using it as a Test Db... either using an existing frozen db or having mocked data in tables. I was wondering about the feasibility of using a test db where the stored procedures are used like Mocks. Instead of duplicating the entire db, just create table names, named by the stored procedure.
The stored procedure would just call one table, and return static data... essentially, trying to emulate the functionality of Mocking data with something like Moq but from a DB perspective.
Can anyone recommend any designs that would include the DB in testing, that are still deterministic?

if you want to use the DB in the tests and have everything be deterministic then you need each test to have its own DB, which means creating (and potentially populating) a new db for each test.
Depending on how your DB layer creates its connection this is feasible. I have done similar by generating a DB using localDb in test setup with a GUID for the name and then deleting the DB again at the end of the test in the tear down.
It ends up being reasonably slow (not surprisingly) but having the DBs created on a Ram disk drive helped with that.
This worked ok for empty dbs, that then had schemas created by EF, but if you need a fixed set of data in the DB then you might need to restore it from a backup in the setup of the test

It seems to me that it's going to be a lot of work setting up your stored procedures to do what you want them to do when they are called for each test, and you still end up with the speed problems that databases always present. I'd recommend you do one or both of the following instead:
Use TypeMock, which has a powerful isolator tool. It basically changes your compilation to make it so that your unit test can mock even static methods.
Instead of just unit tests, try creating "acceptance tests," which focus on mimicking a complete user experience: log in, create object, view object (verify object looks right), update object, view object again (ditto), delete object (verify object is deleted). Begin each of these tests by setting up all the objects you'll need for this particular test, and end by deleting all those objects, so that other tests can run based on an assumed starting state.
The first approach gives you the speed and mockability of true "unit" tests, whereas the second one allows you to exercise much more of your code, increasing the likelihood that you'll catch bugs, even in things like stored procedures.

How would i unit test database logic?

I am still having a issue getting over a small issue when it comes to TDD.
I need a method that will get a certain record set of filtered data from the data layer (linq2SQL). Please note that i am using the linq generated classes from that are generated from the DBML. Now the problem is that i want to write a test for this.
do i:
a) first insert the records in the test and then execute the method and test the results
b) use data that might be in the database. Not to keen on this logic cause it could cause things to break.
c) what ever you suggest?

You should choose option a).
A unit test should be repeatable and has to be fully under your control. So for the test to be meaningful it is absolutely necessary that the test itself prepares the data for its execution - only this way you can rely on the test outcome.

Use a testdatabase and clean it each time you run the tests. Or you might try to create a mock object.

When I run tests using a database, I usually use an in-memory SQLite database.
Using an in memory db generally makes the tests quicker.
Also it is easy to maintain, because the database is "gone" after you close the connection to it.
In the test setup, I set up the db connection and I create the database schema.
In the test, I insert the data needed by the test. (your option a))
In the test teardown, I close the connection to the db.
I used this approach successfully for my NHibernate applications (howto 1 | howto 2 + nice summary), but I'm not that familiar with Linq2SQL.
Some pointers on running SQLite and Linq2SQL are on SO (link 1 | link 2).
Some people argue that a test using a database isn't a unit test. Regardless, I belief that there are situations where you want automated testing using a database:
You can have an architecture / design, where the database is hard to mock out, for instance when using an ActiveRecord pattern, or when you're using Linq2SQL (although there is an interesting solution in one of the comments to Peter's answer)
You want to run integration tests, with the complete system of application and database

What I have done in the past:
Start a transaction
Delete all data from all the tables in the database
Setup the reference data all your tests need
Setup the test data you need in database tables
Run your test
Abort the transaction
This works well provided your database does not have much data in it, otherwise it is slow. So you will wish to use a test database. If you have a test database that is well controlled, you could just run the test in the transaction without the need to delete all data first.
Try to design your system, so you get mock the data access layer for most of your tests. It is valid (and often useful) to unit test database code, however the unit tests for your other code should not need to touch the database.
You should consider if you would get more benefits from “end to end” system tests, with unit tests only for your “logic” code. This depend to an large extent on other factors within the project.

Clear NHibernate database fast

I am using NHibernate for ORM, and everything works fine.
Now I started to write some unit-tests (using the DB, I do not want to put tooo much effort in abstracting this away, I know its not perfect, but it works..).
I need to be sure that the DB is completly empty for some tests. I can, of course, create the whole DB. But that seems to be overkill and I think it takes longer...
Is there a DELETE_ALL command which clears all tables, I can use in NHibernate?
Chris
EDIT: A short update, I decided to go the SQLite way, no problem to change this with NHibernate. There are some pitfalls, I am using this config, and it works. Otherwise you might get "table not found" errors, due to nHibernate closing the connection while in session, resulting in a "lost" database...
For your convinience: Copy and paste...
.Database(SQLiteConfiguration.Standard.ConnectionString("Data Source=:memory:;Version=3;New=True;Pooling=True;Max Pool Size=1;")
.Raw("connection.release_mode", "on_close"))
.Mappings(obj => obj.AutoMappings.Add(_config.APModel));

Drop and recreate the database. You can use schemaexport:
var export = new SchemaExport(config);
export.Drop(false, true);
export.Create(true, true);
Sql lite in memory runs faster for tests than a "normal" database, but the disadvantage is that the sql-lite dialect can be different than the sql dialect in production.

I would recommend you check out ndbunit. It's not a NHibernate-specific, but I've used it for testing NHibernate projects in the past and it works well. Basically, it provides functions to clear the database, prefill it with test data, or restore it to known states after each test. You just have to provide an XSD of the database schema, and optionally some XML data for pre-filling.
I believe I first saw this in the Summer of NHibernate screen-cast series, so check those out to see it in use.

You're not writing unit tests (i.e. tests that test one unit), you're writing integration tests where units interact (i.e. with your database).
Part of your test infrastructure could run a sql script that does one of the following:
Drop db and recreate.
Truncate all tables.
Ideally, you do want to put a bit of work in abstracting the db away, especially since you have NH which makes it much easier than some other frameworks.

Use an in memory database like SQLite, and setup the necessary data in it before each test. The initial setup takes a bit of time, but each test runs very fast afterwards and you can make sure that you start off with a clean slate. Ayende has a few blog posts about how to set it up.

Just in case the Drop/Create DB does not suit your needs (like if the db contains object that NHibernate is not aware of, like SPs, functions etc) you could always make a backup point with the DB empty and after you're done testing just restore to that point