I am trying to find out in Visual Studio 2010 Unit Testing how to keep a transaction of the data I have either added, updated, or deleted during my tests so on my TestCleanup I can rollback their values.
What search terms should I be using to find more about this?
Cheers
Paul
Why do you need to rollback changes? Are your unit tests updating live data? If unit tests are written properly, you shouldn't have to clean up after what your tests changed because the data they're changing should be isolated to your test.
Edit:
It sounds like you've set up a data set for testing and want to make sure that data set is restored back to its original state. I prefer the practice of setting up the test data as part of the test, but I understand that can get difficult for complex tests.
If this in an ADO.NET data source, you could start a transaction then roll back that transaction at the end of your test. For example:
using (var transaction = db.BeginTransaction())
{
// Do tests here
}
// The transaction is rolled back when disposed
Edit 2:
A third option, if you don't have transaction support, is to have a backup of your test data in a place where it won't get modified, then at the end of the test, restore that backup.
For unit testing you should probably try using mocks instead of accessing a test database. Unit tests should generally be completly self-contained (and not rely on external sources e.g., databases).
If you are actually testing calls to the database then this is probably an integration test in which case you could:
Create test data in setup
Run code
Assert test passes
Remove test data inserted in step one
Related
public ApplicationLayout Get(string objectName, string userName)
{
using (var db = new SqlConnection(_connectionString))
{
var sql = "SELECT * FROM dbo.ApplicationLayout WHERE ObjectName = #ObjectName AND UserName=#UserName";
return db.Query<ApplicationLayout>(sql, new {ObjectName = objectName, UserName = userName}).FirstOrDefault();
}
}
I have some code here, and I was wondering if it is possible to perform a unit test in C# using SQL data (any framework). From what I've seen, it is not possible. Can anyone please confirm this?
I always did my testing on a dedicated database. You can do a setup and a tear down, and basically recreate the database (or restore it from a backup) each time you run those tests. It is a lot more involved than just plain C# unit testing. Like some comments stated, it's not really unit testing anymore but stuff needs to be tested one way or another.
Possible, but not recommended for a unit test.
When you talk about a unit test, you're usually talking about a test that can be repeatably executed in isolation; that is, it has no dependencies outside the code assembly where it lives (and ideally each test should not go very far beyond a single class being tested), and has no lasting side effects regardless of how the test is started or stopped (doesn't require use of the file system, or a database, etc). This is because unit tests, being the smallest and most fine-grained tests, are intended to test the code that you yourself wrote. You didn't write SQL Server, so SQL Server shouldn't be in the scope of a unit test.
Now, once you widen the scope to integration tests and acceptance tests, now you're testing that code you wrote plays nicely with all the external dependencies you didn't write. Integration tests can touch external data, like databases, and can write to as well as read from those external stores, as long as they clean up after themselves properly, either by returning a persistent database to its original condition after the run, or just using a portable DB like SQLite or MSS Express, setting up the necessary schema and test data as part of your test setup, and then just blowing the DB away when it's done.
For many, this is splitting hairs, and unit testing frameworks like MSTest or NUnit don't prevent you using them to create a test that has external dependencies. These are merely good ideas for structuring your full test suite into a fine-grained, fast-running, repeatable component that you can run on any change you make, and the longer-running, wider-scoped proof of correctness that might run overnight.
The modern approach to orchestrate testing against a real SQL engine has to work:
locally on dev machines (that run Windows, MacOS or Linux);
build pipelines in the cloud (e.g. Azure DevOps pipelines, GitHub Actions,
etc.).
TL;DR – check out DbSample on GitHub, a sample EF Core based
project with fully automated tests against MS SQL Server and a GitHub
Actions pipeline.
A detailed analysis of orchestrating tests is covered in "Pain & Gain of automated tests against SQL (MS SQL, PostgreSQL)" article. The key steps are:
Launch SQL engine in Docker (use official images for SQL Server or PostgreSQL)
Create a test database and populate the schema, logic, lookup dictionaries, etc (via command line for the containers)
Connect xUnit/NUnit tests to the database and for each test (rinse and repeat):
Seed test data
Perform the testable activity and checks
Revert the database to the pristine state (via Respawn)
Tear down the SQL engine along with the database and other artefacts.
P.S. I know, the OP said "unit tests" but perhaps meant a more broad "automated tests"
I've got a C# library that I've written a lot of unit tests for. The library is a data access layer library and requires SQL Server's full-text indexing capabilities, which means LocalDb will not work. The unit tests are connecting to a local SQL Server instance. The project has an IDatabaseInitializer that drops and re-creates the database for each test, so each test has a fresh set of data to assume it can work against, meaning each test is capable of running on its own - no ordering needed.
A problem that I've had since day one on this, but never tackled yet, is that if I simply run all of the tests at once, some will fail. If I go back and run those tests individually then they succeed. It's not always the same tests that fail when I run all at once.
I've assumed that this is because they're all running so quickly and against the same database. Perhaps the database is not properly deleting and re-creating before the next test runs. On top of just a simple database, I've also got a SQL Server full-text index that some of my tests require in order to pass. The full-text index populates in the background and therefore is not "ready" immediately after populating the test database's tables. The full-text index may take a second or two to build. This causes my tests that target the full-text index to always fail, unless I step through the initializer in a debugger and pause a few seconds while the full-text index builds.
My question is, is there any way to avoid this clashing of database rebuilds? As a bonus, is there a way to "slow down" the tests that need the full-text index, so that it has time to populate?
You can use a database with fixed datas. For your tests you use a transaction that you begin on TestInitialize and rollback on TestCleanup
We are trying to add unit testing into our Business layer. The technology stack is asp.net web forms, WCF, ADO.Net calling stored procedures). The business layer is calling static methods on data classes, so it makes it difficult to introduce DI without making a lot of changes.
It may not be a conventional way to do it, but I'm thinking of keeping the DB in the unit test (dependency), but using it as a Test Db... either using an existing frozen db or having mocked data in tables. I was wondering about the feasibility of using a test db where the stored procedures are used like Mocks. Instead of duplicating the entire db, just create table names, named by the stored procedure.
The stored procedure would just call one table, and return static data... essentially, trying to emulate the functionality of Mocking data with something like Moq but from a DB perspective.
Can anyone recommend any designs that would include the DB in testing, that are still deterministic?
if you want to use the DB in the tests and have everything be deterministic then you need each test to have its own DB, which means creating (and potentially populating) a new db for each test.
Depending on how your DB layer creates its connection this is feasible. I have done similar by generating a DB using localDb in test setup with a GUID for the name and then deleting the DB again at the end of the test in the tear down.
It ends up being reasonably slow (not surprisingly) but having the DBs created on a Ram disk drive helped with that.
This worked ok for empty dbs, that then had schemas created by EF, but if you need a fixed set of data in the DB then you might need to restore it from a backup in the setup of the test
It seems to me that it's going to be a lot of work setting up your stored procedures to do what you want them to do when they are called for each test, and you still end up with the speed problems that databases always present. I'd recommend you do one or both of the following instead:
Use TypeMock, which has a powerful isolator tool. It basically changes your compilation to make it so that your unit test can mock even static methods.
Instead of just unit tests, try creating "acceptance tests," which focus on mimicking a complete user experience: log in, create object, view object (verify object looks right), update object, view object again (ditto), delete object (verify object is deleted). Begin each of these tests by setting up all the objects you'll need for this particular test, and end by deleting all those objects, so that other tests can run based on an assumed starting state.
The first approach gives you the speed and mockability of true "unit" tests, whereas the second one allows you to exercise much more of your code, increasing the likelihood that you'll catch bugs, even in things like stored procedures.
I'm just starting to understand the importance of Unit Testing in a c# environment. Now, I'm wondering how do i implement a black-box unit test that does Inserts,Deletes and updates on a database and then cleaning up the data after a successful test.
How do you actually do a process that rollbacks data inserted/updated/deleted? do you simply reset the index and remove the inserted rows? or restore the original state of the table by creating a script?
please guide me, I appreciate it. thanks!
What we do here in our development cycle. we always have that unit testing and load testing in our mind when ever we are developing application. So we make a column in our every datadase's table with userId or else. Then when we run Load Test or Unit test we insert UserId -1 in that every column, pointing that it is a load test data and -2 in case of unit Test Data. then we have pre Define Jobs at data base end that will clean that data after some time.
As long as your test is concise, and i presume it must be - for testing your DAL, why not just do the insert / update / deletes in a transaction that is rolled back once your test is complete.
Another option is to just use specific Update / Delete scripts in your test cleanup methods to roll back the exact changes that you updated / inserted to their pre-test values.
I think deleting the rows in the CleanUp method should be good choice.
By this you will always be testing your deleting rows code.
I was doing a research recently and found this thread as well. Here are my findings, which might be of some help for the future readers:
Make tests responsible for restoring the data they change. Sth like undo for the command. Tests usually know what data changes are expected, so are able to revert those in theory. This will surely involve additional work and could introduce some noise, unless it's automated, e.g you might try to keep track of the data created/updated in the test somehow generally;
Wrap each test in transaction and revert it afterwards. Pretty much as the one above, but easier to implement with things like TransactionScope. Might be not suitable if app creates own transactions as transactions aren't composable in general and if app doesn't work with TransactionScope (there are issues with Entity Framework for example);
Assert in some smart way on data relevant to test only. Then you won't need to cleanup anything unless there is too much data. E.g you might make your app aware of tests and set specific value to a test-only column added to every table. I've never tried that in practice;
Create and initialize fresh database from scratch for every test;
Use database backups to restore database to the point you need;
Use database snapshots for the restore;
Execute scripts to delete all the data and insert it again.
I personally use the latter and even ended up implementing a Reseed library, which does all the work for me.
Test frameworks usually do allow execute some logic before and after each test/test fixture run, which will mostlikely be needed for the ideas above. E.g for NUnit this implemented with use of OneTimeSetUp, OneTimeTearDown, FixtureSetUp, FixtureTearDown, SetUp, TearDown attributes.
One option is to use a Mock database in place of a real database. Here's a link that describes it.
I am still having a issue getting over a small issue when it comes to TDD.
I need a method that will get a certain record set of filtered data from the data layer (linq2SQL). Please note that i am using the linq generated classes from that are generated from the DBML. Now the problem is that i want to write a test for this.
do i:
a) first insert the records in the test and then execute the method and test the results
b) use data that might be in the database. Not to keen on this logic cause it could cause things to break.
c) what ever you suggest?
You should choose option a).
A unit test should be repeatable and has to be fully under your control. So for the test to be meaningful it is absolutely necessary that the test itself prepares the data for its execution - only this way you can rely on the test outcome.
Use a testdatabase and clean it each time you run the tests. Or you might try to create a mock object.
When I run tests using a database, I usually use an in-memory SQLite database.
Using an in memory db generally makes the tests quicker.
Also it is easy to maintain, because the database is "gone" after you close the connection to it.
In the test setup, I set up the db connection and I create the database schema.
In the test, I insert the data needed by the test. (your option a))
In the test teardown, I close the connection to the db.
I used this approach successfully for my NHibernate applications (howto 1 | howto 2 + nice summary), but I'm not that familiar with Linq2SQL.
Some pointers on running SQLite and Linq2SQL are on SO (link 1 | link 2).
Some people argue that a test using a database isn't a unit test. Regardless, I belief that there are situations where you want automated testing using a database:
You can have an architecture / design, where the database is hard to mock out, for instance when using an ActiveRecord pattern, or when you're using Linq2SQL (although there is an interesting solution in one of the comments to Peter's answer)
You want to run integration tests, with the complete system of application and database
What I have done in the past:
Start a transaction
Delete all data from all the tables in the database
Setup the reference data all your tests need
Setup the test data you need in database tables
Run your test
Abort the transaction
This works well provided your database does not have much data in it, otherwise it is slow. So you will wish to use a test database. If you have a test database that is well controlled, you could just run the test in the transaction without the need to delete all data first.
Try to design your system, so you get mock the data access layer for most of your tests. It is valid (and often useful) to unit test database code, however the unit tests for your other code should not need to touch the database.
You should consider if you would get more benefits from “end to end” system tests, with unit tests only for your “logic” code. This depend to an large extent on other factors within the project.