We Keep Coding

Integration testing garbage data

I have set up integration testing using MSTest. My integration tests create fake data and insert them into the database (real dependencies). For every business object, I have a method like this, which creates a "Fake" and inserts it into the db:
public static EventAction Mock()
{
EventAction action = Fixture.Build<EventAction>().Create();
action.Add(false);
AddCleanupAction(action.Delete);
AppendLog("EventAction was created.");
return action;
}
I clean up all the fakes in [AssemblyCleanup]:
public static void CleanupAllMockData()
{
foreach (Action action in CleanUpActions)
{
try
{
action();
}
catch
{
AppendLog($"Failed to clean up {action.GetType()}. It is possible that it was already cleaned up by parent objects.");
}
}
}
Now, I have a big problem. In my continuous integration environment (TeamCity), we have a separate database for testing, and it cleans itself after every test run, but on my local environment, the integration tests point to my local database. Now, If I cancel the test run for any reason, that leaves a bunch of garbage data in my local database, because CleanupAllMockData() never gets called.
What is the best way to handle this? I couldn't find a way to intercept the test cancellation in MSTest.

I see two options for solving your problem:
Cleanup mock data before each start. Only before start.
Each test is wrapped as a db-transaction, which is never commited. I explain
this option here

Unit test fails when running all, but passes when running individually

I have about 12 unit tests for different scenarios, and I need to call one async method in these tests (sometimes multiple times in one test). When I do "Run all", 3 of them will always fail. If I run them one by one using "Run selected test", they will pass. The exception in output I'm getting is this:
System.AppDomainUnloadedException: Attempted to access an unloaded
AppDomain. This can happen if the test(s) started a thread but did not
stop it. Make sure that all the threads started by the test(s) are
stopped before completion.
I can't really share the code, as it's quite big and I don't know where to start, so here is example:
[TestMethod]
public async Task SampleTest()
{
var someProvider = new SomeProvider();
var result = await someProvider.IsSomethingValid();
Assert.IsTrue(result == SomeProvider.Status.Valid);
NetworkController.Disable();
result = await someProvider.IsSomethingValid();
Assert.IsTrue(result == SomeProvider.Status.Valid);
NetworkController.Enable();
}
EDIT:
The other 2 failing methods set time to the future and to the past respectively.
[TestMethod]
public async Task SetTimeToFutureTest()
{
var someProvider = new SomeProvider();
var today = TimeProvider.UtcNow().Date;
var result = await someProvider.IsSomethingValid();
Assert.IsTrue(result == SomeProvider.Status.Valid);
TimeProvider.SetDateTime(today.AddYears(1));
var result2 = await someProvider.IsSomethingValid();
Assert.IsTrue(result2 == SomeProvider.Status.Expired);
}
Where TimeProvider looks like this:
public static class TimeProvider
{
/// <summary> Normally this is a pass-through to DateTime.Now, but it can be overridden with SetDateTime( .. ) for testing or debugging.
/// </summary>
public static Func<DateTime> UtcNow = () => DateTime.UtcNow;
/// <summary> Set time to return when SystemTime.UtcNow() is called.
/// </summary>
public static void SetDateTime(DateTime newDateTime)
{
UtcNow = () => newDateTime;
}
public static void ResetDateTime()
{
UtcNow = () => DateTime.UtcNow;
}
}
EDIT 2:
[TestCleanup]
public void TestCleanup()
{
TimeProvider.ResetDateTime();
}
Other methods are similar, I will simulate time/date change, etc.
I tried calling the method synchronously by getting .Result() out of it, etc, but it didn't help. I read ton material on the web about this but still struggling.
Did anyone run into the same problem? Any tips will be highly appreciated.

I can't see what you're doing with your test initialization or cleanup but it could be that since all of your test methods are attempting to run asynchronously, the test runner is not allowing all tasks to finish before performing cleanup.
Are the same few methods failing when you run all of the tests or is it random? Are you sure you are doing unit testing and not integration testing? The class "NetworkController" gives me the impression that you may be doing more of an integration test. If that were the case and you are using a common class, provider, service, or storage medium (database, file system) then interactions or state changes caused by one method could affect another test method's efficacy.

When running tests in async/await mode, you will incur some lag. It looks like all your processing is happening in memory. They're probably passing one an one-by-one basis because the lag time is minimal. When running multiple in async mode, the lag time is sufficient to cause differentiation in the time results.
I've run into this before doing NUnit tests run by NCrunch where a DateTime component is being tested. You can mitigate this by reducing the scope of your validation / expiration logic to match to second instead of millisecond, as long as this is permissible within your acceptance criteria. I can't tell from your code what the logic is driving validation status or expiration date, but I'm willing to bet the async lag is the root cause of the test failure when run concurrently.

Both tests shown use the same static TimeProvider, thus interference by methods like ResetDateTime in the cleanup and TimeProvider.SetDateTime(today.AddYears(1)); in a test are to be expected. Also the NetworkController seems to be a static resource, and connecting/disconnecting it could interfere with your tests.
You can solve the issues in several ways:
get rid of static resources, use instances instead
lock the tests such that only one test can be run at a time
Aside from that, almost every test framework offers more than just Assert.IsTrue. Doesn't your framework offer an Assert.AreEqual? That improves readabilty. Also, with more than one Assert in a test, custom messages indicating which of the test failed (or that an Assert is for pre-condition, not the actual test) are recommended.

Attempt at an Asynchronous method is failing

I have an MVC3/.NET 4 application which uses Entity Framework (4.3.1 Code First)
I have wrapped EF into a Repository/UnitOfWork pattern as described here…
http://www.asp.net/mvc/tutorials/getting-started-with-ef-using-mvc/implementing-the-repository-and-unit-of-work-patterns-in-an-asp-net-mvc-application
Typically, as it explains in the article, when I require the creation of a new record I’ve been doing this…
public ActionResult Create(Course course)
{
unitOfWork.CourseRepository.Add(course);
unitOfWork.Save();
return RedirectToAction("Index");
}
However, when more than simply saving a record to a database is required I wrap the logic into what I’ve called an IService. For example…
private ICourseService courseService;
public ActionResult Create(Course course)
{
courseService.ProcessNewCourse(course);
return RedirectToAction("Index");
}
In one of my services I have something like the following…
public void ProcessNewCourse(Course course)
{
// Save the course to the database…
unitOfWork.CourseRepository.Add(course);
unitOfWork.Save();
// Generate a PDF that email some people about the new course being created, which requires more use of the unitOfWork…
var someInformation = unitOfWork.AnotherRepository.GetStuff();
var myPdfCreator = new PdfCreator();
IEnumerable<People> people = unitOfWork.PeopleRepository.GetAllThatWantNotifiying(course);
foreach(var person in people)
{
var message = “Hi ” + person.FullName;
var attachment = myPdfCreator.CreatePdf();
etc...
smtpClient.Send();
}
}
The above isn’t the actual code (my app has nothing to do with courses, I’m using view models, and I have separated the PDF creation and email message out into other classes) but the gist of what is going on is as above!
My problem is that the generation of the PDF and emailing it out is taking some time. The user just needs to know that the record has been saved to the database so I thought I would put the code below the unitOfWork.Save(); into an asynchronous method. The user can then be redirected and the server can happily take its time processing the emails, and attachments and whatever else I require it to do post save.
This is where I’m struggling.
I’ve tried a few things, the current being the following in ICourseService…
public class CourseService : ICourseService
{
private delegate void NotifyDelegate(Course course);
private NotifyDelegate notifyDelegate;
public CourseService()
{
notifyDelegate = new NotifyDelegate(this.Notify);
}
public void ProcessNewCourse(Course course)
{
// Save the course to the database…
unitOfWork.CourseRepository.Add(course);
unitOfWork.Save();
notifyDelegate.BeginInvoke(course);
}
private void Notify(Course course)
{
// All the stuff under unitOfWork.Save(); moved here.
}
}
My Questions/Problems
I’m randomly getting the error: "There is already an open DataReader associated with this Command which must be closed first." in the Notify() method.
Is it something to do with the fact that I’m trying to share the unitOrWork and therefore a dbContext across threads?
If so, can someone be kind enough to explain why this is a problem?
Should I be giving a new instance of unitOfWork to the Notify method?
Am I using the right patterns/classes to invoke the method asynchronously? Or should I be using something along the lines of....
new System.Threading.Tasks.Task(() => { Notify(course); }).Start();
I must say I've become very confused with the terms asynchronous, parallel, and concurrent!!
Any links to articles (c# async for idiots) would be appreciated!!
Many thanks.
UPDATE:
A little more digging got me to this SO page: https://stackoverflow.com/a/5491978/192999 which says...
"Be aware though that EF contexts are not thread safe, i.e. you cannot use the same context in more than one thread."
...so am I trying to achieve the impossible? Does this mean I should be creating a new IUnitOfWork instance for my new thread?

You could create a polling background thread that does the lengthy operation separately from your main flow. This thread could scan the database for new items (or items marked to process). This solution is pretty simple and ensures that jobs get done even if you application crashes (it will be picked up when the polling thread is started again).
You could also use a Synchronised Queue if it's not terrible if the request is 'lost', in the case your application crashes after the doc is requested and before it's generated/sent.
One thing is almost sure - as rikitikitik said - you will need to use a new unit of work, which means a separate transaction.
You could also look at Best threading queue example / best practice .

How do I write a unit test that relies on file system events?

I have the following code that I'd like to test:
public class DirectoryProcessor
{
public string DirectoryPath
{
get;
set;
}
private FileSystemWatcher watcher;
public event EventHandler<SourceEventArgs> SourceFileChanged;
protected virtual void OnSourceFileChanged(SourceEventArgs e)
{
EventHandler<SourceEventArgs> handler = SourceFileChanged;
if(handler != null)
{
handler(this, e);
}
}
public DirectoryProcessor(string directoryPath)
{
this.DirectoryPath = directoryPath;
this.watcher = new FileSystemWatcher(directoryPath);
this.watcher.Created += new FileSystemEventHandler(Created);
}
void Created(object sender, FileSystemEventArgs e)
{
// process the newly created file
// then raise my own event indicating that processing is done
OnSourceFileChanged(new SourceEventArgs(e.Name));
}
}
Basically, I want to write an NUnit test that will do the following:
Create a directory
Setup a DirectoryProcessor
Write some files to the directory (via File.WriteAllText())
Check that DirectoryProcessor.SourceFileChanged has fired once for each file added in step 3.
I tried doing this and adding Thread.Sleep() after step 3, but it's hard to get the timeout correct. It correctly processes the first file I write to the directory, but not the second (and that's with the timeout set to 60s). Even if I could get it working this way, it seems like a terrible way to write the test.
Does anyone have a good solution to this problem?

Typically, you are concerned with testing the interaction with the file system and there is no need to test the framework classes and methods that actually perform the operations.
If you introduce a layer of abstraction into your classes, you can then mock the file system in your unit tests to verify that the interactions are correct without actually manipulating the file system.
Outside of testing, the "real" implementation calls into those framework methods to get the work done.
Yes, in theory you'll need to integration test that "real" implementation, but it should in practice be low-risk, not subject to much change, and verifiable through a few minutes of manual testing. If you use an open source file system wrapper, it may include those tests for your peace of mind.
See How do you mock out the file system in C# for unit testing?

If you are looking to test another object that uses this class my answer is not relevant.
When I write unit tests to operations I prefer using the ManualResetEvent
The unit test will be something like:
...
DirectoryProcessor.SourceFileChanged+=onChanged;
manualResetEvent.Reset();
File.WriteAllText();
var actual = manualResetEvent.WaitOne(MaxTimeout);
...
when manualResetEvent is the ManualResetEvent and the MaxTimeout is some TimeSpan (my advice always use the time out).
now we are missing the "onChanged":
private void onChanged(object sender, SourceEventArgs e)
{
manualResetEvent.Set();
}
I hope this is helpful

How to unit test client network code?

I'm working on a piece of networking code which listens to a TCP connection, parses the incoming data and raises the appropriate event. Naturally, to avoid blocking the rest of the application, the listening and parsing are performed in a background worker. When trying to unit test this code I run into the problem that, seeing as the network code has more work to do than the unit test, the unit test completes before the adapter has a chance to raise the event and so the test fails.
Adapter class:
public class NetworkAdapter : NetworkAdapterBase //NetworkAdapterBase is just an abstract base class with event definitions and protected Raise... methods.
{
//Fields removed for brevity.
public NetworkAdapter(TcpClient tcpClient)
{
_tcpConnection = tcpClient;
//Hook up event handlers for background worker.
NetworkWorker.DoWork += NetworkWorker_DoWork;
if (IsConnected)
{
//Start up background worker.
NetworkWorker.RunWorkerAsync();
}
}
private void NetworkWorker_DoWork(object sender, DoWorkEventArgs e)
{
while (IsConnected)
{
//Listen for incoming data, parse, raise events...
}
}
}
Attempted test code:
[TestMethod]
public void _processes_network_data()
{
bool newConfigurationReceived = false;
var adapter = new NetworkAdapter(TestClient); //TestClient is just a TcpClient that is set up in a [TestInitialize] method.
adapter.ConfigurationDataReceived += (sender, config) =>
{
newConfigurationReceived = true;
};
//Send fake byte packets to TestClient.
Assert.IsTrue(newConfigurationReceived, "Results: Event not raised.");
}
How should I go about trying to test this sort of thing?
Thanks,
James

Well, first, this is not a strict "unit test"; your test depends upon layers of architecture that have side effects, in this case transmitting network packets. This is more of an integration test.
That said, your unit test could sleep for a certain number of millis, as Tony said. You could also see if you can get a handle to the background worker, and Join on it, which will cause your unit test to wait as long as it takes for the background worker to finish.

You could wait for some timeout period, then run the assertion, thusly:
//Send fake byte packets to TestClient
Thread.Sleep(TIMEOUT);
Assert.IsTrue(newConfigurationReceived, "Results: Event not raised.");
Where TIMEOUT is the number of milliseconds you want to wait.

You could use some timeout, but as always what duration should the timeout be to be sure you're test will always pass, but still not slow down your tests too much ?
I would simply test the parsing code apart. This is probably where you're going to have the most bugs, and where you most need unit tests. And it's simple to test !
Then for code that is listening on a socket ... well you could have bugs here ... but if it simply dispatches data to a function/class I'm not sure you really need to test it. And if you want to be really thorough, how are you gonna unit test that your class behaves well if the connection is lost between the client and the server for example ?

In our unit tests, we use .NET 4's parallelization library. You can say:
Parallel.Invoke(() => Dosomething(arguments), () => DosomethingElse(arguments));
And the framework will take care of spawning these actions as different threads, executing them in a number of threads ideal to the particular processes you're working on, and then joining them so that the next instruction doesn't execute until they've all finished.
However, it looks like you may not have direct access to the thread. Instead, you want to wait until the given callback method gets called. You can use an AutoResetEvent or a ManualResetEvent to accomplish this.
See Unit testing asynchronous function