I'm working on a piece of networking code which listens to a TCP connection, parses the incoming data and raises the appropriate event. Naturally, to avoid blocking the rest of the application, the listening and parsing are performed in a background worker. When trying to unit test this code I run into the problem that, seeing as the network code has more work to do than the unit test, the unit test completes before the adapter has a chance to raise the event and so the test fails.
Adapter class:
public class NetworkAdapter : NetworkAdapterBase //NetworkAdapterBase is just an abstract base class with event definitions and protected Raise... methods.
{
//Fields removed for brevity.
public NetworkAdapter(TcpClient tcpClient)
{
_tcpConnection = tcpClient;
//Hook up event handlers for background worker.
NetworkWorker.DoWork += NetworkWorker_DoWork;
if (IsConnected)
{
//Start up background worker.
NetworkWorker.RunWorkerAsync();
}
}
private void NetworkWorker_DoWork(object sender, DoWorkEventArgs e)
{
while (IsConnected)
{
//Listen for incoming data, parse, raise events...
}
}
}
Attempted test code:
[TestMethod]
public void _processes_network_data()
{
bool newConfigurationReceived = false;
var adapter = new NetworkAdapter(TestClient); //TestClient is just a TcpClient that is set up in a [TestInitialize] method.
adapter.ConfigurationDataReceived += (sender, config) =>
{
newConfigurationReceived = true;
};
//Send fake byte packets to TestClient.
Assert.IsTrue(newConfigurationReceived, "Results: Event not raised.");
}
How should I go about trying to test this sort of thing?
Thanks,
James
Well, first, this is not a strict "unit test"; your test depends upon layers of architecture that have side effects, in this case transmitting network packets. This is more of an integration test.
That said, your unit test could sleep for a certain number of millis, as Tony said. You could also see if you can get a handle to the background worker, and Join on it, which will cause your unit test to wait as long as it takes for the background worker to finish.
You could wait for some timeout period, then run the assertion, thusly:
//Send fake byte packets to TestClient
Thread.Sleep(TIMEOUT);
Assert.IsTrue(newConfigurationReceived, "Results: Event not raised.");
Where TIMEOUT is the number of milliseconds you want to wait.
You could use some timeout, but as always what duration should the timeout be to be sure you're test will always pass, but still not slow down your tests too much ?
I would simply test the parsing code apart. This is probably where you're going to have the most bugs, and where you most need unit tests. And it's simple to test !
Then for code that is listening on a socket ... well you could have bugs here ... but if it simply dispatches data to a function/class I'm not sure you really need to test it. And if you want to be really thorough, how are you gonna unit test that your class behaves well if the connection is lost between the client and the server for example ?
In our unit tests, we use .NET 4's parallelization library. You can say:
Parallel.Invoke(() => Dosomething(arguments), () => DosomethingElse(arguments));
And the framework will take care of spawning these actions as different threads, executing them in a number of threads ideal to the particular processes you're working on, and then joining them so that the next instruction doesn't execute until they've all finished.
However, it looks like you may not have direct access to the thread. Instead, you want to wait until the given callback method gets called. You can use an AutoResetEvent or a ManualResetEvent to accomplish this.
See Unit testing asynchronous function
Related
I have about 12 unit tests for different scenarios, and I need to call one async method in these tests (sometimes multiple times in one test). When I do "Run all", 3 of them will always fail. If I run them one by one using "Run selected test", they will pass. The exception in output I'm getting is this:
System.AppDomainUnloadedException: Attempted to access an unloaded
AppDomain. This can happen if the test(s) started a thread but did not
stop it. Make sure that all the threads started by the test(s) are
stopped before completion.
I can't really share the code, as it's quite big and I don't know where to start, so here is example:
[TestMethod]
public async Task SampleTest()
{
var someProvider = new SomeProvider();
var result = await someProvider.IsSomethingValid();
Assert.IsTrue(result == SomeProvider.Status.Valid);
NetworkController.Disable();
result = await someProvider.IsSomethingValid();
Assert.IsTrue(result == SomeProvider.Status.Valid);
NetworkController.Enable();
}
EDIT:
The other 2 failing methods set time to the future and to the past respectively.
[TestMethod]
public async Task SetTimeToFutureTest()
{
var someProvider = new SomeProvider();
var today = TimeProvider.UtcNow().Date;
var result = await someProvider.IsSomethingValid();
Assert.IsTrue(result == SomeProvider.Status.Valid);
TimeProvider.SetDateTime(today.AddYears(1));
var result2 = await someProvider.IsSomethingValid();
Assert.IsTrue(result2 == SomeProvider.Status.Expired);
}
Where TimeProvider looks like this:
public static class TimeProvider
{
/// <summary> Normally this is a pass-through to DateTime.Now, but it can be overridden with SetDateTime( .. ) for testing or debugging.
/// </summary>
public static Func<DateTime> UtcNow = () => DateTime.UtcNow;
/// <summary> Set time to return when SystemTime.UtcNow() is called.
/// </summary>
public static void SetDateTime(DateTime newDateTime)
{
UtcNow = () => newDateTime;
}
public static void ResetDateTime()
{
UtcNow = () => DateTime.UtcNow;
}
}
EDIT 2:
[TestCleanup]
public void TestCleanup()
{
TimeProvider.ResetDateTime();
}
Other methods are similar, I will simulate time/date change, etc.
I tried calling the method synchronously by getting .Result() out of it, etc, but it didn't help. I read ton material on the web about this but still struggling.
Did anyone run into the same problem? Any tips will be highly appreciated.
I can't see what you're doing with your test initialization or cleanup but it could be that since all of your test methods are attempting to run asynchronously, the test runner is not allowing all tasks to finish before performing cleanup.
Are the same few methods failing when you run all of the tests or is it random? Are you sure you are doing unit testing and not integration testing? The class "NetworkController" gives me the impression that you may be doing more of an integration test. If that were the case and you are using a common class, provider, service, or storage medium (database, file system) then interactions or state changes caused by one method could affect another test method's efficacy.
When running tests in async/await mode, you will incur some lag. It looks like all your processing is happening in memory. They're probably passing one an one-by-one basis because the lag time is minimal. When running multiple in async mode, the lag time is sufficient to cause differentiation in the time results.
I've run into this before doing NUnit tests run by NCrunch where a DateTime component is being tested. You can mitigate this by reducing the scope of your validation / expiration logic to match to second instead of millisecond, as long as this is permissible within your acceptance criteria. I can't tell from your code what the logic is driving validation status or expiration date, but I'm willing to bet the async lag is the root cause of the test failure when run concurrently.
Both tests shown use the same static TimeProvider, thus interference by methods like ResetDateTime in the cleanup and TimeProvider.SetDateTime(today.AddYears(1)); in a test are to be expected. Also the NetworkController seems to be a static resource, and connecting/disconnecting it could interfere with your tests.
You can solve the issues in several ways:
get rid of static resources, use instances instead
lock the tests such that only one test can be run at a time
Aside from that, almost every test framework offers more than just Assert.IsTrue. Doesn't your framework offer an Assert.AreEqual? That improves readabilty. Also, with more than one Assert in a test, custom messages indicating which of the test failed (or that an Assert is for pre-condition, not the actual test) are recommended.
If you call the Start()-Method of a MyClass-Object the Object will start sending data with the DataEvent.
class MyClass {
// Is called everytime new Data comes
public event DataEventHandler DataEvent;
// Starts de Data Process
public void StartDataDelivery()
{
}
}
How do I write a Test for that functionality if i can Guarantee that the DataEvent will be Invoked at least three times during a fix time period.
I haven't done any asynchronous Unittests yet. How is that done, assuming that someone else needs to understand the test later?
MSTest hasn't had any serious updates for some time and I don't see that changing.
I'd strongly recommend moving to xUnit. It supports async tests (just return a Task from the test and await to your heart's content), and is used by many new Microsoft projects.
I am using a System.IO.FileSystemWatcher in one of my services. I want to test that when a file being monitored is changed, I get a notification.
I was thinking about having a background thread change the file. In the test, I would join on that thread. Then I can assert that the correct events are called. I could subscribe a callback to capture if the event was called.
I have not done any testing involving threads so I am not sure if this is the best way to handle it or if there are some built in ways in Moq or MSpec that will help test.
Moq or MSpec don't have anything specifically built in that will help you do this, except for some interesting syntax or features that will help you organize your test. I think you're on the right path.
I'm curious how your service exposes file changed notifications. Does it expose them publicly for testing? Or is the FileSystemWatcher entirely hidden inside the service? If the service doesn't simply pass the event notification up and out, you should extract your file monitoring so that it can be easily tested.
You can do that with .NET events or callbacks or whatever. No matter how you go about it, I would write the test something like this...
[Subject("File monitoring")]
public class When_a_monitored_file_is_changed
{
Establish context = () =>
{
// depending on your service file monitor design, you would
// attach to your notification
_monitor.FileChanged += () => _changed.Set();
// or pass your callback in
_monitor = new ServiceMonitor(() => _changed.Set());
}
Because of = () => // modify the monitored file;
// Wait a reasonable amount of time for the notification to fire, but not too long that your test is a burden
It should_raise_the_file_changed_event = () => _changed.WaitOne(TimeSpan.FromMilliseconds(100)).ShouldBeTrue();
private static readonly ManualResetEvent _changed = new ManualResetEvent();
}
I'm testing an application. A [TearDown] methods contains another method which sends a request a server. This one is pretty slow. Meanwhile a server is unable to handle more than 3 requests at the same time.
So I decided to use a semaphore.
[TestFixture]
public class TestBase
{
private const int MaxThreadsCount = 3;
private readonly Semaphore _semaphore = new Semaphore(MaxThreadsCount, MaxThreadsCount);
[SetUp]
public virtual void Setup()
{
}
[TearDown]
public void CleanUp()
{
//...some code
new Thread(_ => SendRequestAsync("url/of/a/server", parameters)).Start();
}
private void SendRequestAsync(string url, NameValueCollection parameters)
{
_semaphore.WaitOne();
string result = MyServerHelper.SendRequest(url, parameters);
Assert.That(string.IsNullOrEmpty(result), Is.False, "SendRequest returned false");
}
[Test]
public void Test01()
{
Assert.AreEqual(1, 1);
}
[Test]
public void Test02()
{
Assert.AreEqual(1, 1);
}
[Test]
public void Test03()
{
Assert.AreEqual(1, 1);
}
//...........................
[Test]
public void TestN()
{
Assert.AreEqual(1, 1);
}
}
However it seems like it does not work properly. Now in log file on a server there are no records, which means a server does not receive any requests.
1) What did I do wrong?
2) How do I initialize a semaphore:
private readonly Semaphore _semaphore = new Semaphore(MaxThreadsCount, MaxThreadsCount);
or
private readonly Semaphore _semaphore = new Semaphore(0, MaxThreadsCount);
1) What did I do wrong?
The test runner is probably ending the test process before the thread has finished (or even started). You can verify it using something like Fiddler to verify there is no communication between the test and the server.
Is there a reason why you need to run it in a separate thread? Unless you are specifically testing threaded code, avoid it because it just creates complexities. Call it as you would normally. It also means any exception or error thrown will be caught by the test runner and reported in the test results.
If a test is taking too long (and you cannot fix the root cause like the server speed) consider an IoC container like AutoFac or reference counting to share this between tests that need it. Also consider running the tests in parallel, too.
2) How do I initialize a semaphore:
The first argument to the Semaphore class constructor is the initial number of requests. In this case, you probably want to initialize it to 0 since you have no requests running initially.
Note that a semaphore may help the test not send more than three requests but it will not help the server if tests are running concurrently but you probably realise that.
My understanding is that unit tests should be testing small units of functionality. You shouldn't need to create multiple threads to get your tests working. If you have slow external dependencies (like a network connection or database); you can define interfaces to abstract that away.
You should be able to test the behaviour you want without the threads. We can just assume that threads work - we're worried about the code you have. Presumably, somewhere in your code you have a counter that indicates the number of active connections or requests; or some other way of determining if you can accept another connection.
You want to test what happens when a request comes in, when you are already at the max.
So write a test that does that. Set that counter to the max, call the open connection code, and verify that it fails with the error you expect.
Needed:
A Windows Service That Executes Jobs from a Job Queue in a DB
Wanted:
Example Code, Guidance, or Best Practices for this type of Application
Background:
A user will click on an ashx link that will insert a row into the DB.
I need my windows service to periodically poll for rows in this table, and it should execute a unit of work for each row.
Emphasis:
This isn't completely new terrain for me.
EDIT: You can assume that I know how to create a Windows Service and basic data access.
But I need to write this service from scratch.
And I'd just like to know upfront what I need to consider.
EDIT: I'm most worried about jobs that fail, contention for jobs, and keeping the service running.
Given that you are dealing with a database queue, you have a fair cut of the job already done for you due to the transactional nature of databases. Typical queue driven application has a loop that does:
while(1) {
Start transction;
Dequeue item from queue;
process item;
save new state of item;
commit;
}
If processing crashes midway, the transaction rolls back and the item is processed on the next service start up.
But writing queues in a database is actually a lot trickier than you believe. If you deploy a naive approach, you'll find out that your enqueue and dequeue are blocking each other and the ashx page becomes unresponsive. Next you'll discover the dequeue vs. dequeue are deadlocking and your loop is constantly hitting error 1205. I strongly urge you to read this article Using Tables as Queues.
Your next challenge is going to be getting the pooling rate 'just right'. Too aggressive and your database will be burning hot from the pooling requests. Too lax and your queue will grow at rush hours and will drain too slowly. You should consider using an entirely different approach: use a SQL Server built-in QUEUE object and rely on the magic of the WAITFOR(RECEIVE) semantics. This allows for completely poll free self load tuning service behavior. Actually, there is more: you don't need a service to start with. See Asynchronous Procedures Execution for an explanation on what I'm talking about: launching processing asynchronously in SQL Server from a web service call, in a completely reliable manner. And finally, if the logic must be in C# process then you can leverage the External Activator, which allows the processing to be hosted in standalone processes as opposed to T-SQL procedures.
First you'll need to consider
How often to poll for
Does your service just stop and start or does it support pause and continue.
Concurrency. Services can increase the likelihood of a encountering a problem
Implementation
Use a System.Timers.Timer not a Threading.Timer
Maker sure you set the Timer.AutoReset to false. This will stop the reentrant problem.
Make sure to include execution time
Here's the basic framework of all those ideas. It includes a way to debug this which is a pain
public partial class Service : ServiceBase{
System.Timers.Timer timer;
public Service()
{
timer = new System.Timers.Timer();
//When autoreset is True there are reentrancy problme
timer.AutoReset = false;
timer.Elapsed += new System.Timers.ElapsedEventHandler(DoStuff);
}
private void DoStuff(object sender, System.Timers.ElapsedEventArgs e)
{
Collection stuff = GetData();
LastChecked = DateTime.Now;
foreach (Object item in stuff)
{
try
{
item.Dosomthing()
}
catch (System.Exception ex)
{
this.EventLog.Source = "SomeService";
this.EventLog.WriteEntry(ex.ToString());
this.Stop();
}
TimeSpan ts = DateTime.Now.Subtract(LastChecked);
TimeSpan MaxWaitTime = TimeSpan.FromMinutes(5);
if (MaxWaitTime.Subtract(ts).CompareTo(TimeSpan.Zero) > -1)
timer.Interval = MaxWaitTime.Subtract(ts).TotalMilliseconds;
else
timer.Interval = 1;
timer.Start();
}
protected override void OnPause()
{
base.OnPause();
this.timer.Stop();
}
protected override void OnContinue()
{
base.OnContinue();
this.timer.Interval = 1;
this.timer.Start();
}
protected override void OnStop()
{
base.OnStop();
this.timer.Stop();
}
protected override void OnStart(string[] args)
{
foreach (string arg in args)
{
if (arg == "DEBUG_SERVICE")
DebugMode();
}
#if DEBUG
DebugMode();
#endif
timer.Interval = 1;
timer.Start();
}
private static void DebugMode()
{
Debugger.Break();
}
}
EDIT Fixed loop in Start()
EDIT Turns out Milliseconds is not the same as TotalMilliseconds
You may want to have a look at Quartz.Net to manage scheduling the jobs. Not sure if it will fit your particular situation, but it's worth a look.
Some things I can think of, based on your edit:
Re: job failure:
Determine whether a job can be retried and do one of the following:
Move the row to an "error" table for logging / reporting later OR
Leave the row in the queue so that it will be reprocessed by the job service
You could add a column like WaitUntil or something similar to delay retrying the job after a failure
Re: contention:
Add a timestamp column such as "JobStarted" or "Locked" to track when the job was started. This will prevent other threads (assuming your service is multithreaded) from trying to execute the job simultaneously.
You'll need to have some cleanup process that goes through and clears stale jobs for re-processing (in the event the job service fails and your lock is never released).
Re: keeping the service running
You can tell windows to restart a service if it fails.
You can detect previous failure upon startup by keeping some kind of file open while the service is running and deleting it upon successful shutdown. If your service starts up and that file already exists, you know the service previously failed and can alert an operator or perform the necessary cleanup operations.
I'm really just poking around in the dark here. I'd strongly suggest prototyping the service and returning with any specific questions about the way it functions.