I am trying to test a method with unit testing, however to also make sure internally it was executed in an expected way. Here is the method simplified; It returns a value from a database but also saves it to the cache, so if it's requested again within 5 seconds, the value will be retrieved from the cache and not the DB.
public static string GetValue()
{
var cache = HttpRuntime.Cache;
string value = (string)cache["test"];
// if value is null then it was never in cache or it expired.
if (value == null)
{
// Imagine here complex code that retrieves and sets "value"
value = "OK";
// Add it to cache to retrieve it faster if requested again within 5 sec.
cache.Add("test", value, null, DateTime.Now + TimeSpan.FromSeconds(5),
System.Web.Caching.Cache.NoSlidingExpiration, CacheItemPriority.NotRemovable, null);
Debug.WriteLine("From DB");
}
else
{
// Value was in cache, so it's ready to return
Debug.WriteLine("From Cache");
}
return value;
}
This basically returns the value "OK" from a hypothetical database. However, because it uses absolute expiration of HttpRuntime.Cache, if the value is requested again within 5 seconds, it's returned from the Cache and not from the DB.
Now my question is, how to write a TestMethod that not only verifies that it returns OK, but also it the caching logic is working correctly.
Notice that depending on whether it used the DB or the Cache to get the value, a corresponding debug line is added to debug output.
So the testing method should like something like this:
[TestMethod]
public void GetValue_OK()
{
Assert.IsTrue(Helpers.GetValue() == "OK");
Thread.Sleep(4000);
Assert.IsTrue(Helpers.GetValue() == "OK");
// Assert that it wrote "From Cache"
Assert.IsTrue(LastDebugLine().Contains("Cache"));
Thread.Sleep(2000);
Assert.IsTrue(Helpers.GetValue() == "OK");
// Assert that it wrote "From DB", because theoretically over 5 seconds passed
// So it has expired and the routine loaded it again from the DB.
Assert.IsTrue(LastDebugLine().Contains("DB"));
}
string LastDebugLine()
{
// Imagine a string named LastDebugOutput that contains the
// last line of output from Debug.WriteLine
// The code below retrieves somehow this line.
return String.Empty;
}
In the testmethod, I not only verify the correct output but also whether it was retrieved from the cache or the DB. I run the method 3 times, inserting delays between each retrieval. I want to also test if the value will be retrieved from the DB or the Cache.
The way I thought this could happen was with a method I call LastDebugLine() that retrieves the last debug info from the tested method. By reading it's status the test method knows the internals of the method so it can compare with the expected result.
Now my question has two parts:
1) What is the correct way to test all of this? Is my idea of using the debug output and checking it in the unit test correct? I could be very wrong here about the general concept so maybe this can be done a better way.
2) If however my concept is correct, what exactly should be the code in LastDebugLine() to get the last line from Debug.WriteLine?
Even if I am correct, there is still a problem as unit tests might run multithreaded, so reading the debug output like this might bring up unexpected results.
How to test this method correctly?
There are a couple of ways to approach this. To do the whole thing as an integration test you could
create a test which calls the method to put the value in the DB/Cache.
then I would have the test delete the value from the db
then call the method again.
If the value is got from the cache then you'll get the right value. if the method calls the db the test will fail.
As an alternative you could isolate the components which access the db and the cache and inject the objects which do this into your class under test. You could then provide mocks in your tests to assert that the objects are called in the right order.
It will be difficult in your current situation as you have a static method and so injecting the dependency is hard, but if you change your GetValue method so that is not static, you can pass some object that wraps your HttpRuntime.Cache and implements a similar interface, then pass a mock of that object and validate that when the object doesn't exist in the cache the other code is called and when it does exist in the cache it isn't called
If you can't change your model so that GetValue can't be static then you could look at using the MS Fakes framework or one of the other commercial products which can mock static methods.
Related
I have the following method in my WCF service. I need to do a unit test for it, but am unsure how to do this since it returns a list of objects Mountain.
The method takes in a grid reference e.g. NN - and searches the list mountslist for those mountains with grid reference starting with NN.
Also can anyone help me as to how I can do a try catch for this? I am confused as to what to return since it expects a Mountain object. (If not found return - mountain not in list for example).
public IEnumerable<Mountain> GetMountainLoc(string mtloc)
{
IEnumerable<Mountain> resultMts =
mountsList
.Where(x => x.Grid_ref.Substring(0, 2) == mtloc)
.ToList();
return resultMts;
}
In this case your method is simply applying a filter againsts a list of type IEnumerable<Mountain>, held in a variable named mountsList.
So one way to test this, then, is to make assertions against the filtered list returned by the method. However, in order to do this, you need to know the state of the variable mountsList at the time the unit test runs.
So how does this variable get set in the containing class? Is it passed in? Is it constructed by some other means? Whichever, unless you know the state of this list at test time, you will need to supplant or otherwise inject a known representation of this list so that you can make accurate assertions against it.
Im running some tests on my code at the moment. My main test method is used to verify some data, but within that check there is a lot of potential for it to fail at any one point.
Right now, I've set up multiple Assert.Fail statements within my method and when the test is failed, the message I type is displayed as expected. However, if my method fails multiple times, it only shows the first error. When I fix that, it is only then I discover the second error.
None of my tests are dependant on any others that I'm running. Ideally what I'd like is the ability to have my failure message to display every failed message in one pass. Is such a thing possible?
As per the comments, here are how I'm setting up a couple of my tests in the method:
private bool ValidateTestOne(EntityModel.MultiIndexEntities context)
{
if (context.SearchDisplayViews.Count() != expectedSdvCount)
{
Assert.Fail(" Search Display View count was different from what was expected");
}
if (sdv.VirtualID != expectedSdVirtualId)
{
Assert.Fail(" Search Display View virtual id was different from what was expected");
}
if (sdv.EntityType != expectedSdvEntityType)
{
Assert.Fail(" Search Display View entity type was different from what was expected");
}
return true;
}
Why not have a string/stringbuilder that holds all the fail messages, check for its length at the end of your code, and pass it into Assert.Fail? Just a suggestion :)
The NUnit test runner (assuming thats what you are using) is designed to break out of the test method as soon as anything fails.
So if you want every failure to show up, you need to break up your test into smaller, single assert ones. In general, you only want to be testing one thing per test anyways.
On a side note, using Assert.Fail like that isn't very semantically correct. Consider using the other built-in methods (like Assert.Equals) and only using Assert.Fail when the other methods are not sufficient.
None of my tests are dependent on any others that I'm running. Ideally
what I'd like is the ability to have my failure message to display
every failed message in one pass. Is such a thing possible?
It is possible only if you split your test into several smaller ones.
If you are afraid code duplication which is usually exists when tests are complex, you can use setup methods. They are usually marked by attributes:
NUnit - SetUp,
MsTest - TestInitialize,
XUnit - constructor.
The following code shows how your test can be rewritten:
public class HowToUseAsserts
{
int expectedSdvCount = 0;
int expectedSdVirtualId = 0;
string expectedSdvEntityType = "";
EntityModelMultiIndexEntities context;
public HowToUseAsserts()
{
context = new EntityModelMultiIndexEntities();
}
[Fact]
public void Search_display_view_count_should_be_the_same_as_expected()
{
context.SearchDisplayViews.Should().HaveCount(expectedSdvCount);
}
[Fact]
public void Search_display_view_virtual_id_should_be_the_same_as_expected()
{
context.VirtualID.Should().Be(expectedSdVirtualId);
}
[Fact]
public void Search_display_view_entity_type_should_be_the_same_as_expected()
{
context.EntityType.Should().Be(expectedSdvEntityType);
}
}
So your test names could provide the same information as you would write as messages:
Right now, I've set up multiple Assert.Fail statements within my
method and when the test is failed, the message I type is displayed as
expected. However, if my method fails multiple times, it only shows
the first error. When I fix that, it is only then I discover the
second error.
This behavior is correct and many testing frameworks follow it.
I'd like to recommend stop using Assert.Fail() because it forces you to write specific messages for every failure. Common asserts provide good enough messages so you can replace you code with the following lines:
// Act
var context = new EntityModelMultiIndexEntities();
// Assert
Assert.Equal(expectedSdvCount, context.SearchDisplayViews.Count());
Assert.Equal(expectedSdVirtualId, context.VirtualID);
Assert.Equal(expectedSdvEntityType, context.EntityType);
But I'd recommend start using should-frameworks like Fluent Assertions which make your code mere readable and provide better output.
// Act
var context = new EntityModelMultiIndexEntities();
// Assert
context.SearchDisplayViews.Should().HaveCount(expectedSdvCount);
context.VirtualID.Should().Be(expectedSdVirtualId);
context.EntityType.Should().Be(expectedSdvEntityType);
When writing a test for persistently stored data, I come up with a test along the lines of:
[TestMethod]
public void DoCreateDeleteTest() {
PersistentDataStore pds = new PersistentDataStore();
bool createSuccess = pds.Save("id", "payload");
Assert.AreEqual(true, createSuccess);
bool deleteSuccess = pds.Delete("id");
Assert.AreEqual(true, deleteSuccess);
}
As long as everything works, this seems fine. The function has no prior dependencies and it cleans up after itself. The problem is: when the .Save() method performs the save but returns false/failure. The assertion fires and the delete is not called so it doesn't clean up after itself.
After this, there is persisted data in the database with name "id" and all future saves fail.
The only way I can think to get around it is to do a precautionary delete before the save, but that seems like way to large a hack.
Put the delete in a method marked with the TestCleanup attribute (I assume you are using MSTest).
By the way your test is also testing two different things: whether the save works and it also tests the delete. Tests should only test one thing at a time.
Wrap both within the one transaction? Do a delete in a catch?
What is the best way to unit test a method that doesn't return anything? Specifically in c#.
What I am really trying to test is a method that takes a log file and parses it for specific strings. The strings are then inserted into a database. Nothing that hasn't been done before but being VERY new to TDD I am wondering if it is possible to test this or is it something that doesn't really get tested.
If a method doesn't return anything, it's either one of the following
imperative - You're either asking the object to do something to itself.. e.g change state (without expecting any confirmation.. its assumed that it will be done)
informational - just notifying someone that something happened (without expecting action or response) respectively.
Imperative methods - you can verify if the task was actually performed. Verify if state change actually took place. e.g.
void DeductFromBalance( dAmount )
can be tested by verifying if the balance post this message is indeed less than the initial value by dAmount
Informational methods - are rare as a member of the public interface of the object... hence not normally unit-tested. However if you must, You can verify if the handling to be done on a notification takes place. e.g.
void OnAccountDebit( dAmount ) // emails account holder with info
can be tested by verifying if the email is being sent
Post more details about your actual method and people will be able to answer better.
Update: Your method is doing 2 things. I'd actually split it into two methods that can now be independently tested.
string[] ExamineLogFileForX( string sFileName );
void InsertStringsIntoDatabase( string[] );
String[] can be easily verified by providing the first method with a dummy file and expected strings. The second one is slightly tricky.. you can either use a Mock (google or search stackoverflow on mocking frameworks) to mimic the DB or hit the actual DB and verify if the strings were inserted in the right location. Check this thread for some good books... I'd recomment Pragmatic Unit Testing if you're in a crunch.
In the code it would be used like
InsertStringsIntoDatabase( ExamineLogFileForX( "c:\OMG.log" ) );
Test its side-effects. This includes:
Does it throw any exceptions? (If it should, check that it does. If it shouldn't, try some corner cases which might if you're not careful - null arguments being the most obvious thing.)
Does it play nicely with its parameters? (If they're mutable, does it mutate them when it shouldn't and vice versa?)
Does it have the right effect on the state of the object/type you're calling it on?
Of course, there's a limit to how much you can test. You generally can't test with every possible input, for example. Test pragmatically - enough to give you confidence that your code is designed appropriately and implemented correctly, and enough to act as supplemental documentation for what a caller might expect.
As always: test what the method is supposed to do!
Should it change global state (uuh, code smell!) somewhere?
Should it call into an interface?
Should it throw an exception when called with the wrong parameters?
Should it throw no exception when called with the right parameters?
Should it ...?
Try this:
[TestMethod]
public void TestSomething()
{
try
{
YourMethodCall();
Assert.IsTrue(true);
}
catch {
Assert.IsTrue(false);
}
}
Void return types / Subroutines are old news. I haven't made a Void return type (Unless I was being extremely lazy) in like 8 years (From the time of this answer, so just a bit before this question was asked).
Instead of a method like:
public void SendEmailToCustomer()
Make a method that follows Microsoft's int.TryParse() paradigm:
public bool TrySendEmailToCustomer()
Maybe there isn't any information your method needs to return for usage in the long-run, but returning the state of the method after it performs its job is a huge use to the caller.
Also, bool isn't the only state type. There are a number of times when a previously-made Subroutine could actually return three or more different states (Good, Normal, Bad, etc). In those cases, you'd just use
public StateEnum TrySendEmailToCustomer()
However, while the Try-Paradigm somewhat answers this question on how to test a void return, there are other considerations too. For example, during/after a "TDD" cycle, you would be "Refactoring" and notice you are doing two things with your method... thus breaking the "Single Responsibility Principle." So that should be taken care of first. Second, you might have idenetified a dependency... you're touching "Persistent" Data.
If you are doing the data access stuff in the method-in-question, you need to refactor into an n-tier'd or n-layer'd architecture. But we can assume that when you say "The strings are then inserted into a database", you actually mean you're calling a business logic layer or something. Ya, we'll assume that.
When your object is instantiated, you now understand that your object has dependencies. This is when you need to decide if you are going to do Dependency Injection on the Object, or on the Method. That means your Constructor or the method-in-question needs a new Parameter:
public <Constructor/MethodName> (IBusinessDataEtc otherLayerOrTierObject, string[] stuffToInsert)
Now that you can accept an interface of your business/data tier object, you can mock it out during Unit Tests and have no dependencies or fear of "Accidental" integration testing.
So in your live code, you pass in a REAL IBusinessDataEtc object. But in your Unit Testing, you pass in a MOCK IBusinessDataEtc object. In that Mock, you can include Non-Interface Properties like int XMethodWasCalledCount or something whose state(s) are updated when the interface methods are called.
So your Unit Test will go through your Method(s)-In-Question, perform whatever logic they have, and call one or two, or a selected set of methods in your IBusinessDataEtc object. When you do your Assertions at the end of your Unit Test you have a couple of things to test now.
The State of the "Subroutine" which is now a Try-Paradigm method.
The State of your Mock IBusinessDataEtc object.
For more information on Dependency Injection ideas on the Construction-level... as they pertain to Unit Testing... look into Builder design patterns. It adds one more interface and class for each current interface/class you have, but they are very tiny and provide HUGE functionality increases for better Unit-Testing.
You can even try it this way:
[TestMethod]
public void ReadFiles()
{
try
{
Read();
return; // indicates success
}
catch (Exception ex)
{
Assert.Fail(ex.Message);
}
}
it will have some effect on an object.... query for the result of the effect. If it has no visible effect its not worth unit testing!
Presumably the method does something, and doesn't simply return?
Assuming this is the case, then:
If it modifies the state of it's owner object, then you should test that the state changed correctly.
If it takes in some object as a parameter and modifies that object, then your should test the object is correctly modified.
If it throws exceptions is certain cases, test that those exceptions are correctly thrown.
If its behaviour varies based on the state of its own object, or some other object, preset the state and test the method has the correct Ithrough one of the three test methods above).
If youy let us know what the method does, I could be more specific.
Use Rhino Mocks to set what calls, actions and exceptions might be expected. Assuming you can mock or stub out parts of your method. Hard to know without knowing some specifics here about the method, or even context.
Depends on what it's doing. If it has parameters, pass in mocks that you could ask later on if they have been called with the right set of parameters.
What ever instance you are using to call the void method , You can just use ,Verfiy
For Example:
In My case its _Log is the instance and LogMessage is the method to be tested:
try
{
this._log.Verify(x => x.LogMessage(Logger.WillisLogLevel.Info, Logger.WillisLogger.Usage, "Created the Student with name as"), "Failure");
}
Catch
{
Assert.IsFalse(ex is Moq.MockException);
}
Is the Verify throws an exception due to failure of the method the test would Fail ?
I'm using MSTEST inside Visual Studio 2008. How can I have each unit test method in a certain test class act as if it were the first test to run so that all global state is reset before running each test? I do not want to explicitly clean up the world using TestInitialize, ClassInitialize, AssemblyInitialize, etc. For example:
[TestClass]
public class MyClassTests
{
[TestMethod]
public void Test1()
{
// The "Instance" property creates a new instance of "SomeSingleton"
// if it hasn't been created before.
var i1 = SomeSingleton.Instance;
...
}
[TestMethod]
public void Test2()
{
// When I select "Test1" and "Test2" to run, I'd like Test2
// to have a new AppDomain feel so that the static variable inside
// of "SomeSingleton" is reset (it was previously set in Test1) on
// the call to ".Instance"
var i2 = SomeSingleton.Instance;
// some code
}
Although a similar question appeared on this topic, it only clarified that tests do not run in parallel. I realize that tests run serially, but there doesn't seem to be a way to explicitly force a new AppDomain for each method (or something equivalent to clear all state).
Ideally, I'd like to specify this behavior for only a small subset of my unit tests so that I don't have to pay the penalty of a new AppDomain creation for tests that don't care about global state (the vast majority of my tests).
In the end, I wrote a helper that used AppDomain.CreateDomain and then used reflection to call the unit test under a different AppDomain. It provides the isolation I needed.
This post on MSDN's forums shows how to handle the situation if you only have a few statics that need to be reset. It does mention some options (e.g. using Reflection and PrivateType ).
I continue to welcome any further ideas, especially if I'm missing something obvious about MSTEST.
Add a helper in your tests that uses reflection to delete the singleton instance (you can add a reset method to the singleton as well, but I would be concerned about its use). Something like:
public static class SingletonHelper {
public static void CleanDALFactory()
{
typeof(DalFactory)
.GetField("_instance",BindingFlags.Static | BindingFlags.NonPublic)
.SetValue(null, null);
}
}
Call this in your TestInitialize method. [ I know this is "cleaning up the world", but you only have to write the method once in a helper per singleton, its very trivial and gives you explicit control ]
I think you are looking for the TestIntialize attribute and the TestCleanUp attribute. Here is an MSDN blog showing the execution orderlink text
We had a similar issue arise with our MSTests. We handled it by calling a function at the beginning and end of the specific tests that needed it.
We are storing a test expiration date in our app configuration. Three tests needed this date to fall into a specific range to determine the appropriate values. The way our application is set up, the configuration values would only be reset if there was not a value assigned in session. So, we created two new private static functions - one to explicitly set the configuration value to a specified date and one to clear that date from session after the test runs. In our three tests, we called these two functions. When the next test runs, the application sees an empty value for the date and refetches it from the configuration file.
I'm not sure if that's helpful, but that was how we worked around our similar issue.