This code should takes a screenshot when test fail:
[TestClass]
public class UnitTest1
{
[OneTimeTearDown]
public void TestFail()
{
IWebDriver driver = new ChromeDriver();
if (NUnit.Framework.TestContext.CurrentContext.Result.Outcome != ResultState.Success)
{
string screensLocation = #"D:\";
string testName = NUnit.Framework.TestContext.CurrentContext.Test.Name;
var screenshot = ((ITakesScreenshot)driver).GetScreenshot();
screenshot.SaveAsFile(screensLocation + testName + ".png");
}
}
[TestMethod]
public void TestMethod1()
{
// my code, here test is failed
}
}
But it is not working. I don't have any screen in location D:\
Otherwise is there a way to debug code under OneTimeTearDown Attribute? Because when the test is fail, debugging ends and I don't know what's going on in the method TestFail().
Thanks for your help.
OneTimeTearDownAttribute is a feature of NUnit.
Although your tag says "nunit", your code is not actually using it. TestClassAttribute and TestMethodAttribute are features of MS Test. If you tried to run this test with NUnit, it would not recognize the tests at all.
Obviously, your test assembly does reference the NUnit framework, since it would not otherwise compile.
So... bottom line, your test code references two different frameworks in such a way that it cannot be run successfully by either runner!!! You have to choose which of the two you want to use, remove the other reference and use a runner for the framework you choose to keep.
I am using NUnit to test one functionality where I need to load XML file to object. The XML file is in location of the Console Application.
I have Following method where configuration will be read :
public string GetConfiguration(TempFlexProcessor processor)
{
var exePath = Path.GetDirectoryName(System.Reflection.Assembly.GetEntryAssembly().Location);
var configPath = Path.Combine(Path.GetFullPath(exePath), "configuration");
var configFile = string.Format(#"{0}.xml", processor.GetType().Name);
}
Now in my NUnit Test I have test method where I test GetConfiguration :
[Test]
public void TempFlexProcessorExecuteTest()
{
#region Given
#endregion
#region When
var tempFlexProcessor = new TempFlexProcessor();
var actual = tempFlexProcessor.GetConfiguration(tempFlexProcessor);
#endregion
Assert.AreEqual("path of the file", actual);
}
But System.Reflection.Assembly.GetEntryAssembly() is null, please help.
I used AppDomain.CurrentDomain.BaseDirectory instead of System.Reflection.Assembly.GetEntryAssembly().Location
I suspect the problem is that NUnit is running your tests in a different AppDomain, but without using ExecuteAssembly. From the documentation for Assembly.GetEntryAssembly:
Gets the process executable in the default application domain. In other application domains, this is the first executable that was executed by AppDomain.ExecuteAssembly.
It's not clear which assembly you really want to get - even if this did return something "appropriate" for NUnit, that's likely to be the NUnit executable, which would be well away from any configuration directories you happen to have.
Basically, I think that you should at least provide an alternative way of specifying the configuration directory - and you might want to reconsider whether using GetEntryAssembly is a good idea anyway. (Aside from anything else, it's slightly odd that you're calling GetConfiguration on a processor and passing in another processor... that may be suitable for your design, but it's at least somewhat unusual, given that in your test case you're passing in a reference to the same object.)
I recently asked a question but but did not get an answer that I could act on. I think this was due to the long code sample included. I have decided to post another question with a much smaller code sample. I want to unit test the below method to make sure that it does work and to make sure that it deletes all .xml files in a specified directory.
private static void DeleteXmlFiles(string XmlFileDirectory)
{
foreach (var file in Directory.GetFiles(XmlFileDirectory, "*.Xml"))
{
File.Delete(file);
}
}
Does anybody have any unit testing code snippet that I can look at which would help me in this case?
The below is all i have in the Test method which basically is not much:
[Test]
public void can_delete_all_files_from_specified_directory()
{
string inputDir = #"C:\TestFiles\";
var sut = new FilesUtility();
var deleteSuccess = sut.
}
In order to unit-test your method, you should test it in isolation. I.e. there should not be any real classes like Directory or File your SUT interacts with. So, you have three options:
Create abstraction which your class will depend on, and mock that abstraction for tests (Moq will do that)
Mock static methods for test. Moq can't help here, but that is possible with TypeMock, Moles, JustMock etc
Do acceptance or integration testing instead of unit testing (Specflow is good for writing acceptance tests)
Last approach is pretty simple - create new folder before each test runs, and delete it after test run
private string path = #"C:\TestFiles\";
[SetUp]
public void Setup()
{
Directory.CreateDirectory(path);
}
[TearDown]
public void Deardown()
{
Directory.Delete(path);
}
[Test]
public void ShouldRemoveAllFilesFromSpecifiedDirectory()
{
// seed directory with sample files
FileUtility.DeleteXmlFiles(path);
Assert.False(Directory.EnumerateFiles(path).Any());
}
There's no way of testing methods like these with Moq or any other free mocking framework. That's because they cannot mock methods other than virtuals or interface implementations (which is roughly the same under the hood anyway).
To fake (not mock) system methods like File.Delete(...) or static methods of any kind, you'll need something like Typemock (commercial) or MS Moles (not very user-friendly).
As a workaround, you could create a test directory along with some files in your test, call DeleteXmlFiles(...) on it, and then check if the directory is empty. But that would be slow and also is not really a unit test but more like an integration test.
one approach to such a test might be:
Create a directory(folder) (in the unit test assembly folder)
Stick some xml files into it (you might copy thwem from another folder in the unit test assembly folder)
call you method that should delete them.
Check to see if they are gone.
a. If they are, report success,
b. If not report failure
Delete the folder created in step 1
I am checking TestContext.CurrentTestOutcome in my TestCleanup method in order to perform an action if the test did not pass (in this case, the tests are using Selenium to exercise a website and I am saving a screenshot if the test does not pass).
private static TestContext _testContext;
private static IWebDriver _driver;
[ClassInitialize]
public static void SetupTests(TestContext testContext)
{
_testContext = testContext;
_driver = new FirefoxDriver();
}
[TestCleanup]
public void TeardownTest()
{
if (_testContext.CurrentTestOutcome != UnitTestOutcome.Passed)
{
var fileName = Path.Combine(
Environment.CurrentDirectory,
string.Format("{0}.{1}.gif", _testContext.FullyQualifiedTestClassName, _testContext.TestName));
((ITakesScreenshot)driver).GetScreenshot().SaveAsFile(fileName, ImageFormat.Gif);
Console.WriteLine("Test outcome was {0}, saved image of page to '{1}'", _testContext.CurrentTestOutcome, fileName);
}
}
This works well when run on a local development PC using ReSharper, but on our build server (which uses TeamCity) the UnitTestOutcome is always Unknown, although TeamCity reports them as passed.
The documentation on MSDN is not very helpful. What can cause this value to be set to Unknown?
According to http://confluence.jetbrains.com/display/TCD8/MSTest+Support TeamCity does not support on-the-fly reporting of individual test results, it parses the tests results file to provide the results to the build step.
That would explain how TeamCity is able to report the tests as passed even though UnitTestOutcome may be unknown at the time an individual test has completed.
The link above mentions "specifics of MSTest tool" as the reason for non-on-the-fly test result reporting so I can only theorize that the same specifics may mean that TestContext is unavailable when running from your build server.
Also, the MSDN documentation for TestContext.CurrentTestOutcome does mention that Full Trust for the immediate caller is required. TeamCity could be executing the tests in a manner that is only partially trusted and therefore causing the test outcome to be Unknown.
A quick way to check if MSTest is your problem would be to switch to NUnit using:
#if NUNIT
using NUnit.Framework;
using TestClass = NUnit.Framework.TestFixtureAttribute;
using TestMethod = NUnit.Framework.TestAttribute;
using TestInitialize = NUnit.Framework.SetUpAttribute;
using TestCleanup = NUnit.Framework.TearDownAttribute;
using IgnoreAttribute = NUnit.Framework.IgnoreAttribute;
#else
using Microsoft.VisualStudio.TestTools.UnitTesting;
using IgnoreAttribute = Microsoft.VisualStudio.TestTools.UnitTesting.IgnoreAttribute;
#endif
source http://www.anotherchris.net/tools/using-team-city-for-staging-and-test-builds-with-asp-net-and-selenium/
You would have to do something similar in your TeardownTest method to use the NUnit TestContext.CurrentContext.Result.Status though.
The fix for this issue is to use a public property for TestContext, rather than using the parameter passed to the [ClassInitialize] method.
i.e.
public TestContext TestContext { get; set; }
The test runner will automatically set the property.
(This is related to another question I posted on SO)
I have an integration test LoadFile_DataLoaded_Successfully(). And I want to refactor it to the unit test for breaking dependency with filesytem.
P.S. I am new in TDD:
Here are my production class :
public class LocalizationData
{
private bool IsValidFileName(string fileName)
{
if (fileName.ToLower().EndsWith("xml"))
{
return true;
}
return false;
}
public XmlDataProvider LoadFile(string fileName)
{
if (IsValidFileName(fileName))
{
XmlDataProvider provider =
new XmlDataProvider
{
IsAsynchronous = false,
Source = new Uri(fileName, UriKind.Absolute)
};
return provider;
}
return null;
}
}
and my test class (Nunit)
[TestFixture]
class LocalizationDataTest
{
[Test]
public void LoadFile_DataLoaded_Successfully()
{
var data = new LocalizationData();
string fileName = "d:/azeri.xml";
XmlDataProvider result = data.LoadFile(fileName);
Assert.IsNotNull(result);
Assert.That(result.Document, Is.Not.Null);
}
}
Any idea how to refactor it to break filesystem dependency
What you're missing here is inversion of control. For instance, you can introduce the dependency injection principle into your code:
public interface IXmlDataProviderFactory
{
XmlDataProvider Create(string fileName);
}
public class LocalizationData
{
private IXmlDataProviderFactory factory;
public LocalizationData(IXmlDataProviderFactory factory)
{
this.factory = factory;
}
private bool IsValidFileName(string fileName)
{
return fileName.ToLower().EndsWith("xml");
}
public XmlDataProvider LoadFile(string fileName)
{
if (IsValidFileName(fileName))
{
XmlDataProvider provider = this.factory.Create(fileName);
provider.IsAsynchronous = false;
return provider;
}
return null;
}
}
In the code above the creation of the XmlDataProvider is abstracted away using an IXmlDataProviderFactory interface. An implementation of that interface can be supplied in the constructor of the LocalizationData. You can now write your unit test as follows:
[Test]
public void LoadFile_DataLoaded_Succefully()
{
// Arrange
var expectedProvider = new XmlDataProvider();
string validFileName = CreateValidFileName();
var data = CreateNewLocalizationData(expectedProvider);
// Act
var actualProvider = data.LoadFile(validFileName);
// Assert
Assert.AreEqual(expectedProvider, actualProvider);
}
private static LocalizationData CreateNewLocalizationData(
XmlDataProvider expectedProvider)
{
return new LocalizationData(FakeXmlDataProviderFactory()
{
ProviderToReturn = expectedProvider
});
}
private static string CreateValidFileName()
{
return "d:/azeri.xml";
}
The FakeXmlDataProviderFactory looks like this:
class FakeXmlDataProviderFactory : IXmlDataProviderFactory
{
public XmlDataProvider ProviderToReturn { get; set; }
public XmlDataProvider Create(string fileName)
{
return this.ProviderToReturn;
}
}
Now in your test environment you can (and probably should) always create the class under test manually. However, you want to abstract the creation away in factory methods to prevent you having to change many tests when the class under test changes.
In your production environment however, it can become very cumbersome very soon when you manually have to create the class. Especially when it contains many dependencies. This is where IoC / DI frameworks shine. They can help you with this. For instance, when you want to use the LocalizationData in your production code, you might write code like this:
var localizer = ServiceLocator.Current.GetInstance<LocalizationData>();
var data = data.LoadFile(fileName);
Note that I'm using the Common Service Locator as an example here.
The framework will take care of the creation of that instance for you. Using such a dependency injection framework however, you will have to let the framework know which 'services' your application needs. For instance, when I use the Simple Service Locator library as an example (shameless plug that is), your configuration might look like this:
var container = new SimpleServiceLocator();
container.RegisterSingle<IXmlDataProviderFactory>(
new ProductionXmlDataProviderFactory());
ServiceLocator.SetLocatorProvider(() => container);
This code will usually go in the startup path of your application. Of course the only missing piece of the puzzle is the actual ProductionXmlDataProviderFactory class. Here is it:
class ProductionXmlDataProviderFactory : IXmlDataProviderFactory
{
public XmlDataProvider Create(string fileName)
{
return new XmlDataProvider
{
Source = new Uri(fileName, UriKind.Absolute)
};
}
}
Please also not that you will probably don't want to new up your LocalizationData in your production code yourself, because this class is probably used by other classes that depend on this type. What you would normally do is ask the framework to create the top most class for you (for instance the command that implements a complete use case) and execute it.
I hope this helps.
The problem here is that you are not doing TDD. You wrote the production code first, and now you want to test it.
Erase all that code and start again. Write a test first, and then write the code that passes that test. Then write the next test, etc.
What is your goal? Given a string that ends in "xml" (why not ".xml"?) you want an XML data provider based upon a file whose name is that string. Is that your goal?
The first tests would be the degenerate case. Given a string like "name_with_wrong_ending" your function should fail. How should it fail? Should it return null? Or should it throw an exception? You get to think about this and decide in your test. Then you make the test pass.
Now, what about a string like this: "test_file.xml" but in the case where no such file exists? What do you want the function to do in that case? Should it return null? Should it throw an exception?
The simplest way to test this, of course, is to actually run the code in a directory that does not have that file in it. However, if you'd rather write the test so that it does not use the file system (a wise choice) then you need to be able to ask the question "Does this file exist", and then your test needs to force the answer to be "false".
You can do that by creating a new method in your class named "isFilePresent" or "doesFileExist". Your test can override that function to return 'false'. And now you can test that your 'LoadFile' function works correctly when the file doesn't exist.
Of course now you'll have to test that the normal implementation of "isFilePresent" works correctly. And for that you'll have to use the real file system. However, you can keep file system tests out of your LocalizationData tests by creating a new class named FileSystem and moving your 'isFilePresent' method into that new class. Then your LocalizationData test can create a derivative of that new FileSystem class and override 'isFilePresent' to return false.
You still have to test the regular implementation of FileSystem, but that's in a different set of tests, that only get run once.
OK, what's the next test? What does your 'loadFile' function do when the file does exist, but does not contain valid xml? Should it do anything? Or is that a problem for the client? You decide. But if you decide to check it, you can use the same strategy as before. Make a function named isValidXML and have the test override it to return false.
Finally we need to write the test that actually returns the XMLDataProvider. So the final function that 'loadData' should call, after all those other function is, createXmlDataProvider. And you can override that to return an empty or dummy XmlDataProvider.
Notice that in your tests you have never gone to the real file system and really created an XMLDataProvider based on a file. But what you have done is to check every if statement in your loadData function. You've tested the loadData function.
Now you should write one more test. A test that uses the real file system and a real valid XML file.
When I look at the following code:
public class LocalizationData
{
private static bool IsXML(string fileName)
{
return (fileName != null && fileName.ToLower().EndsWith("xml"));
}
public XmlDataProvider LoadFile(string fileName)
{
if (!IsXML(fileName)) return null*;
return new XmlDataProvider{
IsAsynchronous = false,
Source = new Uri(fileName, UriKind.Absolute)
};
}
}
(* I'm not thrilled about the return null. Yuck! that smells.)
Anyway, I would ask the following questions to myself:
What could possibly break with this code? Are there any complex logic or fragile code that I should safe-guard myself against?
Is there anything complicated to understand or worth highlighting via a test that the code is not able to communicate?
Once I've written this code, how frequently do I think I'll revisit (change) it?
The IsXML function is extremely trivial. Probably does not even belong to this class.
The LoadFile function creates a synchronous XmlDataProvide if it gets a valid XML filename.
I would first search who uses LoadFile and from where fileName is being passed. If its external to our program, then we need some validation. If its internal and somewhere else we are already doing the validation, then we are good to go. As Martin suggested, I would recommend refactoring this to take Uri as the parameter instead of a string.
Once we address that, then all we need to know is if there is any special reason why the XMLDataProvider is in the synchronous mode.
Now, is there anything worth testing? XMLDataProvider is not a class we built, we expect it to work fine when we give a valid Uri.
So frankly, I would not waste my time writing test for this. In the future, if we see more logic creeping in, we might revisit this again.
In one of my (Python) projects, I assume that all unit tests are run in a special directory that contains the folders "data" (input files) and "output" (output files). I'm using a test script that first checks whether those folders exists (i.e. if the current working directory is correct) and then runs the tests. My unit tests can then use relative filenames like "data/test-input.txt".
I don't know how to do this in C#, but maybe you can test for existence of the file "data/azeri.xml" in the test SetUp method.
It has nothing to do with your testing (x), but consider using Uri instead of String as parameter type for your API.
http://msdn.microsoft.com/en-us/library/system.uri(v=VS.100).aspx
x: I think Steven covered that topic pretty very well.
Why do you use the XmlDataProvider? I don't think that it's a valuable unit test, as it stands now. Instead, why don't you test whatever you would do with that data provider?
For example, if you use the XML data to load out a list of Foo objects, make an interface:
public interface IFooLoader
{
IEnumerable<Foo> LoadFromFile(string fileName);
}
You can then test your implementation of this class using a test file you generate during a unit test. In this way you can break your dependency on the filesystem. Delete the file when your test exits (in a finally block).
And as for collaborators that use this type, you can pass in a mock version. You can either hand code the mock, or use a mocking framework such as Moq, Rhino, TypeMock or NMock. Mocking is great, but if you're new to TDD then it's fine to hand code your mocks while you learn what they're useful for. Once you have that, then you are in a good position to understand the good, bad and ugly of mocking frameworks. They can be a bit gnarly to work with when you're starting TDD. Your mileage may vary.
Best of luck.
In this case, you are basically at the lower level of dependency. You are testing that a file exist and that an xmlprovider can be created with the file as source.
The only way that you could break the dependency, would be to inject something to create the XmlDataProvider. You could then mock it to return a XmlDataProvider that you created (as opposed to read). As simplistic example would be:
class XmlDataProviderFactory
{
public virtual XmlDataProvider NewXmlDataProvider(string fileName)
{
return new XmlDataProvider
{
IsAsynchronous = false,
Source = new Uri(fileName, UriKind.Absolute)
};
}
class XmlDataProviderFactoryMock : XmlDataProviderFactory
{
public override XmlDataProvider NewXmlDataProvider(string fileName)
{
return new XmlDataProvider();
}
}
public class LocalizationData
{
...
public XmlDataProvider LoadFile(string fileName, XmlDataProviderFactory factory)
{
if (IsValidFileName(fileName))
{
return factory.NewXmlDataProvider(fileName);
}
return null;
}
}
[TestFixture]
class LocalizationDataTest
{
[Test]
public void LoadFile_DataLoaded_Succefully()
{
var data = new LocalizationData();
string fileName = "d:/azeri.xml";
XmlDataProvider result = data.LoadFile(fileName, new XmlDataProviderFactoryMock());
Assert.IsNotNull(result);
Assert.That(result.Document, Is.Not.Null);
}
}
Using an injection framework could simplify the call to LoadFile by injecting the factory in the class constructor or elsewhere.
I Like #Steven's answer except I think He didn't go far enough:
public interface DataProvider
{
bool IsValidProvider();
void DisableAsynchronousOperation();
}
public class XmlDataProvider : DataProvider
{
private string fName;
private bool asynchronousOperation = true;
public XmlDataProvider(string fileName)
{
fName = fileName;
}
public bool IsValidProvider()
{
return fName.ToLower().EndsWith("xml");
}
public void DisableAsynchronousOperation()
{
asynchronousOperation = false;
}
}
public class LocalizationData
{
private DataProvider dataProvider;
public LocalizationData(DataProvider provider)
{
dataProvider = provider;
}
public DataProvider Load()
{
if (provider.IsValidProvider())
{
provider.DisableAsynchronousOperation();
return provider;
}
return null;
}
}
By not going far enough I mean that he didn't follow the Last Possible Responsible Moment. Push as much down into the implemented DataProvider class as possible.
One thing I didn't do with this code, is drive it with unit tests and mocks. That is why you're still checking the state of the provider to see if it is valid.
Another thing is that I tried to remove the dependencies on having the LocalizationData know that the provider is using a file. What if it was a web service or database?
So first of all let us understand what we need to test. We need to verify that given a valid filename, your LoadFile(fn) method returns an XmlDataProvider, otherwise it returns null.
Why is the LoadFile() method difficult to test ? Because it creates a XmlDataProvider with a URI created from the filename. I have not worked much with C#, but am assuming that if the file does not actually exist on the system, we will get an Exception. The real problem is, your production method LoadFile() is creating something which is difficult to fake. Not being able to fake it is a problem because we cannot ensure the existence of a certain file in all test environments, without having to enforce implicit guidelines.
So the solution is - we should be able to fake the collaborators (XmlDataProvider) of the loadFile method. However, if a method creates it's collaborators it cannot fake them, hence a method should never create it's collaborators.
If a method does not create it's collaborators, how does it get them ? - In one of these two ways:
They should be injected into the method
They should be obtained from some factory
In this case it does not make sense for the XmlDataProvider to be injected into the method, since that is exactly what it is returning. So we should get it from a global Factory - XmlDataProviderFactory.
Here comes the interesting part. When your code is running in production, the factory should return an XmlDataProvider, and when your code is running in a test environment, the factory should return a fake object.
Now the only part of the puzzle is, how to ensure that the factory behaves in different ways in different environments ? One way is to use some properties which have different values in both environments, and the other way is to configure the factory for what it should return. I personally prefer the former way.
Hope this helps.
This time, don't try to break your dependency on the file system. This behavior clearly depends on the file system, and appears to be at the integration point with the file system, so test it with the file system.
Now, I second Bob's advice: throw this code away and try test-driving it. It makes for great practice and is exactly how I trained myself to do it. Good luck.
Instead of returning XmlDataProvider which ties you a specific tech, hide this implementation detail. It looks like you need a repository Role to
LocalizationData GetLocalizationData(params)
You can have an implementation for this Role, which internally uses Xml. You'd need to write integration tests to test whether XmlLocalizationDataRepository can read actual Xml data stores. (Slow).
The rest of your code can mock out GetLocalizationData()