I have the following basic setup for a uni-test: it tests a class that is responsible for indexing files in a directory, and keep giving the correct one on demand.
To do this I use a mock file system provided by system.io.abstractions library. The basic setup is as follows:
[TestFixture]
public class ImageDataTests
{
private static MockFileSystem fs;
private const int TESTNUM = 3;
[SetUp]
public static void Init() {
var allnamegroups = TestSettings.NameGroupFactory(TESTNUM);
fs = new MockFileSystem();
foreach (string[] namegroup in allnamegroups) {
var dname = ImageData.MakeDirectoryName(namegroup[0], namegroup[1]);
TestSettings.AddTestFilesToMockFileSystem(fs, dname);
}
}
}
Now each test works on a case for this, and to test (say) a "reload" function I add the following method to above class:
public static IEnumerable<ImageData> ImgDatSource() {
while (true) {
yield return new ImageData("test", fs);
}
}
[Test, Sequential]
public void GetAllOfTypeTest(
[ValueSource(nameof(ImgDatSource))] ImageData img,
[Values("foo", "bar", "this")]string type,
[Values(1, 2, 0)]int ex) {
Assert.That(() => img.GetAllOfType(type).Count(), Is.EqualTo(ex));
}
This should (right now) list all files starting with respectively "foo", "bar" and "this". - It fails, stating that the directory is not found. - Even before the first test is run: at the initializing of Imagedata.
Debugging verifies what I thought: the Init() is not run yet when the ValueSource tries to initialize the input.
Is there a way to do "such" a test, short of initializing the Imagedata inside the test body? (And either losing the parametric construction + test indexing, or having to write many similar tests manually).
Notice that I really NEED to run the filesystem generator each test: while above test is simple and doesn't do much. Some tests will adapt/change the filesystem to test the class against more edge-cases. - Also during the test.
NUnit's execution has two phases: loading (discovering) tests and running tests. OneTimeSetUp, SetUp, test method execution, TearDown and OneTimeTearDown are all part of running the test.
Test loading takes place, as you might expect, before running the tests. Your data sources (TestCaseSource or ValueSource) are all run during the load phase.
This leads to two conclusions:
1. It is not possible for anything in your SetUp and other test execution methods to have an effect on the data that is used for a test.
2. Your data sources need to provide data parameters that will allow you to create objects at runtime, not the objects themselves.
In your example, I would change the test method to take four parameters, replacing img with "test" and fs. I would add a line to the start of each test to create the imagedata by calling a helper method modeled after your Init.
I am using NUnit to test one functionality where I need to load XML file to object. The XML file is in location of the Console Application.
I have Following method where configuration will be read :
public string GetConfiguration(TempFlexProcessor processor)
{
var exePath = Path.GetDirectoryName(System.Reflection.Assembly.GetEntryAssembly().Location);
var configPath = Path.Combine(Path.GetFullPath(exePath), "configuration");
var configFile = string.Format(#"{0}.xml", processor.GetType().Name);
}
Now in my NUnit Test I have test method where I test GetConfiguration :
[Test]
public void TempFlexProcessorExecuteTest()
{
#region Given
#endregion
#region When
var tempFlexProcessor = new TempFlexProcessor();
var actual = tempFlexProcessor.GetConfiguration(tempFlexProcessor);
#endregion
Assert.AreEqual("path of the file", actual);
}
But System.Reflection.Assembly.GetEntryAssembly() is null, please help.
I used AppDomain.CurrentDomain.BaseDirectory instead of System.Reflection.Assembly.GetEntryAssembly().Location
I suspect the problem is that NUnit is running your tests in a different AppDomain, but without using ExecuteAssembly. From the documentation for Assembly.GetEntryAssembly:
Gets the process executable in the default application domain. In other application domains, this is the first executable that was executed by AppDomain.ExecuteAssembly.
It's not clear which assembly you really want to get - even if this did return something "appropriate" for NUnit, that's likely to be the NUnit executable, which would be well away from any configuration directories you happen to have.
Basically, I think that you should at least provide an alternative way of specifying the configuration directory - and you might want to reconsider whether using GetEntryAssembly is a good idea anyway. (Aside from anything else, it's slightly odd that you're calling GetConfiguration on a processor and passing in another processor... that may be suitable for your design, but it's at least somewhat unusual, given that in your test case you're passing in a reference to the same object.)
I am trying to do Unit Testing with Excel as data source. I am getting the following exception. How do we correct it?
The unit test adapter failed to connect to the data source or to read
the data. For more information on troubleshooting this error, see
"Troubleshooting Data-Driven Unit Tests"
[TestMethod]
[Owner("Lijo ")]
[TestProperty("TestCategory", "Developer"),
DataSource("Microsoft.ACE.OLEDB.12.0",
"Data Source=C:/Sheets/DataSheet.xlsx;Extended Properties=Excel 12.0;",
"[Sheet1$]",
DataAccessMethod.Sequential)]
public void ChangePasswordTest()
{
int a = Convert.ToInt32(TestContext.DataRow[0]); //(int)Column.UserId
int b = Convert.ToInt32(TestContext.DataRow[1]);
int expectedResult = Convert.ToInt32(TestContext.DataRow[2]);
MyClass myObj = new MyClass(1, "P#ssw0rd");
int actualResult = myObj.GetAdditionResult(a, b);
Assert.AreEqual<int>(expectedResult, actualResult, "The addition result is incorrect.");
}
Readings:
Unit Testing Error - The unit test adapter failed to connect to the data source or to read the data
Data driven unit tests problem
How to create Startup and Cleanup script for Visual Studio Test Project?
How Does MSTEST/Visual Studio 2008 Team Test Decide Test Method Execution Order?
Visual Studio 2010 Ultimate - Data Generation Plan Setting Incorrect Data Type for Column
How should I unit-test a simple CRUD-class?
I faded a same task today. And after some headaches I was able to solve without app.config:
[TestMethod]
[DataSource("System.Data.OleDB",
#"Provider=Microsoft.ACE.OLEDB.12.0; Data Source=C:\Sheets\DataSheet.xlsx; Extended Properties='Excel 12.0;HDR=yes';",
"Sheet1$",
DataAccessMethod.Sequential
)]
public void ChangePasswordTest()
{
//Arrange
//Act
//Assert
}
If you use excel files as resources in your testproject, you have to set the Copy to Output Directory property of the file to Copy always or Copy if newer. And add the DeploymentItem attribute to your test:
[TestMethod]
[DataSource("System.Data.OleDB",
#"Provider=Microsoft.ACE.OLEDB.12.0; Data Source=.\DataSheet.xlsx; Extended Properties='Excel 12.0;HDR=yes';",
"Sheet1$",
DataAccessMethod.Sequential
)]
[DeploymentItem(".\DataSheet.xlsx")]
public void ChangePasswordTest()
{
//Arrange
//Act
//Assert
}
Solved it myself in a different way. Other answers are welcome.
Refer: Walkthrough: Using a Configuration File to Define a Data Source http://msdn.microsoft.com/en-us/library/ms243192.aspx
[TestMethod]
[DeploymentItem("C:/Sheets/DataSheet.xlsx")]
[DataSource("MyExcelDataSource")]
public void ChangePasswordTest()
{
int a = Convert.ToInt32(TestContext.DataRow[0]); //(int)Column.UserId
int b = Convert.ToInt32(TestContext.DataRow[1]);
int expectedResult = Convert.ToInt32(TestContext.DataRow[2]);
MyClass myObj = new MyClass(1, "P#ssw0rd");
int actualResult = myObj.GetAdditionResult(a, b);
Assert.AreEqual<int>(expectedResult, actualResult, "The addition result is incorrect.");
}
App.Config
<configuration>
<configSections>
<section name="microsoft.visualstudio.testtools"
type="Microsoft.VisualStudio.TestTools.UnitTesting.TestConfigurationSection, Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"/>
</configSections>
<connectionStrings>
<add name="MyExcelConn"
connectionString="Dsn=Excel Files;dbq=C:/Sheets/DataSheet.xlsx;defaultdir=.; driverid=790;maxbuffersize=2048;pagetimeout=5" providerName="System.Data.Odbc" />
</connectionStrings>
<microsoft.visualstudio.testtools>
<dataSources>
<add name="MyExcelDataSource"
connectionString="MyExcelConn"
dataTableName="Sheet1$"
dataAccessMethod="Sequential"/>
</dataSources>
</microsoft.visualstudio.testtools>
</configuration>
For VS 2010, TestTools version used is Version=10.0.0.0
Although it's not 100% related to the question and perhaps somewhat trivial, I'd like to throw in my two cents on this general topic (this was the most relevant question I could find to post in).
In the project I'm currently working on I came across the need to do some simple data-driven unit tests (e.g. with perhaps 20 rows or so for a given test). My wish-list for a data-driven "framework" was:
Nice and easy integration in the Visual Studio Test Explorer
Ability to keep using my existing keyboard shortcut keys to run/debug the test method in which the cursor lies
No external files or databases to have to manage (i.e. no "DataSources")
"Native" support in C# and Visual Studio, i.e. no additional packages necessary
Despite wish #4, I looked into external libraries like xUnit and NUnit. They seemed to solve wish #3, but didn't do a great job at wishes #1 and #2.
Frustrated with the lack of an easy solution, I decided to implement an extremely basic data-driven helper myself:
public static void DataDrivenTest(Action<List<object>> testAction, List<List<object>> dataRows)
{
foreach (var dataRow in dataRows)
testAction(dataRow);
}
I use it like this:
[TestMethod]
public void Unit_Can_Add_Two_Numbers()
{
UnitTestUtilities.DataDrivenTest(
dataRow =>
{
// Tests a+b=c
var a = (int)dataRow[0];
var b = (int)dataRow[1];
var c = (int)dataRow[2];
Assert.AreEqual(a + b, c);
},
new List<List<object>>
{
// Rows of arguments a,b,c respectively
new List<object>{1,2,3},
new List<object>{4,5,9}
});
}
Although it fulfills all of my wishes above, it has disadvantages:
Casting is required inside the test action definition (could probably be fixed with some generic argument magic)
Updating the order of the arguments in the data rows means that the accessors in the action method also have to be updated
Clearly not appropriate for large data-driven tests (e.g. this helper would be messy for more than, say, 20 test rows)
Clearly lacks "sophistication", i.e. I'm sure that libraries like xUnit have some funky features that this doesn't have
It runs all rows as one unit test (i.e. no separate unit test and reporting for each data row)
Regardless, it solved my basic problem in a simple way. I hope that helps somebody looking for a simple solution like I was. After all, if you find that you need a tonne of test rows for testing a method, it might be worth considering a refactor to break down the function under test into smaller components (and then use in conjunction with mocks/fakes, dependency injection etc).
I have an integration test LoadFile_DataLoaded_Successfully(). And I want to refactor it to the unit test for breaking dependency with filesytem.
P.S. I am new in TDD:
Here are my production class :
public class LocalizationData
{
private bool IsValidFileName(string fileName)
{
if (fileName.ToLower().EndsWith("xml"))
{
return true;
}
return false;
}
public XmlDataProvider LoadFile(string fileName)
{
if (IsValidFileName(fileName))
{
XmlDataProvider provider =
new XmlDataProvider
{
IsAsynchronous = false,
Source = new Uri(fileName, UriKind.Absolute)
};
return provider;
}
return null;
}
}
and my test class (Nunit)
[TestFixture]
class LocalizationDataTest
{
[Test]
public void LoadFile_DataLoaded_Successfully()
{
var data = new LocalizationData();
string fileName = "d:/azeri.xml";
XmlDataProvider result = data.LoadFile(fileName);
Assert.IsNotNull(result);
Assert.That(result.Document, Is.Not.Null);
}
}
Any idea how to refactor it to break filesystem dependency
What you're missing here is inversion of control. For instance, you can introduce the dependency injection principle into your code:
public interface IXmlDataProviderFactory
{
XmlDataProvider Create(string fileName);
}
public class LocalizationData
{
private IXmlDataProviderFactory factory;
public LocalizationData(IXmlDataProviderFactory factory)
{
this.factory = factory;
}
private bool IsValidFileName(string fileName)
{
return fileName.ToLower().EndsWith("xml");
}
public XmlDataProvider LoadFile(string fileName)
{
if (IsValidFileName(fileName))
{
XmlDataProvider provider = this.factory.Create(fileName);
provider.IsAsynchronous = false;
return provider;
}
return null;
}
}
In the code above the creation of the XmlDataProvider is abstracted away using an IXmlDataProviderFactory interface. An implementation of that interface can be supplied in the constructor of the LocalizationData. You can now write your unit test as follows:
[Test]
public void LoadFile_DataLoaded_Succefully()
{
// Arrange
var expectedProvider = new XmlDataProvider();
string validFileName = CreateValidFileName();
var data = CreateNewLocalizationData(expectedProvider);
// Act
var actualProvider = data.LoadFile(validFileName);
// Assert
Assert.AreEqual(expectedProvider, actualProvider);
}
private static LocalizationData CreateNewLocalizationData(
XmlDataProvider expectedProvider)
{
return new LocalizationData(FakeXmlDataProviderFactory()
{
ProviderToReturn = expectedProvider
});
}
private static string CreateValidFileName()
{
return "d:/azeri.xml";
}
The FakeXmlDataProviderFactory looks like this:
class FakeXmlDataProviderFactory : IXmlDataProviderFactory
{
public XmlDataProvider ProviderToReturn { get; set; }
public XmlDataProvider Create(string fileName)
{
return this.ProviderToReturn;
}
}
Now in your test environment you can (and probably should) always create the class under test manually. However, you want to abstract the creation away in factory methods to prevent you having to change many tests when the class under test changes.
In your production environment however, it can become very cumbersome very soon when you manually have to create the class. Especially when it contains many dependencies. This is where IoC / DI frameworks shine. They can help you with this. For instance, when you want to use the LocalizationData in your production code, you might write code like this:
var localizer = ServiceLocator.Current.GetInstance<LocalizationData>();
var data = data.LoadFile(fileName);
Note that I'm using the Common Service Locator as an example here.
The framework will take care of the creation of that instance for you. Using such a dependency injection framework however, you will have to let the framework know which 'services' your application needs. For instance, when I use the Simple Service Locator library as an example (shameless plug that is), your configuration might look like this:
var container = new SimpleServiceLocator();
container.RegisterSingle<IXmlDataProviderFactory>(
new ProductionXmlDataProviderFactory());
ServiceLocator.SetLocatorProvider(() => container);
This code will usually go in the startup path of your application. Of course the only missing piece of the puzzle is the actual ProductionXmlDataProviderFactory class. Here is it:
class ProductionXmlDataProviderFactory : IXmlDataProviderFactory
{
public XmlDataProvider Create(string fileName)
{
return new XmlDataProvider
{
Source = new Uri(fileName, UriKind.Absolute)
};
}
}
Please also not that you will probably don't want to new up your LocalizationData in your production code yourself, because this class is probably used by other classes that depend on this type. What you would normally do is ask the framework to create the top most class for you (for instance the command that implements a complete use case) and execute it.
I hope this helps.
The problem here is that you are not doing TDD. You wrote the production code first, and now you want to test it.
Erase all that code and start again. Write a test first, and then write the code that passes that test. Then write the next test, etc.
What is your goal? Given a string that ends in "xml" (why not ".xml"?) you want an XML data provider based upon a file whose name is that string. Is that your goal?
The first tests would be the degenerate case. Given a string like "name_with_wrong_ending" your function should fail. How should it fail? Should it return null? Or should it throw an exception? You get to think about this and decide in your test. Then you make the test pass.
Now, what about a string like this: "test_file.xml" but in the case where no such file exists? What do you want the function to do in that case? Should it return null? Should it throw an exception?
The simplest way to test this, of course, is to actually run the code in a directory that does not have that file in it. However, if you'd rather write the test so that it does not use the file system (a wise choice) then you need to be able to ask the question "Does this file exist", and then your test needs to force the answer to be "false".
You can do that by creating a new method in your class named "isFilePresent" or "doesFileExist". Your test can override that function to return 'false'. And now you can test that your 'LoadFile' function works correctly when the file doesn't exist.
Of course now you'll have to test that the normal implementation of "isFilePresent" works correctly. And for that you'll have to use the real file system. However, you can keep file system tests out of your LocalizationData tests by creating a new class named FileSystem and moving your 'isFilePresent' method into that new class. Then your LocalizationData test can create a derivative of that new FileSystem class and override 'isFilePresent' to return false.
You still have to test the regular implementation of FileSystem, but that's in a different set of tests, that only get run once.
OK, what's the next test? What does your 'loadFile' function do when the file does exist, but does not contain valid xml? Should it do anything? Or is that a problem for the client? You decide. But if you decide to check it, you can use the same strategy as before. Make a function named isValidXML and have the test override it to return false.
Finally we need to write the test that actually returns the XMLDataProvider. So the final function that 'loadData' should call, after all those other function is, createXmlDataProvider. And you can override that to return an empty or dummy XmlDataProvider.
Notice that in your tests you have never gone to the real file system and really created an XMLDataProvider based on a file. But what you have done is to check every if statement in your loadData function. You've tested the loadData function.
Now you should write one more test. A test that uses the real file system and a real valid XML file.
When I look at the following code:
public class LocalizationData
{
private static bool IsXML(string fileName)
{
return (fileName != null && fileName.ToLower().EndsWith("xml"));
}
public XmlDataProvider LoadFile(string fileName)
{
if (!IsXML(fileName)) return null*;
return new XmlDataProvider{
IsAsynchronous = false,
Source = new Uri(fileName, UriKind.Absolute)
};
}
}
(* I'm not thrilled about the return null. Yuck! that smells.)
Anyway, I would ask the following questions to myself:
What could possibly break with this code? Are there any complex logic or fragile code that I should safe-guard myself against?
Is there anything complicated to understand or worth highlighting via a test that the code is not able to communicate?
Once I've written this code, how frequently do I think I'll revisit (change) it?
The IsXML function is extremely trivial. Probably does not even belong to this class.
The LoadFile function creates a synchronous XmlDataProvide if it gets a valid XML filename.
I would first search who uses LoadFile and from where fileName is being passed. If its external to our program, then we need some validation. If its internal and somewhere else we are already doing the validation, then we are good to go. As Martin suggested, I would recommend refactoring this to take Uri as the parameter instead of a string.
Once we address that, then all we need to know is if there is any special reason why the XMLDataProvider is in the synchronous mode.
Now, is there anything worth testing? XMLDataProvider is not a class we built, we expect it to work fine when we give a valid Uri.
So frankly, I would not waste my time writing test for this. In the future, if we see more logic creeping in, we might revisit this again.
In one of my (Python) projects, I assume that all unit tests are run in a special directory that contains the folders "data" (input files) and "output" (output files). I'm using a test script that first checks whether those folders exists (i.e. if the current working directory is correct) and then runs the tests. My unit tests can then use relative filenames like "data/test-input.txt".
I don't know how to do this in C#, but maybe you can test for existence of the file "data/azeri.xml" in the test SetUp method.
It has nothing to do with your testing (x), but consider using Uri instead of String as parameter type for your API.
http://msdn.microsoft.com/en-us/library/system.uri(v=VS.100).aspx
x: I think Steven covered that topic pretty very well.
Why do you use the XmlDataProvider? I don't think that it's a valuable unit test, as it stands now. Instead, why don't you test whatever you would do with that data provider?
For example, if you use the XML data to load out a list of Foo objects, make an interface:
public interface IFooLoader
{
IEnumerable<Foo> LoadFromFile(string fileName);
}
You can then test your implementation of this class using a test file you generate during a unit test. In this way you can break your dependency on the filesystem. Delete the file when your test exits (in a finally block).
And as for collaborators that use this type, you can pass in a mock version. You can either hand code the mock, or use a mocking framework such as Moq, Rhino, TypeMock or NMock. Mocking is great, but if you're new to TDD then it's fine to hand code your mocks while you learn what they're useful for. Once you have that, then you are in a good position to understand the good, bad and ugly of mocking frameworks. They can be a bit gnarly to work with when you're starting TDD. Your mileage may vary.
Best of luck.
In this case, you are basically at the lower level of dependency. You are testing that a file exist and that an xmlprovider can be created with the file as source.
The only way that you could break the dependency, would be to inject something to create the XmlDataProvider. You could then mock it to return a XmlDataProvider that you created (as opposed to read). As simplistic example would be:
class XmlDataProviderFactory
{
public virtual XmlDataProvider NewXmlDataProvider(string fileName)
{
return new XmlDataProvider
{
IsAsynchronous = false,
Source = new Uri(fileName, UriKind.Absolute)
};
}
class XmlDataProviderFactoryMock : XmlDataProviderFactory
{
public override XmlDataProvider NewXmlDataProvider(string fileName)
{
return new XmlDataProvider();
}
}
public class LocalizationData
{
...
public XmlDataProvider LoadFile(string fileName, XmlDataProviderFactory factory)
{
if (IsValidFileName(fileName))
{
return factory.NewXmlDataProvider(fileName);
}
return null;
}
}
[TestFixture]
class LocalizationDataTest
{
[Test]
public void LoadFile_DataLoaded_Succefully()
{
var data = new LocalizationData();
string fileName = "d:/azeri.xml";
XmlDataProvider result = data.LoadFile(fileName, new XmlDataProviderFactoryMock());
Assert.IsNotNull(result);
Assert.That(result.Document, Is.Not.Null);
}
}
Using an injection framework could simplify the call to LoadFile by injecting the factory in the class constructor or elsewhere.
I Like #Steven's answer except I think He didn't go far enough:
public interface DataProvider
{
bool IsValidProvider();
void DisableAsynchronousOperation();
}
public class XmlDataProvider : DataProvider
{
private string fName;
private bool asynchronousOperation = true;
public XmlDataProvider(string fileName)
{
fName = fileName;
}
public bool IsValidProvider()
{
return fName.ToLower().EndsWith("xml");
}
public void DisableAsynchronousOperation()
{
asynchronousOperation = false;
}
}
public class LocalizationData
{
private DataProvider dataProvider;
public LocalizationData(DataProvider provider)
{
dataProvider = provider;
}
public DataProvider Load()
{
if (provider.IsValidProvider())
{
provider.DisableAsynchronousOperation();
return provider;
}
return null;
}
}
By not going far enough I mean that he didn't follow the Last Possible Responsible Moment. Push as much down into the implemented DataProvider class as possible.
One thing I didn't do with this code, is drive it with unit tests and mocks. That is why you're still checking the state of the provider to see if it is valid.
Another thing is that I tried to remove the dependencies on having the LocalizationData know that the provider is using a file. What if it was a web service or database?
So first of all let us understand what we need to test. We need to verify that given a valid filename, your LoadFile(fn) method returns an XmlDataProvider, otherwise it returns null.
Why is the LoadFile() method difficult to test ? Because it creates a XmlDataProvider with a URI created from the filename. I have not worked much with C#, but am assuming that if the file does not actually exist on the system, we will get an Exception. The real problem is, your production method LoadFile() is creating something which is difficult to fake. Not being able to fake it is a problem because we cannot ensure the existence of a certain file in all test environments, without having to enforce implicit guidelines.
So the solution is - we should be able to fake the collaborators (XmlDataProvider) of the loadFile method. However, if a method creates it's collaborators it cannot fake them, hence a method should never create it's collaborators.
If a method does not create it's collaborators, how does it get them ? - In one of these two ways:
They should be injected into the method
They should be obtained from some factory
In this case it does not make sense for the XmlDataProvider to be injected into the method, since that is exactly what it is returning. So we should get it from a global Factory - XmlDataProviderFactory.
Here comes the interesting part. When your code is running in production, the factory should return an XmlDataProvider, and when your code is running in a test environment, the factory should return a fake object.
Now the only part of the puzzle is, how to ensure that the factory behaves in different ways in different environments ? One way is to use some properties which have different values in both environments, and the other way is to configure the factory for what it should return. I personally prefer the former way.
Hope this helps.
This time, don't try to break your dependency on the file system. This behavior clearly depends on the file system, and appears to be at the integration point with the file system, so test it with the file system.
Now, I second Bob's advice: throw this code away and try test-driving it. It makes for great practice and is exactly how I trained myself to do it. Good luck.
Instead of returning XmlDataProvider which ties you a specific tech, hide this implementation detail. It looks like you need a repository Role to
LocalizationData GetLocalizationData(params)
You can have an implementation for this Role, which internally uses Xml. You'd need to write integration tests to test whether XmlLocalizationDataRepository can read actual Xml data stores. (Slow).
The rest of your code can mock out GetLocalizationData()