I am doing an AI project that currently had the following namespace/package structure:
My ideia is to have some Unit/Integration tests for each one of the modules (probably not for all of them. Startup probably won't be tested).
Each one of these "main" packages is in a different physical file. I'd like to know what you think would be better:
Having for each physical file X.dll a X.Tests.dll
Having a Tests physical file that has the whole lot of tests (doesn't seem like a good idea).
Incorporating Breakthrough's tests in Breakthrough.dll, AI.GeneticAlgorithms in AI.GeneticAlgorithms.dll, etc. I kind of like this solution, as then I won't have my Visual Studio solution explorer cluttered with the the whole set of Test projects.
Thanks!
I would have one Test.dll for each .dll.
It's the way I'm used to work, and I think it's a good method. You now where what is tested.
As Liam already pointed out, the implementation and the tests shouldn't be in the same dll. It increases the size of the delivered dll unnecessarily and you have to deliver all the dependencies of the tests (e. g. unit testing framework, mocking etc.)
Personally, I would have one project/assembly for startup, one for tests, and one for everything else.
Build time increases significantly with multiple projects, so the less projects the better. Unless of course you need to be able to deploy the different packages separately.
Tests are generally not included in the project they are testing because it requires the assembly to take a dependency on any testing frameworks you are using.
I have several class files in App_Code in an ASP.net website running in Microsoft Visual Studio 2005 Professional.
In liu of using a full unit test suite I just want to somehow compile those project-wide classses into an .EXE so that I can nightly run unit tests on them.
I do know how to create a separate C# library project consisting of those files and how to include them into my website--but that is not desirable--I don't want to give up the ability to make on-the-fly code changes of those library classes when running the website in the debugger. As far as I know .Net debugger isn't powerful enough to modify code in included libraries with instant auto re-compilation on page re-load.
So, I want my cake and eat it, too:
Command-line unit testing of website class files in App_Code directory
Being able to modify those class files w/o stopping/re-starting the web debugger.
Is it possible to have both?
You should put the code in an altogether separate class library/assembly, then reference it from your web project and the command-line utility. As far as I know, it makes no difference where you modify your code, when stopped in the debugger. Never had problems myself.
Hope that helps.
Your project is under source control, right? Right? In that case, you can use your source control system to include a link to your asp.net project's app_code folder as part of a separate unit testing project. The exact linking mechanism varies by source control platform, but done right it means there's exactly one instance of your App_Code folder in source control that's visible from two different projects. This way, everything stays up to date.
This has the advantage of allowing you to keep easy, uncompiled code right there just like you always have, but still making the code available for testing.
Is there a Test Stripper (as defined in xUnit Test Patterns) available that supports removing classes containing methods tagged as [Fact]s etc. plus the dependency on xunit.dll from binaries [as part of a build process] ?
Further details of the full requirements and context are at this xUnit CodePlex post.
Failing that (something that removes the tests and the reference to the DLL), does anyone have a utility/proven clean approach to removing the xunit.dll dependecy without too much monkeying around in .vcproj files etc.
Open source preferred. A cleanly written one for NUnit might potentially serve as a base.
NB: I'm aware of the negatives of embedding code into assemblies - this question isnt about that (unless you feel after reading the codeplex thread that some key points are being missed).
If your unit tests are placed in a project resp. assembly of their own there should be no need to strip test classes prior to deployment. What's wrong about this approach?
As noted in the codeplex post where I specified the requirements, I implemented a basic stripper with Mono.Cecil but ended up leveraging the fact that unused references get optimised out in a release build to 'strip' tests out by excluding them based on a #define, which I control from outside via the MSBuild invocation
I try to use NUnit with SnippetCompiler http://www.sliver.com/dotnet/SnippetCompiler/
I have added references to nunit.framework.dll in snippetcompiler (Menu Tools, References) and compiled nunit sample http://www.nunit.org/index.php?p=quickStart&r=2.5.2 to bank.dll
but when I open bank.dll in NUnit GUI it fails saying it cannot load NUnit assembly or one of its dependencies.
Is it possible to fix this ?
I couldn't even get v2.0.8.3 of SnippetCompiler to include the reference. It let me do it, but it wouldn't compile.
In any case writing unit tests isn't the purpose of SnippetCompiler. It's designed to do quick spikes - try something and see if it works. In other words, it's throwaway code.
In addition, the version for .NET 3.5 (the one I'm using) is an alpha release; the developer does not seem to be maintaining this. (Not to put down the author - this was an awesome tool that saved me lots of time!)
For writing spikes against modern versions of .NET, I've switched to LINQPad. Change Edit/Preferences/Query to C# Program and it's very similar to SnippetCompiler. The basic version is free; for a small fee, the registered version provides IntelliSense.
Even if you're writing learning tests with NUnit, you'll want to preserve those tests. Use Visual Studio (or another IDE) and create a separate Class Library project for your tests.
Maybe this is the solution
http://weblogs.asp.net/rosherove/archive/2008/02/21/ad-hoc-unit-tests-with-snippet-compiler.aspx
Ad hoc Unit Tests with Snippet
Compiler
If you're a fan of snippet compiler
(if you're not you should seriously
check it out)Travis Illig published a
little template for writing Typemock
Isolator test inside this handy little
tool.
the reasons you'd need a specialized
template in the first place to write
these kinds of tests in snippet
compiler:
1) Typemock Isolator uses the .NET
profiling APIs to work its magic, so
the .net process running your tests
need to have a couple of environment
variables enabled to work
2) His code template actually creates
and runs a new process that triggers
nunit-console.exe with the path of the
current code you just wrote in snippet
compiler allowing you to effecively
write and run unit tests in snippet
compiler!
3) the nunit-console process will
already have the env. vars as
mentioned in the first item set to it.
Travis' template will work for
anything nunit can run, not just
typemock isolator tests, which is
pretty cool.
How do you usually go about separating your codebase and associated unit tests? I know people who create a separate project for unit tests, which I personally find confusing and difficult to maintain. On the other hand, if you mix up code and its tests in a single project, you end up with binaries related to your unit test framework (be it NUnit, MbUnit or whatever else) and your own binaries side by side.
This is fine for debugging, but once I build a release version, I really do not want my code to reference the unit testing framework any more.
One solution I found is to enclose all your unit tests within #if DEBUG -- #endif directives: when no code references an unit testing assembly, the compiler is clever enough to omit the reference in the compiled code.
Are there any other (possibly more comfortable) options to achieve a similar goal?
I definitely advocate separating your tests out to a separate project. It's the only way to go in my opinion.
Yes, as Gary says, it also forces you to test behavior through public methods rather than playing about with the innards of your classes
As the others point out, a seperate test project (for each normal project) is a good way to do it. I usually mirror the namespaces and create a test class for each normal class with 'test' appended to the name. This is supported directly in the IDE if you have Visual Studio Team System which can automatically generate test classes and methods in another project.
One thing to remember if you want to test classes and methods with the 'internal' accessor is to add the following line to the AssemblyInfo.cs file for each project to be tested:
[assembly: InternalsVisibleTo("UnitTestProjectName")]
The .Net framework after v2 has a useful feature where you can mark an assembly with the InternalsVisibleTo attribute that allows the assembly to be accessed by another.
A sort of assembly tunnelling feature.
Yet another alternative to using compiler directives within a file or creating a separate project is merely to create additional .cs files in your project.
With some magic in the project file itself, you can dictate that:
nunit.framework DLLs are only referenced in a debug build, and
your test files are only included in debug builds
Example .csproj excerpt:
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="3.5">
...
<Reference Include="nunit.framework" Condition=" '$(Configuration)'=='Debug' ">
<SpecificVersion>False</SpecificVersion>
<HintPath>..\..\debug\nunit.framework.dll</HintPath>
</Reference>
...
<Compile Include="Test\ClassTest.cs" Condition=" '$(Configuration)'=='Debug' " />
...
</Project>
I would recommend a separate project for unit tests (and yet more projects for integration tests, functional tests etc.). I have tried mixing code and tests in the same project and found it much less maintainable than separating them into separate projects.
Maintaining parallel namespaces and using a sensible naming convention for tests (eg. MyClass and MyClassTest) will help you keeping the codebase maintainable.
As long as your tests are in a seperate project, the tests can reference the codebase, but the codebase never has to reference the tests. I have to ask, what's confusing about maintaining two projects? You can keep them in the same solution for organization.
The complicated part, of course, is when the business has 55 projects in the solution and 60% of them are tests. Count yourself lucky.
I put the tests in a separate project but in the same solution. Granted, in big solutions there might be a lot of projects but the solution explorer is good enough on separating them and if you give everything reasonable names I don't really think it's an issue.
One thing yet to be considered is versions of VisualStudio prior to 2005 did not allow EXE assembly projects to be referenced from other projects. So if you are working on a legacy project in VS.NET your options would be:
Put unit tests in the same project and use conditional compilation to exclude them from release builds.
Move everything to dll assemblies so your exe is just an entry point.
Circumvent the IDE by hacking the project file in a text editor.
Of the three conditional compilation is the least error prone.
I've always keep my unit tests in a seperate project so it compiles to it's own assembly.
For each project there is a corresponding .Test project that contains tests on it.
E.g. for the assembly called, say "Acme.BillingSystem.Utils", there would be a test assembly called "Acme.BillingSystem.Utils.Test".
Exclude it from the shipping version of your product by not shipping that dll.
If the #if(DEBUG) tag allows for a clean "release" version, why would you need a separate project for tests. The nunit LibarryA/B example (yeah, I know its a example) does this. Currently wrestling with the scenario. Had been using a separate project, but this seems to possibly allow for some productivity improvements. Still hummin and hawin.
I definitely agree with everyone else that you should separate the tests from your production code. If you insist on not, however, you should define a conditional comiplation constant called TEST, and wrap all of your unit test classes with a
#if TEST
#endif
first to ensure that the test code does not compile in a production scenario. Once that is done, you should either be able to exclude the test dlls from your production deployment, or even better (but higher maintenance), create an NAnt or MSBuild for production that compiles without the references to the test dlls.
I always create a separate Acme.Stuff.Test project that is compiled separately.
The converse argument is: Why do you want to take the tests out? Why not deliver the test? If you deliver the tests along with a test runner you have some level of acceptance test and self test delivered with the product.
I've heard this argument a few times and thought about it but I personally still keep tests in a separate project.