I am using Visual Studio 2015 Enterprise RTM to write unit tests for a project that uses Unity Container.
I have discovered that the simple act of adding a fakes assembly for Unity, not even actually using the fake, is sufficient to generate this exception:
System.InvalidProgramException: Common Language Runtime detected an invalid program.
Consider the following steps to reproduce:
Using Visual Studio 2015 Enterprise RTM create a Unit Test project targeting .NET 4.6
Add the NuGet package "Unity" version 3.5.1404.0
Add the NuGet package "CommonServiceLocator" version 1.2.0
Write a single unit test like so:
[TestClass]
public class UnitTest1 : IDisposable
{
[TestMethod]
public void TestMethod1()
{
new ResolvedArrayParameter<IDisposable>(new IDisposable[] {this});
}
void IDisposable.Dispose()
{
}
}
Verify the test passes
Right click on the Microsoft.Practices.Unity reference and choose "Add Fakes Assembly"
Re-run the test
Observe the following remarkable test failure:
Test Name: TestMethod1
Test FullName: UnitTestProject11.UnitTest1.TestMethod1
Test Source: c:\temp\UnitTestProject11\UnitTestProject11\UnitTest1.cs : line 12
Test Outcome: Failed
Test Duration: 0:00:00.0572447
Result StackTrace:
at Microsoft.Practices.Unity.ResolvedArrayParameter..ctor(Type arrayParameterType, Type elementType, Object[] elementValues)
at Microsoft.Practices.Unity.ResolvedArrayParameter`1..ctor(Object[] elementValues)
at UnitTestProject11.UnitTest1.TestMethod1() in c:\temp\UnitTestProject11\UnitTestProject11\UnitTest1.cs:line 13
Result Message:
Test method UnitTestProject11.UnitTest1.TestMethod1 threw exception:
System.InvalidProgramException: Common Language Runtime detected an invalid program.
The most extraordinary feature of this problem is evidently fakes don't even need to appear directly in the code for the failure to manifest.
An extensive amount of fiddling reveals that retargeting the test project to .NET 4.5 "fixes" the problem, which is a non-starter for me because of another issue I posted some weeks back.
Even more fiddling with virtually all fakes settings (code contracts, etc.) yielded no solution.
Any advice on this issue would be very much appreciated.
The only general solution is to make sure that all parts match the version of CLR you are using very closely and that VS has the latest updates.
There is no magic bullet for this problem. You need to know (dig out) the exact CLR version compatibility of all parts that are connected in your project when you inject fakes. Mind you that "compatibility" might be just the matter of manifest but more frequently they are the matter of nuances of how was/is the final code generated and for which version of the virtual machine.
These things normally don't matter for running and debugging since there are several layers that make sure that minor version differences either don't matter or you get a silent switch to whatever your code is declared to be compatible with.
But, when you use Fakes, the "system" injects raw code into your (which includes 3rd party libraries involved) and that means that it skips most checks - couldn't work otherwise. But, when the time comes to actually run the code, the engine (virtual machine) has to do some checks for it's own safety/integrity and it tends to get paranoid and bail out if it looks like declarations don't match closely enough.
This is the reason why someone asked whether assemblies involved are strongnamed or signed. That's the only level of guarantee that "system" will really trust. Without that it will do a degree of guessing and while it mostly doesn't matter for normal runs if matters a lot for code injection.
I'm still not talking possible real problems - this is all assuming that actual code is fine and just declarations are confused. You could try to play with that but it would take a lot of time and effort. Much easier to go checking if you can get versions of assemblies that are better matching.
The fact that errors vanished when you switched your flavor back to 4.5 tells you that either some of the assemblies involved are not "close enough" for 4.6 or there can be some glitches with code injection that were fixed by updates you haven't taken in yet.
Yes it involves a lot of pain but that's the price of wanting to be at the frontier.
Related
I'm struggling to get my unit tests to work. I've wrestled with this issue for several hours now, and I have no explanation as to why things don't work. I had a fairly major refactor on my codebase and have since gone through and fixed all the unit tests. The test project builds, it outputs a new unit test dll. However, when I go to run the tests in the test explorer, I get this message:
[2/27/2019 5:08:05 PM Warning] [MSTest][Discovery][C:\pathtotest.dll] Failed to discover tests from assembly C:\pathtotest.dll. Reason:Could not load file or assembly 'System.Runtime, Version=4.1.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
[2/27/2019 5:08:05 PM Warning] No test matches the given testcase filter FullyQualifiedName=<namespace.namespace...testmethod> in C:\pathtotest.dll
Here is what I know:
I recently updated Visual Studio (within the past two weeks, don't remember exactly when I did it).
All packages in the test project have been updated and are running the latest versions of things.
I have another unit test project that is .Net Core 2.1, this is .Net Framework 4.7.2. The other project works.
Some suggestions other posts give are to make sure your test architecture is correct and to delete a folder in %TEMP% (don't recall the exact name, except is was something about VisualStudioExtensions). The folder they suggest to delete was not found in %TEMP% and I tried running my tests on both architectures with the same result.
So the next step was to take a sanity test and make sure the built test dll exists. It does.
At this point I'm about ready to just start a new test project and copy paste over all the tests one by one and see if maybe one is throwing a silent error. I can't find any useful information with my own Google-fu skills and I'm hoping someone has some useful insight or tricks to try.
From the comment above:
Using this C:\pathtotest.dll
Check also if it 32bit or 64bit runtime.
Most cases are using 32bit dlls.
Hope it helps
I had a similar issue with a brand new test project in an existing solution - all of my other test projects compiled and tested correctly, but the new one repeatedly came up with the error:
No test matches the given testcase filter...
The answer came from this post VSTest: A testsettings file or a runsettings with a ForcedLegacyMode set to true is not supported with the MSTest V2 Adapter. No test is available which suggested switching off the testsettings file, which somehow had been selected on the new project.
As soon as I deselected the Test -> Test Settings -> c:...\Repos...testsettings option in Visual Studio the tests were runnable.
In my case, the issue ended up being a mix of two different test engines. I inadvertently decorated my test methods with [TestMethod] (MSTest) when I was using NUnit. Once I changed my test methods to just [Test] and used the proper test runner, I was finally in business.
I had a similar issue. In my case, the problem was the return type.
The test method needed to use async but the return type was void instead of Task. Simple and small think that might eat a lot of time.
Attempting to setup a unit test that calls a WCF service. Running under NET4.6 The client uses the basicHttpsBinding.
Have copied the system.serviceModel config section across (which includs the basicHttpsBinding tags to the unit Test's app.config.
When I run the test (xUnit) I get the following error when the Client constructor runs.
Exception thrown: 'System.Configuration.ConfigurationErrorsException' in System.Configuration.dll
Additional information: Configuration binding extension 'system.serviceModel/bindings/basicHttpsBinding' could not be found.
Verify that this binding extension is properly registered in system.serviceModel/extensions/bindingExtensions and that it is spelled correctly.
Another answer suggested adding the following (to ensure at least net4.5 is in executd for the test.
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6"/>
</startup>
Any clues?
Edit: This is only failing under resharper 10 Test Runer. If I use CTRL-R, T and run against the InBuild runner it works fine.
I have tried a few options in Resharper,
force CLR to 4.5,
disable/enable shadow copy, and
run in own
appDomain.
none changed the result.
I don't see any obvious reasons for this to fail - the xunit plugin (now a part of ReSharper itself as of version 2016.1) does work with app.config files. If you could log an issue, and ideally attach the solution (you can do so privately, so only JetBrains staff can see, or we can do NDA if required) I can take a look for you.
I suspect it is similar to why my similar WCF ordered tests fail under the Microsoft tools...Running a single test works because it is run directly out of \bin\Debug...but running multiple units as an ordered test fails because to run it Microsoft uses an executable sandbox called QTAgentXXXXX.exe.
To find out what executable is running your tests, simply break point early on, and dump out the AppDomain.CurrentDomain.SetupInformation (note the TargetFrameworkName and ApplicationName properties specifically) and also dump out a call to System.Diagnostics.Process.GetCurrentProcess().MainModule.FileVersionInfo to get the executable path.
Load this executable into ILSpy and you can check out the framework it was compiled against.
The Microsoft QTAgent apps are under C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\ and there seems to exist several varieties depending on the .Net Framework being used or whatever weird detection scheme they use to pick one....
UNFORTUNATELY, I observe that the most recent QTAgent32_40.exe is compiled under the old 4.0 Framework and before you ask, yes there is a QTAgent32_40.exe.config and no, adding the supportedRuntime setting with sku=4.6 does not encourage it to operate any differently (Maybe you would see different results tweaking your resharper 10 Test Runer config though). And I've tried the assemblyBinding appliesTo attribute trick mentioned in several threads to no avail as well.
I attempted to use ILSpy to recompile my own version of QTAgent against 4.6.2 but a lot of the Microsoft.VisualStudio.blah.blah assemblies being referenced have a lot of internal classes that you cannot build against. These assemblies have this crappy assembly attribute called InternalsVisibleTo which lists what external libraries and executables are allowed access to these internal classes. The QTAgent executables are among these attributes, which is why they can use them from those assemblies. So it would be a very long and deep rabbit hole to see if one could essentially decompile ALL these related assemblies and recompile the unit testing framework. That is a lot of research for such a small gain. I say screw Microsoft's Unit Testing framework. It is fraught with other issues as well. My co-worker keeps suggesting to use NUnit. I think I may give it a try.
For the last few months I've been developing a side project for my company, but the higher-ups have now decided it would be a good fit in an existing product.
I've been developing the side project using Microsoft's Code Contracts for static type checking (partly because I hadn't used them before and was eager to learn).
My problem is that if I check in my code to the code base with Contracts in place, will every other developer need the Code Contracts tools installed to be able to continue developing? I know for a fact that none of them have it installed, and I'm the junior here so I doubt I could convince them all to take it up.
I'm using .Net 4.5, so the Code Contract libraries are included, but I'm wondering if Visual Studio will complain that they're not building with CONTRACTS_FULL specified in the build options every time they go to build, or, if I leave CONTRACTS_FULL in the build options, what will happen when another developer tries to build? Additionally I'm wondering how the end-product will act when a Contract fails, but the code has not been built with the Code Contracts Rewriter.
I created a new solution with just one project. Created a simple function that fired a code contract violation, with code contracts uninstalled and CONTRACTS_FULL not specified. Built and ran it and received the following error:
Run-time exception (line 8): An assembly (probably "hdxticim") must be rewritten using the code contracts binary rewriter (CCRewrite) because it is calling Contract.Requires<TException> and the CONTRACTS_FULL symbol is defined. Remove any explicit definitions of the CONTRACTS_FULL symbol from your project and rebuild. CCRewrite can be downloaded from http://go.microsoft.com/fwlink/?LinkID=169180.
After the rewriter is installed, it can be enabled in Visual Studio from the project's Properties page on the Code Contracts pane. Ensure that "Perform Runtime Contract Checking" is enabled, which will define CONTRACTS_FULL
I think the error message needs rewriting, as CONTRACTS_FULL is most definitely not specified.
Thanks to Matías Fidemraizer, we've worked out that this happens when using Contract.Requires<TException>() and not on Contract.Requires().
Ideally, I'd like to modify this behaviour so that the Contract fires the provided exception as if it were a normal guard statement instead of complaining about the rewriter.
Here's a fiddle demonstrating the problem: https://dotnetfiddle.net/cxrAPe
The short answer is: Yes. If you check in the code with Code Contracts, then all developers that might build that code must also have Code Contracts installed in order to build the code.
In contradiction to what #CBauer wrote in his answer, there is a "blessed" package for Code Contracts. No, it's not a NuGet package—it's an MSI installer based installation.
Finally, if you are in a Continuous Integration environment for debug builds (e.g. Development, QA/QC and/or Testing), then those build servers will also need to have Code Contracts installed.
When you use Code Contracts, Debug builds always require the use of Code Contracts. Please note that this is not necessarily the case for Release builds. It depends on what form of contract checking you're using and the options specified in the Project Properties.
The Code Contracts manual has all the details. It's quite good and I highly recommend taking the time to read and understand it.
It should be noted that if you use the Contract.Requires<TException>(bool condition) form of preconditions, you must have Code Contracts enabled for Release builds (see Section 5: Usage Guidelines, specifically, page 20, Usage 2 scenario).
Since you are integrating this code into an existing codebase that has not been developed with Code Contracts, you should consider revising your Code Contracts Project Property settings to conform to Usage Scenario 3 outlined on page 20 of the Code Contracts manual, and reformulate your contracts using the "legacy" if-then-throw pattern. This will enable your team to best tranisition the code base to using Code Contracts everywhere, allowing you to eventually replace "legacy" if-then-throw precondition checks with actual Code Contracts Contract.Requires(bool condition) checks, and if you like, Contract.Requires<TException>(bool condition) checks.
UPDATE: There Will Soon Be A NuGet Package For Code Contracts
I was at the new GitHub repository for Code Contracts today. For those of you who don't know, Microsoft has open-sourced this and it is now a community-driven effort.
They recently (back in January) announced v1.10.xxxx.RC1 release. You can find information about it here on their GitHub repository.
Unfortunately it's difficult to keep a library that is not installed via nuget up to date without forcing your coworkers to maintain their personal environment. Code contracts does not appear to have an official Microsoft-blessed package out there for use.
fourpastmidnight's post below has a recently updated answer to my original concern above, but the bottom half I think remains relevant even now. Get your buy-in, people!
In my (admittedly subjective) experience, tools like this come to be reviled if buy-in is not established among coworkers in advance. You might try broaching the subject slowly to see if they would be receptive to using it.
Contract.Requires<TException>() will fail if the code contracts rewriter is not installed, regardless of setting CONTRACTS_FULL or not.
One solution is to build a separate class FailableContract which tests the preprocessor condition CONTRACTS_FULL, which is set by Code Contracts.
Then all you have to do is remember to remove CONTRACTS_FULL from your build parameters before committing your code to the code base. You still get code contracts, everybody else gets guard statements.
A small portion of FailableContract (which I'm still writing):
public class FailableContract
{
public static void Requires(bool condition)
{
#if CONTRACTS_FULL
Contract.Requires(condition);
#else
if (!condition) throw new ArgumentException(); //Just go with an argument exception because we haven't been supplied with an exception type.
#endif
}
public static void Requires<TException>(bool condition) where TException : Exception, new()
{
#if CONTRACTS_FULL
Contract.Requires<TException>(condition);
#else
if (!condition) throw new TException();
#endif
}
}
Then a simple Find, Replace Contract. to FailableContract.should get most issues.
Summary: I can run unit tests and code-coverage, but the report only includes NUnit classes, not my application classes.
I have successfully used PartCover in the past. Not so this time. I tried the latest PartCover (4.0), downgraded to the next latest (2.0), both with NUnit 2.5.6.
I created a simple .NET 4.0 class library (also tried this with a web application that has a project that's a class library) with a single class in some namespace, and two test methods in another library in another class.
NUnit/PartCover installed correctly; I can run the NUnit tests both in NUnit, and through PartCover (I can see them running and saying "2 passed"), but the report only shows me NUnit namespaces. (Yes, I'm using +[] as my coverage rule.)
Any ideas? As much as I like NUnit, I'd like to see coverage for my own classes :o)
And I also tried aligning the test-DLL and code-DLL namespaces, to no avail.
Edit: I tried re-running my previously working code-covered sample from a year ago; all the tests run, but the actual project namespaces don't appear. There's a hint here, which seems to imply that it depends on the NUnit version you use: http://sourceforge.net/projects/partcover/forums/forum/605222/topic/3308367 (and yes, I already tried the appdomain-reporting checkbox)
I've tried NUnit 2.5.5.x and 2.5.6.x and both give me the same results.
Edit: It seems this fork of the official 4.0 version seems to work, albeit sporadically (google for PartCover fork, I can't add more hyperlinks)
Madness. Apparently, pressing Pause/Break on your keyboard after NUnit prints out the summary of total passed/failed, and waiting approximately one second for the second "CoreProfiler is turned off" message, makes it all work.
Surely, this cannot be the real solution. Sure, I can rig up a batch file that'll sleep ~1 second after executing NUnit, but this seems like a major hack.
The correct way to handle this is to add the required runtime to nunit's configuration. You will notice that NUnit is running in CLR 2.0 instead of 4.0. There are numerous answers to this question on SO, but I found this one first. Doing this alone fixed it for me. Note, your version of the runtime may be slightly different. You may need to confirm.
I try to use NUnit with SnippetCompiler http://www.sliver.com/dotnet/SnippetCompiler/
I have added references to nunit.framework.dll in snippetcompiler (Menu Tools, References) and compiled nunit sample http://www.nunit.org/index.php?p=quickStart&r=2.5.2 to bank.dll
but when I open bank.dll in NUnit GUI it fails saying it cannot load NUnit assembly or one of its dependencies.
Is it possible to fix this ?
I couldn't even get v2.0.8.3 of SnippetCompiler to include the reference. It let me do it, but it wouldn't compile.
In any case writing unit tests isn't the purpose of SnippetCompiler. It's designed to do quick spikes - try something and see if it works. In other words, it's throwaway code.
In addition, the version for .NET 3.5 (the one I'm using) is an alpha release; the developer does not seem to be maintaining this. (Not to put down the author - this was an awesome tool that saved me lots of time!)
For writing spikes against modern versions of .NET, I've switched to LINQPad. Change Edit/Preferences/Query to C# Program and it's very similar to SnippetCompiler. The basic version is free; for a small fee, the registered version provides IntelliSense.
Even if you're writing learning tests with NUnit, you'll want to preserve those tests. Use Visual Studio (or another IDE) and create a separate Class Library project for your tests.
Maybe this is the solution
http://weblogs.asp.net/rosherove/archive/2008/02/21/ad-hoc-unit-tests-with-snippet-compiler.aspx
Ad hoc Unit Tests with Snippet
Compiler
If you're a fan of snippet compiler
(if you're not you should seriously
check it out)Travis Illig published a
little template for writing Typemock
Isolator test inside this handy little
tool.
the reasons you'd need a specialized
template in the first place to write
these kinds of tests in snippet
compiler:
1) Typemock Isolator uses the .NET
profiling APIs to work its magic, so
the .net process running your tests
need to have a couple of environment
variables enabled to work
2) His code template actually creates
and runs a new process that triggers
nunit-console.exe with the path of the
current code you just wrote in snippet
compiler allowing you to effecively
write and run unit tests in snippet
compiler!
3) the nunit-console process will
already have the env. vars as
mentioned in the first item set to it.
Travis' template will work for
anything nunit can run, not just
typemock isolator tests, which is
pretty cool.