I recently added Code Contracts to my solution. After some modifications our build runs without any problems but our unit tests fail because of Code Contracts.
Environment:
Source control and build server are hosted on Visual Studio Team Service (VSTS) Scripted Build (formerly VSO)
VS 2013 Premium Code (now VS 2015 Enterprise) contracts enabled on configurations Debug and Release
Code contract turned off with DoNotBuild flag set for custom configurations BuildServer
Visual Studio Team Services build definition:
Has step Visual Studio
has flags set /p:CodeContractsEnableRuntimeChecking=false;CodeContractsReferenceAssembly=false
Example error text from build server:
Test method
Web.Tests.AccountControllerTests.CreateAccount_Pass_NoPasswordSend_Test
threw exception: System.Diagnostics.Contracts.ContractException: An
assembly (probably "Cms.Web") must be rewritten using the code
contracts binary rewriter (CCRewrite) because it is calling
Contract.Requires and the CONTRACTS_FULL symbol is
defined. Remove any explicit definitions of the CONTRACTS_FULL symbol
from your project and rebuild.
I have examined the diagnostic output from the build server and there is no symbol CONTRACTS_FULL that can be found anywhere.
I do not want to enable code contracts on the build server mostly because I had tried that before with no success and due to time constraints I gave up (after more searching I came to the conclusion that this is not possible as Microsoft does not allow for custom extensions to be installed on the build server). I simply want to ignore code contracts on the build server but the Unit Test execution does not seem to do this.
Any one have any ideas on where I need to look to ensure code contracts are ignored by the unit tests? Seems like this should be possible right? Maybe there is another flag I could set in the build definition that will be picked up by the unit tests?
Edit
Please note the build server is HOSTED in the cloud as a Microsoft service as a part of Visual Studio Team Services (formerly Visual Studio Online). This means it is not our build server, we have no control over what is installed on the build server. Even so we want to use Code Contracts. The accepted answer below is a way to allow for this.
Please, please read the Code Contracts manual. It tells you everything you need to know. You don't need any ConditionalAttribute defined on your methods. You can use Contract.Requires<TException>(bool condition) if you know what the requirements for doing so are.
As #Mixxiphoid said in their answer, because they were using the generic form of Requires, Code Contracts is required to be installed on the build server and Peform Runtime Contract Checking needs to be enabled for the Release configuration in addition to the Debug configuration.
If you are using Code Contracts, then you will always need Code Contracts installed on the build servers that build Debug releases. The same goes for your team: all developers who need to compile the code will be required to have Code Contracts installed for Debug builds.
It's all in Section 5: Usage Guidelines in the Code Contracts manual. There's a helpful diagram on page 20 that gives a great summary of the various usage scenarios and their requirements.
About Contract.Requires<TException>(bool condition)
First off, the developer that wrote this method for Code Contracts did not forget to apply the ConditionalAttribute to the method, as the accepted answer states. Far from it. The very fact that this method is not decorated with that attribute is by design.
If you read Section 5: Usage Guidelines, you see that if you use this form of the Requires method, then you are required to use the binary rewriter—ccrewrite—when building your assembly. This implies that any machine which will build your assembly must have ccrewrite available.
So, what about build servers that you may not control? I'm glad you asked.
TFS Hosted Build Servers
With respect to TFS hosted build servers: It's true that you cannot simply install Code Contracts on the build servers. So what are your options?
You could limit yourself to using only the "legacy" form of code contracts, if-then-throw blocks followed by Contract.EndContractBlock() or limit yourself to only using Contract.Requires(bool condition) (the non-generic form), or a mixture of the two.
In either case, you'll probably want to have a separate Debug build configuration for the TFS-hosted build server. This build configuration would have Perform Runtime Contract Checking disabled for that build configuration.
You could do a little "bing-ing" or "googling" and come up with this search result, which describes how to enable building your solution with Code Contracts on TFS Online.
The blog post linked to above has instructions for modifying a TFS Hosted Build Controller hosted build.
UPDATE: THERE WILL SOON BE A NUGET VERSION OF CODE CONTRACTS
For those of you using hosted build services, such as Visual Studio Online, AppVeyor, etc., the new community-driven Code Contracts GitHub repo recently announced v1.10.xxxxx.RC1. In the release notes, they mention a NuGet version of code contracts. Enjoy.
After double checking this build configuration also has the same errors in the unit test as on the build server. After some digging I came up with the answer.
The answer lies in Contract.Requires. The generic version of method Contract.Requires<T> does not apply the attribute [Conditional("CONTRACTS_FULL")] where as the non-generic method Contract.Requires does. This means that Contract.Requires<T> is always evaluated and there is nothing you can do about it. See Cutting Edge - Code Contracts Settings in Visual Studio 2010 for more detail.
After I reverted all of my code to make use of Contract.Requires instead of Contract.Requires<T> all my unit tests were able to execute again. You can still make use of ContractClassFor and ContractClass attributes and make use of the generic Contract.Requires<T> implementation within the contract class without any problem, this because the contract class will not get built without CONTRACTS_FULL.
Got the exact same problem this morning. The solution was to enable runtime contract checking on release mode instead of only debug mode.
Beside that, CodeContracts had to be installed on the build server as well.
Related
Attempting to setup a unit test that calls a WCF service. Running under NET4.6 The client uses the basicHttpsBinding.
Have copied the system.serviceModel config section across (which includs the basicHttpsBinding tags to the unit Test's app.config.
When I run the test (xUnit) I get the following error when the Client constructor runs.
Exception thrown: 'System.Configuration.ConfigurationErrorsException' in System.Configuration.dll
Additional information: Configuration binding extension 'system.serviceModel/bindings/basicHttpsBinding' could not be found.
Verify that this binding extension is properly registered in system.serviceModel/extensions/bindingExtensions and that it is spelled correctly.
Another answer suggested adding the following (to ensure at least net4.5 is in executd for the test.
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6"/>
</startup>
Any clues?
Edit: This is only failing under resharper 10 Test Runer. If I use CTRL-R, T and run against the InBuild runner it works fine.
I have tried a few options in Resharper,
force CLR to 4.5,
disable/enable shadow copy, and
run in own
appDomain.
none changed the result.
I don't see any obvious reasons for this to fail - the xunit plugin (now a part of ReSharper itself as of version 2016.1) does work with app.config files. If you could log an issue, and ideally attach the solution (you can do so privately, so only JetBrains staff can see, or we can do NDA if required) I can take a look for you.
I suspect it is similar to why my similar WCF ordered tests fail under the Microsoft tools...Running a single test works because it is run directly out of \bin\Debug...but running multiple units as an ordered test fails because to run it Microsoft uses an executable sandbox called QTAgentXXXXX.exe.
To find out what executable is running your tests, simply break point early on, and dump out the AppDomain.CurrentDomain.SetupInformation (note the TargetFrameworkName and ApplicationName properties specifically) and also dump out a call to System.Diagnostics.Process.GetCurrentProcess().MainModule.FileVersionInfo to get the executable path.
Load this executable into ILSpy and you can check out the framework it was compiled against.
The Microsoft QTAgent apps are under C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\ and there seems to exist several varieties depending on the .Net Framework being used or whatever weird detection scheme they use to pick one....
UNFORTUNATELY, I observe that the most recent QTAgent32_40.exe is compiled under the old 4.0 Framework and before you ask, yes there is a QTAgent32_40.exe.config and no, adding the supportedRuntime setting with sku=4.6 does not encourage it to operate any differently (Maybe you would see different results tweaking your resharper 10 Test Runer config though). And I've tried the assemblyBinding appliesTo attribute trick mentioned in several threads to no avail as well.
I attempted to use ILSpy to recompile my own version of QTAgent against 4.6.2 but a lot of the Microsoft.VisualStudio.blah.blah assemblies being referenced have a lot of internal classes that you cannot build against. These assemblies have this crappy assembly attribute called InternalsVisibleTo which lists what external libraries and executables are allowed access to these internal classes. The QTAgent executables are among these attributes, which is why they can use them from those assemblies. So it would be a very long and deep rabbit hole to see if one could essentially decompile ALL these related assemblies and recompile the unit testing framework. That is a lot of research for such a small gain. I say screw Microsoft's Unit Testing framework. It is fraught with other issues as well. My co-worker keeps suggesting to use NUnit. I think I may give it a try.
I am using Visual Studio 2015 Enterprise RTM to write unit tests for a project that uses Unity Container.
I have discovered that the simple act of adding a fakes assembly for Unity, not even actually using the fake, is sufficient to generate this exception:
System.InvalidProgramException: Common Language Runtime detected an invalid program.
Consider the following steps to reproduce:
Using Visual Studio 2015 Enterprise RTM create a Unit Test project targeting .NET 4.6
Add the NuGet package "Unity" version 3.5.1404.0
Add the NuGet package "CommonServiceLocator" version 1.2.0
Write a single unit test like so:
[TestClass]
public class UnitTest1 : IDisposable
{
[TestMethod]
public void TestMethod1()
{
new ResolvedArrayParameter<IDisposable>(new IDisposable[] {this});
}
void IDisposable.Dispose()
{
}
}
Verify the test passes
Right click on the Microsoft.Practices.Unity reference and choose "Add Fakes Assembly"
Re-run the test
Observe the following remarkable test failure:
Test Name: TestMethod1
Test FullName: UnitTestProject11.UnitTest1.TestMethod1
Test Source: c:\temp\UnitTestProject11\UnitTestProject11\UnitTest1.cs : line 12
Test Outcome: Failed
Test Duration: 0:00:00.0572447
Result StackTrace:
at Microsoft.Practices.Unity.ResolvedArrayParameter..ctor(Type arrayParameterType, Type elementType, Object[] elementValues)
at Microsoft.Practices.Unity.ResolvedArrayParameter`1..ctor(Object[] elementValues)
at UnitTestProject11.UnitTest1.TestMethod1() in c:\temp\UnitTestProject11\UnitTestProject11\UnitTest1.cs:line 13
Result Message:
Test method UnitTestProject11.UnitTest1.TestMethod1 threw exception:
System.InvalidProgramException: Common Language Runtime detected an invalid program.
The most extraordinary feature of this problem is evidently fakes don't even need to appear directly in the code for the failure to manifest.
An extensive amount of fiddling reveals that retargeting the test project to .NET 4.5 "fixes" the problem, which is a non-starter for me because of another issue I posted some weeks back.
Even more fiddling with virtually all fakes settings (code contracts, etc.) yielded no solution.
Any advice on this issue would be very much appreciated.
The only general solution is to make sure that all parts match the version of CLR you are using very closely and that VS has the latest updates.
There is no magic bullet for this problem. You need to know (dig out) the exact CLR version compatibility of all parts that are connected in your project when you inject fakes. Mind you that "compatibility" might be just the matter of manifest but more frequently they are the matter of nuances of how was/is the final code generated and for which version of the virtual machine.
These things normally don't matter for running and debugging since there are several layers that make sure that minor version differences either don't matter or you get a silent switch to whatever your code is declared to be compatible with.
But, when you use Fakes, the "system" injects raw code into your (which includes 3rd party libraries involved) and that means that it skips most checks - couldn't work otherwise. But, when the time comes to actually run the code, the engine (virtual machine) has to do some checks for it's own safety/integrity and it tends to get paranoid and bail out if it looks like declarations don't match closely enough.
This is the reason why someone asked whether assemblies involved are strongnamed or signed. That's the only level of guarantee that "system" will really trust. Without that it will do a degree of guessing and while it mostly doesn't matter for normal runs if matters a lot for code injection.
I'm still not talking possible real problems - this is all assuming that actual code is fine and just declarations are confused. You could try to play with that but it would take a lot of time and effort. Much easier to go checking if you can get versions of assemblies that are better matching.
The fact that errors vanished when you switched your flavor back to 4.5 tells you that either some of the assemblies involved are not "close enough" for 4.6 or there can be some glitches with code injection that were fixed by updates you haven't taken in yet.
Yes it involves a lot of pain but that's the price of wanting to be at the frontier.
The DiagnosticAnalyzer is a custom Roslyn based extension whose DiagnosticDescriptor with DiagnosticSeverity.Error is as shown below
internal static DiagnosticDescriptor Rule = new DiagnosticDescriptor(DiagnosticId, Description, MessageFormat, Category, DiagnosticSeverity.Error);
When the extension is used it does show the red squiggles over the codes that is against the Diagnostic Analyzers custom rule emphasising that it is an error, it is even shown in the Error List window of Visual Studio.
But when compiled, this code gets a compilation succeeded message in the Output Window of Visual Studio. This is working against the whole concept of emphasising an error as severity for a diagnostic analyzer.
If this is the default way of working for Roslyn, then do we have a work around for stopping the compilation.
I'm using:
Visual Studio 2013 Update 3
Roslyn End User Preview.vsix
VSIX extension containing the DiagnosticAnalyzer made using Roslyn SDK Project Templates.vsix
The integration of diagnostics into the actual build pipeline has been happening as part of VS "14". If you try the same scenario there, it will cause a compilation error.
Additionally, diagnostics are now attached to projects and distributed via NuGet so that the errors will happen consistently for the entire team, continuous integration build, etc.
I'm using VS2015 Update 3 and this issue is still there. I wanted to enforce local variable and parameter naming conventions, it turned out Roslyn doesn't support these by default, only by workaround: SO link
Then I wanted to fail the build if there is such an error, but Roslyn is not capable of this.
Roslyn could have been a very robust and powerful tool, but it is still failing in the most basic tasks. What a pity.
(For the moderators: I still don't have 50 rep to comment and I don't think I will ever have)
For the last few months I've been developing a side project for my company, but the higher-ups have now decided it would be a good fit in an existing product.
I've been developing the side project using Microsoft's Code Contracts for static type checking (partly because I hadn't used them before and was eager to learn).
My problem is that if I check in my code to the code base with Contracts in place, will every other developer need the Code Contracts tools installed to be able to continue developing? I know for a fact that none of them have it installed, and I'm the junior here so I doubt I could convince them all to take it up.
I'm using .Net 4.5, so the Code Contract libraries are included, but I'm wondering if Visual Studio will complain that they're not building with CONTRACTS_FULL specified in the build options every time they go to build, or, if I leave CONTRACTS_FULL in the build options, what will happen when another developer tries to build? Additionally I'm wondering how the end-product will act when a Contract fails, but the code has not been built with the Code Contracts Rewriter.
I created a new solution with just one project. Created a simple function that fired a code contract violation, with code contracts uninstalled and CONTRACTS_FULL not specified. Built and ran it and received the following error:
Run-time exception (line 8): An assembly (probably "hdxticim") must be rewritten using the code contracts binary rewriter (CCRewrite) because it is calling Contract.Requires<TException> and the CONTRACTS_FULL symbol is defined. Remove any explicit definitions of the CONTRACTS_FULL symbol from your project and rebuild. CCRewrite can be downloaded from http://go.microsoft.com/fwlink/?LinkID=169180.
After the rewriter is installed, it can be enabled in Visual Studio from the project's Properties page on the Code Contracts pane. Ensure that "Perform Runtime Contract Checking" is enabled, which will define CONTRACTS_FULL
I think the error message needs rewriting, as CONTRACTS_FULL is most definitely not specified.
Thanks to Matías Fidemraizer, we've worked out that this happens when using Contract.Requires<TException>() and not on Contract.Requires().
Ideally, I'd like to modify this behaviour so that the Contract fires the provided exception as if it were a normal guard statement instead of complaining about the rewriter.
Here's a fiddle demonstrating the problem: https://dotnetfiddle.net/cxrAPe
The short answer is: Yes. If you check in the code with Code Contracts, then all developers that might build that code must also have Code Contracts installed in order to build the code.
In contradiction to what #CBauer wrote in his answer, there is a "blessed" package for Code Contracts. No, it's not a NuGet package—it's an MSI installer based installation.
Finally, if you are in a Continuous Integration environment for debug builds (e.g. Development, QA/QC and/or Testing), then those build servers will also need to have Code Contracts installed.
When you use Code Contracts, Debug builds always require the use of Code Contracts. Please note that this is not necessarily the case for Release builds. It depends on what form of contract checking you're using and the options specified in the Project Properties.
The Code Contracts manual has all the details. It's quite good and I highly recommend taking the time to read and understand it.
It should be noted that if you use the Contract.Requires<TException>(bool condition) form of preconditions, you must have Code Contracts enabled for Release builds (see Section 5: Usage Guidelines, specifically, page 20, Usage 2 scenario).
Since you are integrating this code into an existing codebase that has not been developed with Code Contracts, you should consider revising your Code Contracts Project Property settings to conform to Usage Scenario 3 outlined on page 20 of the Code Contracts manual, and reformulate your contracts using the "legacy" if-then-throw pattern. This will enable your team to best tranisition the code base to using Code Contracts everywhere, allowing you to eventually replace "legacy" if-then-throw precondition checks with actual Code Contracts Contract.Requires(bool condition) checks, and if you like, Contract.Requires<TException>(bool condition) checks.
UPDATE: There Will Soon Be A NuGet Package For Code Contracts
I was at the new GitHub repository for Code Contracts today. For those of you who don't know, Microsoft has open-sourced this and it is now a community-driven effort.
They recently (back in January) announced v1.10.xxxx.RC1 release. You can find information about it here on their GitHub repository.
Unfortunately it's difficult to keep a library that is not installed via nuget up to date without forcing your coworkers to maintain their personal environment. Code contracts does not appear to have an official Microsoft-blessed package out there for use.
fourpastmidnight's post below has a recently updated answer to my original concern above, but the bottom half I think remains relevant even now. Get your buy-in, people!
In my (admittedly subjective) experience, tools like this come to be reviled if buy-in is not established among coworkers in advance. You might try broaching the subject slowly to see if they would be receptive to using it.
Contract.Requires<TException>() will fail if the code contracts rewriter is not installed, regardless of setting CONTRACTS_FULL or not.
One solution is to build a separate class FailableContract which tests the preprocessor condition CONTRACTS_FULL, which is set by Code Contracts.
Then all you have to do is remember to remove CONTRACTS_FULL from your build parameters before committing your code to the code base. You still get code contracts, everybody else gets guard statements.
A small portion of FailableContract (which I'm still writing):
public class FailableContract
{
public static void Requires(bool condition)
{
#if CONTRACTS_FULL
Contract.Requires(condition);
#else
if (!condition) throw new ArgumentException(); //Just go with an argument exception because we haven't been supplied with an exception type.
#endif
}
public static void Requires<TException>(bool condition) where TException : Exception, new()
{
#if CONTRACTS_FULL
Contract.Requires<TException>(condition);
#else
if (!condition) throw new TException();
#endif
}
}
Then a simple Find, Replace Contract. to FailableContract.should get most issues.
I have a C# .NET assembly, in visual studio 2008. If required I could upgrade it to VS 2010.
How can I find which code in the assembly is not called? Ideally if something could analyse the whole solution in one go. It is too much for me to analyse it all myself manually.
The assembly is shared between a client and server project so I need to understand which code is used where.
In VS 2010 you can use code coverage together with unit tests for that - so if you can refactor your client code into actual unit tests you can use that.
To quote from MSDN:
To determine what proportion of your
project's code is actually being
tested using unit tests, you can use
the code coverage feature of Visual
Studio Application Lifecycle
Management. To do this, first edit the
run configuration to indicate the
assembly that contains the code whose
coverage you want to measure. Then run
tests on that code. Detailed code
coverage statistics appear in a
window, and you can also see,
line-by-line, which code has been
tested.
Try getting resharper. That does that for you. You can get a free trial version from their website. http://www.jetbrains.com/resharper/download/
http://www.jetbrains.com/resharper/
Providing you don't mind buying something to do it, ReSharper will go off and find unused code for you.
FxCOP is a free tool from Microsoft that includes dead code detection.
For best resulsts, don't mark classes/methods public unless they really need to be public. It assumes that any public method is used by external assemblies.
Check Code Analysis in Visual Studio, help here.
You could try the safe delete function of ReSharper, keeping in mind it will say it's safe to delete code which is only used via reflection (such as Fluent automapping conventions).
Couldn't hurt to download a 30 day trial to see if it does what you need anyhow.