VS2012 Test explorer locks native .dll, making rebuilds fail - c#

I am using Visual Studio 2012 for a solution with a C# and a C++/CLI .dll, with the C++/CLI dll referencing native .dlls such as boost. The C++ code is compiled as x64.
When I open VS, I can clean and build my project.
Using test explorer, I can run my tests.
As soon as I've used test explorer to run tests once, I cannot rebuild the project. It seems that VS2012 Test Explorer keeps a lock on my C++/CLI-dll, and there I get the following error:
LNK1104: cannot open file 'C:\Dev\LockExample\bin\Debug\cli.dll'
As a result of this, whenever I have run the tests using Test Explorer, I need to restart VS2012 before I can continue developing. Obviously this is not a sustainable development process.
Testing and rebuilding works without problem with C#-only dlls - as far as I can tell the problem only occurs with DLLs that use native x64 code.
After some more testing, I found that the villain here is vstest.executionengine.exe. Using handle (from SysInternals), I see that vstest.executionengine.exe holds locks for the .dll and the .pdb of the cli-dll. It does not hold any locks for managed-only dlls.
How can I get Visual Studio Test Explorer to release the locks on the C++/Cli dlls after the test runs completed?

In Visual Studio 2013 this problem can easily be fixed by unchecking the option "Keep Test Execution Engine Running" under "Test -> Test Settings" in the menu.
I found the answer in another post:
vstest.executionengine.x86.exe not closing

After some more searching, I found this post on connect.microsoft.com. The last hint in workarounds does solve the problem, although it's an ugly hack.
I can rebuild if I add the following as pre-build events to my C++/CLI dll:
taskkill /F /IM vstest.executionengine.exe /FI "MEMUSAGE gt 1"
taskkill /F /IM vstest.executionengine.x86.exe /FI "MEMUSAGE gt 1"
This will kill the vstest.executionengine.exe process, thereby freeing the lock on my .dll file.

I've also encountered this issue while testing involving native dlls. The workaround (solution?) I found was to add a DeploymentItemAttribute to the tests - not sure if this is generally true, but it certainly worked for me. It's a bit of a pain if there are lots of them (I had 6 in my case), but once done it was easy to copy and paste to the other tests.
So my unit test class looks something like this:
[TestClass]
public class TestMyClass
{
[TestMethod]
[DeploymentItem("firstnative.dll")]
[DeploymentItem("secondnative.dll")]
public void TestMyMethod()
{
//Code which (indirectly) uses the above native dlls.
}
}

I've also been fighting this problem and initially used the "taskkill" workaround, but I just stumbled over an option in the VS2013 settings which seems to more elegantly solve this problem:
Remove the check-mark on the
Keep test execution engine running between test runs
option found at
Tools / Options / Web Performance Test Tools

I had written a C# utility to be able to load and unload native libraries in here
I used it in the test project as below. Since it unloads dll in Dispose, I don't get build errors.
public interface INativeCrypto : INativeImport
{
[ImportFunction("mynative.dll"]
int NativeMethod();
}
[TestClass]
public class UnitTest1
{
public void TestMethod1()
{
INativeCrypto impl = NativeImport.Create<INativeCrypto>("");
// Use methods in impl
int i = impl.NativeMethod();
//...
}
}

Adding some stuff to the answer of #frodesto, (in case of VS2013), "Test>Test Setting>Keep Test Executin Engine running" configuration is stored in the user configuration (SUO file). This can be a bit nasty in case of this error happens in the TFS Build agent, beacause it uses a service default user.
To fix this case, first kill the existing vstest.executionengine.exe, modify the user used by TFS Build agent to execute with the user you are logged on. Open the solution stored in TFS Build agent workspace and unselect the option.
After that, TFS Build agent will read the "keep test execution engine" option beacause the SUO file is for the same user.

Related

I need a CSharp script (.csx) runner, like csi.exe, for a build agent machine

I'm writing .csx build scripts. On my local machine I run them with CSharp interactive from visual studio or via C:\Program Files (x86)\MSBuild\14.0\bin\csi.exe. I've tried running .csx scripts on a TeamCity build agent machine (agent machines perform actual build tasks like compilation), which has Microsoft Built Tools 2015 installed. Surprisingly, there was no csi.exe in MSBuild folders.
I'm looking for a way to install csi.exe or a compatible standalone .csx runner on my build agent machines. The options found in Google, like https://github.com/cake-build/cake , all seem to have their distinct flavor of scripts or lots of additional functionality, whereas I'm looking for a csi.exe clone ideally.
Solution turned out to be easy: Roslyn NuGet package contains csi.exe.
It will be located at packages\microsoft.net.compilers.2.4.0\tools\csi.exe.
One thing to note - installation script for the package changes .csproj file to use the compiler provided by the package. This is likely not what you want, so make sure to roll-back such changes if they are made during installation.
You can compile your own runner with any customization you like using Roslyn API.
This code would do the basic job of running a script:
static void Main(string[] args)
{
Task<object> result = CSharpScript.EvaluateAsync(File.ReadAllText(args[0]));
result.Wait();
}
There is a very good tutorial in the Microsoft Documentation:
Cross-Platform Code Generation with Roslyn and .NET Core
It's somewhat equivalent to using csi.exe but with added benefits.
In their example, you have a variable const string code. This could be a string from your form or the contents of a script file.
I prefer this approach myself because:
It allows you to access and control the resources of your runner, instance it properly, measure its performance, etc. Simply running CSI could potentially explode in the memory if left untouched, and is much more opaque than managing it in code.
It returns code analysis in cases of errors. You can let the user easily pinpoint the source of an unsuccessful build.
It's cross-platform, unlike e.g. csi.exe sitting on a some hard drive
You can choose to compile at run-time, or create a DLL that runs much faster if pre-built
TeamCity 2021.2 provides a new C# Script runner, I hope what you need

Test Explorer Discovery Fails Unless Run As Administrator in Visual Studio 2017

I am running Visual Studio 2017 & the Test Runner cannot discover the Unit Tests unless Visual Studio is running as an Administrator. Running as Administrator is next to impossible at my new workplace.
As such...
Q: How can get the Test Explorer to discover Unit Tests without being an administrator?
UPDATE:
I have confirmed the following...
POSITIVE: The Unit Test Project is set to build
POSITIVE: The Test Project is a genuine Unit Test Project (w/ magic guids)
NEGATIVE: The Test View is not an available option in VS2017 Community Edition
A summary of various fixes for this no tests found issue.
You have 2 versions of the Microsoft.TestAdapter installed (reference Microsoft.VisualStudio.TestPlatform.TestFramework and Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions). Check NuGet Package Manager, version 1.1.11 doesn't work, you need to use 1.1.18 (at time of writing for it to work).
Also check your Bin folder &/or your Build Agents references folder for duplicate versions.
Make sure that your test project is set to build. If the test assembly isn't being built, VS won't be able to find any tests:
Magic GUIDs are needed in the Test Project:
C#:
{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}
VB:
{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{F184B08F-C81C-45F6-A57F-5ABD9991F28F}
Trying to get ignored tests running again. Same error message occurs if you remove the Ignore label. Does not automatically re-enable the test. This article takes you through the last step. http://richallen.blogspot.com/2008/05/ms-test-re-enabling-ignored-tests.html
Ie
Open the Test List Editor, selected the ignored test, right click and select "Enable" from the context menu.
Make sure your method belongs to a class with the [TestClass] Attribute and the method it's marked with the [TestMethod] Attribute.
Use Test View to find your test.
Open your Properties window (F4), and make sure your test is enabled
Some people have accidentially added a Class Library project instead of Test Project. Running Tests in a Class Library doesn't work, it has to be a Test Project, probably because of the Magic GUIDs.
Check whether the solution is running on 64bit. If so change it to x86.
Marking tests as static makes them fail to appear in the test list.
Close down Visual Studio and delete the .vsmdi (visual studio test meta data) file. This will be regenerated.

A pain-free way of debugging a "plug-in" application?

I'm about to start developing a desktop application (WPF) based on a "plugin" architecture, and was going to use MEF (and its DirectoryCatalog) to discover and load plugin assemblies. We're going to be developing many plugins, so it seems sensible to keep them in separate VS solutions rather than bloat the "core" application solution, but having only ever worked on single, standalone solutions, I suspect this is going to make debugging a bit tricky. I'm using VS2013 if that makes a difference.
I'm assuming that I'll still be able to step into a plugin in scenarios where the "core" application calls a method in that plugin? And I'm guessing that once in there, I'll be able to set breakpoints in those source code files that have been "visited"? But what if I want to add a breakpoint to a different source code file - one that hasn't been visited while stepping-through? How can I open that file? In a single solution I could just open it via Solution Explorer, but not (I'm guessing) when it's in a separate assembly.
I'm trying to pre-empt any problems I might have with this multi-solution approach, and wondered if VS had any clever features to simplify some of this stuff. Having separate solutions also means first compiling the plugin solution(s) that I want to test, then compiling and running the "core" application solution. While it's only a couple of extra mouse clicks, are there (again) any VS features that could help here?
This is a common scenario and not tricky at all.
In the project properties of your plug-ins, go to Debug -> Start Action and set Start external program to the executable of your core application.
This way, you only have to compile your core application once (probably using a build script that just builds everything), and debugging a plug-in will start the core application with the debugger attached and you can debug the plug-in (as soon as your core apllication loads the plug-in assembly).
Also keep in mind that you can dettach the debugger from the running application, switch to another instance of Visual Studio with another solution opened, and again attach to your running application. This comes in handy if you e.g. debug your plug-in and want to set or use existing break points in your core application.
As long as Visual Studio is able to find the debugging symbols (the *.pdb files), stepping through the code of e.g. your core application while debugging your plug-in is also no problem.
I see two ways to do this.
The more comfortable option:
1. You can add the external solution to the core solution.
Walkthrough: Adding an existing Visual Studio solution to another solution
By doing this you can organize your solution to reference the code and still keep each plugin solution separate at the same time.
You just reference those plugin solutions from your core solution that you currently want to work on. Also, using this approach you can organize the other solutions just like you would with normal projects and move thembetween virtual solutios folders to your liking until you have the most adequate folder structure.
Quote from the article:
The nice thing about this approach is that not only are all the
projects now in one solution but at any time, you can open the
separate solutions without impacting the "master" solution and vice
versa.
The files in the references solution can be opened and edited just like any other file from your "normal" projects, and of course, you can set breakpoint like in any other code file, too.
This way you can both edityour code and step through it, which I personally find much more convenient than switching and attaching to multiple processes.
2. Add the PDB files.
Put the DLLs with their corresponding PDBs of those plugins you want to debug into a folder and configure your core application to use that folder for the DirectoryCatalog. This enables you to step into the plugin code, but you will not be able to edit them.
#Andrew
Regarding debugging, it shouldn't be an issue as long as you drop the .pdb files with assembly in directory which you are using as DirectoryCatalog.
Regarding building plugin solution before Core- as you have 1 build file for each solution, you should check if you can write msbuild commands in a .bat file to get it executed one after other.
Besides all the above suggestions, another way to debug is to attach your addin solution to the running core process. Attach to Running Processes with the Visual Studio Debugger

NUnit is not releasing the .dll for my unit test project

I have the NUnit GUI runner to run tests on my unit test assembly every time that it is built. The problem is that when I try to build, the .dll in the Debug folder is in use by NUnit, which prevents it from being built, which prevents the automatic test run. Is there a way around this?
Can I suggest a slightly different approach but along the same lines as #UvarajGopu....
As long as you are using Visual Studio higher than "Express" version, and if you write your UnitTests in a separate project (typically alongside the project being tested, with a ".UnitTests" suffix), then do this:
Set your UnitTests project as being the StartUp project (right-click in Solution Explorer, "Set as Startup Project")
In the Project Properties for that project, in the "Debug" tab, choose "Start External Program", and choose the NUnit GUI executable. Put the name of UnitTests assembly in "Command line arguments".
Now you can simply press F5 (to start debugging) which will build your projects, and launch the NUnit GUI for you. This has the added advantage that if your tests fail, you can add breakpoints, and step through using the debugger (without needing to attach the debugger manually).
Whenever you are building your solution. I suggest you to close the Nunit, which is loaded with your .dll. After unloading try to build, then your build will succeed.
As Nunit is using you .dll. It won't allow your build to be success.
I had a similar problem a while ago and my final solution was to resign from using NUnit GUI runner and just run them directly from code.
In my project I referenced to NUnit libraries needed by those freshly built dlls plus nunit.core and nunit.utils. The code itself is very simple:
TestResult ExecuteTests(string testAssemblyPath) {
CoreExtensions.Host.InitializeService();
TestPackage testPackage = new TestPackage(testAssemblyPath);
testPackage.BasePath = Path.GetDirectoryName(testAssemblyPath);
RemoteTestRunner testRunner = new RemoteTestRunner();
testRunner.Load(testPackage);
TestResult testResult = testRunner.Run(new NullListener(), TestFilter.Empty, true, LoggingThreshold.Warn);
testRunner.Unload();
CoreExtensions.Host.UnloadService();
return testResult;
}
The TestResult object is very powerful. Among other things, it contains all the results, subresults, tests themselves etc. In order to analyze them, you can either create a simple parser or use one of the possibilities given by NUnit libraries. My favorite is XmlResultWriter but there are others available as well. All of them can be found in nunit.util.dll.
Unfortunately, that would still block the loaded dll from being rebuilt. I avoided this problem by running this in a separate AppDomain and unloading this domain after the tests are finished running. Then dll is nicely freed and you can do with it whatever you want.

Can't load DLL while executing tests with MS-Test

In my program, I use SevenZipSharp to generate zip files. SevenZipSharp is a managed DLL which loads another DLL, 7z.dll. I am manually setting SevenZipSharp's path to 7z.dll using SevenZipCompressor.SetLibraryPath.
When I execute my program in Debug mode, this all works fine, and it generates the zip file as nice as you please. However, when I execute my unit tests with mstest, SevenZipSharp always gives me the following error:
Test method threw exception:
SevenZip.SevenZipLibraryException: Can not load 7-zip library or
internal COM error! Message: failed to load library..
I suspect that MSTest might be doing something that is preventing SevenZipSharp from being able to load 7z.dll, like running in a security-tight sandbox (or something. I'm new to C# and MSTest...)
Does anyone have an idea about what might be happening?
Thank you!
Though the question posits a questionable scenario, the general problem of MSTest not loading required DLLs seems to be a common one, deserving of a less dismissive answer.
By default MSTest will copy the assemblies it believes are required by the test container to the Out folder of the default results folder, which changes for each run.
MSTest does not always automatically infer the necessary assemblies correctly; if there is no explicit direct reference to an assembly it won't be copied. Also, native DLLs are typically not detected.
I am not aware of a direct option to set the MSTest search path. You can determine the search path using procmon.exe as suggested above (it is basically the standard Windows DLL search).
Unintuitively, the default search path does not include the launch directory and I think this is a cause of confusion. When the tests are running the current directory is the test results "Out" directory, not the MSTest launch directory.
However, it is possible to control MSTest search behaviour (and the copying behaviour) with a test settings file. You can easily create and edit these through Visual Studio (see the Test menu) and then specify the created settings file on the MSTest command line. You can use different settings files for Visual Studio and MSTest.
By this means you can control exactly what DLLs are copied to your test directory.
See Create Test Settings to Run Automated Tests from Visual Studio to get more information on this.
Of course, DLL load failures may be due to missing dependencies, and the DLL mentioned in the error message may itself be present. You can use the dependency viewer or procmon to pick up unexpected dependencies in DLLs.
Consider using Process Monitor (aka procmon.exe) from the excellent SysInternals tools to monitor your test harness (MSTest). It will show you where the executable is looking for 7z.dll.
Visual Studio 2017 (and possibly 2015) provides two new ways to indicate that a native dll or other file is required by your tests without the need for a test settings file:
1: Add a link to the dll to your test project and tell VS to copy it to the output directory. Right-click the project in Solution Explorer and choose Add > Existing Item. Browse to 7z.dll, click the down arrow next to the Add button, and choose Add as Link. Then select the new 7z.dll item in Solution Explorer, alt-Enter to bring up Properties, and set "Copy to Output Directory" to "Copy if newer" (or "Copy always").
2: Attach a DeploymentItemAttribute to your test class. This attribute's constructor takes a single string argument, which is the path to the file you want to make available to tests in the class. It is relative to the test project's output directory.
Are you sure you want your unit tests to involve external libraries? Ideally you should have mechanisms to replace external stuff with e.g. mock objects, because testing external libraries like that actually turns your test into an integration test.
For popular libraries such as SevenZipSharp you can assume that it is properly tested, and you can have manual integration tests to verify that it performs correctly in your program.
I would consider getting rid of that dependency through dependency injection, mocking frameworks, etc, and let your unit tests solely test your own code.
Investigate the factory method or abstract factory design patterns for tips on how to easily replace such dependencies.
A good start would be to create your own ICustomZipInterface, and use the wrapper pattern to encapsulate your zip logic for production code. In your unit tests, replace that wrapper class with a dummy implementation. The dummy implementation might for example record how you access your zip component and you could use that information to validate your code, rather than checking if a zip file is actually created.

Categories