I would like to be able to pass a system argument - "host" to the MStest suite. To create automated jobs for continuous integration, I want to be able to specify the host as a parameter so the tests are run on that specific host. I couldn't find any such option with mstest.
In Java, -Dhost="localhost" would work which can be specified as a parameter for the running VM. Is there a similar way in MStest for C#?
There is not an equivalent to the Java system properties that you mention. Here are a couple of ideas on how to approximate what you are looking for:
[1]
Visual Studio test support does include Test Run Configurations (renamed Test Settings in Visual Studio 2010). This is a file that specifies many settings that control aspects of the test run. For example, you can deploy additional files alongside your test, or run a "setup" batch script before your test run begins.
If you have a finite set of hosts, you could have a separate test run config/test settings for each host. Each config/settings would deploy a file that contains the name of a different host. You could then read in that file as part of your unit test setup, perhaps from your [TestInitialize] method. A bit hokey, but maybe it would do what you want.
[2]
You could set a system environment variable (e.g., "TESTHOST") before running the test, and then read that environment variable from your tests. You could wrap all of this up in a simple program or batch script that accepts an argument to set the environment variable, invoke mstest, and unset the environment variable afterwards. For example, this StackOverflow post may give you some ideas on how you might do something like this using PowerShell.
I don't believe there is an exact equivalent. Instead, try leveraging .NET configuration files:
Add an application configuration file (App.config) to your MSTest project. Add your "system" properties as keys in the appSettings section. Reference these values in your tests using the ConfigurationManager.AppSettings collection.
Related
I have a .NET 4.7.1 C# class library with some associated tests in a separate project.
I would like to be able to run the tests in ServerGC mode to replicate the standard usage of the class library by consumers (both for normal and performance-related tests).
As per Microsoft documentation here it is possible to configure ServerGC in a number of ways.
The runtimeconfig.json isn't an option because this is a .NET Framework project.
After some playing around with regular projects and tests projects. It appears to be the case that ServerGC must be set for whatever the parent process is for it to take effect. In the case of a regular application this is fine and makes it trivial to use the MSBuild property ServerGarbageCollection or the app.config file. In the case of running a project via VSTest, this makes the test runner (vstest.console.exe) the parent process which therefore means that the ServerGarbageCollection property has no effect. I would prefer not to have to change the app.config for vstest.console.exe because it is used by all test runs on that machine and could therefore interfere with other solutions.
The Microsoft documentation suggests that it is possible to set the environment variable: COMPlus_gcServer with a value of 1 to enable ServerGC. It is possible to set environment variables for VSTest in the runsettings file as shown by the example below and at the bottom of the documentation here. The documentation also states that specifying an environment variable will enforce the /InIsolation flag to ensure that the environment variables are set and then the test run process is started. However this doesn't appear to work.
<RunSettings>
<RunConfiguration>
<EnvironmentVariables>
<COMPlus_gcServer>1</COMPlus_gcServer>
</EnvironmentVariables>
</RunConfiguration>
</RunSettings>
I have added a simple test case which checks the GCSettings.IsServerGC property and still gives false despite the environment variable being successfully read as having a value of 1 within the same test.
Edit: Having done more reading, it looks like the runtimeconfig.json property, the MSBuild property and the Environment variable are all .NETCore only. This leaves the only option as somehow providing VSTest with an app.config that contains the GCServer property. Is there any way to do this via runsettings or command line arguments to VSTest?
What I have
I have a unit test Visual Studio C# Solution (which runs webdriver tests, not that that's necessarily relevant). It runs via TeamCity. Currently the environment is hard-coded to "Dev" in one of the .cs files, and I manually change the code locally to run elsewhere when required.
What I want
A way to setup two projects in TeamCity - one to run on "Dev" environment and the other on "Test" environment. Obviously I can't use hard-coded values so I need some sort of set of configuration files that can be chosen at runtime, or possibly some sort of build parameters - but I have no clue how to do this or what will work.
(I didn't mention TeamCity in the question as it is not 100% relevant / just provides context --- as long as I can run the unit tests eg from the command prompt with parameters that can be passed in, that would do the trick.)
What I've tried
From what I've asked around, I don't believe I can use web.config as it's not a web solution but a unit test solution. I believe there is a mechanism to tell Configuration Manager what web.config file to use, so I'm hoping there's a similar mechanism that can be used for Unit Test projects. I've tried hunting down information on "build configurations" on "unit test projects" and a range of other searches, but it's a nightmare finding anything relevant.
Can someone point me in the right direction? I'm good with my basic programming, but if it requires messing around with configurations or build parameters, then I might need a more explicit 'how-to' from you.
Thanks in advance.
Check this extension for Visual Studio
https://visualstudiogallery.msdn.microsoft.com/579d3a78-3bdd-497c-bc21-aa6e6abbc859
This allows you to create different config files for different build type. You need to create different build types each with a config file specific to that build type.
Ok as per my comments above, I tried a solution here: https://visualstudiogallery.msdn.microsoft.com/69023d00-a4f9-4a34-a6cd-7e854ba318b5
IT WORKED.
I'm now able to create App.Config transforms that automatically link to a Build Configuration. I can then specify the build configuration inside TeamCity.
The only trick was that the "Rebuild" target didn't work if there were no code changes but there was a difference in Configuration. (They use the same directory of the same Agent, and a rebuild is necessary). The workaround for this is to tick the 'Clean all files before build' option in TeamCity Version Control settings.
I'm calling nunit test from Jenkins and I need to be able to specify just one URL address as parameter which I can use inside test. Is there possibility to do that?
For example I'm calling "Execute Windows batch command" in Jenkins like that:
"C:\Program Files (x86)\NUnit 2.6.4\bin\nunit-console.exe" "D:\selenium\SeleniumTest.dll" /run:SeleniumTest.Test.MyTest
Any ideas?
As far as I currently know of, there is no solution to provide just that what you describe that you want. Best option is to use NUnit project files, modify settings there and pass the solution file to the runner.
On the other hand, you could try to use the Environment variable.
Use set from the command-line. Then read the value using Environment.GetEnvironmentVariable() and use that within your testmethod.
In my program, I use SevenZipSharp to generate zip files. SevenZipSharp is a managed DLL which loads another DLL, 7z.dll. I am manually setting SevenZipSharp's path to 7z.dll using SevenZipCompressor.SetLibraryPath.
When I execute my program in Debug mode, this all works fine, and it generates the zip file as nice as you please. However, when I execute my unit tests with mstest, SevenZipSharp always gives me the following error:
Test method threw exception:
SevenZip.SevenZipLibraryException: Can not load 7-zip library or
internal COM error! Message: failed to load library..
I suspect that MSTest might be doing something that is preventing SevenZipSharp from being able to load 7z.dll, like running in a security-tight sandbox (or something. I'm new to C# and MSTest...)
Does anyone have an idea about what might be happening?
Thank you!
Though the question posits a questionable scenario, the general problem of MSTest not loading required DLLs seems to be a common one, deserving of a less dismissive answer.
By default MSTest will copy the assemblies it believes are required by the test container to the Out folder of the default results folder, which changes for each run.
MSTest does not always automatically infer the necessary assemblies correctly; if there is no explicit direct reference to an assembly it won't be copied. Also, native DLLs are typically not detected.
I am not aware of a direct option to set the MSTest search path. You can determine the search path using procmon.exe as suggested above (it is basically the standard Windows DLL search).
Unintuitively, the default search path does not include the launch directory and I think this is a cause of confusion. When the tests are running the current directory is the test results "Out" directory, not the MSTest launch directory.
However, it is possible to control MSTest search behaviour (and the copying behaviour) with a test settings file. You can easily create and edit these through Visual Studio (see the Test menu) and then specify the created settings file on the MSTest command line. You can use different settings files for Visual Studio and MSTest.
By this means you can control exactly what DLLs are copied to your test directory.
See Create Test Settings to Run Automated Tests from Visual Studio to get more information on this.
Of course, DLL load failures may be due to missing dependencies, and the DLL mentioned in the error message may itself be present. You can use the dependency viewer or procmon to pick up unexpected dependencies in DLLs.
Consider using Process Monitor (aka procmon.exe) from the excellent SysInternals tools to monitor your test harness (MSTest). It will show you where the executable is looking for 7z.dll.
Visual Studio 2017 (and possibly 2015) provides two new ways to indicate that a native dll or other file is required by your tests without the need for a test settings file:
1: Add a link to the dll to your test project and tell VS to copy it to the output directory. Right-click the project in Solution Explorer and choose Add > Existing Item. Browse to 7z.dll, click the down arrow next to the Add button, and choose Add as Link. Then select the new 7z.dll item in Solution Explorer, alt-Enter to bring up Properties, and set "Copy to Output Directory" to "Copy if newer" (or "Copy always").
2: Attach a DeploymentItemAttribute to your test class. This attribute's constructor takes a single string argument, which is the path to the file you want to make available to tests in the class. It is relative to the test project's output directory.
Are you sure you want your unit tests to involve external libraries? Ideally you should have mechanisms to replace external stuff with e.g. mock objects, because testing external libraries like that actually turns your test into an integration test.
For popular libraries such as SevenZipSharp you can assume that it is properly tested, and you can have manual integration tests to verify that it performs correctly in your program.
I would consider getting rid of that dependency through dependency injection, mocking frameworks, etc, and let your unit tests solely test your own code.
Investigate the factory method or abstract factory design patterns for tips on how to easily replace such dependencies.
A good start would be to create your own ICustomZipInterface, and use the wrapper pattern to encapsulate your zip logic for production code. In your unit tests, replace that wrapper class with a dummy implementation. The dummy implementation might for example record how you access your zip component and you could use that information to validate your code, rather than checking if a zip file is actually created.
I have a component that reads some configuration from the standard .NET configuration (app.config) file.
when I run unit tests (NUnit) for this component (using TD.NET), i noticed that the configuration file is not read.
Upon inspection of AppDomain.CurrentDomain.SetupInformation.ConfigurationFile
I have noticed that its' value is set to C:\Users\ltal\AppData\Local\Temp\tmp6D2F.tmp (some temp random locaiton).
Is there a reason for why this is happening? (Is it NUnit or TD.NET's fault?)
I suppose i could set this SetupInformation object myself for the sake of the test, haven't tried yet, but still wondering why is it being created like that and not as default.
To workaround this, you can create an app.config in your unit test project. This will then be called in place of the main app.config by your unit tests. You can then change values in that app.config in your unit tests making it easier to test different values and configurations i.e. you can setup your test app.config with certain values before running your test.
ConfigurationManager.AppSettings[""] = "";
Another option might be to place settings in the Settings.setting file of your main project. You do not have to change anything in your unit test project then. Some links about the difference between settings and app.config - MSDN forums, StackOverflow, User Settings - MSDN
And of course a third option would be to remove the dependency on the app.config from your component by introducing an interface and inject the dependency into the component making it easy to mock it out and unit test.
By default the .NET runtime looks in the working directory of the AppDomain, which is being managed by NUnit in the temp location.
This link offers two solutions about how to get config files picked up:
http://blogs.msdn.com/b/josealmeida/archive/2004/05/31/loading-config-files-in-nunit.aspx
Basically, they need to live in the testing directory.