I am using Approval Tests. On my dev machine I am happy with DiffReporter that starts TortoiseDiff when my test results differ from approved:
[UseReporter(typeof (DiffReporter))]
public class MyApprovalTests
{ ... }
However when the same tests are running on Teamcity and results are different tests fail with the following error:
System.Exception : Unable to launch: tortoisemerge.exe with arguments ...
Error Message: The system cannot find the file specified
---- System.ComponentModel.Win32Exception : The system cannot find the file
specified
Obviously it cannot find tortoisemerge.exe and that is fine because it is not installed on build agent. But what if it gets installed? Then for each fail another instance of tortoisemerge.exe will start and nobody will close it. Eventually tons of tortoisemerge.exe instances will kill our servers :)
So the question is -- how tests should be decorated to run Tortoise Diff on local machine
and just report errors on build server? I am aware of #IF DEBUG [UseReporter(typeof (DiffReporter))] but would prefer another solution if possible.
There are a couple of solutions to the question of Reporters and CI. I will list them all, then point to a better solution, which is not quite enabled yet.
Use the AppConfigReporter. This allows you to set the reporter in your AppConfig, and you can use the QuietReporter for CI.
There is a video here, along with many other reporters. The AppConfigReporter appears at 6:00.
This has the advantage of separate configs, and you can decorate at the assembly level, but has the disadvantage of if you override at the class/method level, you still have the issue.
Create your own (2) reporters. It is worth noting that if you use a reporter, it will get called, regardless as to if it is working in the environment. IEnvironmentAwareReporter allows for composite reporters, but will not prevent a direct call to the reporter.
Most likely you will need 2 reporters, one which does nothing (like a quiet reporter) but only works on your CI server, or when called by TeamCity. Will call it the TeamCity Reporter. And One, which is a multiReporter which Calls teamCity if it is working, otherwise defers to .
Use a FrontLoadedReporter (not quite ready). This is how ApprovalTests currently uses NCrunch. It does the above method in front of whatever is loaded in your UseReporter attribute. I have been meaning to add an assembly level attribute for configuring this, but haven't yet (sorry) I will try to add this very soon.
Hope this helps.
Llewellyn
I recently came into this problem myself.
Borrowing from xunit and how they deal with TeamCity logging I came up with a TeamCity Reporter based on the NCrunch Reporter.
public class TeamCityReporter : IEnvironmentAwareReporter, IApprovalFailureReporter
{
public static readonly TeamCityReporter INSTANCE = new TeamCityReporter();
public void Report(string approved, string received) { }
public bool IsWorkingInThisEnvironment(string forFile)
{
return Environment.GetEnvironmentVariable("TEAMCITY_PROJECT_NAME") != null;
}
}
And so I could combine it with the NCrunch reporter:
public class TeamCityOrNCrunchReporter : FirstWorkingReporter
{
public static readonly TeamCityOrNCrunchReporter INSTANCE =
new TeamCityOrNCrunchReporter();
public TeamCityOrNCrunchReporter()
: base(NCrunchReporter.INSTANCE,
TeamCityReporter.INSTANCE) { }
}
[assembly: FrontLoadedReporter(typeof(TeamCityOrNCrunchReporter))]
I just came up with one small idea.
You can implement your own reporter, let's call it DebugReporter
public class DebugReporter<T> : IEnvironmentAwareReporter where T : IApprovalFailureReporter, new()
{
private readonly T _reporter;
public static readonly DebugReporter<T> INSTANCE = new DebugReporter<T>();
public DebugReporter()
{
_reporter = new T();
}
public void Report(string approved, string received)
{
if (IsWorkingInThisEnvironment())
{
_reporter.Report(approved, received);
}
}
public bool IsWorkingInThisEnvironment()
{
#if DEBUG
return true;
#else
return false;
#endif
}
}
Example of usage,
[UseReporter(typeof(DebugReporter<FileLauncherReporter>))]
public class SomeTests
{
[Test]
public void test()
{
Approvals.Verify("Hello");
}
}
If test is faling, it still would be red - but reporter would not came up.
The IEnvironmentAwareReporter is specially defined for that, but unfortunatelly whatever I return there, it still calls Report() method. So, I put the IsWorkingInThisEnvironment() call inside, which is a little hackish, but works :)
Hope that Llywelyn can explain why it acts like that. (bug?)
I'm using CC.NET and I do have TortoiseSVN installed on the server.
I reconfigured my build server to allow the CC.NET service to interact with the desktop. When I did that, TortiseMerge launched. So I think what's happening is that Approvals tries to launch the tool, but it cant because CC.NET is running as a service and the operating system prevents that behavior by default. If TeamCity runs as a service, you should be fine, but you might want to test.
Related
I've been using Selenium and NUnit to do some automated testing, and up until now everything has been fine. The change I made recently was adding more than one test to a test class.
I'm pretty certain the issue lies with the code in my "Teardown" function in the test class. When I comment out
BrowserFactory.CloseAllDrivers();
Things run just fine.
This is the code for my "FrontEndAddItemToCartTest":
class FrontEndAddItemToCartTest : PageTest
{
[SetUp]
public void Initialize()
{
SetBrowser(BrowserFactory.BrowserType.Chrome); // Not headless
SetServer("testUrlNotGivenForSecurityPurposes");
StartTest(TestType.FrontEnd);
SetSize(MobileSize.XXLarge);
}
[Test]
public void StandardQuantityTest()
{
OrderItem standardQuantity = new OrderItem(new Product("500", ".25"), 500);
FrontEndActions.AddItemToCart(standardQuantity);
}
[Test]
public void CustomQuantityTest()
{
OrderItem customQuantity = new OrderItem(new Product("482", ".25"), 225);
FrontEndActions.AddItemToCart(customQuantity);
}
[TearDown]
public void EndTest()
{
BrowserFactory.CloseAllDrivers();
}
}
This is the error I get:
Message: OpenQA.Selenium.WebDriverException : Unexpected error. System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it
with a bunch of other stuff that I don't believe is relevant.
That being said, I have code in "SetBrowser(...)" that initializes a
new ChromeDriver();
but that doesn't seem to be enough.
The methods at the top are there to avoid having to do too much Selenium-y looking stuff in each of the tests - to make things more maintainable by people other than just myself - but it's what you would expect from typical Driver setup. I'm not sure why the second test is what causes the issue, but since things work fine w/o the CloseAllDrivers() being run, I'm imagining it's that.
By the looks of the attributes, you're using MSTest? Is it executing tests in parallel?
I guess regardless, for good test isolation you would want to change the CloseAllDrivers method to only dispose the driver used in the test it's cleaning up. I'd recommend starting there and see if that has the same problem.
Also, is CloseAllDrivers calling driver.Quit() or driver.Dispose()? Either of those are the ones you want to use.
OK so the issue was a combination of things:
1. I had Drivers[driver].Close(); and Drivers[driver].Quit(); -- only having .Quit() resolved one issue. Not really sure why Close() was causing problems, to be quite honest. The other (window not properly closing in the end) was due to the following code in my BrowserFactory:
if (driver == null)
{
Driver = new ChromeDriver((ChromeOptions)options); // options created elsewhere
Drivers.Add("Chrome", Driver); // This adds the driver to the list of Drivers currently up.
}
else
{
Driver = new ChromeDriver((ChromeOptions)options); // same as before
Drivers["Chrome"] = Driver; // **this** wasn't here before. This was the issue. Essentially, I was calling ```Quit()``` on the first instance of the driver, not on the fresh one created by the second test.
}
Thanks for all the help, guys. A combination of me determined to figure this out and your responses got me to the solution :)
I was wondering if it was possible to have a start-up script before running any load tests? For example, perhaps to seed some data or clear anything down prior to the tests executing.
In my instance I have a mixed bag of designer and coded tests. Put it simply, I have:
Two coded tests
A designer created web test which points to these coded tests
A load test which runs the designer
I have tried adding a class and decorating with the attributes [TestInitialize()], [ClassInitialize()] but this code doesn't seem to get run.
Some basic code to show this in practice (see below). Is there a way of doing this whereby I can have something run only the once before test run?
[TestClass]
public class Setup : WebTest
{
[TestInitialize()]
public static void Hello()
{
// Run some code
}
public override IEnumerator<WebTestRequest> GetRequestEnumerator()
{
return null;
}
}
Probably should also mention that on my coded tests I have added these attributes and they get ignored. I have come across a workaround which is to create a Plugin.
EDIT
Having done a little more browsing around I found this article on SO which shows how to implement a LoadTestPlugin.
Visual Studio provides a way of running a script before and also after a test run. They are intended for use in deploying data for a test and cleaning up after a test. The scripts are specified on the "Setup and cleanup" page in the ".testsettings" file.
A load test plugin can contain code to run before and after any test cases are executed, also at various stages during test execution. The interface is that events are raised at various points during the execution of a load test. User code can be called when these events occur. The LoadTestStarting event is raised before any test cases run. See here for more info.
If you are willing to use NUnit you have SetUp/TearDown for a per test scope and TestFixtureSetUp/TestFixtureTearDown to do something similar for a class (TestFixture)
Maybe a bit of a hack, but you can place your code inside the static constructor of your test class as it will automatically run exactly once before the first instance is created or any static members are referenced:
[TestClass]
public class Setup : WebTest
{
static Setup()
{
// prepare data for test
}
public override IEnumerator<WebTestRequest> GetRequestEnumerator()
{
return null;
}
}
I am trying to write an attribute to apply security to a method. Something that would look like this:
[CustomAuthorization(SecurityAction.Demand)]
public void DoSomething()
{
//Do Something
}
so I have my Attribute on another assembly:
public sealed class AuthorizationAttribute : CodeAccessSecurityAttribute
{
public override IPermission CreatePermission()
{
if (!/*authorize here*/)
{
return new CustomPermission(PermissionState.Unrestricted);
}
throw new Exception("IdentificationFailure.");
}
}
public AuthorizationAttribute(SecurityAction securityAction)
: base(securityAction) { }
}
So far it works.
I run my main program and it does its job.
Now I go and to modify the assembly having the attribute, build it. no problem.
I go back in my main program try to build and there it fails. It cannot copy the new built dll because the old one is still in use by a process.
Does anybody have any idea what would be happening here?
If you're using VS2010, there is an issue with vhost.exe not releasing the instance. You can end process on it for now until MS comes out with a fix.
It sounds like you haven't exited your main program before trying to rebuild it. Check your running processes for references to your main program or your security attribute dll. Process Explorer can be a real big help here.
Just been trouble shooting the same issue and it boiled down to the fact that we were using testaccessors to test private methods. When unloading the unittest projects, the assembly is released. Our assembly gets locked when compiling. Haven't found a solution to this yet, but have submitted a bug to ms. Are you using testaccessors?
Also see Assembly is being used by another process and https://stackoverflow.com/questions/6895038/testaccessor-impl-of-codeaccesssecurityattribute-locks-assembly
MS bug:
https://connect.microsoft.com/VisualStudio/feedback/details/682485/use-of-testaccessor-and-impl-of-codeaccesssecurityattribute-locks-assembly#details
I'm trying to write a custom decoupled WMI provider in C#. I've followed the examples and yet whenever I try to access my WMI class from either WMI Studio, through PowerShell, or via wmic, it just hangs there indefinitely until I terminate the provider host app, at which point I get an error ranging from "Invalid Class" to "The remote procedure call failed".
I can see my WMI provider fine if I don't try to actually access an instance of it, so I know it's registering with WMI.
Here's the code I'm running:
[assembly: WmiConfiguration(#"root\CIMV2", HostingModel=ManagementHostingModel.Decoupled)]
[RunInstaller(true)]
public class WmiInstaller : DefaultManagementInstaller { }
[ManagementEntity(Singleton=true)]
[ManagementQualifier("Description", Value="Accesses and manipulates licenses held in the SLN license database.")]
public class SoftwareLicensingNetworkLicenseDatabase {
[ManagementBind]
public SoftwareLicensingNetworkLicenseDatabase() { Test = "OMG!"; }
[ManagementProbe]
public string Test;
}
And then in my main function:
[STAThread]
static void Main() {
InstrumentationManager.RegisterType(typeof(SoftwareLicensingNetworkLicenseDatabase));
Console.ReadLine();
InstrumentationManager.UnregisterType(typeof(SoftwareLicensingNetworkLicenseDatabase));
}
I've tried any number of things to diagnose this issue: switch to .Net 3.5 (I'm using .Net 4.0), change namespace names, use a multi-instance class instead of a singleton, etc.
Any help would be sincerely appreciated!
Nevermind, I figured it out:
Your Main function cannot have STAThread as an attribute. I added it when I was debugging something that required it and had not taken it off. It figures it would take me so long to figure out something so simple and obvious once you think about it.
I have two Windows services written in C#. One service is "Console Application" and second one is "Windows Application" (can't change it).
Both service applications can be executed in few modes (depending on command line arguments and Environment.UserInteractive flag):
when (Environment.UserInteractive == false) and...
when no cmd-line parameter is provided - standard out-generated code (required for SCM) is executed - ServiceBase.Run(ServicesToRun)
when (Environment.UserInteractive == true) and...
when "-i" cmd-line parameter is provided - service application installs itself (something like installutil -i [scv_app])
when "-u" cmd-line parameter is provided - service application uninstalls itself (something like installutil -u [scv_app])
when "-c" cmd-line parameter is provided - service is executed in "console" mode (for debug purposes)
Both services use static method from class library to choose and process described execution path.
However, I have one problem in this class library - when application has type "Windows Application", then Console.WriteLine() has no visible effect. In that case I could use Win32 AttachConsole() or something like that, but I prefer to show summarized messages via MessageBox.Show().
Thus, I think that in class library I need to know whether application is "Console Application" or "Windows Application"... Do you have any idea how to do it?
At the beginning, instead of trying to detect app type I have tried to write something like that:
if (string.IsNullOrEmpty(Console.Title) == false) Console.WriteLine(msg);
It works in Win7 but it does not work under Win2k3. So maybe there is a better way to detect if Console.WriteLine(msg) / Console.ReadLine works as expected?
I’ve seen some other suggestions (sublinked here), but none of them looks good to me. I am looking for "nice" solution (!= p/invoke; != accessing any Console object property (e.g. Title) inside a try/catch block).
Windows services are not supposed to have ui. This includes consoles and messageboxes.
With that out of the way...
Have you considered hooking up an appropriate trace listener to System.Diagnostics.Trace.TraceListeners? Based on your command line you could add a MessageBox tracelistener or a tracelistener that dumps trace messages to a console? You'll be leveraging built-in debugging mechanisms that have been extensively tested and are, in their own way, extremely extensible as well. You'll also implicitly get to distinguish between messages that are displayed in release mode vs. messages displayed in debug mode (via System.Diagnostics.Trace.Write() or System.Diagnostics.Debug.Write()).
The when dealing with a class library, I pass in a UI object depending on the environment that is calling the library.
public interface IAlerts {
void Show(string message);
}
public class EventLogAlerts : IAlerts { ... }
public class WindowsAlerts : IAlerts { ... }
public class ConsoleAlerts : IAlerts { ... }
public class MyLibrary {
private IAlerts alertImpl = new EventLogAlerts();
public void SetUserInterface(IMyUserInterface impl)
{
this.alertImpl = impl;
}
public void DoSomething()
{
try
{
...
}
catch (Exception)
{
this.alertImpl.Show("Whoops!");
}
}
}