We Keep Coding

C# - MsTestV2 - problem with datatestmethod

i'm having a problem using DataTestMethods (MStestV2), the behaviour is the following:
i run all test cases without no problems, but since once exception is thrown (e.g one assert for example), when i try to re-run the test i'm getting the following error:
Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.ObjectModel.TestFailedException: Only data driven test methods can have parameters. Did you intend to use [DataRow] or [DynamicData]?
But, when i clean the project and run the tests again, all works fine, until i get another test failed.
Anyone can help? thank you.

Unit Tests failing when I Run All Tests but pass when I Debug

I'm using NUnit3 in Visual Studio 2017 and doing TDD. Something really strange is happening since I updated my code to make my latest test pass.
Now, 3 of my other tests are failing when I click Run All Tests, as below:
It is telling me that the actual and expected values in my Assert method are not equal.
However, when I put a breakpoint at the line where the Assert method is and start debugging, the stacktrace is showing that expected and actual are the same value and then the test passes, as below:
Am I doing something stupid or could there be a bug in VS2017 or NUnit or something?
This ever happen to anyone else?
[Edit: I should probably add that I have written each test as a separate class]

The failing tests share a resource that affects them all when tested together. Recheck the affected tests and their subjects.
You should also look into static fields or properties in the subjects. They tends to cause issues if not used properly when designing your classes.

Some subtle differences might occur. For instance if a first test change a state which affects the behavior of a second test, then the outcome of this 2nd test may not be the same if I run it alone.
An idea to help understand a test failure when a breakpoint can't be used, could be to add logging.
Anyway, to answer your questions:
This ever happen to anyone else?
Yes
Am I doing something stupid or could there be a bug in VS2017 or NUnit or something?
I bet that it's neither: just a case a bit more subtle

I experienced a similar issue in Visual Studio 2017 using MSTest as the testing framework. Assertions in unit tests were failing when the tests were run but would pass when the unit tests were debugged. This was occurring for a handful of unit tests but not all of them. In addition to the Assertion failures, many of the units tests were also failing due to a System.TypeLoadException (Could not load type from assembly error). I ultimately did the following which solved the problem:
Open the Local.testsettings file in the solution
Go to the "Unit Test" settings
Uncheck the "Use the Load Context for assemblies in the test directory." checkbox
After taking these steps all unit tests started passing when run.

I encountered this phenomenon myself, but found the cause quite easily. More concretely, I tested some matrix calculations, and in my test class I defined data to calculate with as a class variable and performed my calculations with it. My matrix routines, however, modified the original data, so when I used "run tests" on the test class, the first test corrupted the data, and the next test could not succeed.
The sample code below is an attempt to show what I mean.
[TestFixture]
public void MyTestClass()
{
[Test]
public void TestMethod1()
{
MyMatrix m = new MyMatrix();
// Method1() modifies the data...
m.Method1(_data);
}
[Test]
public void TestMethod2()
{
MyMatrix m = new MyMatrix();
// here you test with modified data and, in general, cannot expect success
m.Method2(_data);
}
// the data to test with
private double[] _data = new double[1, 2, 3, 4]{};
}

Visual Studio test explorer runs tests in foreign language

I have created C# unit tests (using VS test tools). Among others, there are tests checking exceptions like that:
Assert.AreEqual("Object reference not set to an instance of an object.", e.Message);
When I open my unit test file and click "Run tests" in the context menu, everything works fine. When I use the test explorer of Visual Studio and click "Run tests" there, I get exceptions saying that the message of the exception is different. Actually, it is the same as the one I am checking for but in another language. The tests running in both cases are the same, there are no language changes or anything else, just simple logic tests. My OS is in English, Visual Studio as well. I have similar tests in another solution and there everything works fine. This solution and one of the projects that I am writing unit tests for were created by a colleague which has the system in this foreign language. However, I can't find settings like that anywhere. Does someone have an idea where these could be? How do I get the tests running always in English?

Try using HResult instead.
Assert.AreEqual(val, e.HResult)
Shouldn't change depending on language.
HResult is a coded numerical value assigned to a specific exception, so using HResult you know which exception is being thrown.
Edit:
Or if you know what exception you are expecting, you could write your own Exception-"expecter".
protected static void ThrowsException<T>(Action action) where T : Exception
{
var thrown = false;
try
{
action();
}
catch (T)
{
thrown = true;
}
Assert.IsTrue(thrown);
}
And then use it like this:
ThrowsException<IndexOutOfRangeException>(() => MyController.MyAction());

Here is my solution for the tests, in case someone else need it. It is not really good when the tests are language dependent, so I now check for the exception type:
Assert.IsInstanceOfType(e, typeof(ArgumentNullException));
However, I would still like to know where the language is taken from. When I have everything in English, I really expect it to be in English.

Why don't you just use Throws with specific exception? Checking for a string message is really bad. In my opinion it's even cleaner than using IsInstanceOfType
var testDelegate = () => MyService.Method(params);
Assert.Throws<NullException>(testDelegate);

Result of "is" expression returns false when run, but true when inspected

I have the following code. The CustomControlHelper generates an instance of an object via reflection. At this stage we don't know what type of object we are dealing with. We do know it will be a CustomControl, but we don't know if it implements any particular interface or if it extends any other classes. The following code is trying to establish whether the loaded control implements the IRichAdminCustomControl interface.
Object obj = CustomControlHelper.GetControl(cc.Id, cc.ControlClass);
if(obj != null)
{
bool isWhatWeWant = (obj is IRichAdminCustomControl);
return isWhatWeWant;
}
That's all fine, but I've noticed that when I know I have an object that implements IRichAdminCustomControl, the expression evaluates to false.
Okay, this is where it gets really weird. If I inspect the code when debugging, the expression evaluates to true, but then if I immediately let the code run and inspect the result, it evaluates to false (I've attached an animated gif below to illustrate).
Has anyone come across weirdness like this before and if so, what on earth is causing it?
Incidentally, I believe the product I'm using uses Spring.NET to provide dependency injection in the CustomControlHelper.

If you are using Visual Studio 2010 SP1, I came across this bug:
Misreporting of variable values when debugging x64 code
There is a workaround on that page, posted by Microsoft:
You can either set all projects to compile to x86, or create an intermediate initialised variable declaration to ensure the debugger reports the correct value of the variable being examined.
Try this as a workaround:
bool isWhatWeWant = true;
isWhatWeWant &= (obj is IRichAdminCustomControl);
bool finalValue = isWhatWeWant; // this line should fix isWhatWeWant too in the debugger
return finalValue;
EDIT: seems like VS2012 also encounters similar problems in specific conditions.

Two possibilities come to mind. The first is that your interface name is generic enough that it could already be in the namespace somewhere. Try fully qualifying the interface in the is clause. The second possibility is that you might be running the code as part of a constructor, or being called indirectly by a constructor. Any reflection like stuff needs to be done after we are certain the application has fully loaded.

So I found the answer. It was because I had two copies of the dll in different locations. I had one copy in the bin of my back-end application and one in a shared external directory that gets dynamically loaded by the backend app.
I should explain; this application consists of two apps running in tandem, a frontend app and a backend app. Ordinarily, you place "Custom Controls" into your frontend app. These controls are then copied on application start to a external directory that is accessible to the backend app.
In this case, I had logic in my Custom Control library that needed to be accessed in the backend app - so I had to make a reference to it... which ended up with the backend app having two references to the same class. D'oh! And OF COURSE that's going to work when you're debugging.
Solution was to split my extra logic into its own project and reference THAT in the backend app.
I'm not going to "accept" my own answer here because, although it solved my specific problem, the solution is a bit TOO specific to the apps I'm working with and would be unlikely to help anyone else.

This happened to me once and even though I never came to a conclusion as to why it was happening I believed the PDB files that were being loaded with the debugging symbols where out of sync. So, by "cleaning" the solution and then rebuilding the solution this weird issue went away.

C#: catch test failure during classcleanup

Is it possible to nest TestMethods in ClassCleanup and have them run/behave as TestMethods instead of regular method calls?
I have a TestClass to test an AppMngr class I created to manage a process. I test for the ability to Open/Close the app (e.g. MyNotepadMngrClass.Open() and ...Close()). I have several more classes that do work inside that process (e.g. MyNotepadWorkerClass.WriteLine() or ...DoSomething() ). When testing the other classes I need to start notepad and close it when done. ClassInitialize/ClassCleanup are obvious places for this. But I want to confirm notepad closed.
So I created a static [TestMethod] for the Close operation. I call it from ClassCleanup in MyNotepadWorkerTestClass. It performs the close operation fine. But if I add something like -- Assert.IsFalse(true); -- to the body of my close method the test run does not fail.
Let me know if what I am trying to do fundamentally wrong. Appreciate any you can give help.
P.S. Hey TestStand guys, I am looking for Setup/Ceanup behaviour during RunSelectedStep. TestDriven.NET gives me RunSelectedStep. So how do I catch failures during ClassInitialize and ClassCleanup.

ClassCleanup is by definition to be used "after all the tests in the test class have run" (from msdn), so you probably can't add new test methods in there. You'll need to restructure your test.