Following method shall only be called if it has been verified that there are invalid digits (by calling another method). How can I test-cover the throw-line in the following snippet? I know that one way could be to merge together the VerifyThereAreInvalidiDigits and this method. I'm looking for any other ideas.
public int FirstInvalidDigitPosition {
get {
for (int index = 0; index < this.positions.Count; ++index) {
if (!this.positions[index].Valid) return index;
}
throw new InvalidOperationException("Attempt to get invalid digit position whene there are no invalid digits.");
}
}
I also would not want to write a unit test that exercises code that should never be executed.
If the "throw" statement in question is truly unreachable under any possible scenario then it should be deleted and replaced with:
Debug.Fail("This should be unreachable; please find and fix the bug that caused this to be reached.");
If the code is reachable then write a unit test that tests that scenario. Error-reporting scenarios for public-accessible methods are perfectly valid scenarios. You have to handle all inputs correctly, even bad inputs. If the correct thing to do is to throw an exception then test that you are throwing an exception.
UPDATE: according to the comments, it is in fact impossible for the error to be hit and therefore the code is unreachable. But now the Debug.Fail is not reachable either, and it doesn't compile because the compiler notes that a method that returns a value has a reachable end point.
The first problem should not actually be a problem; surely the code coverage tool ought to be configurable to ignore unreachable debug-only code. But both problem can be solved by rewriting the loop:
public int FirstInvalidDigitPosition
{
get
{
int index = 0;
while(true)
{
Debug.Assert(index < this.positions.Length, "Attempt to get invalid digit position but there are no invalid digits!");
if (!this.positions[index].Valid) return index;
index++;
}
}
}
An alternative approach would be to reorganize the code so that you don't have the problem in the first place:
public int? FirstInvalidDigitPosition {
get {
for (int index = 0; index < this.positions.Count; ++index) {
if (!this.positions[index].Valid) return index;
}
return null;
}
}
and now you don't need to restrict the callers to call AreThereInvalidDigits first; just make it legal to call this method any time. That seems like the safer thing to do. Methods that blow up when you don't do some expensive check to verify that they are safe to call are fragile, dangerous methods.
I don't understand why you wouldn't want to write a unit test that exercises "code that should not happen".
If it "should not happen", then why are you writing the code at all? Because you think it might happen of course - perhaps an error in a future refactoring will break your other validation and this code will be executed.
If you think it's worth writing the code, it's worth unit testing it.
It's sometimes necessary to write small bits of code that can't execute, just to appease a compiler (e.g. if a function which always throw an exception, exits the application, etc. is called from a function which is supposed to return a value, the compiler may insist that the caller include a "return 0", "Return Nothing", etc. statement which can never execute. I wouldn't think one should be required to test such "code", since it may be impossible to make it execute without seriously breaking other parts of the system.
A more interesting question comes with code that the processor should never be able to execute under normal circumstances, but which exists to minimize harm from abnormal circumstances. For example, some safety-critical applications I've written start with code something like (real application is machine code; given below is pseudo-code)
register = 0
test register for zero
if non-zero goto dead
register = register - 1
test register for zero
if non-zero goto okay1
dead:
shut everything down
goto dead
okay1:
register = register + 1
if non-zero goto dead
I'm not sure how exactly one would go about testing that code in the failure case. The purpose of the code is to ensure that if the register has suffered electrostatic or other damage, or is otherwise not working, the system will shut down safely. Absent some very fancy equipment, though, I have no idea how to test such code.
Part of the purpose of testing is to test both things that should happen and things that can happen.
If you have code that should never execute, but could under the right (or wrong) conditions, you can verify that the exception occurs in the right conditions (in MS's Unit Test Framework) by decorating your test method with:
[ExpectedException(typeof(InvalidOperationException))]
To rephrase the issue, basically: how do you test an invalid state of your system when you've already gone to a lot of trouble to make sure that the system can't get into that invalid state in the first place?
It will make for slightly slower tests, but I would suggest using reflection to put your system into the invalid state in order to test it.
Also notice that reflection is precisely one of the possible ways for your system to get into that invalid state, which will execute your supposedly impossible to reach code.
If your code was truly impossible to reach, which is not quite the case in your example as far as I see, then you should remove the code. Eg:
public int GetValue() {
return 5;
throw new InvalidOperationException("Somehow the return statement did not work!");
}
I also would not want to write a unit test that exercises code that should never be executed.
What do you mean by "should" here? Like, that it ought not to be executed? Surely it should be ok for the code to be executed in a test. It should be executed in a test.
Related
Is it possible to get run-time information about where a method has returned?
I mean, if the method returned after running all its lines of code, or because of an earlier
return statement that occurred due to some condition.
The scenario is using interceptor for creating UnitOfWork that should exists in method scope.
For example, lets consider this code:
[UnitOfWork]
public void Foo()
{
// insert some values to the database, using repositories interfaces...
DoSomeChangesInTheDataBaseUsingRepositories();
var result = DoSomethingElse();
if (!result) return;
DoMoreLogicBecuseTheResultWasTrue();
}
I have interceptor class that opens thread static unit of work for methods that are flagged with [UnitOfWork] and when the scope of the method ends it run commit on the UoW and dispose it.
This is fine, but lets consider the scenario above, where for some reason a programmer decided to return in the middle of the method due to some condition, and in that scenario the changes made by the repositories should not be persisted.
I know that this can indicate wrong design of the method, but be aware that it is a possible scenario to be written by a programmer and I want to defend my database from these kind of scenarios.
Also, I don't want to add code to the method itself that will tell me where it ended. I want to infer by the method info somehow its returned point, and if it is not at the end of its scope the interceptor will know not to commit.
The simple answer is use BREAKPOINTS and Debugging.
Edit:- As mentioned by Mels in the comments. This could be a useful suggestion.
If your application is very timing-sensitive, set conditional breakpoints such that they never actually stop the flow of execution. They do keep track of Hit Count, which you can use to backtrace the flow of execution.
Just for your attention. From the microsoft site:-
For those out there who have experience debugging native C++ or VB6
code, you may have used a feature where function return values are
provided for you in the Autos window. Unfortunately, this
functionality does not exist for managed code. While you can work
around this issue by assigning the return values to a local variable,
this is not as convenient because it requires modifying your code. In
managed code, it’s a lot trickier to determine what the return value
of a function you’ve stepped over. We realized that we couldn’t do the
right thing consistently here and so we removed the feature rather
than give you incorrect results in the debugger. However, we want to
bring this back for you and our CLR and Debugger teams are looking at
a number potential solutions to this problem. Unfortunately this is
will not be part of Visual Studio 11.
There are a couple ways that normally indicate that a method exited early for some reason, one is to use the actual return value, if the value is a valid result that then your method probably finished correctly, if its another value then probably not, this is the pattern that most TryXXX methods follow
int i;
//returns false as wasn't able to complete
bool passed = int.TryParse("woo", out i);
the other is to catch/trhow an exception, if an exception is found, then the method did not complete as you'd expect
try
{
Method();
}
catch(Exception e)
{
//Something went wrong (e.StackTrace)
}
Note: Catching Exception is a bad idea, the correct exceptions should be caught, i.e NullReferenceException
EDIT:
In answer to your update, if your code is dependant on the success of your method you should change the return type to a boolean or otherwise, return false if unsuccessful
Generally you should use trace logs to watch you code flow if you cant debug it.
You could always do something like this:
private Tuple<int, MyClass> MyMethod()
{
if (condition)
{
return new Tuple<int, MyClass>(0,new MyClass());
}
else if(condition)
{
return new Tuple<int, MyClass>(1, new MyClass());
}
return new Tuple<int, MyClass>(2,new MyClass());
}
This way you´ll have an index of which return was returning your MyClass object. All depends on what you are trying to accomplish and why - which is at best unclear. As someone else mentioned - that is what return values are for.
I am curios to know what you are trying to do...
So, I have a piece of code using WeakReferences. I know of the common .IsAlive race condition, so I didn't use that. I basically have something like this:
WeakReference lastString=new WeakReference(null);
public override string ToString()
{
if(lastString!=null)
{
var s=lastString.Target as string; //error here
if(s!=null)
{
return s;
}
}
var str=base.ToString();
lastString=new WeakReference(str);
return str;
}
Somehow, I'm getting a null reference exception at the marked line. By debugging it, I can confirm that lastString is indeed null, despite being wrapped in a null check and lastString never actually being set to null.
This only happens in a complex flow as well, which makes me think garbage collection is somehow taking my actual WeakReference object, and not just it's target.
Can someone enlighten me as to how this is happening and what the best course of action is?
EDIT:
I can't determine at all the cause of this. I ended up wrapping the error code in a try-catch just fix it for now. I'm very interested in the root cause of this though. I've been trying to reproduce this in a simple test case, but it's proven very difficult to do. Also, this only appears to happen when running under a unit test runner. If I take the code and trim it down to the minimum, it will continue to crash when running using TestDriven and Gallio, but will not fail when put into a console application
This ended up being a very hard to spot logic bug that was in plain sight.
The offending if statement really was more like this:
if(lastString!=null && limiter==null || limiter=lastLimiter)
The true grouping of this is more like this:
if((lastString!=null && limiter==null) || limiter=lastLimiter)
And as Murphy's law would dictate, somehow, in this one unrelated test case, lastLimiterand lastString got set to null by a method used no where but this one single test case.
So yea, no bug in the CLR, just my own logic bug that was very hard to spot
I've read this question about simulating throwing exceptions. Answers suggest I create a mock object that pretends being a real object.
That's not what I want to have. If I replace a real object I lose part of real code. I want to invoke real code with minimal changes and have an exception thrown from inside it at random point of that code.
Is it possible to have an exception thrown at random point of code being called from a Unit test?
don't put randomness into your unit tests. it will only bring you troubles. a unit tests should always have some consistence. especially if you want him to tell you what went wrong when he went red. if you implement a random exception throwing into your unit tests it can happen that he SOMETIMES crashes and returns a red bar. maybe the next run it the error is gone again. this is really not what unit tests are here for and you will have big troubles finding the issue for that once-occurred failed test.
a much better approach is to systematically test critical parts of your code. replace each crucial method with a mocked object, which throws the type of exception you want to test for and cover each test case like this. this will give you much more information about what went wrong when you are looking for that error.
hope this helps.
I can think about this work around.
1- Make a function called RandomException(); that throw exception if a random value is divisible by 3 otherwise it won't throw the exception. make sure this function is encapsulcate in this block of code.
#if DEBUG
void RandomException()
{
// gen random value
// if random value % 3 == 0 throw the exception now
}
#else
void RandomException()
{
}
#endif
that way when you release the code these function calls won't affect the program.
I hope that idea helps you.
It seems like you are trying to test the exception handling, i dont think this is a good approach to throw an exception randomly and check if it gets cought, if you need to test handling in certain conditions you need to simulate the condition in a real object, like an illegal situation which should cause an exception.
You still can produce this behaviour through a local varibale which you can set from outside (in your Unittest) which will cause the code to throw an exception depending on this variable, but as saied i dont think its a good approach.
I'm starting to use Code Contracts, and whilst Contract.Requires is pretty straight forward, I'm having trouble seeing what Ensures actually does.
I've tried creating a simple method like this:
static void Main()
{
DoSomething();
}
private static void DoSomething()
{
Contract.Ensures(false, "wrong");
Console.WriteLine("Something");
}
I never see the message "wrong" though, nor does it throw exceptions or anything else.
So what does it actually do ?
It's odd for it to not throw anything - if you're running the rewriter tool with the appropriate settings. My guess is that you're running in a mode which doesn't check postconditions.
The confusing thing about Contract.Ensures is that you write it at the start of the method, but it executes at the end of the method. The rewriter does all the magic to make sure it executes appropriately, and is given the return value if necessary.
Like many things about Code Contracts, I think it's best to run Reflector on the results of the rewriter tool. Make sure you've got the settings right, then work out what the rewriter has done.
EDIT: I realise I haven't expressed the point of Contact.Ensures yet. Simply put, it's to ensure that your method has done something by the end - for example, it could ensure that it's added something to a list, or (more likely) that the return value is non-null, or positive or whatever. For example, you might have:
public int IncrementByRandomAmount(int input)
{
// We can't do anything if we're given int.MaxValue
Contract.Requires(input < int.MaxValue);
Contract.Ensures(Contract.Result<int>() > input);
// Do stuff here to compute output
return output;
}
In the rewritten code, there will be a check at the point of return to ensure that the returned value really is greater than the input.
This question is probably language-agnostic, but I'll focus on the specified languages.
While working with some legacy code, I often saw examples of the functions, which (to my mind, obviously) are doing too much work inside them. I'm talking not about 5000 LoC monsters, but about functions, which implement prerequisity checks inside them.
Here is a small example:
void WorriedFunction(...) {
// Of course, this is a bit exaggerated, but I guess this helps
// to understand the idea.
if (argument1 != null) return;
if (argument2 + argument3 < 0) return;
if (stateManager.currentlyDrawing()) return;
// Actual function implementation starts here.
// do_what_the_function_is_used_for
}
Now, when this kind of function is called, the caller doesn't have to worry about all that prerequisities to be fulfilled and one can simply say:
// Call the function.
WorriedFunction(...);
Now - how should one deal with the following problem?
Like, generally speaking - should this function only do what it is asked for and move the "prerequisity checks" to the caller side:
if (argument1 != null && argument2 + argument3 < 0 && ...) {
// Now all the checks inside can be removed.
NotWorriedFunction();
}
Or - should it simply throw exceptions per every prerequisity mismatch?
if (argument1 != null) throw NullArgumentException;
I'm not sure this problem can be generalized, but still I want to here your thoughts about this - probably there is something I can rethink.
If you have alternative solutions, feel free to tell me about them :)
Thank you.
Every function/method/code block should have a precondition, which are the precise circumstances under which it is designed to work, and a postcondition, the state of the world when the function returns. These help your fellow programmers understand your intentions.
By definition, the code is not expected to work if the precondition is false, and is considered buggy if the postcondition is false.
Whether you write these down in your head, on paper in a design document, in comments, or in actual code checks is sort of a matter of taste.
But a practical issue, long-term, life is easier if you code the precondition and post-condition as explicit checks. If you code such checks, because the code is not expected to work
with a false precondition, or is buggy with a false postcondition, the pre- and post- condition checks should cause the program to report an error in a way that makes it easy to discover the point of failure. What code should NOT do is simply "return" having done nothing, as your example shows, as this implies that it has somehow executed correctly.
(Code may of course be defined to exit having done nothing, but if that's the case, the pre- and post- conditions should reflect this.)
You can obviously write such checks with an if statement (your example comes dangerously close):
if (!precondition) die("Precondition failure in WorriedFunction"); // die never comes back
But often the presence of a pre- or post-condition is indicated in the code by defining a special function/macro/statement... for the language called an assertion, and such special construct typically causes a program abort and backtrace when the assertion is false.
The way the code should have been written is as follows:
void WorriedFunction(...)
{ assert(argument1 != null); // fail/abort if false [I think your example had the test backwards]
assert(argument2 + argument3 >= 0);
assert(!stateManager.currentlyDrawing());
/* body of function goes here */
}
Sophisticated functions may be willing to tell their callers that some condition has failed. That's the real purpose of exceptions. If exceptions are present, technically the postcondition should say something to the effect of "the function may exit with an exception under condition xyz".
That's an interesting question. Check the concept of "Design By Contract", you may find it helpful.
It depends.
I'd like to seperate my answer between case 1, 3 and case 2.
case 1, 3
If you can safely recover from an argument problem, don't throw exceptions. A good example are the TryParse methods - if the input is wrongly formatted, they simply return false. Another example (for the exception) are all LINQ methods - they throw if the source is null or one of the mandatory Func<> are null. However, if they accept a custom IEqualityComparer<T> or IComparer<T>, they don't throw, and simply use the default implementation by EqualityComparer<T>.Default or Comparer<T>.Default. It all depends on the context of usage of the argument and if you can safely recover from it.
case 2
I'd only use this way if the code is in an infrastructure like environment. Recently, I started reimplementing the LINQ stack, and you have to implement a couple of interfaces - those implementations will never be used outside my own classes and methods, so you can make assumptions inside them - the outside will always access them via the interface and can't create them on their own.
If you make that assumption for API methods, your code will throw all sorts of exceptions on wrong input, and the user doesn't have a clue what is happening as he doesn't know the inside of your method.
"Or - should it simply throw exceptions per every prerequisity mismatch?"
Yes.
You should do checks before calling and function and if you own the function, you should make it throw exceptions if arguments passed are not as expected.
In your calling code these exceptions should be handled. Ofcourse arguments passed should be verified before the call.