I have a CodedUI test. It sporadic fail by exception(can't focus element). Can I do something like this
[TestMethod]
public void MySporadicFailedTest()
{
try {
//Some Test action
}
catch((Exception ex)) {
if (ex is System.Exception.ElementNotFocused){
//retry test
}
}
}
This is something I deal with frequently when writing Coded UI tests. I almost always end up writing a simple extension method to handle retrying specific actions (not the entire test!). Sometimes, especially on pages with weird, non-standard markup or lots of AJAXy things happening, you'll just hit a situation where an action will fail one second because something isn't ready yet, then pass the next.
public static class TestRetryExtensions
{
public static void WithRetry<T>(this Action thingToTry, int timeout = 30) where T: Exception
{
var expiration = DateTime.Now.AddSeconds(timeout)
while (true)
{
try
{
thingToTry();
return;
}
catch (T)
{
if (DateTime.Now > expiration)
{
throw;
}
Thread.Sleep(1000);
}
}
}
}
Then, within my actual test:
uiMap.ClickSomeThing();
uiMap.EnterSomeText();
Action clickSomeOtherThingAction = () => uiMap.ClickSomeOtherThingThatFailsForNoReason();
clickSomeOtherThingAction.WithRetry<UITestControlHiddenException>(60);
It tries to perform the action. If it fails with an exception that you aren't aware of being an occasional "normal" thing, it throws the exception as normal. If it fails with an exception that you're telling it to retry on, it will keep trying that action (with a 1 second delay between retries) until the timeout is exceeded, at which point it just gives up and rethrows the exception.
As long as you can catch which ever exception is thrown, you can wrap your test code in a retry loop. Then it will try the test code a certain number of times before giving up:
for (var i = 0; i < TimesToRetry; i++)
{
try{
//perform test
//test ran correctly - break out loop to end test
break;
}
catch(Exception){
//might want to log exception
}
}
If the codedUI test continuously failing without proper reason you can added some code to enhance the test and make it fail safe. If test failing specifically when focusing to an element try to give focus to upper level element first and then try focusing child elements. This LINK may help you to write fail safe test cases.
we can include below line of code in test clean up method to re run the failed script
if (TestContext.CurrentTestOutCome==TestContext.unittestoutcome.failed)
{
var type=Type.GetType(TestContext.FullyQualifiedTestClassName);
if (type !=null)
{
var method=Type.GetMethod(TestContext.TestName);
var event=Activator.CreateInstance(type);
}
method.invoke(event);
}
Related
I am working with a large and complex event driven body of code and there are piles of opportunity to accidentally create a recursive condition.
Sometimes the recursive condition is temporary and the application catches up with itself but even that usually creates unnecessary lag. Other times it creates a stackoverflow which is often very difficult to debug when it happens at a client site.
I would like to have a way to either blacklist or whitelist sections of code that are permitted to recurse. If the recursive condition happens during DEV then I want it to assert so that I can correct the code.
What I am considering is having the application examine its own stack to ensure that the method it just entered is not already on the stack.
Any pointers would be appreciated.
Note: This is for a Web Application but I have run into this challenge in multiple environments.
You can inspect stack like this:
[MethodImpl(MethodImplOptions.NoInlining)]
// optionally decorate with Conditional to only be used in Debug configuration
[Conditional("DEBUG")]
public static void FailIfCallerIsRecursive() {
var trace = new StackTrace();
// previous frame is the caller
var caller = trace.GetFrame(1).GetMethod();
// inspect the rest
for (int i = 2; i < trace.FrameCount; i++) {
// if found caller somewhere up the stack - throw
if (trace.GetFrame(i).GetMethod() == caller)
throw new Exception("Recursion detected");
}
}
Then just call it a the beginning:
void MyPotentiallyRecursiveMethod() {
FailIfCallerIsRecursive()
}
But note that it's quite expensive. However since you are going to use that only in dev (debug) configuration - why not. You can also modify it a bit to throw only when certain level of recursion is detected (so caller appears X time up the stack).
You could call the RuntimeHelpers.EnsureSufficientExecutionStack method and then catch the InsufficientExecutionStackException that is thrown if the next method call would cause a (not catchable) StackOverflowException.
You could create an extension method for it:
public static T EnsureSafeRecursiveCall<T>(this Func<T> method)
{
try
{
RuntimeHelpers.EnsureSufficientExecutionStack();
return method();
}
catch (InsufficientExecutionStackException ex)
{
string msg = $"{method.Method.Name} would cause a {nameof(StackOverflowException)} on the next call";
Debug.Fail(msg);
// logging here is essential here because Debug.Fail works only with debug
throw new StackOverflowException(msg, ex); // wrap in new exception to avoid that we get into this catch again and again(note we are in a recursive call)
}
}
Now your original method remains almost unchanged:
public static IEnumerable<T> YourRecursiveMethod<T>(IEnumerable<T> seq)
{
var method = new Func<IEnumerable<T>>(() => YourRecursiveMethod(seq));
return method.EnsureSafeRecursiveCall();
}
In C# you can catch an exception in the default test suite like this:
[TestMethod]
[ExpectedException(typeof (ArgumentNullException))]
public void TestNullComposite()
{
IApi api = new Api();
api.GetDataUsingDataContract(null); // this method throws ArgumentNullException
}
But when you analyze the code coverage, it says that you only get 66.67% coverage because the last curly brace was not covered.
How would I go about achieving 100% coverage on this unit test?
Usually when people are measuring code coverage, they are looking at the code covered by the tests and not the tests themselves.
As this case shows, it doesn't really make sense to require 100% coverage on test units.
The test is supposed to throw. That is what you are testing.
If you really want the entire method to be executed I guess you could test whether the exception was thrown manually. Something like this (haven't tested it, but I don't see why it shouldn't work):
[TestMethod]
public void TestNullComposite()
{
IApi api = new Api();
bool didThrow = false;
try
{
api.GetDataUsingDataContract(null); // this method throws ArgumentNullException
}
catch(ArgumentNullException)
{
didThrow = true;
}
Assert.IsTrue(didThrow);
}
But it seems like extra work for no good reason. I would suggest you re-evaluate your testing practices.
Within NUnit you have a method Assert.Throws<Exception>(), which checks if the desired exception was thrown. It also returns that exception as return value, so that you could have further assertations if you like:
[Test]
public void Api_GetDataUsingDataContractWithNullParameter_ThrowsArgumentNullException()
{
var api = new Api();
var exception = Assert.Throws<ArgumentNullException>(() => api.GetDataUsingDataContract(null));
Assert.That(exception.Message, Is.Not.Null.Or.Empty);
Assert.That(exception.Message, Is.StringContaining("source"));
}
Due to the fact, that the method does not throw by itself, your coverage would be 100%.
What MAV is saying is true. Additionally, there is a way to exclude the test class from being analyzed from Code Coverage.
Just adorn your [TestClass] with the Attribute [ExcludeFromCodeCoverage]!
This way it is at least theoretically possible to reach 100% CC.
Yes You Can But First You have to understand see assert throw will fully cover when
The expected exception match with the exception thrown by your function
e.g
assertThrows(NullPointerException.class,() -> userProfileService.getUserDetailById(userProfile.getAssociateId()));
When the expected and actual exception are not match or no any exception are thrown by assertthrow.
Consider the above scenario example suppose called function will throw Other exception than NullPointerException then assertthrow will fail.
Now you have to think to fail and pass at the same time for assertthrow.
So a solution that I find is
userProfile = new UserProfile("101", "Amit Kumar Gupta", "amit#gmail.com", "Software Intern", "No",
"999999999");
userProfile2 = new UserProfile("102", "Satyam Sharma", "Satyam#gmail.com", "Software Engineer", "Yes",
"8769356421");
Mockito.when(userProfileDAO.findById(Mockito.anyString())).thenReturn(Optional.ofNullable(userProfile))
.thenReturn(Optional.ofNullable(userProfile2));
List<Class<? extends Exception>> exceptionClassList = new ArrayList<Class<? extends Exception>>();
exceptionClassList.add(NullPointerException.class);
exceptionClassList.add(IllegalArgumentException.class);
for (int i = 0; i < 2; i++) {
try {
assertThrows(exceptionClassList.get(i),
() -> userProfileService.getUserDetailById(userProfile.getAssociateId()));
} catch (AssertionError e) {
assertNotNull(e.getMessage());
}
}
In this case Mockito will return two values and both will be used by function call inside the assertThrow first time assertThrow will match the expected exception i.e. NullPointerException.class
Second time assertThrow will Fail because the lambda function will not throw any exception but assertThrow is expecting a IllegalArgumentException.
It generates the AssertionError So that will caught by catch block.
Now You have 100% coverage.
I got a function named inner_logic() that can throw many types of exception , each of one of two types: critical and non-critical. The logic of inner_logic() includes an "infinite" loop, so in case of a non-critical exception I'd wish to step back in to it. At that point you can ask why not catching the non-critical exception within the loop, and the reason is that it's an abstract class with a few inheritors, each have a unique loop. for clarification, what I want to do is something like:
public void logic()
{
try
{
inner_logic();
}
catch (non_critical e)
{
// log
logic();
}
catch(critical e)
{
// log
return;
}
public virtual void inner_logic(){}
As far as I can tell, I seems that it should work, and that all calls are "catchable", but it feels kinda hackish.
My Question is: Is there a better way for doing that, or a reason I should reconsider my suggestion?
I did something similar in the past. I try to avoid recursion so I did a loop instead.
public void logic()
{
bool running = true;
while(running)
{
running = false;
try
{
inner_logic();
}
catch(non_critical e)
{
running = true;
}
}
}
Actually, I did it a bit diffrently. Instead of a boolean, I counted the number of non_critical errors. If the number exceded 3 then I handled it as a critical error.
You should not do it this way, as if the current state of the application means that the non- critical exception is thrown repeatedly, a stack overflow exception will occur.
Catch the exception outside the method and repeat if needed.
It feels to me like it should just be looping:
while (true) // Or probably have a maximum number of retries
{
try
{
// Logic
// Break out of the loop by returning if we succeed
return;
}
catch (NonCriticalException e)
{
// Log, then let the loop continue
}
catch (CriticalException e)
{
// Log, then rethrow (in preference to returning)
}
}
With your original approach, if the non-critical exception just keeps happening, you'll get a StackOverflowException. I would definitely suggest adding a maximum retry count though. (Keeping track of the number of retries is good for logging, too.)
I've noticed this problem happening a lot in most things I do, so I'm thinking there must be a design pattern for this.
Basically if an exception is thrown, attempt to solve the problem and retry. If I place it in the try, all it will do is catch the exception, but I want to retry whatever it was doing and if it fails again, retry again a certain number of times.
Is there a common pattern for this sort of stuff?
check this SO answer.. hope that helps u
Cleanest way to write retry logic?
public static class RetryUtility
{
public static void RetryAction(Action action, int numRetries, int retryTimeout)
{
if(action == null)
throw new ArgumenNullException("action");
do
{
try
{
action();
return;
}
catch
{
if(numRetries <= 0)
throw; // Avoid silent failure
else
{
Thread.Sleep(retryTimeout);
numRetries--;
}
}
}
while(numRetries > 0);
}
}
Call
RetryUtility.RetryAction( () => SomeFunctionThatCanFail(), 3, 1000 );
Credit goes to LBushkin
This runs indefinately but it would be easy to add a loop counter to the while clause
var solved = false;
var tries = 0;
while (!solved)
{
try
{
//Do Something
solved = true;
}
catch
{
//Fix error
}
finally
{
if(solved || IsRediculous(tries))
break;
tries++;
}
}
try/catch inside a loop, with a counter for retries?
EDIT: And your requirement of "retry whatever it was doing," you need custom logic for that, how to retry varies wildly (ie, reopen a stream, recreate the object, pause for X milliseconds, etc...), so you need it's own try/catch inside a loop for every atomic operation.
By "atomic operation" I mean a set of related statements, such as read a file. The whole file read into memory might be an atomic operation, for example.
On some limited basis, you might want to put your try/catch into a loop, and force break if is ultimately successful. Such might be for internet access testing and you want user to have another attempt at connection.
Something like this, maybe:
int MAX_RETRIES = 5;
for (var attempt=1; attempt <= MAX_RETRIES; attempt++) {
try {
DoSomethingThatMightThrow();
}
catch (AnExceptionIKnowHowToHandle) {
if (attempt < MAX_RETRIES)
continue;
throw;
}
}
Depends what you are trying, but typically you want to check for the possibility of an exception happening PRIOR to executing the code that could cause an exception.
For example, check that a file exists before accessing it, and create it (or whatever) if it doesn't.
Are you sure exception handling is the proper methodology here? If you can "solve the problem" you can probably detect the error condition prior to calling the exception-generatiing code.
Exception handling is most natural for things which are truly exceptional. A failed Internet connection (as in the previous answer) is something that can be detected and handled before calling exception-throwing code.
Yes, it is quite common to have a loop with a number of retries where you break out of the loop on success. A couple of things:
You might want to add a delay before retrying so that you don't use up all your retries in just a few milliseconds before the temporary problem had time to fix itself.
If you eventually fail, you should throw the first exception you caught, not the last one. The second exception could be the result of failing to recover correctly from the first failure and might not help to debug the original problem.
Coding what others have already mentioned:
var success = false;
var attempts = 0;
var maxAttempts = 0;
do {
attempts++;
try {
/* your code */
success = condition;
} catch(SuperciliousException e) {
/* recover */
}
} while(!success && attempts < maxAttempts);
Working on a project where a sequential set of methods must be run every x seconds. Right now I have the methods contained within another "parent method", and just sequentially call them right after another.
class DoTheseThings()
{
DoThis();
NowDoThat();
NowDoThis();
MoreWork();
AndImSpent();
}
Each method must run successfully without throwing an exception before the next step can be done. So now I wrapped each of those methods with a while and try..catch, then in the catch execute that method again.
while( !hadError )
{
try
{
DoThis();
}
catch(Exception doThisException )
{
hadError = true;
}
}
This seems smelly and not very dry. Is there a better way to do this so I'm not wrapping any new functionality in the same methods. Isn't some kind of Delegate collection the proper way to implement this?
Is there a more "proper" solution?
Action[] work=new Action[]{new Action(DoThis), new Action(NowDoThat),
new Action(NowDoThis), new Action(MoreWork), new Action(AndImSpent)};
int current =0;
while(current!=work.Length)
{
try
{
work[current]();
current++;
}
catch(Exception ex)
{
// log the error or whatever
// maybe sleep a while to not kill the processors if a successful execution depends on time elapsed
}
}
Isn't some kind of Delegate collection the proper way to implement this?
Delegate is a possible way to solve this problem.
Just create a delegate something like:
public delegate void WorkDelegate();
and put them in arraylist which you can iterate over.
I have a personal religious belief that you shouldn't catch System.Exception, or more accurately, you should only catch the exceptions you know how to handle.
That being said, I am going to assume that each one of the methods that you are calling are doing something different, and could result in different exceptions being thrown. Which means you would likely need to have different handlers for each method.
If you follow my religion as well, and the second statement is true, then you are not repeating code unnecessarily. Unless you have other requirements, my recommendations to improve your code would be:
1) Put the try-catch in each method, not around each method call.
2) Have the catches within each method catch ONLY the exceptions you know about.
http://blogs.msdn.com/fxcop/archive/2006/06/14/631923.aspx
http://blogs.msdn.com/oldnewthing/archive/2005/01/14/352949.aspx
http://www.joelonsoftware.com/articles/Wrong.html
HTH ...
your example seems ok.. its a dry one but will do the job well!! actually if this methods execute db access.. you can use transaction to ensure integrity...
if your dealing with shared variables for multi threader programs.. it is cleaner to use synchronization.. the most important thing in coding is that you write the proper code... that has less bugs.. and will do the task correctly..
public void DoTheseThings()
{
SafelyDoEach( new Action[]{
DoThis,
NowDoThat,
NowDoThis,
MoreWork,
AndImSpent
})
}
public void SafelyDoEach( params Action[] actions )
{
try
{
foreach( var a in actions )
a();
}
catch( Exception doThisException )
{
// blindly swallowing every exception like this is a terrible idea
// you should really only be swallowing a specific MyAbortedException type
return;
}
}
What would be the reason that an error was occuring?
If this were a resource issue, such as access to something like a connection or object, then you might want to look at using monitors, semaphores, or just locking.
lock (resource)
{
Dosomething(resource);
}
This way if a previous method is accessing the resource, then you can wait until it releases the resource to continue.
Ideally, you shouldn't have to run a loop to execute something each time it fails. It is failing at all, you would want to know about the issue and fix it. Having a loop to always just keep trying is not the right way to go here.
I'd do what Ovidiu Pacurar suggests, only I'd use a foreach loop and leave dealing with array indexes up to the compiler.
Simple delegate approach:
Action<Action> tryForever = (action) => {
bool success;
do {
try {
action();
success = true;
} catch (Exception) {
// should probably log or something here...
}
} while (!success);
};
void DoEverything() {
tryForever(DoThis);
tryForever(NowDoThat);
tryForever(NowDoThis);
tryForever(MoreWork);
tryForever(AndImSpent);
}
Stack approach:
void DoEverything() {
Stack<Action> thingsToDo = new Stack<Action>(
new Action[] {
DoThis, NowDoThat, NowDoThis, MoreWork, AndImSpent
}
);
Action action;
while ((action = thingsToDo.Pop()) != null) {
bool success;
do {
try {
action();
success = true;
} catch (Exception) {
}
} while (!success);
}