I'm attempting to extend this answer on SO to make a WCF client retry on transient network failures and handle other situations that require a retry such as authentication expiration.
Question:
What are the WCF exceptions that need to be handled, and what is the correct way to handle them?
Here are a few sample techniques that I'm hoping to see instead of or in addition to proxy.abort():
Delay X seconds prior to retry
Close and recreate a New() WCF client. Dispose the old one.
Don't retry and rethrow this error
Retry N times, then throw
Since it's unlikely one person knows all the exceptions or ways to resolve them, do share what you know. I'll aggregate the answers and approaches in the code sample below.
// USAGE SAMPLE
//int newOrderId = 0; // need a value for definite assignment
//Service<IOrderService>.Use(orderService=>
//{
// newOrderId = orderService.PlaceOrder(request);
//}
/// <summary>
/// A safe WCF Proxy suitable when sessionmode=false
/// </summary>
/// <param name="codeBlock"></param>
public static void Use(UseServiceDelegateVoid<T> codeBlock)
{
IClientChannel proxy = (IClientChannel)_channelFactory.CreateChannel();
bool success = false;
try
{
codeBlock((T)proxy);
proxy.Close();
success = true;
}
catch (CommunicationObjectAbortedException e)
{
// Object should be discarded if this is reached.
// Debugging discovered the following exception here:
// "Connection can not be established because it has been aborted"
throw e;
}
catch (CommunicationObjectFaultedException e)
{
throw e;
}
catch (MessageSecurityException e)
{
throw e;
}
catch (ChannelTerminatedException)
{
proxy.Abort(); // Possibly retry?
}
catch (ServerTooBusyException)
{
proxy.Abort(); // Possibly retry?
}
catch (EndpointNotFoundException)
{
proxy.Abort(); // Possibly retry?
}
catch (FaultException)
{
proxy.Abort();
}
catch (CommunicationException)
{
proxy.Abort();
}
catch (TimeoutException)
{
// Sample error found during debug:
// The message could not be transferred within the allotted timeout of
// 00:01:00. There was no space available in the reliable channel's
// transfer window. The time allotted to this operation may have been a
// portion of a longer timeout.
proxy.Abort();
}
catch (ObjectDisposedException )
{
//todo: handle this duplex callback exception. Occurs when client disappears.
// Source: https://stackoverflow.com/questions/1427926/detecting-client-death-in-wcf-duplex-contracts/1428238#1428238
}
finally
{
if (!success)
{
proxy.Abort();
}
}
}
EDIT: There seems to be some inefficiencies with closing and reopening the client multiple times. I'm exploring solutions here and will update & expand this code if one is found. (or if David Khaykin posts an answer I'll mark it as accepted)
After tinkering around with this for a few years, the code below is my preferred strategy (after seeing this blog posting from the wayback machine) for dealing with WCF retries and handling exceptions.
I investigated every exception, what I would want to do with that exception, and noticed a common trait; every exception that needed a "retry" inherited from a common base class. I also noticed that every permFail exception that put the client into an invalid state also came from a shared base class.
The following example traps every WCF exception a client could through, and is extensible for your own custom channel errors.
Sample WCF Client Usage
Once you generate your client side proxy, this is all you need to implement it.
Service<IOrderService>.Use(orderService=>
{
orderService.PlaceOrder(request);
}
ServiceDelegate.cs
Add this file to your solution. No changes are needed to this file, unless you want to alter the number of retries or what exceptions you want to handle.
public delegate void UseServiceDelegate<T>(T proxy);
public static class Service<T>
{
public static ChannelFactory<T> _channelFactory = new ChannelFactory<T>("");
public static void Use(UseServiceDelegate<T> codeBlock)
{
IClientChannel proxy = null;
bool success = false;
Exception mostRecentEx = null;
int millsecondsToSleep = 1000;
for(int i=0; i<5; i++) // Attempt a maximum of 5 times
{
// Proxy cann't be reused
proxy = (IClientChannel)_channelFactory.CreateChannel();
try
{
codeBlock((T)proxy);
proxy.Close();
success = true;
break;
}
catch (FaultException customFaultEx)
{
mostRecentEx = customFaultEx;
proxy.Abort();
// Custom resolution for this app-level exception
Thread.Sleep(millsecondsToSleep * (i + 1));
}
// The following is typically thrown on the client when a channel is terminated due to the server closing the connection.
catch (ChannelTerminatedException cte)
{
mostRecentEx = cte;
proxy.Abort();
// delay (backoff) and retry
Thread.Sleep(millsecondsToSleep * (i + 1));
}
// The following is thrown when a remote endpoint could not be found or reached. The endpoint may not be found or
// reachable because the remote endpoint is down, the remote endpoint is unreachable, or because the remote network is unreachable.
catch (EndpointNotFoundException enfe)
{
mostRecentEx = enfe;
proxy.Abort();
// delay (backoff) and retry
Thread.Sleep(millsecondsToSleep * (i + 1));
}
// The following exception that is thrown when a server is too busy to accept a message.
catch (ServerTooBusyException stbe)
{
mostRecentEx = stbe;
proxy.Abort();
// delay (backoff) and retry
Thread.Sleep(millsecondsToSleep * (i + 1));
}
catch (TimeoutException timeoutEx)
{
mostRecentEx = timeoutEx;
proxy.Abort();
// delay (backoff) and retry
Thread.Sleep(millsecondsToSleep * (i + 1));
}
catch (CommunicationException comException)
{
mostRecentEx = comException;
proxy.Abort();
// delay (backoff) and retry
Thread.Sleep(millsecondsToSleep * (i + 1));
}
catch(Exception e)
{
// rethrow any other exception not defined here
// You may want to define a custom Exception class to pass information such as failure count, and failure type
proxy.Abort();
throw e;
}
}
if (success == false && mostRecentEx != null)
{
proxy.Abort();
throw new Exception("WCF call failed after 5 retries.", mostRecentEx );
}
}
}
I started a project on Codeplex that has the following features
Allows efficient reuse of the client proxy
Cleans up all resources, including EventHandlers
Operates on Duplex channels
Operates on Per-call services
Supports config constructor, or by factory
http://smartwcfclient.codeplex.com/
It is a work in progress, and is very heavily commented. I'll appreciate any feedback regarding improving it.
Sample usage when in instance mode:
var reusableSW = new LC.Utils.WCF.ServiceWrapper<IProcessDataDuplex>(channelFactory);
reusableSW.Reuse(client =>
{
client.CheckIn(count.ToString());
});
reusableSW.Dispose();
we have a WCF client that deal with almost any type of failure at the server. The Catch list is very long but does not have to be. If you look closely, you will see that many exceptions are child definitions of the Exception Class (and a few other classes).
Thus you can simplify things a lot if you want to. That said, here are some typical errors that we catch:
Server timeout
Server too busy
Server unavailable.
Below links may help to handle WCF Exceptions:
http://www.codeproject.com/KB/WCF/WCFErrorHandling.aspx
http://msdn.microsoft.com/en-us/library/cc949036.aspx
Related
I have a socket and I'd like to send messages and read from it.
When I read/write with the socket while the other side is offline, I get the same Exception: System.IO.IOException: Unable to read data from the transport connection: Operation on non-blocking socket would block.
How can I identify in which of the two it happened besides having two separate try-catch blocks? Can't I just get a Timeout Exception when the reading timeout is over?
example:
try
{
SendData("!GetLocation!");
string data = GetData();
}
catch (Exception ex)
{
if (ex is System.IO.IOException)
{
//How can I identify if the exception was raised at the read method or the write method?
}
}
Yeah, exception handling is heavy resource wise, but sometimes is not so bad.
If you stick to only one try-catch you can check the error message.
Note: I have also added a second try-catch for generic (non IO) errors
try
{
SendData("!GetLocation!");
string data = GetData();
}
catch (System.IO.IOException ex)
{
if (ex.Message.IndexOf("Unable to read") != -1)
{
// GetData error
}
else if (ex.Message.IndexOf("Unable to write") != -1)
{
// SendData error
}
else
{
//Other IO errors
}
}
catch(Exception exc)
{
// Unspected errors
}
you could also set a boolean variable and check its value to know where it
broke your code.
bool sendCalled = false;
try
{
SendData("!GetLocation!");
sendCalled = true;
string data = GetData();
}
catch (System.IO.IOException ex)
{
if (sendCalled)
{
// GetData error
}
else
{
// SendData error
}
}
Not that I endorse either of these solutions, but an answer is an answer: you can either
analyze the stack trace of the exception to find out which call failed (e.g. name of the method at the top of the stack frame
set a flag after the write, and do logic based on that flag
Neither of these is as straight forward as wrapping each method call. In fact, wrapping each call conveys your intent. In the catch of your first call, you can return/break/skip the read call, which explicitly tells the reader you're bailing out fast.
How can i catch the exception that occurs when starting a windows service. I am unable to get the exception here in my below code even though i am throwing exception in the Onstart() method of the service.
public class InterOpIntegrationWinService : ServiceBase
{
protected override void OnStart(string[] args)
{
throw new InvalidOperationException(message);
}
}
Calling thread code
try
{
using (ServiceController controller = new ServiceController())
{
controller.ServiceName = objServiceConfig.ServiceName;
controller.Start();
System.Windows.Forms.Application.DoEvents();
//controller.WaitForStatus(ServiceControllerStatus.Running, new TimeSpan(0, 0, 15));
//controller.WaitForStatus(ServiceControllerStatus.Running);
//if (!string.IsNullOrEmpty(LogUtilities.ServiceOnStartException))
//{
// MessageBox.Show("Error with starting service : " + LogUtilities.ServiceOnStartException);
// LogUtilities.ServiceOnStartException = string.Empty;
//}
}
}
catch (System.InvalidOperationException InvOpExcep)
{
DisplayError(InvOpExcep.Message);
LogUtilities.DisplayMessage("Failed to start service. " + LogUtilities.ServiceOnStartException, InvOpExcep);
LogUtilities.ServiceOnStartException = string.Empty;
}
catch (Exception ex)
{
DisplayError(ex.Message);
LogUtilities.DisplayMessage("Failed to start service. " + LogUtilities.ServiceOnStartException, ex);
LogUtilities.ServiceOnStartException = string.Empty;
}
i check for application license in the onstart() method and throws a licensing error if it fails. i want this to shared to my calling thread so i could show the message in a DialogBox. Any ideas of how to do this if i cannot handle the exceptions in my calling process.
Separate your service into (at least) two components - a component that deals with IPC in some form (e.g. Remoting, WCF endpoint, REST service, etc) and (one or more) components that do its actual job.
If the licensing check fails, don't start the other components - but do still start the component that offers IPC. After starting your service (which should now always at least start), you forms-based application can connect to the service and (through whatever means you want) determine that the service is currently refusing to provide any functionality due to a failed licensing check.
I use RabbitMQ as my queue message server, I use .NET C# client.
When there is error in processing message from queue, message will not ackknowleage and still stuck in queue not be processed again as the document I understand.
I don't know if I miss some configurations or block of codes.
My idea now is auto manual ack the message if error and manual push this message to queue again.
I hope to have another better solution.
Thank you so much.
my code
public void Subscribe(string queueName)
{
while (!Cancelled)
{
try
{
if (subscription == null)
{
try
{
//try to open connection
connection = connectionFactory.CreateConnection();
}
catch (BrokerUnreachableException ex)
{
//You probably want to log the error and cancel after N tries,
//otherwise start the loop over to try to connect again after a second or so.
log.Error(ex);
continue;
}
//crate chanel
channel = connection.CreateModel();
// This instructs the channel not to prefetch more than one message
channel.BasicQos(0, 1, false);
// Create a new, durable exchange
channel.ExchangeDeclare(exchangeName, ExchangeType.Direct, true, false, null);
// Create a new, durable queue
channel.QueueDeclare(queueName, true, false, false, null);
// Bind the queue to the exchange
channel.QueueBind(queueName, exchangeName, queueName);
//create subscription
subscription = new Subscription(channel, queueName, false);
}
BasicDeliverEventArgs eventArgs;
var gotMessage = subscription.Next(250, out eventArgs);//250 millisecond
if (gotMessage)
{
if (eventArgs == null)
{
//This means the connection is closed.
DisposeAllConnectionObjects();
continue;//move to new iterate
}
//process message
channel.BasicAck(eventArgs.DeliveryTag, false);
}
}
catch (OperationInterruptedException ex)
{
log.Error(ex);
DisposeAllConnectionObjects();
}
}
DisposeAllConnectionObjects();
}
private void DisposeAllConnectionObjects()
{
//dispose subscription
if (subscription != null)
{
//IDisposable is implemented explicitly for some reason.
((IDisposable)subscription).Dispose();
subscription = null;
}
//dipose channel
if (channel != null)
{
channel.Dispose();
channel = null;
}
//check if connection is not null and dispose it
if (connection != null)
{
try
{
connection.Dispose();
}
catch (EndOfStreamException ex)
{
log.Error(ex);
}
catch (OperationInterruptedException ex)//handle this get error from dispose connection
{
log.Error(ex);
}
catch (Exception ex)
{
log.Error(ex);
}
connection = null;
}
}
I think you may have misunderstood the RabbitMQ documentation. If a message does not get ack'ed from the consumer, the Rabbit broker will requeue the message onto the queue for consumption.
I don't believe your suggested method for ack'ing and then requeuing a message is a good idea, and will just make the problem more complex.
If you want to explicitly "reject" a message because the consumer had a problem processing it, you could use the Nack feature of Rabbit.
For example, within your catch exception blocks, you could use:
subscription.Model.BasicNack(eventArgs.DeliveryTag, false, true);
This will inform the Rabbit broker to requeue the message. Basically, you pass the delivery tag, false to say it is not multiple messages, and true to requeue the message.
If you want to reject the message and NOT requeue, just change true to false.
Additionally, you have created a subscription, so I think you should perform your ack's directly on this, not through the channel.
Change:
channel.BasicAck(eventArgs.DeliveryTag, false);
To:
subscription.Ack();
This method of ack'ing is much cleaner since you are then keeping everything subscription-related on the subscription object, rather than messing around with the channel that you've already subscribed to.
I have been handing the closing and aborting of channels this way:
public async Task<MyDataContract> GetDataFromService()
{
IClientChannel channel = null;
try
{
IMyContract contract = factory.CreateChannel(address);
MyDataContract returnValue = await player.GetMyDataAsync();
channel = (IClientChannel);
return returnValue;
}
catch (CommunicationException)
{
// ex handling code
}
finally
{
if (channel != null)
{
if (channel.State == CommunicationState.Faulted)
{
channel.Abort();
}
else
{
channel.Close();
}
}
}
}
Assume only a single thread uses the channel. How do we know the channel will not fault right after checking the state? If such a thing were to happen, the code would try to Close() and Close() will throw an exception in the finally block. An explanation about why this is safe/unsafe and examples of a better, safer way would be appreciated.
Yes, the state is a "snapshot" of the current state when you get it. In the time between when you access the CommunicationState and when you go to make a logical decision based on it, the state can have easily changed. A better WCF pattern is:
try
{
// Open connection
proxy.Open();
// Do your work with the open connection here...
}
finally
{
try
{
proxy.Close();
}
catch
{
// Close failed
proxy.Abort();
}
}
In this way you don't rely on the state in order to make decisions. You try to do the most likely thing (a healthy close) and if that fails (which it will when the CommunicationState is Faulted), you call Abort to ensure proper cleanup.
In C# ASP.NET 3.5 web application running on Windows Server 2003, I get the following error once in a while:
"Object reference not set to an instance of an object.: at System.Messaging.Interop.MessagePropertyVariants.Unlock()
at System.Messaging.Message.Unlock()
at System.Messaging.MessageQueue.ReceiveCurrent(TimeSpan timeout, Int32 action, CursorHandle cursor, MessagePropertyFilter filter, MessageQueueTransaction internalTransaction, MessageQueueTransactionType transactionType)
at System.Messaging.MessageEnumerator.get_Current()
at System.Messaging.MessageQueue.GetAllMessages()".
The line of code that throws this error is:
Message[] msgs = Global.getOutputQueue(mode).GetAllMessages();
where Global.getOutputQueue(mode) gives the messagequeue I want to get messages from.
Update:
Global.getPool(mode).WaitOne();
commonClass.log(-1, "Acquired pool: " + mode, "Report ID: " + unique_report_id);
............../* some code /
..............
lock(getLock(mode))
{
bool yet_to_get = true;
int num_retry = 0;
do
{
try
{
msgs = Global.getOutputQueue(mode).GetAllMessages();
yet_to_get = false;
}
catch
{
Global.setOutputQueue(mode);
msgs = Global.getOutputQueue(mode).GetAllMessages();
yet_to_get = false;
}
++num_retry;
}
while (yet_to_get && num_retry < 2);
}
... / some code*/
....
finally
{
commonClass.log(-1, "Released pool: " + mode, "Report ID: " + unique_report_id);
Global.getPool(mode).Release();
}
Your description and this thread suggests a timing issue. I would create the MessageQueue object infrequently (maybe only once) and have Global.getOutputQueue(mode) return a cached version, seems likely to get around this.
EDIT: Further details suggest you have the opposite problem. I suggest encapsulating access to the message queue, catching this exception and recreating the queue if that exception occurs. So, replace the call to Global.getOutputQueue(mode).GetAllMessages() with something like this:
public void getAllOutputQueueMessages()
{
try
{
return queue_.GetAllMessages();
}
catch (Exception)
{
queue_ = OpenQueue();
return queue_.GetAllMessages();
}
}
You'll notice I did not preserve your mode functionality, but you get the idea. Of course, you have to duplicate this pattern for other calls you make to the queue, but only for the ones you make (not the whole queue interface).
This is an old thread, but google brought me here so I shall add my findings.
I agree with user: tallseth that this is a timing issue.
After the message queue is created it is not instantly available.
try
{
return _queue.GetAllMessages().Length;
}
catch (Exception)
{
System.Threading.Thread.Sleep(4000);
return _queue.GetAllMessages().Length;
}
try adding a pause if you catch an exception when accessing a queue which you know has been created.
On a related note
_logQueuePath = logQueuePath.StartsWith(#".\") ? logQueuePath : #".\" + logQueuePath;
_queue = new MessageQueue(_logQueuePath);
MessageQueue.Create(_logQueuePath);
bool exists = MessageQueue.Exists(_logQueuePath);
running the MessageQueue.Exists(string nameofQ); method immediately after creating the queue will return false. So be careful when calling code such as:
public void CreateQueue()
{
if (!MessageQueue.Exists(_logQueuePath))
{
MessageQueue.Create(_logQueuePath);
}
}
As it is likely to throw an exception stating that the queue you are trying to create already exists.
-edit: (Sorry I don't have the relevant link for this new info)
I read that a newly created MessageQueue will return false on MessageQueue.Exists(QueuePath)until it has received at least one message.
Keeping this and the earlier points i mentioned in mind has gotten my code running reliably.