Weird WCF behaviour regarding Timouts and Exceptions - c#

I have created an WCF service hosted inside a normal Windows service. This service is deployed to customers and set up on their servers. Therefore (afaik) I need to establish the WCF proxy dynamically and cannot rely on some prebuilt proxy created by VS or the Silverlight tools. The clients in this case are mobile apps built with Xamarin.Forms.
The Code to create the "Channel":
public void Init(int timeout = 15)
{
ea = new EndpointAddress(string.Format("http://{0}:{1}/{2}", _settingsService.ConnectionIP, _settingsService.ConnectionPort, _settingsService.ConnectionEndpoint));
bhttpb = new BasicHttpBinding(BasicHttpSecurityMode.None);
bhttpb.SendTimeout = TimeSpan.FromSeconds(timeout);
cfIMMC = new ChannelFactory<IMaintMobileContract>(bhttpb, ea);
cfIMMC.Opened += cfIMMC_Opened;
cfIMMC.Faulted += cfIMMC_Faulted;
cfIMMC.Closed += cfIMMC_Closed;
immc = cfIMMC.CreateChannel(ea);
immc.Ping(); // This function is defined by me in the Contract. It only returns true, if the server can be reached.
}
So far everything works fine if the service is running, but the app has to run "offline" and then it gets weird.
When the connection is established there is no EndpointException or anything, and when a function is called it just sits there waiting until the timeout hits.
It would be really nice to get some information whether the WCF service is actually there or not. I have function calls that can take up to multiple minutes and it would be fatal for the app to wait that long when the WCF server is not there at all. How can I achieve that?
Update:
Right now it got even weirder. Now, aprox. 30 seconds after the Ping() fails, I get System.Net.Sockets.SocketException: Connection timed out and System.Net.WebException: Error: ConnectFailure (Connection timed out) out of nowhere.
Update 2 :
Here a pic of the CallStack:

If you need fast feedback regarding whether service is alive or not, then setup additional endpoint (with separate contract containing only Ping method) and set small timeouts for it.
And important part is to set send/receive timeouts to small value as well - this will ensure that Ping method returns/throws fast if service is not available.
As far as I remember WCF does not open channel (== does not connect to server) until you call one of the methods - that's why you don't have exceptions before Ping is called.
About exception after 30 seconds. Where do you see it? I mean is it Visual Studio that breaks there or do you have your application failing with unhandled exception? I'm asking it because I see this in the Xamarin/Mono code:
initConn = new WaitCallback (state => {
try {
InitConnection (state);
} catch {}
});
And it means that even though this exception is thrown after 30 seconds - it'll be swallowed. What really happens is that when request is sent (i.e. when you call Ping()) the runtime tries to open connection in background (your call stack confirms that) and 30 seconds is default Windows timeout for connection. WCF will fail earlier if it has lower timeout set (like in your case), but connection attempt will last for 30 seconds and will complete with exception.
So, my opinion is that you should not care about this exception, unless it somehow stops your application.

Related

Excessive timeout on simple byte array reply from SOAP / WSDL Web Service call ... as if it started replying but never finishes?

My C# client (running on .NET Framework 4.5.1 or later) calls a WSDL-defined SOAP web service call that returns a byte[] (with length typically about 100000). We make hundreds of calls to this web service just fine -- they normally take just a few seconds to return. But very intermittently, the call sits there for exactly 5 minutes and then throws an InvalidOperationException indicating that "There is an error in XML document (1, 678)", with an InnerException that is a WebException "The operation has timed out." We've wrapped a try-catch around this call, look for those particular Exceptions, and then ask the user if they'd like us to retry it, and usually it works just fine on the next try.
Looking at the logging on the server, the logs for the good calls and the intermittent bad calls look exactly the same. In particular, in both cases we get the log statement at the very end of the web service, right before the "return byteArray;"... and it is doing that in the typical 3-15 seconds from the start of the call. So, it seems the web service returns the byte array successfully, but the client that called the web service just never receives it.
However, the client does NOT get the typical SoapException or WebException... for example, if we pause the web service in the debugger right before that return, then after 60 seconds the client will get a WebException "The operation has timed out." But we don't get that in this case... instead we are stuck there for a full 5 minutes before we finally get the InvalidOperationException mentioned above. So, it is as if it started receiving the reply, so it doesn't consider it timed out the normal way, but it never gets the rest of the reply, and the parsing/deserializing of the XML containing the reply eventually times out.
Question #1: Any suggestions on what's happening here? Or what we might be doing wrong in our web service that would result in a byte[] reply getting stuck mid-return intermittently? I'd obviously love to fix the root problem.
Question #2: What controls the length of that 5 minute timeout?? Our exception handling for this would be okay except for the ridiculous 5 minute timeout. After about 10 seconds, the user knows it is stuck because it normally returns in 10 seconds or less. But they have to sit there and wait for 5 minutes before they can do anything. We have set every timeout setting we could find to just 60 seconds, but none seem to control this. We have set:
In the server Web.config: <httpRuntime executionTimeout="60">
In the server Global.asax.cs: HttpContext.Current.Server.ScriptTimeout = 60;
In both server and client: ServicePointManager.MaxServicePointIdleTime = 60000;
In the client, right after we new up the WSDL-defined class derived from SoapHttpClientProtocol with all the web service calls, we call: service.Timeout = 60000;
We previously had those at their defaults or set to 100 / 100000 ... we lowered them all to 60 / 60000 to see if the 5 minute wait would come down at all (just in case one or more of them were being added into that 5 minutes). But no, no matter what we changed any of those timeouts to, the timeout in this case remains exactly 5 minutes, every time it gets stuck.
Does anybody know where the length of the timeout is set for when it generates an InvalidOperationException on the XML document containing the returned byte array due to an InnerException WebException with the timeout?? (please!)

Remoting call never returns. Possibly remoting conflict?

I am trying to establish a .Net remoting call to a thirdparty app. Here's the example code I have been given for that connection (With proprietary names removed):
IDictionary props = new ListDictionary();
props["port"] = 0; // have the Remoting system pick a unique listening port (required for callbacks)
props["name"] = string.Empty; // have the Remoting system pick a unique name
BinaryServerFormatterSinkProvider serverProv = new BinaryServerFormatterSinkProvider();
serverProv.TypeFilterLevel = System.Runtime.Serialization.Formatters.TypeFilterLevel.Full;
_channel = new TcpChannel(props, new BinaryClientFormatterSinkProvider(), serverProv);
ChannelServices.RegisterChannel(_channel, true);
IThirdparty _thirdparty = (IThirdparty)Activator.GetObject(typeof(IThirdparty), "tcp://localhost:9090/Thirdparty.AppIntegration");
//Example API call
_thirdparty.Minimized = !_thirdparty.Minimized;
When this code gets called normally, it hangs at _thirdparty.Minimized and outputs a SocketException in the diagnostics window with the message:
No connection could be made because the target machine actively refused it
The call only returns if I close the Thirdparty app.
I checked netstat -bano and the only app running on port 9090 is the one I am trying to connect to.
So I moved the call to the first few lines of the Main() function in my app and it works just fine. Problem is, that's not where it's supposed to be.
My app contains a lot of other remoting calls to a different server (not on port 9090) as well as a WCF service. My guess is that one of these things are interfering.
Any ideas on how I can figure out why this remoting call never returns?
Update:
I have determined that the SocketException is likely a red herring as these exceptions are created when it works in the 3rd party test app. Also, it looks like the reason why it is hanging is because it is waiting for a Socket.Read() which never gets any data.
It turns out, in .NET remoting, there can only be one TCPClientChannel per AppDomain. Activator.GetObject() uses the first channel that was registered.
The reason why this was blocking is because I already had a TCPClientChannel setup in my AppDomain. This channel had the secure set to false when it was registered. i.e.
ChannelServices.RegisterChannel(_channel, false);
The service I was trying to talk to had security enabled and therefore the remoting call would hang trying to listen to a response that would never come.
The solution is to load my integration into a new AppDomain so that I can configure the TCPChannel differently.

HPC BrokerClient takes a long time to execute and throws an error on trying to get results

I am trying to get my sample WCF service to run on a on-premise HPC 2012 SP1 cluster. It's an 2012 HPC WCF application. My test service is deployed to all Compute nodes in a
cluster and configuration file is deployed on a head node too. In my client application I am using BrokerClient wrapper over my service proxy client. By using HPC Cluster
Manager I can see that my service is invoked and ran, but it is very slow, it takes him about 5 minutes to finish executing (and it's a simple test service, without intensive
operations). And also in my client application I am getting he following error after tasks finished executing on a cluster:
"Broker is unavailable due to loss of heartbeat.
Make sure you can connect to the broker node, the HpcBroker service is running on the broker node and the session is still running."
here is my client code:
SessionStartInfo info = new SessionStartInfo(this.textBox1.Text, this.textBox2.Text);
info.SessionResourceUnitType = SessionUnitType.Core;
info.Secure = false;
info.MinimumUnits = 1;
info.MaximumUnits = 1;
using (Session s = Session.CreateSession(info))
{
NetTcpBinding binding = new NetTcpBinding(SecurityMode.None, false);
using (BrokerClient<IHpcTestService> client = new BrokerClient<IHpcTestService>(s, binding))
{
ApplyCalculationsRequest request = new ApplyCalculationsRequest("test");
client.SendRequest<ApplyCalculationsRequest>(request);
client.EndRequests();
foreach (var resp in client.GetResponses<ApplyCalculationsResponse>())
{
this.label3.Text = resp.Result.ApplyCalculationsResult;
break;
}
}
}
P.S. when I try to use plain proxy client, without BrokerClient wrapper, everything works just fine.
Any ideas on the above?
Regrading the slowness, you'd need to provide more specifics about what kind of slow down you are talking about. You can see slowness in various areas and each one is caused by something different.
If it is slowness in Job Creation, this happens on the Head Node
If it is slowness in submission of tasks to job and executing EndRequests, this happens on the Broker node as the requests are written to the MSMQ folder on the broker node brokering the job.
If it is slowness in job runtime, then this is an issue with the worker nodes; this could be caused by a slow database connection, or general slowness in your application, or if you are using central shared location for the service instead of a local copy this could also cause slowness.
As for the error you are getting:
"Broker is unavailable due to loss of heartbeat. Make sure you can connect to the broker node, the HpcBroker service is running on the broker node and the session is still running."
When using the BrokerClient wrapper for whatever reason HPC sends 2 final callback messages instead of just the one. For any given job the final task in the list (but not necessarily the last executed task in the list) has a bool that designates it as the last task. When you have received all callbacks and you re-attach to the job and close it, it sends out a final message which you cannot open because the service channel was closed. I do not know why it does this when you have a BrokerClient wrapper in place; I've yet to figure that out. Essentially if I don't see that message more than once per job I ignore it at this point.
Out of curiosity can you post your code related to using a Proxy Client?

WCF timeout error with code running fine

I have a project that uses a WCF service to do some database queries, builds an "Environment" object (which consists of different database class objects) and returns it inside a "Workspace" object to the client. It's been running fine.
I added another "Database" type to the service with all the correct contract and method updates. Now when I call the method the client times out after 1 minute. In debugging it take about 3-5 seconds to hit the end of the service method. Then nothing happens for the rest of the minute until on the client side we see a timeout problem. There are no errors/exceptions thrown.
Please see below:
Calling from client:
490 m_ScanWorkspace = m_Connection.ScanProxy.CreateEnvironments
End of service method:
477 return tWorkspace;
478 }
It takes 3-5 seconds to get to line 478 in the service. F10 shows it's complete.
Nothing happens until 1 minute later when line 490 in the client shows a timeout error. while debugging I can see a valid object in tWorkspace.
Firstly, set up WCF tracing using the Diagnostics namespace. Just use the first example on that tutorial and WCF will dump out a log of all activity, which you can open up in the log viewer. It will tell you exactly where the call is failing, which will help you pinpoint the problem.
WCF is great, but the error messages it gives are cryptic and often close to useless. A timeout after 1 minute doesn't necessarily mean what a timeout would normally mean - i.e. couldn't find the server. It could be other issues.
More than likely there will be a threshold exceeded which causes the response object to be incomplete. This could be array length, string content length, message size, and so on. You will find some of these detailed here: https://stackoverflow.com/a/480191/146077
Good luck!

Problem about checking a WCF connection is opened

I have a problem about checking a WCF connection is opened. My WCF Connection is bi-directional. I use State property to check the connection's state at client. My function:
private bool ConnectionIsOpen()
{
if (m_Service != null && (m_Service.State | CommunicationState.Opened) == CommunicationState.Opened)
{
return true;
}
return false;
}
I create a service which is a thread running every 10 seconds to check the connection's state. I use the method ConnectionIsOpen() for checking. Everything is well on running on Windows XP. However there is a problem when running on Windows 7.
When I unplug the network cable to create to disconnect, If running application on Windows XP, checking connection's State is Faulted but if running on Windows 7, checking connection' State is still Opened.
Anyone can help me how to check a connection is openned or not in this case. Thanks.
This will always be true:
(m_Service.State | CommunicationState.Opened) == CommunicationState.Opened
Example, m_Service.State = 0:
0 | CommuncationState.Opened == CommuncationState.Opened
You want to use & (AND) instead.
We ran into a similar problem in our own system; disconnecting the network cable or placing either the client machine or the server in sleep mode does not generate a channel fault.
From what I can tell, it seems that the connection state only indicates the state of the connection after the last call and not the current connection state. The only way to know the current state is to actually call the service.
If your client doesn’t need to call the service that often but must react if the connection is lost one solution is to implement a dummy call on the client side which periodically polls the service. If the connection is unavailable when the dummy call is made you’ll get a channel fault that you can then deal with.
The catch is you can’t simply use the dummy call to guarantee that the next call to the service will work:
public void SomeMethode()
{
if (ConnectionIsOpen())
{
m_Service.Dummy();
// Connection is lost here
m_Service.SomeMethode();
}
}
To get around this problem, we implemented a system that automatically re-executes any failed service calls which generate a channel fault after the connection has been restored.
The best and asured way to confirm the Communication state is Open or not is to call the Faulted event like below :
proxyInstance.InnerChannel.Faulted -= new EventHandler(ProxyChannelFaulted);
But this works only with those bindings that support ReliableMessaging like WsHttpBinding.
For detail refer the link : WCF Proxy Client taking time to create, any cache or singleton solution for it
Thanks,
Jai Kumar
The fact that you are getting completely different results on windows 7 is not surprising. Microsoft completely re-engineered the TCP stack with windows vista, so the functionality is quite different from xp in the core networking functionality.
The first thing that I would do is use wireshark to see what is actually going across the wire. See if your TCP connection actually terminates when you pull the plug. Windows might be doing some kind of connection persistence / buffering in case the connection comes back quickly.

Categories