SignalR hub method not reporting all calls to IProgress.Report - c#

I have a situation here where my javascript application is calling back into the server using web socket with the signalR proxy.
I have a long running method for which I report progress 10 times.
However I noticed that the client progress callback is not always called 10 times even though the server method definitely calls it 10 times.
Here is the server method pseudo code simplified for clarity:
public async Task LongRunning(IProgress<Something> progress)
{
var things = await Get10ThingsFromDb();
foreach (var thing in things)
{
var something = ComputeSomething(thing);
progress.Report(something);
}
}
Notice that right after that last call to progress.Report, the method finishes immediately after.
Most of the time it works fine, but maybe once every 5 times I get less than 10 report calls, sometimes even just 1.
I have inspected the network tab in Chrome, and it seems that the problem is that the server is sometimes sending the method completion result BEFORE it has sent all the reports.
So in a normal case the server sends the following frames
-Report 10 times
-Completion 1 time
But in a failed case the server sends the following frames.
-Report 7 times
-Completion 1 time
-Report 3 times
My guess is that the js client discards any progress report that happens after the completion.
I ended up adding await Task.Delay(100)
at the end of the server method to avoid the race condition. This is not ideal...
Is this a bug or am I missing something?
Thanks

Related

Excessive timeout on simple byte array reply from SOAP / WSDL Web Service call ... as if it started replying but never finishes?

My C# client (running on .NET Framework 4.5.1 or later) calls a WSDL-defined SOAP web service call that returns a byte[] (with length typically about 100000). We make hundreds of calls to this web service just fine -- they normally take just a few seconds to return. But very intermittently, the call sits there for exactly 5 minutes and then throws an InvalidOperationException indicating that "There is an error in XML document (1, 678)", with an InnerException that is a WebException "The operation has timed out." We've wrapped a try-catch around this call, look for those particular Exceptions, and then ask the user if they'd like us to retry it, and usually it works just fine on the next try.
Looking at the logging on the server, the logs for the good calls and the intermittent bad calls look exactly the same. In particular, in both cases we get the log statement at the very end of the web service, right before the "return byteArray;"... and it is doing that in the typical 3-15 seconds from the start of the call. So, it seems the web service returns the byte array successfully, but the client that called the web service just never receives it.
However, the client does NOT get the typical SoapException or WebException... for example, if we pause the web service in the debugger right before that return, then after 60 seconds the client will get a WebException "The operation has timed out." But we don't get that in this case... instead we are stuck there for a full 5 minutes before we finally get the InvalidOperationException mentioned above. So, it is as if it started receiving the reply, so it doesn't consider it timed out the normal way, but it never gets the rest of the reply, and the parsing/deserializing of the XML containing the reply eventually times out.
Question #1: Any suggestions on what's happening here? Or what we might be doing wrong in our web service that would result in a byte[] reply getting stuck mid-return intermittently? I'd obviously love to fix the root problem.
Question #2: What controls the length of that 5 minute timeout?? Our exception handling for this would be okay except for the ridiculous 5 minute timeout. After about 10 seconds, the user knows it is stuck because it normally returns in 10 seconds or less. But they have to sit there and wait for 5 minutes before they can do anything. We have set every timeout setting we could find to just 60 seconds, but none seem to control this. We have set:
In the server Web.config: <httpRuntime executionTimeout="60">
In the server Global.asax.cs: HttpContext.Current.Server.ScriptTimeout = 60;
In both server and client: ServicePointManager.MaxServicePointIdleTime = 60000;
In the client, right after we new up the WSDL-defined class derived from SoapHttpClientProtocol with all the web service calls, we call: service.Timeout = 60000;
We previously had those at their defaults or set to 100 / 100000 ... we lowered them all to 60 / 60000 to see if the 5 minute wait would come down at all (just in case one or more of them were being added into that 5 minutes). But no, no matter what we changed any of those timeouts to, the timeout in this case remains exactly 5 minutes, every time it gets stuck.
Does anybody know where the length of the timeout is set for when it generates an InvalidOperationException on the XML document containing the returned byte array due to an InnerException WebException with the timeout?? (please!)

Send constant number of http requests without waiting

Hi guys I am trying to use a simple console application to send requests to service without out waiting. The goal is to send 100 requests per second without waiting for the response. My idea is to create and run 10 threads at a time, each thread will create a http client and send request. after threads created then wait for 100 ms, then create another 10 threads. The code looks like:
while (true)
{
try
{
for (int i = 0; i < 10; i++)
{
Task.Factory.StartNew(SendRequest);
}
}
catch (Exception)
{
// ignored
}
Thread.Sleep(100);
}
Each SendRequest method will print http status code the result to console after get the result from http request. But I found that this application will successfully print first 20 result, which means that threads has finished and destroyed. But after this, it will no longer print anything out, but the memory is increasing, which means that thread is kept creating but spinning. So, could you help answer:
1) why the thread is hanging after 20 requests finished, is it because of the connection limit at http client?
2) how to send constant requests per second without waiting by using code?
May be off topic, but if you don't need to do this in C#, you can use apache benchmark to do this and test out. You can control concurrency, time and number of requests.
ab -c 10 -t 60 -n 6000 http://www.website.com/
If it has to be in C# .. then sorry and nevermind

SignalR Groups.Add times out and fails

I'm trying to add a member to a Group using SignalR 2.2. Every single time, I hit a 30 second timeout and get a "System.Threading.Tasks.TaskCanceledException: A task was canceled." error.
From a GroupSubscriptionController that I've written, I'm calling:
var hubContext = GlobalHost.ConnectionManager.GetHubContext<ProjectHub>();
await hubContext.Groups.Add(connectionId, groupName);
I've found this issue where people are periodically encountering this, but it happens to me every single time. I'm running the backend (ASP.NET 4.5) on one VS2015 launched localhost port, and the frontend (AngularJS SPA) on another VS 2015 launched localhost port.
I had gotten SignalR working to the point where messages were being broadcast to every connected client. It seemed so easy. Now, adding in the Groups part (so that people only get select messages from the server) has me pulling my hair out...
That task cancellation error could be being thrown because the connectionId can't be found in the SignalR registry of connected clients.
How are you getting this connectionId? You have multiple servers/ports going - is it possible that you're getting your wires crossed?
I know there is an accepted answer to this, but I came across this once for a different reason.
First off, do you know what Groups.Add does?
I had expected Groups.Add's task to complete almost immediately every time, but not so. Groups.Add returns a task that only completes, when the client (i.e. Javascript) acknowledges that it has been added to a group - this is useful for reconnecting so it can resubscribe to all its old groups. Note this acknowledgement is not visible to the developer code and nicely covered up for you.
The problem is that the client may not respond because they have disconnected (i.e. they've navigated to another page). This will mean that the await call will have to wait until the connection has disconnected (default timeout 30 seconds) before giving up by throwing a TaskCanceledException.
See http://www.asp.net/signalr/overview/guide-to-the-api/working-with-groups for more detail on groups

Is it possible to implement Async Webservice call in .Net not only by one connection?

I have a .net Webservice. It has a method that does a task that takes 60 seconds and after that, returns the string result to client.
In some networks especially low band widths I get timeout error in 40 seconds before webservice method do its task in 60 second.
Now I want to implement webservice Async call to support low band width networks.
In Async webservice call an approach is using a thread that runs webservice method and returns webservice result to main thread that is shown in the following picture.
But my problem will not be solved in this approach because that thread uses one connection.
I need another approach. A thread in my client call webservice method and method starts its operation and when the task is done, 1) webservice sends a message that your response is ready or, 2) client checks if the webservice response is ready (I think polling mechanism) like the following picture.
How can I implement the second approach in .net? Is it possible?
Thanks.
Create a table on your database to store the state of the process.
UniqClientId, ProcessId, StartTime, EndTime and any other state if required.
Client sends a request to the server by passing its unique id.
Server logs the process on the above table and initiates the process.
Clients contacts the server instantly(2-3 sec or 15-20 sec depending on your application) to check the process completion.
If the client get a response that process has been completed, then it requests the server to send the response.
In between, the server does the following job.
When the process completes, stores the EndTime on above table.
Provides a method to send the process state by checking the above table.
Provides a method to send the response.
I'm not sure what exactly your service is doing, but if the operation of your process is just to modify some table on the database, then is is not difficult to implement this.

Application throwing time out exception on sending bulk data

I have a Console Application which consumes a BizTalk Web Service. The Problem is that when I send the BizTalk Service object data in bulk, my console application throws the exception:
Application has either timed out or is Timing out.
My application actually needs to wait for the Biztalk service to finish processing its job. Increasing the obj.Timeout value was of no help. Is there anything else other than using Thread.Sleep method (which I want to avoid)?
Below is the relevant code snippet from my application:
pumpSyncService.Timeout = 750000;
outputRecords = pumpSyncService.PumpSynchronization(pumpRecords);
The pump records contain an array of objects. When the count is around 30, I get a correct response, but when the count increases to around 150 I get the exception.
Try sending smaller chunks in a loop. Instead of sending 150 all at once, send 30 records 5 times. The timeout might be happening because it takes too long to send 150 records.
you should be able to send all 30 at once , if the service allows you to. I am assuming you have verified that the event kicking this off is not firing 5 times . try it asynchronously and process your results when they come back.

Categories