Set StackyClient Timeout - c#

I use the StackyClient library to loop over newest questions in Stack Overflow. Generally this works:
StackyClient client = new StackyClient("1.1", "", Sites.MetaStackOverflow, new UrlClient(), new JsonProtocol());
var o = new QuestionOptions();
...
IPagedList<Question> l = null;
while (!stop) {
l = client.GetQuestions(o);
var newQuestions = (from Question q in l
orderby q.Id descending
select q).ToList();
...
}
But at some point this seems to fail. Maybe due to network connectivity problems or I don't know what. The program hangs at client.GetQuestions(o).
I would like to detect that problem and reconnect again. I thought about setting a timeout somehow to prevent the program from hanging. How can it be done?

There are two possible answers here:
Are you sure this is hanging? Did you run it under a debugger and make sure it's not throwing an uncaught exception? Did you check with ProcessExplorer and/or Fiddler to see that the request and socket are still open? It seems more likely to me that the socket is being closed or you're getting an exception and it's not being caught.
If it is confirmed that this is hanging at GetQuestions, then a review of the code indicates that you'll have to ask the author of Stacky to implement this feature or you're going to have to implement it yourself... the feature does not exist today, from what I can tell.
In Stacky's UrlClient.cs there is a blocking call to HttpWebRequest.GetResponse and there is a blocking call to ReadToEnd on the response stream. No timeout is being set for either call, nor on the HttpWebRequest object itself:
Stacky/UrlClient.cs
In addition, I don't see anything in the connectionManagement settings that would allow setting a timeout on responses and reading from the response stream of HttpWebRequest:
connectionManagement setting
The Stack UrlClient code is re-throwing WebExceptions other than 404 and 407 (proxy auth required), so if you got a 408 (the server timing you out) from StackOverflow it would throw out of the Stacky GetQuestions call, which is why I suspect it might be that you're seeing an uncaught exception and not a hang.
I periodically see 408's from StackOverflow when just browsing questions in Chrome, so it is possible that this is happening to your program and you just need to catch the exception and retry after a delay.
Hope this helps - Harold

Related

Are sync exceptions possible in HttpClient?

Right now, I'm not talking about ArgumentNullException or InvalidOperationException, but more or less only about HttpRequestException.
var responseAsync = httpClient.SendAsync(something);
try
{
var response = await responseAsync;
}
catch(Exception)
{
}
Will this code ever throw any exceptions, or can we safely assume that all the possible exceptions can only happen during the await?
"but more or less only about HttpRequestException"
MSDN:
The doco makes it quite clear:
HttpRequestException
The request failed due to an underlying issue such as network connectivity, DNS failure, server certificate validation or timeout.
So the answer would be "yes" if say your cat pulled out the network lead for example.
Will this code ever throw any exceptions,
Yes.
or can we safely assume that all the possible exceptions can only happen during the await?
It can occur on the following line too because by the time SendAsync returns (not to be confused with when the Task is complete), the Task has already been created and there is a tiny chance of it throwing before you get to the next line.
var responseAsync = httpClient.SendAsync(something);
EDIT: (from my comment below)
Also, for all we know, SendAsync might perform some "initial check" prior to the Task being created. In which case you'll need a try/catch on the above.

How to get meaningful information out of SendAsync exceptions

I've written some code which does a post to a remote webservice. I've tried using both HttpClient.PostAync as well as HttpClient.SendAsync, where with the former I just provide everything up front, and with the later I build an HttpRequestMessage with the appropriate values. In my testing today I'm getting some type of exception here. The same exception with both implementations. Presumably a timeout, or other error on the remote side / transmission. When I get this exception, it comes with the generic message of 'A task was cancelled'.
The code looks something like this:
using (MultipartFormDataContent formData = new MultipartFormDataContent())
{
//formData.Add(clientIDContent, "client_id");
//formData.Add(clientSecretContent, "client_secret");
formData.Add(quoteIDContent, "CPQuoteID");
formData.Add(poNumContent, "PONumber");
formData.Add(dealerNumContent, "DealerNumber");
formData.Add(orderUserContent, "OrderingUser");
formData.Add(fileContent, "order-file");
try
{
var response = await client.PostAsync(actionURL, formData);
int I = 0;
}
catch (Exception ex)
{
int x = 0;
}
}
the exception I get back is of type TaskCanceledException. The inner exception also carries no meaningful information. There must be a way to get better information back from this. It's frustrating to debug issues when our logs contain only generic messages like this. How do I get this info?
Just a bit more info. I'm operating under the assumption here that the HTTP client does this operation with a Task. At some point the task times out. I don't know if this is because the HTTP timeout is greater than the task timeout in some way, or if the HttpClient object catches the timeout exception, consumes it, then cancels the task throwing the cancellation exception. Either way, it would seem that meaningful information about the underlying issue is lost. Info I'd like to log.

ObjectDisposedException for TCPClient.Close in System.dll?

Im using a Request System called xNet and it seems ObjectDisposedExceptions are occuring which on-occurence causes a HUGE cpu spike, sometimes continuously keeping CPU at 99-100% causing freezes and lag.
The script mentioned is the following:
https://github.com/PR4GM4/xNet-Ameliorated
An example code is:
using (HttpRequest httpRequest = new HttpRequest()) {
string url = "https://httpbin.org";
string[] proxysplit = proxy.Split(':');
httpRequest.Proxy = new InternalProxyClient(ProxyType.HTTP, proxysplit[0], int.Parse(proxysplit[1]), null, null);
httpRequest.UserAgent = Http.ChromeUserAgent();
httpRequest.KeepAlive = true;
httpRequest.ConnectTimeout = 15000;
httpRequest.AllowAutoRedirect = true;
HttpResponse hr = httpRequest.Start(HttpMethod.GET, new Uri(url, UriKind.Absolute), new StringContent(""));
if (hr == null) return "2";
string sr = hr.ToString();
if (sr == null) return "2";
}
(If a list of half/dead proxies are needed, I can provide it, just im not sure if linking to them is allowed or not.)
Big note here, it seems to only ever occur whenever theres some kind of other exception like failing to connect to the proxy, or a general bad response, so good connection proxies and/or no proxy at all never has this issue (unless again a general failed error).
If you loop this code, and give it a dead proxy (And to speed things up, multi-thread it to around 5 a time) it will eventually cause an exception like bad response or a timeout and eventually an objectdisposedexception.
I tried debugging in Visual Studio but it gives me almost no information, Historical Debugging gives no information just "Source not found".
Call Stack for the First Exception Thrown of the ObjectDisposedException in the screenshot above.
Seems to be related to line 1430 in ~Http/HttpRequest.cs or line 217 in ~Proxy/ProxyClient.cs as it's the only line I know to exist thats to do with EndConnect socket and also coincidentally can produce ObjectDisposedException. Just not sure how to properly handle the exception here without causing the rest of the script to fail. Also why does a simple exception here cause so much CPU spike?
Strangely enough, no matter how I wrap an exception handler for ObjectDisposedException it never gets triggered, no matter how much code or where I wrap? (On both scripts)
try
{
tcpClient.EndConnect(ar);
}
catch (Exception ex)
{
connectException = ex;
}
I found out why, it wasnt because of the .EndConnect on either of the 2 files, it was actually caused by the .Close() calls, since it does .EndConnect inside of that, thats why I couldnt see any "Source" etc.
So it was causeed because the socket connection wasnt connected, so doing .Close() would cause the Exception.
It was a simple fix.
(Where tcp = a TcpClient)
Do the following instead of just tcp.Close()
On Timeouts (Where it's most likely if never at all connected):
if (tcp.Client.Connected) {
tcp.GetStream().Close();
tcp.Close();
}
When it might be properly connected:
if (!tcp.Connected) {
if (tcp.Client.Connected) tcp.GetStream().Close();
tcp.Close();
}

How to handle an interrupted HTTP connection in C#

Suppose I create a HTTPWebRequest, call its GetResponse() and start reading from the response stream. If the connection is interrupted while reading from the stream, do I have to wait for it to time out, or can I know right away that something's gone wrong? No exception is thrown when I interrupt the connection (e.g. I disconnect my computer from the network).
It depends on the situation.
In general you'll need to be prepared for both situations (immediate and late interruption).
If, for example, the server disconnects you, you'll know relatively quickly.
See http://msdn.microsoft.com/en-us/library/system.net.webexceptionstatus for the kinds of errors that can occur (the WebRequest classes throw WebExceptions on errors)
You have a variety of options:
Use the async methods (BeginGet... and EndGet...) and model your application around this. Basically you'll be notified "at some point" if there was a success or error. Do something else in the meantime
If you want absolute control you can specify a ReadTimeout on the acquired stream (See comment on the other answer, set Timeout on the request as well). Re-try or whatever.
Just wait
You dont have to worry if the request is interrupted or not.
You can specify explicit timeout as follows.
If its interrupted you will get exception.
try
{
var request = HttpWebRequest.Create(url);
request.Timeout = 3000;
var response = request.GetResponse() as HttpWebResponse;
if (response.StatusCode.Equals(HttpStatusCode.OK))
{
//do stuff
}
}
catch (Exception exception)
{
exception.ToLog();
}
Most probably you have to wait for the timeout

The I/O operation has been aborted because of either a thread exit or an application request

My application is working as a client application for a bank server. The application is sending a request and getting a response from the bank. This application is normally working fine, but sometimes
The I/O operation has been aborted because of either a thread exit or
an application request
error with error code as 995 comes through.
public void OnDataReceived(IAsyncResult asyn)
{
BLCommonFunctions.WriteLogger(0, "In :- OnDataReceived",
ref swReceivedLogWriter, strLogPath, 0);
try
{
SocketPacket theSockId = (SocketPacket)asyn.AsyncState;
int iRx = theSockId.thisSocket.EndReceive(asyn); //Here error is coming
string strHEX = BLCommonFunctions.ByteArrToHex(theSockId.dataBuffer);
}
}
Once this error starts to come for all transactions after that same error begin to appear, so
please help me to sort out this problem. If possible then with some sample code
Regards,
Ashish Khandelwal
995 is an error reported by the IO Completion Port. The error comes since you try to continue read from the socket when it has most likely been closed.
Receiving 0 bytes from EndRecieve means that the socket has been closed, as does most exceptions that EndRecieve will throw.
You need to start dealing with those situations.
Never ever ignore exceptions, they are thrown for a reason.
Update
There is nothing that says that the server does anything wrong. A connection can be lost for a lot of reasons such as idle connection being closed by a switch/router/firewall, shaky network, bad cables etc.
What I'm saying is that you MUST handle disconnections. The proper way of doing so is to dispose the socket and try to connect a new one at certain intervals.
As for the receive callback a more proper way of handling it is something like this (semi pseudo code):
public void OnDataReceived(IAsyncResult asyn)
{
BLCommonFunctions.WriteLogger(0, "In :- OnDataReceived", ref swReceivedLogWriter, strLogPath, 0);
try
{
SocketPacket client = (SocketPacket)asyn.AsyncState;
int bytesReceived = client.thisSocket.EndReceive(asyn); //Here error is coming
if (bytesReceived == 0)
{
HandleDisconnect(client);
return;
}
}
catch (Exception err)
{
HandleDisconnect(client);
}
try
{
string strHEX = BLCommonFunctions.ByteArrToHex(theSockId.dataBuffer);
//do your handling here
}
catch (Exception err)
{
// Your logic threw an exception. handle it accordinhly
}
try
{
client.thisSocket.BeginRecieve(.. all parameters ..);
}
catch (Exception err)
{
HandleDisconnect(client);
}
}
the reason to why I'm using three catch blocks is simply because the logic for the middle one is different from the other two. Exceptions from BeginReceive/EndReceive usually indicates socket disconnection while exceptions from your logic should not stop the socket receiving.
In my case, the request was getting timed out. So all you need to do is to increase the time out while creating the HttpClient.
HttpClient client = new HttpClient();
client.Timeout = TimeSpan.FromMinutes(5);
I had the same issue with RS232 communication. The reason, is that your program executes much faster than the comport (or slow serial communication).
To fix it, I had to check if the IAsyncResult.IsCompleted==true. If not completed, then IAsyncResult.AsyncWaitHandle.WaitOne()
Like this :
Stream s = this.GetStream();
IAsyncResult ar = s.BeginWrite(data, 0, data.Length, SendAsync, state);
if (!ar.IsCompleted)
ar.AsyncWaitHandle.WaitOne();
Most of the time, ar.IsCompleted will be true.
I had this problem. I think that it was caused by the socket getting opened and no data arriving within a short time after the open. I was reading from a serial to ethernet box called a Devicemaster. I changed the Devicemaster port setting from "connect always" to "connect on data" and the problem disappeared. I have great respect for Hans Passant but I do not agree that this is an error code that you can easily solve by scrutinizing code.
In my case the issue was caused by the fact that starting from .NET 5 or 6 you must either call async methods for async stream, or sync methods for sync strem.
So that if I called FlushAsync I must have get context using GetContextAsync
What I do when it happens is Disable the COM port into the Device Manager and Enable it again.
It stop the communications with another program or thread and become free for you.
I hope this works for you. Regards.
I ran into this error while using Entity Framework Core with Azure Sql Server running in Debug mode in Visual Studio. I figured out that it is an exception, but not a problem. EF is written to handle this exception gracefully and complete the work. I had VS set to break on all exceptions, so it did. Once I unchecked the check box in VS to not break on this exception, my C# code, calling EF, using Azure Sql worked every time.

Categories