I am trying to do "long polling" with an HttpWebRequest object.
In my C# app, I am making an HTTP GET request, using HttpWebRequest. And then afterwards, I wait for the response with beginGetResponse(). I am using ThreadPool.RegisterWaitForSingleObject to wait for the response, or to timeout (after 1 minute).
I have set the target web server to take a long time to respond. So that, I have time to disconnect the network cable.
After sending the request, I pull the network cable.
Is there a way to get an exception when this happens? So I don't have to wait for the timeout?
Instead of an exception, the timeout (from RegisterWaitForSingleObject) happens after the 1 minute timeout has expired.
Is there a way to determine that the network connection went down? Currently, this situation is indistinguishable from the case where the web server takes more than 1 minute to respond.
I found a solution:
Before calling beginGetResponse, I can call the following on the HttpWebRequest:
req.ServicePoint.SetTcpKeepAlive( true, 10000, 1000)
I think this means that after 10 seconds of inactivity, the client will send a TCP "keep alive" over to the server. That keep alive will fail if the network connection is down because the network cable was pulled.
So, when the cable is pulled, I a keep alive gets sent within 10 seconds (at most), and then the callback for BeginGetResponse happens. In the callback, I get and exception when I call req.EndGetResponse().
I guess this defeats one of the benefits of long polling, though. Since we're still sending packets around.
I'll leave it to you to try pulling the plug on this.
ManualResetEvent done = new ManualResetEvent(false);
void Main()
{
// set physical address of network adapter to monitor operational status
string physicalAddress = "00215A6B4D0F";
// create web request
var request = (HttpWebRequest)HttpWebRequest.Create(new Uri("http://stackoverflow.com"));
// create timer to cancel operation on loss of network
var timer = new System.Threading.Timer((s) =>
{
NetworkInterface networkInterface =
NetworkInterface.GetAllNetworkInterfaces()
.FirstOrDefault(nic => nic.GetPhysicalAddress().ToString() == physicalAddress);
if(networkInterface == null)
{
throw new Exception("Could not find network interface with phisical address " + physicalAddress + ".");
}
else if(networkInterface.OperationalStatus != OperationalStatus.Up)
{
Console.WriteLine ("Network is down, aborting.");
request.Abort();
done.Set();
}
else
{
Console.WriteLine ("Network is still up.");
}
}, null, 100, 100);
// start asynchronous request
IAsyncResult asynchResult = request.BeginGetResponse(new AsyncCallback((o) =>
{
try
{
var response = (HttpWebResponse)request.EndGetResponse((IAsyncResult)o);
var reader = new StreamReader(response.GetResponseStream(), System.Text.Encoding.UTF8);
var writer = new StringWriter();
writer.Write(reader.ReadToEnd());
Console.Write(writer.ToString());
}
finally
{
done.Set();
}
}), null);
// wait for the end
done.WaitOne();
}
I dont think you are gonna like this. You can test for internet connectivity after you create the request to the slow server.
There are many ways to do that - from another request to google.com (or some ip address in your network) to P/Invoke. You can get more info here: Fastest way to test internet connection
After you create the original request you go in a loop that checks for internet connectivity and until either the internet is down or the original request comes back (it can set a variable to stop the loop).
Helps at all?
Related
When fetching data from a domain that does not resolve, it takes more than 10 seconds to go into the catch block.
try
{
var resultDomain = client.GetAsync("http://nonexistent.nonexistent.nonexistent").Result.Content.ReadAsStringAsync().Result;
}
catch(Exception ex)
{
//outputs
//The remote name could not be resolved: 'nonexistent.nonexistent.nonexistent'
//11,0632079
Console.WriteLine(ex.InnerException.InnerException.Message);
Console.WriteLine(watch.Elapsed.TotalSeconds);
}
This while the command
nslookup nonexistent.nonexistent.nonexistent
finishes almost immediately with the notification that the domain doesnt exist. Is there a way to have the HttpClient/WebClient/... behave as fast as nslookup does? What is .NET waiting for?
According to wireshark, DNS responds immediately
What about setting a timeout lower than the default of 100 seconds?
client.Timeout = TimeSpan.FromMilliseconds(500);
client.GetAsync(...);
I have a windows service that uses a websocket (from http://sta.github.io/websocket-sharp/) to conenct to Slack and monitor messages.
My connection code looks something like this:
ws = new WebSocket(connection.Url);
ws.OnMessage += async (sender, e) =>
{
var msg = JsonConvert.DeserializeObject<MessageFromSlack>(e.Data);
if (msg.Type == "message" && msg.Text != null && msg.User != UserId)
{
if (userMatcher.IsMatch(msg.Text))
{
await ProcessDirectMessage(msg);
}
await ProcessMessage(msg);
}
if (msg.Type == "channel_joined")
{
await ChannelJoined(msg.ChannelModel.Id);
}
};
ws.OnClose += (sender, e) =>
{
var reason = e.Reason;
var code = e.Code;
System.Diagnostics.Debug.WriteLine($"{code}:{reason}");
};
ws.Connect();
Basically it waits for a message and then if it's directed # my bot, it'll call ProcessDirectMessage and if not it'll call ProcessMessage. The details of those functions are, I think, unimportant (they do some matching looking for key phrases and respond by sending a message back).
This all works fine. For a while. But after some period of time (usually more than a day), it just stops responding altogether. My OnMessage handler never gets hit. I thought that maybe what is happening is the websocket is getting closed on the server side, so I added the OnClose handler, but that never seems to get hit either.
Does anybody have an idea what might be happening here? Is there a way to keep the connection alive, or else reconnect it when it dies?
By the nature of TCP connection - the only reliable way to detect its gone is to write something to it. If you are just reading (waiting for data to arrive) - you can do that for a very long time while the other side is long time dead. That happens if that other side did not close connection gracefully (which involves an exchange of some TCP packets).
Web socket protocol defines special Ping frame, and corresponding Pong frame, which you should use to avoid situation described in the question. From time to time you should send Ping frame and wait (for a certain timeout) for server to respond with Pong frame. If it did not respond in given timeout - assume connection is dead and reconnect.
As far as I know - library you use does not automatically send ping requests on your behalf. However, it allows you to do that via Ping method.
You need to configure timeout with
ws.WaitTime = TimeSpan.FromSeconds(5);
And then, from time to time (for example - when you did not receive any new messages in last X seconds), do:
bool isAlive = ws.Ping();
There is also boolean property which does the same:
bool isAlive = ws.IsAlive;
This is a blocking call (both of the above). It will return true if server replied with Pong during ws.WaitTime interval, and false otherwise. Then you can act accordingly.
I am using TCP as a mechanism for keep alive here is my code:
Client
TcpClient keepAliveTcpClient = new TcpClient();
keepAliveTcpClient.Connect(HostId, tcpPort);
//this 'read' is supposed to blocked till a legal disconnect is requested
//or till the server unexpectedly dissapears
int numberOfByptes = keepAliveTcpClient.GetStream().Read(new byte[10], 0, 10);
//more client code...
Server
TcpListener _tcpListener = new TcpListener(IPAddress.Any, 1000);
_tcpListener.Start();
_tcpClient = _tcpListener.AcceptTcpClient();
Tracer.Write(Tracer.TraceLevel.INFO, "get a client");
buffer = new byte[10];
numOfBytes = _tcpClient.GetStream().Read(buffer, 0, buffer.Length);
if(numOfBytes==0)
{
//shouldn't reach here unless the connection is close...
}
I put only the relevant code... Now what that happens is that the client code is block on read as expected, but the server read return immediately with numOfBytes equals to 0, even if I retry to do read on the server it return immediately... but the client read is still block! so in the server side I think mistakenly that the client is disconnected from the server but the client thinks it connected to the server... someone can tell how it is possible? or what is wrong with my mechanism?
Edit: After a failure I wrote to the log these properties:
_tcpClient: _tcpClient.Connected=true
Socket: (_tcpClient.Client properties)
_tcpClient.Client.Available=0
_tcpClient.Client.Blocking=true
_tcpClient.Client.Connected=true
_tcpClient.Client.IsBound=true
Stream details
_tcpClient.GetStream().DataAvailable=false;
Even when correctly implemented, this approach will only detect some remote server failures. Consider the case where the intervening network partitions the two machines. Then, only when the underlying TCP stack sends a transport level keep-alive will the system detect the failure. Keepalive is a good description of the problem. [Does a TCP socket connection have a “keep alive”?] 2 is a companion question. The RFC indicates the functionality is optional.
The only certain way to reliably confirm that the other party is still alive is to occasionally send actual data between the two endpoints. This will result in TCP promptly detecting the failure and reporting it back to the application.
Maybe something that will give clue: it happens only when 10 or more clients
connect the server the same time(the server listen to 10 or more ports).
If you're writing this code on Windows 7/8, you may be running into a connection limit issue. Microsoft's license allows 20 concurrent connections, but the wording is very specific:
[Start->Run->winver, click "Microsoft Software License Terms"]
3e. Device Connections. You may allow up to 20 other devices to access software installed on the licensed computer to use only File Services, Print Services, Internet Information Services and Internet Connection Sharing and Telephony Services.
Since what you're doing isn't file, print, IIS, ICS, or telephony, it's possible that the previous connection limit of 10 from XP/Vista is still enforced in these circumstances. Set a limit of concurrent connections to 9 in your code temporarily, and see if it keeps happening.
The way I am interpretting the MSDN remarks it seems that behavior is expected. If you have no data the Read the method returns.
With that in mind I think what I would try is to send data at a specified interval like some of the previous suggestions along with a "timeout" of some sort. If you don't see the "ping" within your designated interval you could fail the keepalive. With TCP you have to keep in mind that there is no requirement to deem a connection "broken" just because you aren't seeing data. You could completely unplug the network cables and the connection will still be considered good up until the point that you send some data. Once you send data you'll see one of 2 behaviors. Either you'll never see a response (listening machine was shutdown?) or you'll get an "ack-reset" (listening machine is no longer listening on that particular socket)
https://msdn.microsoft.com/en-us/library/vstudio/system.net.sockets.networkstream.read(v=vs.100).aspx
Remarks:
This method reads data into the buffer parameter and returns the number of bytes successfully read. If no data is available for reading, the Read method returns 0. The Read operation reads as much data as is available, up to the number of bytes specified by the size parameter. If the remote host shuts down the connection, and all available data has been received, the Read method completes immediately and return zero bytes.
As I can see you are reading data on both sides, server and client. You need to write some data from the server to the client, to ensure that your client will have something to read. You can find a small test program below (The Task stuff is just to run the Server and Client in the same program).
class Program
{
private static Task _tcpServerTask;
private const int ServerPort = 1000;
static void Main(string[] args)
{
StartTcpServer();
KeepAlive();
Console.ReadKey();
}
private static void StartTcpServer()
{
_tcpServerTask = new Task(() =>
{
var tcpListener = new TcpListener(IPAddress.Any, ServerPort);
tcpListener.Start();
var tcpClient = tcpListener.AcceptTcpClient();
Console.WriteLine("Server got client ...");
using (var stream = tcpClient.GetStream())
{
const string message = "Stay alive!!!";
var arrayMessage = Encoding.UTF8.GetBytes(message);
stream.Write(arrayMessage, 0, arrayMessage.Length);
}
tcpListener.Stop();
});
_tcpServerTask.Start();
}
private static void KeepAlive()
{
var tcpClient = new TcpClient();
tcpClient.Connect("127.0.0.1", ServerPort);
using (var stream = tcpClient.GetStream())
{
var buffer = new byte[16];
while (stream.Read(buffer, 0, buffer.Length) != 0)
Console.WriteLine("Client received: {0} ", Encoding.UTF8.GetString(buffer));
}
}
}
I'm making a tool to test a connection to certain host using a class "PingReply" in .NET. My problem is it takes a while to get a result if the ping result is a fail. It is LAN environment so i can already assume that the connection is failed if it takes more than 100ms. The code below shows a result after 5 seconds, which is 5000ms, if the connection to the host fails. Can i get the faster result even though the connection is failed?
Ping x = new Ping();
PingReply reply = x.Send(IPAddress.Parse("192.168.0.1"));
if (reply.Status == IPStatus.Success)
{
//Do something
}
You can pass a timeout to the Ping.Send() method. Please check out the overloaded members.
Since we can't see your ping object, ill assume you don't know about TIMEOUT. I usually send an async ping, and set the timeout to 3 seconds.
try
{
Ping ping = new Ping();
ping.PingCompleted += (sender, e) =>
{
if (e.Reply.Status != IPStatus.Success)
// Report fail
else
// Report success
};
ping.SendAsync(target, 3000, target); // Timeout is 3 seconds here
}
catch (Exception)
{
return;
}
Ping.Send() has an overload with a timeout parameter:
PingReply reply = x.Send(IPAddress.Parse("192.168.0.1"), 100);
You could use an async delegate to kick off the Ping. The async delegate has a function called BeginInvoke that will kick off a background thread that will immediately return a IAsyncResult. The IAsyncResult has a wait handler called AsyncWaitHandle which has a method called WaitOne which can be assigned a time to wait. This will freeze the current thread a given time in milliseconds, in your case 100, then you can use the property IsCompleted to check to see if the thread has completed its work. For Example:
Func<PingReply> pingDelegate = () => new Ping().Send(IPAddress.Parse("192.168.0.1"));
IAsyncResult result = pingDelegate.BeginInvoke(r => pingDelegate.EndInvoke(r), null);
//wait for thread to complete
result.AsyncWaitHandle.WaitOne(100);
if (result.IsCompleted)
{
//Ping Succeeded do something
PingReply reply = (PingReply) result;
//Do something with successful reply
}
I created a live host scanner too. It uses ARP to check if a computer is online.
An ARP request is much faster than if you'd ping a host.
Here's the code I used to check if a Host is available:
//You'll need this pinvoke signature as it is not part of the .Net framework
[DllImport("iphlpapi.dll", ExactSpelling = true)]
public static extern int SendARP(int DestIP, int SrcIP,
byte[] pMacAddr, ref uint PhyAddrLen);
//These vars are needed, if the the request was a success
//the MAC address of the host is returned in macAddr
private byte[] macAddr = new byte[6];
private uint macAddrLen;
//Here you can put the IP that should be checked
private IPAddress Destination = IPAddress.Parse("127.0.0.1");
//Send Request and check if the host is there
if (SendARP((int)Destination.Address, 0, macAddr, ref macAddrLen) == 0)
{
//SUCCESS! Igor it's alive!
}
If you're interested Nmap also uses this technique to scan for available hosts.
ARP scan puts Nmap and its optimized algorithms in charge of ARP requests. And if it gets a response back, Nmap doesn't even need to worry about the IP-based ping packets since it already knows the host is up. This makes ARP scan much faster and more reliable than IP-based scans. So it is done by default when scanning ethernet hosts that Nmap detects are on a local ethernet network. Even if different ping types (such as -PE or -PS) are specified, Nmap uses ARP instead for any of the targets which are on the same LAN.
This only works within the current subnet! As long as there is no router between the requesting machine and the target it should work fine.
ARP is a non-routable protocol, and can therefore only be used between systems on the same Ethernet network. [...]
arp-scan can be used to discover IP hosts on the local network. It can discover all hosts, including those that block all IP traffic such as firewalls and systems with ingress filters. - Excerpt from NTA-Monitor wiki
For more information on the SendARP function you can check the pinvoke.net documentation.
I believe after lengthy research and searching, I have discovered that what I want to do is probably better served by setting up an asynchronous connection and terminating it after the desired timeout... But I will go ahead and ask anyway!
Quick snippet of code:
HttpWebRequest webReq = (HttpWebRequest)HttpWebRequest.Create(url);
webReq.Timeout = 5000;
HttpWebResponse response = (HttpWebResponse)webReq.GetResponse();
// this takes ~20+ sec on servers that aren't on the proper port, etc.
I have an HttpWebRequest method that is in a multi-threaded application, in which I am connecting to a large number of company web servers. In cases where the server is not responding, the HttpWebRequest.GetResponse() is taking about 20 seconds to time out, even though I have specified a timeout of only 5 seconds. In the interest of getting through the servers on a regular interval, I want to skip those taking longer than 5 seconds to connect to.
So the question is: "Is there a simple way to specify/decrease a connection timeout for a WebRequest or HttpWebRequest?"
I believe that the problem is that the WebRequest measures the time only after the request is actually made. If you submit multiple requests to the same address then the ServicePointManager will throttle your requests and only actually submit as many concurrent connections as the value of the corresponding ServicePoint.ConnectionLimit which by default gets the value from ServicePointManager.DefaultConnectionLimit. Application CLR host sets this to 2, ASP host to 10. So if you have a multithreaded application that submits multiple requests to the same host only two are actually placed on the wire, the rest are queued up.
I have not researched this to a conclusive evidence whether this is what really happens, but on a similar project I had things were horrible until I removed the ServicePoint limitation.
Another factor to consider is the DNS lookup time. Again, is my belief not backed by hard evidence, but I think the WebRequest does not count the DNS lookup time against the request timeout. DNS lookup time can show up as very big time factor on some deployments.
And yes, you must code your app around the WebRequest.BeginGetRequestStream (for POSTs with content) and WebRequest.BeginGetResponse (for GETs and POSTSs). Synchronous calls will not scale (I won't enter into details why, but that I do have hard evidence for). Anyway, the ServicePoint issue is orthogonal to this: the queueing behavior happens with async calls too.
Sorry for tacking on to an old thread, but I think something that was said above may be incorrect/misleading.
From what I can tell .Timeout is NOT the connection time, it is the TOTAL time allowed for the entire life of the HttpWebRequest and response. Proof:
I Set:
.Timeout=5000
.ReadWriteTimeout=32000
The connect and post time for the HttpWebRequest took 26ms
but the subsequent call HttpWebRequest.GetResponse() timed out in 4974ms thus proving that the 5000ms was the time limit for the whole send request/get response set of calls.
I didn't verify if the DNS name resolution was measured as part of the time as this is irrelevant to me since none of this works the way I really need it to work--my intention was to time out quicker when connecting to systems that weren't accepting connections as shown by them failing during the connect phase of the request.
For example: I'm willing to wait 30 seconds on a connection request that has a chance of returning a result, but I only want to burn 10 seconds waiting to send a request to a host that is misbehaving.
Something I found later which helped, is the .ReadWriteTimeout property. This, in addition to the .Timeout property seemed to finally cut down on the time threads would spend trying to download from a problematic server. The default time for .ReadWriteTimeout is 5 minutes, which for my application was far too long.
So, it seems to me:
.Timeout = time spent trying to establish a connection (not including lookup time)
.ReadWriteTimeout = time spent trying to read or write data after connection established
More info: HttpWebRequest.ReadWriteTimeout Property
Edit:
Per #KyleM's comment, the Timeout property is for the entire connection attempt, and reading up on it at MSDN shows:
Timeout is the number of milliseconds that a subsequent synchronous request made with the GetResponse method waits for a response, and the GetRequestStream method waits for a stream. The Timeout applies to the entire request and response, not individually to the GetRequestStream and GetResponse method calls. If the resource is not returned within the time-out period, the request throws a WebException with the Status property set to WebExceptionStatus.Timeout.
(Emphasis mine.)
From the documentation of the HttpWebRequest.Timeout property:
A Domain Name System (DNS) query may
take up to 15 seconds to return or
time out. If your request contains a
host name that requires resolution and
you set Timeout to a value less than
15 seconds, it may take 15 seconds or
more before a WebException is thrown
to indicate a timeout on your request.
Is it possible that your DNS query is the cause of the timeout?
No matter what we tried we couldn't manage to get the timeout below 21 seconds when the server we were checking was down.
To work around this we combined a TcpClient check to see if the domain was alive followed by a separate check to see if the URL was active
public static bool IsUrlAlive(string aUrl, int aTimeoutSeconds)
{
try
{
//check the domain first
if (IsDomainAlive(new Uri(aUrl).Host, aTimeoutSeconds))
{
//only now check the url itself
var request = System.Net.WebRequest.Create(aUrl);
request.Method = "HEAD";
request.Timeout = aTimeoutSeconds * 1000;
var response = (HttpWebResponse)request.GetResponse();
return response.StatusCode == HttpStatusCode.OK;
}
}
catch
{
}
return false;
}
private static bool IsDomainAlive(string aDomain, int aTimeoutSeconds)
{
try
{
using (TcpClient client = new TcpClient())
{
var result = client.BeginConnect(aDomain, 80, null, null);
var success = result.AsyncWaitHandle.WaitOne(TimeSpan.FromSeconds(aTimeoutSeconds));
if (!success)
{
return false;
}
// we have connected
client.EndConnect(result);
return true;
}
}
catch
{
}
return false;
}