How to make Httpclient fail faster when connecting to an invalid endpoint? - c#

We notice that the .NET HttpClient takes atleast ~20 seconds to fail while trying to connect to an endpoint that is invalid. I was wondering if there is a way to make it fail faster, because our back-end services could be moving to different nodes based on criteria such as load etc. We have a mechanism to retrieve the new location of these services, if there is a communication failure to a particular service. But, since the HttpClient failure time is ~20s, as a result response time for the client request is degrading.
I was looking at the documentation and notice that the HttpClient timeout may take atleast 15s due to the DNS resolution. There was similar post about this here,
HttpClient - how to figure out if server is down faster?
But, all our endpoints have IP addresses in their host that should be preventing the DNS look-up according to this post here,
How to prevent DNS lookup when fetching by using HttpClient
internal static bool TryInternalResolve(string hostName, out IPHostEntry result)
{
if (Logging.On)
Logging.Enter(Logging.Sockets, "DNS", "TryInternalResolve", hostName);
if (string.IsNullOrEmpty(hostName) || hostName.Length > MaxHostName)
{
result = null;
return false;
}
IPAddress address;
if (IPAddress.TryParse(hostName, out address))
{
GlobalLog.Print("Dns::InternalResolveFast() returned address:" + address.ToString());
result = GetUnresolveAnswer(address); // Do not do reverse lookups on IPAddresses.
return true;
}
}
https://referencesource.microsoft.com/#System/net/System/Net/DNS.cs,e177fd4099d6a754,references
We are still trying to capture this behavior in a local environment with fiddler or netmon.
Where could the httpclient be spending bulk of its time to fail?
Are there any master settings that can prevent this time delay on failure?

Related

Does RestSharp perform WINS lookup?

I am trying to connect to an API deployed on a Windows server that does not belong to a domain.
When I initialize my RestClient this way my requests fail. If I provide my machine's IP address, my request works properly.
var options = new RestClientOptions("https://MyMachine/") {
Timeout = 1000
};
var client = new RestClient(options);
My guess is, RestSharp only performs DNS lookup and not WINS lookup. Is my guess right?
Thank you very much for your input, it has been helpful to understand that I was on the wrong track.
I solved my issue by adding the following line:
ServicePointManager.EnableDnsRoundRobin = true;

Set Custom Host Name as part of changing the Endpoint Address of a Soap Client

I am presently in a scenario where I need to have multiple servers that live behind a Load Balancer talk to each other directly, and I need to communicate with specific servers for PUSH notifications. This is for a chat tool that requires users that have been moved to different servers by a load balancer to still be able to talk to one another live.
The actual pushes are being handled with Signal-R, and I have all of that working. So here is the actual complication:
Normally, this would be simple enough to do by targeting them via IP Address to bypass the load balancer. However, this is complicated because the servers expects a specific Host name or it will reject the request.
I know it is possible to do this with a WebRequest, but would like to avoid having to build a proxy if I can help it.
Here is the piece I have where I'm trying to send a global push to tell everyone across all connected servers to update their buddy lists because someone logged in or out.
private void NotifyUsersChangedGlobal()
{
List<string> addressesToNotify = ChatUsers.Select(x => x.ServerIP).Distinct().ToList();
foreach (string address in addressesToNotify)
{
ChatUplinkSoapClient client = BuildClient(address);
client.NotifyUsersChanged();
}
}
And this is the Client Builder where (I assume) I need to handle assigning the custom Host name to ride on top of the IP Address
private ChatUplinkSoapClient BuildClient(string endpointIP)
{
string relativeUrl = "/WebServices/ChatUplink.asmx";
//Turn on HTTPS
HttpBindingBase binding = new BasicHttpBinding(BasicHttpSecurityMode.Transport);
//Link together the IP Address and the asmx route
EndpointAddress endpoint = new EndpointAddress(endpointIP + relativeUrl);
//Make the Client
ChatUplinkSoapClient client = new ChatUplinkSoapClient(binding, endpoint);
//Need to set Host header to "HostHeaderName", or the server will reject the request.
return client;
}
Ideas?

How do I decrease DNS refresh timeout successfully with HttpClient in Xamarin Forms?

I'm working on a fairly complex Xamarin.Forms application. We make a lot of REST requests. It's been reported that our application isn't respecting DNS failover for load balancing in a timely manner, so I started investigating. I'm running dnsmasq so I can look at when the app makes DNS requests. The code is currently using HttpWebRequest, and I noticed it's making DNS queries at least 10 minutes apart.
I understand part of this is because most .NET networking bits use a keepalive connection. I certainly see a higher rate of DNS queries if I force the headers to not use keepalive, but that adds network overhead so it's not a desirable solution. But I didn't initially see a clear way to control how HttpWebRequest makes DNS queries.
It looked promising that I could get its ServicePoint property and set the ConnectionLeaseTimeout on that. Unfortunately, that throws NotImplementedException in Xamarin so it's not going to be part of any solution.
I thought that perhaps HttpClient would be more configurable. I see a lot of discussion about how to use it properly, and that if you do it that way you need to set ServicePointManager.DnsRefreshTimeout to a smaller value for use cases where you want to expect DNS to update frequently. But this is usually done in conjunction with getting the ServicePoint for the deisred endpoint and also modifying ConnectionLeaseTimeout, which isn't possible.
I've been testing with a very simple app that reuses an HttpClient and makes the same request any time I push a button. Slap this ViewModel behind some Xaml with a button:
using System;
using Xamarin.Forms;
using System.Net.Http;
using System.Net;
namespace TestDns {
public class MainPageViewModel {
private const string _URL = "http://www.example.com";
private HttpClient _httpClient;
private ServicePoint _sp;
public MainPageViewModel() {
var sp = ServicePointManager.FindServicePoint(new Uri(_URL));
_sp = sp;
//_sp.ConnectionLeaseTimeout = 100; // throws NIE
_httpClient = new HttpClient();
ServicePointManager.DnsRefreshTimeout = 100;
}
public Command WhenButtonIsClicked {
get {
return new Command(() => SendRequest());
}
}
private async void SendRequest() {
Console.WriteLine($"{_sp.CurrentConnections}");
var url = "http://www.example.com";
var response = await _httpClient.GetAsync(url);
Console.WriteLine($"{response.Content}");
}
}
}
I didn't expect ConnectionLeaseTimeout to throw. I expected this code to only cache DNS requests for 100ms, I was going to choose a more reasonable timeframe like 2-3 minutes in more production-oriented tests. But since I can't get this simple example to function like I want, it seems moot to increase the delays.
Surely someone else has had this problem in a Xamarin app? Is my only solution going to be to look deeper and try to use native networking constructs?
If you're doing this on Android, DNS is cached for 10 minutes, and I don't believe you have any access to the expiration/refresh time from inside of your app. There are a number of ways to force a refresh but they all involve user actions like going into Network Connections and flipping from Static to DHCP and back, etc.
The only way I can think of to be sure of getting a fresh DNS lookup from inside your app is to have 10+ minutes worth of DNS entries that all alias to your server, and cycle your app through them, so every time you ask for a DNS lookup, it's a new name and not in the cache.
For example, look for 1.server.example.com 2.server.example.com, etc. Each new name will force a new lookup and won't be pulled from the cache because it's not there.
It seems that Java has decided the solution to "some people implement DNS improperly and ignore TTL" is to make the problem worse by ensuring devices that use Java implement DNS improperly. There is a single TTL used for all DNS entries in the cache. There's some philosophical debate and what led me to the answer in this question which I adapted for the answer.
In terms of Xamarin projects, add this somewhere (I chose early in MainActivity):
Java.Security.Security.SetProperty("networkaddress.cache.ttl", "<integer seconds>");
Replace "<integer seconds>" with your desired TTL for all DNS entries. Be aware lower values might mean you make more DNS queries than you used to, if you're seriously trying to save networking bytes this could be an issue.
I'm leaving Terry Carmen's answer selected as "the answer".

Why does Request["host"] == "dev.testhost.com:1234" whereas Request.Url.Host == "localhost"

Hi all, I seem to have found a discrepancy when testing ASP.NET applications locally on the built-in web server with Visual Studio 2008 (Cassini).
I've set up a host on my local machine associating dev.testhost.com with 127.0.0.1, since I have an application that needs to change its appearance depending on the host header used to call it.
However, when I request my test application using http://dev.testhost.com:1234/index.aspx, the value of Request.Url.Host is always "localhost". Whereas the value of Request.Headers["host"] is "dev.testhost.com:1234" (as I would expect them both to be).
I'm NOT concerned that the second value includes the port number, but I am mighty confused as to why the HOST NAMES are completely different! Does anyone know if this is a known issue, or by design? Or am I being an idiot?!
I'd rather use Request.Url.Host, since that avoids having to strip out the port number when testing... - Removed due to possibly causing confusion! - Sam
Request.Headers["host"] is the value received from the application that connects to the server, while the other value is the one the server gets when it tries to get the domain name.
The browser uses in the request the domain name entered because that is used in the case of virtual domains. The server reports the one set in the server preferences, or the first one it finds.
EDIT: Looking at the code of Cassini to see if it uses some particular settings, I noticed the following code:
public string RootUrl {
get {
if (_port != 80) {
return "http://localhost:" + _port + _virtualPath;
}
else {
return "http://localhost" + _virtualPath;
}
}
}
//
// Socket listening
//
public void Start() {
try {
_socket = CreateSocketBindAndListen(AddressFamily.InterNetwork, IPAddress.Loopback, _port);
}
catch {
_socket = CreateSocketBindAndListen(AddressFamily.InterNetworkV6, IPAddress.IPv6Loopback, _port);
}
// …
}
The explanation seems to be that Cassini makes explicit reference to localhost, and doesn't try to make a reverse DNS lookup. Differently, it would not use return "http://localhost" + _virtualPath;.
The Request.Headers["host"] is the host as specified in the http header from the browser. (e.g. this is what you'd see if you examined the traffic with Fiddler or HttpWatch)
However, ASP.NET loasds this (and other request info) into a System.Uri instance, which parses the request string into its constituent parts. In this case, "Host" refers to literally the host machine part of the original request (e.g. with the tcp port being in the Port) property.
This System.Uri class is a very useful helper class that takes all the pain out of splitting your request into it's parts, whereas the "Host:" (and for that matter the "GET") from the http header are just raw request data.
Although they both have the same name, they are not meant to be the same thing.
It's a matter of what the w3 specs say versus what the Microsoft Uri.Host property is supposed to contain. The naming does not imply an attempt by MS to provide identical functionality. The function that does include port numbers is Uri.Authority.
With the update you posted, you're still facing the same problem, just examining a different aspect of it. The Uri.Host property is not explicity or implicity stated to perform the same function as the headers that are defined in the w3 specs. In long form, here are some quotes from the Uri.Host MSDN page:
Uri.Host Property
Gets the host component of this instance.
Property Value
Type: System.String
A String that contains the host name. This is usually the DNS host name or IP address of the server.
There's no guarantee that this will match what is in the headers, just that it represents the host name in some form.

Get external IP address over remoting in C#

I need to find out the external IP of the computer a C# application is running on.
In the application I have a connection (via .NET remoting) to a server. Is there a good way to get the address of the client on the server side?
(I have edited the question, to be a little more clear. I'm apologize to all kind people who did their best to respond to the question, when I perhaps was a little too vague)
Solution:
I found a way that worked great for me. By implementing a custom IServerChannelSinkProvider and IServerChannelSink where I have access to CommonTransportKeys.IPAddress, it's easy to add the client ip on the CallContext.
public ServerProcessing ProcessMessage(IServerChannelSinkStack sinkStack,
IMessage requestmessage, ITransportHeaders requestHeaders,
System.IO.Stream requestStream, out IMessage responseMessage,
out ITransportHeaders responseHeaders, out System.IO.Stream responseStream)
{
try
{
// Get the IP address and add it to the call context.
IPAddress ipAddr = (IPAddress)requestHeaders[CommonTransportKeys.IPAddress];
CallContext.SetData("ClientIP", ipAddr);
}
catch (Exception)
{
}
sinkStack.Push(this, null);
ServerProcessing srvProc = _NextSink.ProcessMessage(sinkStack, requestmessage, requestHeaders,
requestStream, out responseMessage, out responseHeaders, out responseStream);
return srvProc;
}
And then later (when I get a request from a client) just get the IP from the CallContext like this.
public string GetClientIP()
{
// Get the client IP from the call context.
object data = CallContext.GetData("ClientIP");
// If the data is null or not a string, then return an empty string.
if (data == null || !(data is IPAddress))
return string.Empty;
// Return the data as a string.
return ((IPAddress)data).ToString();
}
I can now send the IP back to the client.
This is one of those questions where you have to look deeper and maybe rethink the original problem; in this case, "Why do you need an external IP address?"
The issue is that the computer may not have an external IP address. For example, my laptop has an internal IP address (192.168.x.y) assigned by the router. The router itself has an internal IP address, but its "external" IP address is also internal. It's only used to communicate with the DSL modem, which actually has the external, internet-facing IP address.
So the real question becomes, "How do I get the Internet-facing IP address of a device 2 hops away?" And the answer is generally, you don't; at least not without using a service such as whatismyip.com that you have already dismissed, or doing a really massive hack involving hardcoding the DSL modem password into your application and querying the DSL modem and screen-scraping the admin page (and God help you if the modem is ever replaced).
EDIT: Now to apply this towards the refactored question, "How do I get the IP address of my client from a server .NET component?" Like whatismyip.com, the best the server will be able to do is give you the IP address of your internet-facing device, which is unlikely to be the actual IP address of the computer running the application. Going back to my laptop, if my Internet-facing IP was 75.75.75.75 and the LAN IP was 192.168.0.112, the server would only be able to see the 75.75.75.75 IP address. That will get it as far as my DSL modem. If your server wanted to make a separate connection back to my laptop, I would first need to configure the DSL modem and any routers inbetween it and my laptop to recognize incoming connections from your server and route them appropriately. There's a few ways to do this, but it's outside the scope of this topic.
If you are in fact trying to make a connection out from the server back to the client, rethink your design because you are delving into WTF territory (or at least, making your application that much harder to deploy).
Dns.GetHostEntry(Dns.GetHostName()); will return an array of IP addresses. The first one should be the external IP, the rest will be the ones behind NAT.
So:
IPHostEntry IPHost = Dns.GetHostEntry(Dns.GetHostName());
string externalIP = IPHost.AddressList[0].ToString();
EDIT:
There are reports that this does not work for some people. It does for me, but perhaps depending on your network configuration, it may not work.
I found a way that worked great for me. By implementing a custom IServerChannelSinkProvider and IServerChannelSink where I have access to CommonTransportKeys.IPAddress, it's easy to add the client ip on the CallContext.
public ServerProcessing ProcessMessage(IServerChannelSinkStack sinkStack,
IMessage requestmessage, ITransportHeaders requestHeaders,
System.IO.Stream requestStream, out IMessage responseMessage,
out ITransportHeaders responseHeaders, out System.IO.Stream responseStream)
{
try
{
// Get the IP address and add it to the call context.
IPAddress ipAddr = (IPAddress)requestHeaders[CommonTransportKeys.IPAddress];
CallContext.SetData("ClientIP", ipAddr);
}
catch (Exception)
{
}
sinkStack.Push(this, null);
ServerProcessing srvProc = _NextSink.ProcessMessage(sinkStack, requestmessage, requestHeaders,
requestStream, out responseMessage, out responseHeaders, out responseStream);
return srvProc;
}
And then later (when I get a request from a client) just get the IP from the CallContext like this.
public string GetClientIP()
{
// Get the client IP from the call context.
object data = CallContext.GetData("ClientIP");
// If the data is null or not a string, then return an empty string.
if (data == null || !(data is IPAddress))
return string.Empty;
// Return the data as a string.
return ((IPAddress)data).ToString();
}
I can now send the IP back to the client.
Better to just use http://www.whatismyip.com/automation/n09230945.asp it only outputs the IP just for the automated lookups.If you want something that does not rely on someone else put up your own page http://www.unkwndesign.com/ip.php is just a quick script:
<?php
echo 'Your Public IP is: ' . $_SERVER['REMOTE_ADDR'];
?>
The only downside here is that it will only retrieve the external IP of the interface that was used to create the request.
Jonathan Holland's answer is fundamentally correct, but it's worth adding that the API calls behind Dns.GetHostByName are fairly time consuming and it's a good idea to cache the results so that the code only has to be called once.
The main issue is the public IP address is not necessarily correlated to the local computer running the application. It is translated from the internal network through the firewall. To truly obtain the public IP without interrogating the local network is to make a request to an internet page and return the result. If you do not want to use a publicly available WhatIsMyIP.com type site you can easily create one and host it yourself - preferably as a webservice so you can make a simple soap compliant call to it from within your application. You wouldn't necessarily do a screen capture as much as a behind the scenes post and read the response.
If you just want the IP that's bound to the adapter, you can use WMI and the Win32_NetworkAdapterConfiguration class.
http://msdn.microsoft.com/en-us/library/aa394217(VS.85).aspx
Patrik's solution works for me!
I made one important change. In process message I set the CallContext using this code:
// try to set the call context
LogicalCallContext lcc = (LogicalCallContext)requestMessage.Properties["__CallContext"];
if (lcc != null)
{
lcc.SetData("ClientIP", ipAddr);
}
This places the ip address in the correct CallContext, so it can later be retrieved with
GetClientIP().
Well, assuming you have a System.Net.Sockets.TcpClient connected to your client, you can (on the server) use client.Client.RemoteEndPoint. This will give you a System.Net.EndPoint pointing to the client; that should contain an instance of the System.Net.IPEndPoint subclass, though I'm not sure about the conditions for that. After casting to that, you can check it's Address property to get the client's address.
In short, we have
using (System.Net.Sockets.TcpClient client = whatever) {
System.Net.EndPoint ep = client.Client.RemoteEndPoint;
System.Net.IPEndPoint ip = (System.Net.IPEndPoint)ep;
DoSomethingWith(ip.Address);
}
Good luck.
I believe theoretically you are unable to do such a thing while being behind a router (e.g. using invalid ip ranges) without using an external "help".
You can basically parse the page returned by doing a WebRequest of http://whatismyipaddress.com
http://www.dreamincode.net/forums/showtopic24692.htm
The most reliable manner of doing this is checking a site like http://checkip.dyndns.org/ or similar, because until you actually go external to your network, you cannot find your external IP. However, hardcoding such a URL is asking for eventual failure. You may wish to only perform this check if the current IP looks like an RFC1918 private address (192.168.x.x being the most familiar of these.
Failing that, you can implement your own, similar, service sitting external to the firewall, so you will at least know if it's broken.

Categories