How to prevent DNS lookup when fetching by using HttpClient - c#

I am not sure i am doing correctly or not
Would below way prevent DNS lookup when keep-alive is set false?
The host is : tatoeba.org
The url is : http://188.213.24.161/eng/sentences/show/1
Here screenshots
the url is given as above
the host is set as below

I believe that if you specify your host as an ip address (as you did), then .net will skip the dsn look up (regardless of the keep alive or the host header setting).
If you dig a little bit into HttpClient you will see it basically uses HttpWebRequest for making the requests. https://github.com/dotnet/corefx/blob/master/src/System.Net.Http/src/System/Net/Http/HttpClient.cs
HttpWebRequest eventually uses a class called ServicePoint which call a
Dns.TryInternalResolve.
Dns.TryInternalResolve doesn't try to resolve IPAddresses.
For more info refer to:
https://referencesource.microsoft.com/#System/net/System/Net/DNS.cs,f8023b9c19212708
I also tried to verify that by running the following lines and monitor the requests using netmon
using (HttpClient c = new HttpClient())
{
var response = c.GetAsync(url).Result;
}
I saw that indeed for a url that contains an host name .net issue a dns request while for requests with an ipAddress as an host name there is no dns request.

Related

Call a URL without DNS Entry from Azure Function .Net [duplicate]

Basically, I need to be able to make an HTTP Request to a Website on the same machine I am on, without modifying the host file to create a pointer to the domain name.
For example.
I am running the code on one website, let's say www.bobsoft.com which is on a server.
I need to make an HTTP request to www.tedsoft.com which is on the same server.
How can I make a call using a C# HttpClient without modifying the host file? Take into account that the websites are routed by bindings in IIS. I do know the domain I am going to use ahead of time, I just have to make it all internal in the code without server changes.
Thanks!
IIS bindings on the same port but different hostnames are routed based on the http Host header. The best solution here is really to configure local DNS so requests made to www.tedsoft.com don't leave the machine. That being said, if these kinds of configuration aren't an option you can easily set the host header as a part of your HttpRequestMessage.
I have 2 test sites configured on IIS.
Default Web Site - returns text "test1"
Default Web Site 2 - returns text "test2"
The following code uses http://127.0.0.1 (http://localhost also works) and sets the host header appropriately based on the IIS bindings to get the result you're looking for.
class Program
{
static HttpClient httpClient = new HttpClient();
static void Main(string[] args)
{
string test1 = GetContentFromHost("test1"); // gets content from Default Web Site - "test1"
string test2 = GetContentFromHost("test2"); // gets content from Default Web Site 2 - "test2"
}
static string GetContentFromHost(string host)
{
HttpRequestMessage msg = new HttpRequestMessage(HttpMethod.Get, "http://127.0.0.1");
msg.Headers.Add("Host", host);
return httpClient.SendAsync(msg).Result.Content.ReadAsStringAsync().Result;
}
}

C# SoapHttpClientProtocol - set local binding address (source address)

I am trying to make a Soap call to a webservice hosted by business partner. The problem I am having is that our client is hosted on a virtual PC with multiple IPs. I would like to call the hosted webservice with a different (not default) IP address. Why is that? There is VPN connection between our client application and hosted webservice but it is set up LAN2LAN between two IPs. Now I have to change local source IP address to match with the VPN requirements.
I have tried using SoapHttpClientProtocol's WebProxy, setting its value to the new IP but it does not seem to work. I am getting HTTP 404 error.
Here is some of the code:
//soapApiClient is typeof SoapHttpClientProtocol
//endpoint url -> webservice, url from appSettings
var url = SettingsProvider.ClientSapGetUserDataUrl;
soapApiClient.Url = url;
//proxy settings -> setting new IP, defined in appSettings
var proxy = SettingsProvider.ClientProxyAddress;
soapApiClient.Proxy = new WebProxy(proxy);
//credentials
soapApiClient.Credentials = GetCredentials();
I HTTP post user code and return value should be user data from SAP, currently I am getting 404 http error code. For comparison, it works like a charm from SoapUI. Maybe setting proxy is not what I am looking for? Any help would be much appreciated.
EDIT: To be more clear
Currently, by default service is called from default ip 91.185.201.88. Service's IP is 10.67.145.70. I want to change it like so: service is called from 192.168.4.2 to service's 10.67.145.70.
I finally managed to make it work. If anyone else will have problem with this, just follow these instructions.
Before calling remote service, you have to find service point for it. This is done via ServicePointManager like so
Example:
var servicePointUserReader = ServicePointManager.FindServicePoint(new Uri(FULLY QUALIFIED REMOTE SERVICE URL));
servicePointUserReader.BindIPEndPointDelegate = (sp, remote, retryCount) => new IPEndPoint(IPAddress.Parse(SOURCE IP FROM WHICH YOU WANT TO SEND REQUEST), 0);
In my example, fully qualified remote service url was something like http://65.145.63.71:8010/sapService (this is the one we are trying to call). Then I wanted to make a call from a different IP on our virtual PC which has many IPs addressed to it. You just need to input desired IP as shown in the second line of the code, for example 192.168.5.1.
Make sure you use "http://" when calling FindServicePoint with new Uri() constrcutor otherwise it will not work!
After that just proceed with the call to the service. Two lines of code, that's all you need. :)

Is it possible to spoof a hosts file entry within a web application?

I have a rest application sitting on an IIS server (on the public internet) with a configuration as follows:
IP Address: A.B.C.D
Host Header: something-not-public-dns.example.com
(The entry isn't in the public dns simply because it doesn't have to be. This isn't merely security by obscuring host headers.)
In order to get my client to connect to the application, I simply add an entry in my hosts file:
A.B.C.D something-not-public-dns.example.com
With the hosts file entry everything works great.
The problem is one of my clients is sitting inside of an ASP.NET web application, and I cannot modify the hosts file for the web server the application resides on.
Thus my question- is it possible to spoof a hosts file entry inside of a web application?
I was thinking of something like this:
string url = "something-not-public.example.com/myservice/mymethod";
var client = new WebClient();
client.Headers[HttpRequestHeader.ContentType] = "application/json";
// Set the destination IP for this request here!
string json = new System.Web.Script.Serialization.JavaScriptSerializer().Serialize(MyObject);
string response = client.UploadString(url, json);
I would consider the above example request level spoofing, which would be great, but I'm also curious to know if application level spoofing could be done as well. This way the application could set up the host entries and all requests made from within that application would use them.
You're going about this the wrong way around - instead of trying to setup a mapping from the domain name to the IP address you should simply make the request to the IP and explicitly override the Host header of your HTTP request with the domain name you want to use:
string url = "A.B.C.D/myservice/mymethod";
var client = new WebClient();
client.Headers[HttpRequestHeader.ContentType] = "application/json";
client.Headers[HttpRequestHeader.Host] = "something-not-public.example.com";

Inspect headers set by WebClient proxy

I'm executing request through some free proxy servers, and I would like to know what headers each proxy server sets. Right now I'm visiting a page that prints out the result in the html body.
using(WebClient client = new WebClient())
{
WebProxy wp = new WebProxy("proxy url");
client.Proxy = wp;
string str = client
.DownloadString("http://www.pagethatprintsrequestheaders.com");
}
The WebClient doesn't show the modified headers, but the page prints the correct ones. Is there any way to find out what headers that are being set by the proxy without visiting a page that prints them like in my example? Do I have to create my own http listener?
When the proxy server sets its own headers, it is essentially performing its own web request. It can even hide or override some of the headers that you set using your WebProxy.
Consequently, only the target page (pagethatprintsrequestheaders.com) can reliably see the headers being set by the proxy. There is no guarantee that the proxy server will send back the headers that it had sent to the target, back to you.
To put it another way, it really depends on the proxy server implementation. if the proxy server you are using is based on Apache's ProxyPass, you'd probably see the headers being set! If it's a custom implementation, then you may not see it.
You can first try inspecting the client.ResponseHeaders property of the WebClient after your response comes back. If this does not contain headers matching what (pagethatprintsrequestheaders.com) reports, then it's indeed a custom or modified implementation.
You could then create your own proxy servers, but this is more involved. You would probably spin up an EC2 instance, install Squid/TinyProxy/YourCustomProxy on it and use that in your WebProxy call.
You may also want to modify your question and explain why you want to read the headers. There may be solutions to your overall goal that don't require reading headers at all but could be done in some other way.
It looks like your sending a request from your WebClient, through the proxy and its received by the host at www.pagethatprintsrequestheaders.com.
If the proxy is adding headers to the request, your webclient will never see them on it's request.
webclient proxys request
request with headers added
client -----------> proxy ----------------------> destination host
The webclient can only see the state of the request between it and the proxy. The proxy will create a new request to send to the destination host, and its that request to which the headers are added. It also that request that is received by the destination host (which is why when it echoes back the headers it can see those added by the proxy)
When the response comes back, the headers are set by the host. It's possible that the proxy will add some headers to the response, but even if it did, they are not likely to be the same headers it adds to a request.
response response
(forwarded by proxy) (headers set by host)
client <------------------- proxy <------------------------- destination host
Using a host that echo the headers back as part of the response payload is one option.
Another would be to use something between the proxy and the destination host to inspect the request there (e.g a packet sniffer or another proxy like Fiddler that lets you see the request headers).
If the proxy is outside of you network, getting between the proxy and the destination host will be difficult (unless the host is under your control).

Use fiddler to monitor C# requests sent through proxy

Issue:
Consider the following working code.
System.Net.WebProxy proxy = new System.Net.WebProxy(proxyServer[i]);
System.Net.HttpWebRequest objRequest = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(https_url);
objRequest.Method = "GET";
objRequest.Proxy = proxy;
First, notice that proxyServer is an array so each request may use a different proxy.
If I comment out the last line thereby removing the use of any proxies, I can monitor requests in Fiddler just fine, but once I reinstate the line and start using them, Fiddler stops logging outbound requests from my app.
Question:
Do I need to configure something in Fiddler to see the requests or is there a change in .Net I can make?
Notes:
.Net 4.0
requests are sometimes https, but i don't think this is directly relevant to issue
all requests are outbound (not localhost/127.0.0.1)
Fiddler is a proxy itself. By assigning a different proxy to your request.. you're essentially taking Fiddler out of the equation.
If you're looking to capture traffic and use your own proxy.. you can't use a proxy (by definition that makes no sense).. you want a network analyzer, such as WireShark. This captures the traffic instead of having the traffic routed through it (as a proxy does), allowing you to have it monitor traffic and route your requests through your custom proxy.

Categories