I am using WebClient to get data from a web resource thus:
var wc = new System.Net.WebClient();
var stream = wc.OpenRead("http://...");
// etc..
It used to work until recently, when the server would forcibly close the connection.
Based on this StackOverflow answer, I added the ServicePointManager setting thus:
var wc = new System.Net.WebClient();
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
var stream = wc.OpenRead("http://...");
// etc..
… and it once again works as it did before.
I get that TLS is relevant to HTTPS and the effect of the setting is to include TLS1.2 in the handshake, and that the host site must have been recently updated to reject the older vulnerable protocol, but why is it necessary for ordinary, non-secure HTTP?
AllowAutoRedirect is set to true by default in WebClient instances.
Therefore, the request automatically follows redirection responses from your server.
Simply set AllowAutoRedirect to false and you will not follow any redirection, so you will not have to deal with SSL/TLS handshakes.
Of course, if your server does not want to serve your request with HTTP, you will not get the content you are looking for. Anyway, setting AllowAutoRedirect to false will help you confirm that the behaviour you encounter is due to a redirect.
Related
First of all, let me say that I am not used to the HttpClient API, REST protocol nor Windows Authentication. It is possible I want to do something definitely impossible.
From a WPF application configured like this (don't know if it is relevent):
AppDomain.CurrentDomain.SetPrincipalPolicy(PrincipalPolicy.WindowsPrincipal);
I am writing a call to a REST Web Service through HttpClient. I want to provide to the WS the currently logged in user credentials in order to check on the WS side if he can actually call it.
I have read plenty of examples on the internet and I though I just had to provide to the HttpClient a HttpClientHandler configured like this to do so:
var handler = new HttpClientHandler
{
UseDefaultCredentials = true
};
_client = new HttpClient(handler, true);
var response = await _client.GetAsync(concreteUrl);
I also tried a lots of combinations with some other properties I found on the internet:
PreAuthenticate = true
Credentials = CredentialCache.DefaultCredentials
But no matter the combinations tried to configure the handler, with fiddler, when I look at the request, the header is desperately empty:
GET http://my.domain/url/ HTTP/1.1
Host: my.domain
Connection: Keep-Alive
In the Auth tab:
No Proxy-Authorization Header is present.
No Authorization Header is present.
The WS is being developed in PHP by the way, and the client is running under .NET 4.5.2.
Is this even possible? Otherwise, how could I authenticate the user on the WS through HttpClient requests?
As far as I can tell, there seems to be a big limitation in .NET in that there is no way using C# and .NET to make an TLS connection that uses Server Name Indication (SNI). Have I missed something or is my understanding correct?
Does anybody know if and how I could make an SNI connection using OpenSSL.NET, libcurl.NET or some other 3rd party .NET library? Some sample code would be very much appreciated.
In my .Net 4.5 project the following fails for a server using SNI:
var url = "https://www.somesite.com";
System.Net.WebClient client = new System.Net.WebClient();
client.Encoding = Encoding.UTF8;
var data = client.DownloadString(url);
But it works if explicitly specifying TLS1.2 by prefixing it with:
System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
The same applies to webrequest:
WebRequest request = WebRequest.Create("https://www.somesite.com");
and HttpRequestMessage:
var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, "https://www.google.com");
They all need the protocol explicitly set to TLS 1.2 to work with an SNI server (this may have changed in newer .Net versions)
This is a fairly old post but still this answer might help some people, at least it cost me some days.
.NET Framework does support the Server Name Indication by default. (Tested on 4.5.1 but I guess it's same at least for .NET 4.5+)
A short example:
HttpResponseMessage response;
var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, "https://www.google.com");
var handler = new HttpClientHandler
{
CookieContainer = new CookieContainer()
};
using (var client = new HttpClient(handler))
{
response = client.SendAsync(httpRequestMessage).Result;
}
This is a very standard way to create a GET request within C#. You will see, this example does run using, in my case, TLS 1.2 with SNI. If you use Wireshark to see the actual packages which are sent, you will see a Client Hello with the Server Name Indication set to www.google.com.
An issue we ran into: The SNI tag is set by the .NET Framework (or Schannel by Windows, not sure) based on the URL passed in the constructor of HttpRequestMessage. If you know initialize the request based on some URL (for example https://www.google.com) and later on you switch the RequestUri OR the Host header, the SNI tag will still be created based on the original url URL. This might be the case for example if you pass through a request and you remap all original headers to the newly created request.
I am able to fix a problem with a client where they cannot authenticate through a proxy doing the following:
var proxy = WebRequest.GetSystemWebProxy();
proxy.Credentials = CredentialCache.DefaultNetworkCredentials;
service.Proxy = proxy;
This works fine for Windows XP, however on Windows 7 I get a 407 (proxy not authenticated exception). Does anybody know what the difference is, and more importantly, what I need to do to get this to work on both OS?
UPDATE
I am having the users check the following:
In the registry editor, can you go to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon and let me know what the value is for CachedLogonsCount. f
In the Start box, type in Group Policy and an option to Edit Group Policy should pop up, click on it. Then go to Computer Configuration\Administrative Templates\System\User Profiles\Delete cached copies of roaming profiles and let me know if it is configured, and if so, to what is it set?
UPDATE FOR BOUNTY
So, I added the bounty. I can take a solution from here, or just an alternate means to getting through a proxy on Windows 7...
Another Update
I am not sure if this is useful or not, but we are also doing the following:
service.PreAuthenticate = true;
service.Url = "myurl";
service.Credentials = new NetworkCredential(txt_UserName.Text, txt_Password.Text);
My temporary solution
This is not really a solution, but works for now. I am using the app.config and setting the proxy to be default, with a ByPassList so that the proxy is not even used. This is only doable since the proxy does not have a strong firewall currently. For other clients, I need to get the above to work
This piece of code works for me on XP, Win7 and 2008
var webProxy = new WebProxy(WebRequest.DefaultWebProxy.GetProxy(new Uri({TheURLoftheService})));
webProxy.Credentials = CredentialCache.DefaultCredentials;
webProxy.UseDefaultCredentials = true;
service.Proxy = webProxy;
actually looks like they "fixed" it in Win7 :) Can you confirm that both client and server are specifying http 1.1
Now let's discuss as to why the browser works in this scenario. IE
uses WinINet under the hood rather than WinHTTP. If we look at the
network traces we see that IE sends HTTP/1.1, but the proxy replies
with HTTP/1.0. IE still accepts this behavior, because in the internet
scenario there are countless number of clients and servers which still
use HTTP/1.0.
WinHTTP strictly requires HTTP/1.1 compliance for keeping the
connection alive and HTTP Keep-Alives are not supported in HTTP/1.0
protocol. HTTP Keep-Alive feature was introduced in the HTTP/1.1
protocol as per RFC 2616. The server or the proxy which expects the
keep-alive should also implement the protocol correctly. WinHTTP on
Windows 7, Windows 2008 R2 are strict in terms of security wrto
protocol compliance. The ideal solution is to change the server/proxy
to use the right protocol and be RFC compliant.
http://blogs.msdn.com/b/httpcontext/archive/2012/02/21/changes-in-winhttp-on-windows-7-and-onwards-wrto-http-1-0.aspx
Will this work?
I am using this to set proxy, so far we did not encounter an error on all windows platform
Uri address = new Uri("http://your-webservice-address");
//Get User current network credential
ICredentials credentials = CredentialCache.DefaultCredentials;
NetworkCredential credential = credentials.GetCredential(address, "Basic");
//Get HttpWebRequest
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
//Network Credential should be included on the request to avoid network issues when requesting to the web servic
request.Proxy = WebRequest.DefaultWebProxy;
request.Credentials = new NetworkCredential(credential.UserName, credential.Password, credential.Domain);
It's hard to say based on the code you've given. I'd suspect that it's either your IE settings or your proxy variables.
Check http://social.msdn.microsoft.com/Forums/en/netfxnetcom/thread/61b71194-1758-4c7b-89fe-91be7363db13 it may help.
I've been struggling to get a WCF client to work through a web proxy. If I manually specify the proxy as below, I can get the http request to work.
WebProxy proxy = new WebProxy("http://x.x.x.x:3128", false);
proxy.Credentials = new NetworkCredential("user", "pass");
WebRequest.DefaultWebProxy = proxy;
However I have the client service proxy set to use ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.ChainTrust;
Inspecting the packets that get sent out, the client successfully connects to the server, but the then tries to validate the certificate that the service provides. This validation attempt fails because the requests for the chain do not have Proxy-Authorization headers (they fail with 407 errors). How can I get these requests to properly use the DefaultWebProxy that's specified?
If I set the validation mode to None, then it all works of course, but I really don't want to have to do that.
this setting only affects message level certificates. for transport level try something like this http://webservices20.blogspot.co.il/2008/12/wcf-gotcha-disabling-ssl-validation.html
I'm trying to get HTTP calls I'm making from C# .NET to a local address (localhost:3000) to use the proxy I set (so I can go through fiddler). Using the below WebProxy approach works if I point the target URL to a non-local address, however I need to point it to a local web-server I have (at localhost:3000), and when I do this the request is not going through the proxy.
I have inlcuded the "proxyObject.BypassProxyOnLocal = false". This should make it work no? Any suggestions re how to force the request to go through the WebProxy for http calls targetting a local address?
WebProxy proxyObject = new WebProxy("http://localhost:8888/", false);
proxyObject.Credentials = new NetworkCredential();
proxyObject.BypassProxyOnLocal = false;
WebRequest.DefaultWebProxy = proxyObject;
var request = (HttpWebRequest)WebRequest.Create(targetUri);
// I also included this line as a double check
request.Proxy = proxyObject;
Subsequent calls do not go through the proxy however, such as when I do:
var res = (HttpWebResponse)req.GetResponse();
thanks
I get around this simply by appending a "dot" to localhost, so instead of accessing "localhost", I try to access "localhost." (notice the dot at the end of the hostname)
Credit where credit is due:
I got this unusual tip from this thread http://www.west-wind.com/weblog/posts/2009/Jan/14/Monitoring-HTTP-Output-with-Fiddler-in-NET-HTTP-Clients-and-WCF-Proxies#596591
Works fine!
See explanation on https://docs.telerik.com/fiddler/observe-traffic/troubleshooting/notraffictolocalhost
Internet Explorer and the .NET Framework are hardcoded not to send requests for Localhost through any proxies, and as a proxy, Fiddler will not receive such traffic.
The simplest workaround is to use your machine name as the hostname instead of Localhost or 127.0.0.1. So, for instance, rather than hitting http://localhost:8081/mytestpage.aspx, instead visit http://machinename:8081/mytestpage.aspx.