I saw couple of questions asked about core 2.0 on how to make HttpClient to use default proxy configured on the system. But no where found right answer. Posting this question hoping someone who might have encountered this issue might have found the solution by now.
In .net framework versions I've used the following configuration in my web.config and it worked for me.
<system.net>
<defaultProxy useDefaultCredentials="true"></defaultProxy>
</system.net>
But in .net core 2.0 where I've make a web request to external api from my company's intranet my code is failing with 407, proxy authentication required.
After little bit of research I am of the opinion that it is not possible to make your HttpClient to use default proxy settings configured via WPAD in IE. Can someone correct my understanding here?
On this page of https://github.com/dotnet/corefx/issues/7037
It is said as follows :
"The default for HttpClientHandler.UseProxy property is true. And the default value of HttpClientHandler.Proxy is NULL which means to use the default proxy."
But I don't observe this behavior.
Update:
I am finally able to call external web api by specifying the proxy server address and then making the HttpClient call. Still wondering how to use default proxy setup in IE.
using (var handler = new HttpClientHandler {
Credentials = new System.Net.NetworkCredential(user, password, domain),
UseProxy = true,
Proxy = new System.Net.WebProxy(new Uri("http://xxxxxxx:8080"), true)
})
{
handler.Proxy.Credentials = new NetworkCredential("xxxx", "yyyyy", "cccccc");
using (var httpClient = new HttpClient(handler))
{
var request = new HttpRequestMessage()
{
RequestUri = new Uri(destinationUrl),
Method = HttpMethod.Post
};
request.Content = new StringContent(requestXml, Encoding.UTF8, "text/xml");
HttpResponseMessage response = await httpClient.SendAsync(request);
Task<Stream> streamTask = response.Content.ReadAsStreamAsync();
}
}
If any one interested in finding out how I was able to find out the proxy server was, I wrote the following code in .net 4.0 and found out the proxy used.
var proxy = WebRequest.GetSystemWebProxy();
var url = proxy.GetProxy(new Uri("http://google.com"));
Thanks
I hope this is the answer you're looking for: Default Proxy issues #28780
If you simply want to use the default system proxy and need to pass default credentials to that proxy (because the proxy is an authenticated proxy) during HTTP requests, then do this:
var handler = new HttpClientHandler();
handler.DefaultProxyCredentials = CredentialCache.DefaultCredentials;
var client = new HttpClient(handler);
Related
I'm having difficulty understanding how web requests and credentials work in .NET.
I have the following method that is executing a request to a SOAP endpoint.
public WebResponse Execute(NetworkCredential Credentials)
{
HttpWebRequest webRequest = CreateWebRequest(_url, actionUrl);
webRequest.AllowAutoRedirect = true;
webRequest.PreAuthenticate = true;
webRequest.Credentials = Credentials;
// Add headers and content into the requestStream
asyncResult.AsyncWaitHandle.WaitOne();
return webRequest.EndGetResponse(asyncResult);
}
It works well enough. However, users of my applications may have to execute dozens of these requests in short succession. Hundreds over the course of the day. My goal is to implement some of the recommendations I've read about, namely using an HttpClient that exists for the entire lifetime of the application, and to use the CredentialCache to store user's credentials, instead of passing them in to each request.
So I'm starting with the CredentialCache.
Following the example linked above, I instantiated a CredentialCache and added my network credentials to it. Note that this is the exact same NetworkCredential object that I was passing to the request earlier.
NetworkCredential credential = new NetworkCredential();
credential.UserName = Name;
credential.Password = PW;
Program.CredCache.Add(new Uri("https://blah.com/"), "Basic", credential);
Then, when I go to send my HTTP request, I get the credentials from the cache, instead of providing the credentials object directly.
public WebResponse Execute(NetworkCredential Credentials)
{
HttpWebRequest webRequest = CreateWebRequest(_url, actionUrl);
webRequest.AllowAutoRedirect = true;
webRequest.PreAuthenticate = true;
webRequest.Credentials = Program.CredCache;
// more stuff down here
}
The request now fails with a 401 error.
I am failing to understand this on several levels. For starters, I can't seem to figure out whether or not the CredentialCache has indeed passed the proper credentials to the HTTP request.
I suspect part of the problem might be that I'm trying to use "Basic" authentication. I tried "Digest" as well just as a shot in the dark (which also failed), but I'm sure there must be a way to see what kind of authentication the server is expecting.
I have been combing StackOverflow and MDN trying to read up as much as possible about this, but I am having a difficult time separating the relevant information from the outdated and irrelevant information.
If anyone can help me solve the problem that would be most appreciated, but even links to proper educational resources would be helpful.
According to the documentation the CredentialCache class is only for SMTP, it explicitly says that it is not for HTTP or FTP requests:
https://msdn.microsoft.com/en-us/library/system.net.credentialcache(v=vs.110).aspx
Which directly contradicts the info in the later api docs. Which one is right I don't know.
You could try using the HttpClient class. The methods and return types are different, so you would need to tweak your other a code a bit, but it would look a bit like this:
public class CommsClass
{
private HttpClient _httpClient;
public CommsClass(NetworkCredential credentials)
{
var handler = new HttpClientHandler { Credentials = credentials };
_httpclient = new HttpClient(handler);
}
public HttpResponseMessage Execute(HttpRequestMessage message)
{
var response = _httpClient.SendAsync(message).Result;
return response;
}
}
You can do all sorts of other things with the handler, and the client like set request headers or set a base address.
I am trying to send a HTTP GET request to a service secured with BASIC authentication and https. If I use the RESTClient Firefox plugin to do so there is no problem. I am defining the basic-header and sending the GET to the url and I am getting the answer (data in json).
Now I am working on a Windows Store App in C# which is meant to consume the service. I enabled all required capabilities in the manifest and wrote the following method:
private async void HttpRequest()
{
string basic = "Basic ...........";
Uri testuri = new Uri(#"https://...Servlet");
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Add("Authorization", basic);
Task<HttpResponseMessage> response = client.GetAsync(testuri);
var text = await response;
var message = text.RequestMessage;
}
I tried out many different possibilites like getting the response-string but everything lead to an 401 Status Code answer from the Server.
I looked at many similar problems and my understanding of the communication is the following: Client request -> Server response with 401 -> Client sends Authorization header -> Server response with 200 (OK)
What I don't understand is why I am getting the 401 "Unauthorized" Status Code although I am sending the Authorization header right at the beginning. It would be interesting if someone knows how this is handled in the RESTClient.
The BASIC header is definetly correct I was comparing it with the one in the RESTClient.
It would be great if someone could help me with this.
Thanks in advance and kind regards,
Max
Was having a similar problem, i added a HttpClientHandler to HttpClient.
var httpClientHandler = new HttpClientHandler();
httpClientHandler.Credentials = new System.Net.NetworkCredential("","")
var httpClient = new HttpClient(httpClientHandler);
Credentials should be encoded, before adding to the header. I tested it in WPF app, It works...
string _auth = string.Format("{0}:{1}", "username", "password");
string _enc = Convert.ToBase64String(Encoding.UTF8.GetBytes(_auth));
string _basic = string.Format("{0} {1}", "Basic", _enc);
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Add("Authorization",_basic);
What is the equivalent to WebResponse.IsFromCache when using HttpClient and HttpResponseMessage?
Is there some HTTP header in the response that I can look at?
FYI: The Windows.Web.Http HttpClient (a similar API targetted at Windows 8.1 app development) does include an HttpResponseMessage.Source field that specifies where the result came from (common values are "cache" and "network").
The Windows.Web.Http classes are usable from C# and other .NET languages, from C++, and from JavaScript (when running as a WwaHost app like from the Windows app store).
Can I ask what you're trying to achieve? Are trying to avoid caching?
The reason for asking is I've looked at the source code for HttpClient (specifically HttpClientHandler) and the source for HttpWebResponse and I dont believe you can get this information from the headers.
HttpClient/HttpClientHandler does use HttpWebResponse internally however it does not expose all properties from HttpWebResponse :
private HttpResponseMessage CreateResponseMessage(HttpWebResponse webResponse, HttpRequestMessage request)
{
HttpResponseMessage httpResponseMessage = new HttpResponseMessage(webResponse.StatusCode);
httpResponseMessage.ReasonPhrase = webResponse.StatusDescription;
httpResponseMessage.Version = webResponse.ProtocolVersion;
httpResponseMessage.RequestMessage = request;
httpResponseMessage.Content = (HttpContent) new StreamContent((Stream) new HttpClientHandler.WebExceptionWrapperStream(webResponse.GetResponseStream()));
//this line doesnt exist, would be nice
httpResponseMessage.IsFromCache = webResponse.IsFromCache;// <-- MISSING!
...
}
So your options the way I see it are:
a) Look at the source code for HttpWebRequest to determine the logic for IsFromCache and retrofit this somehow into HttpClient (this may not even be possible, depends on what the logic actually does/needs)
b)ask the ASP.NET team for this property to be included with HttpResponseMessage. either directly as a property or perhaps they could 'keep' the HttpWebResponse
Neither of these options are that great sorry, hence my original question, what are you trying to acheive?
I've been struggling with this scenario recently as well.
What I needed was an integration test to verify that:
Responses for a newly created resource had the correct headers set by the server.
Subsequent requests for that resource were fulfilled from the client-cache.
Responses for an existing resource had the correct headers set by the server as well.
What I ended up doing was a twofold check:
A non-caching HttpClient to check the initial response:
new WebRequestHandler
{
AllowAutoRedirect = true,
UseCookies = true,
CookieContainer = new CookieContainer(),
CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.Refresh)
};
var client = new HttpClient(handler)
and a second HTTP client to check the client-side cache:
new WebRequestHandler
{
AllowAutoRedirect = true,
UseCookies = true,
CookieContainer = new CookieContainer(),
CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.Default)
};
var client = new HttpClient(handler)
To verify the source of response messages I compare the HttpResponseMessage.Headers.Date values from steps 1 and 2 (which will be the same if the response came from the client cache). For my third step I can just re-use the client from the first step and append an arbitrary string to the URL.
Disclaimer: this applies to .NET Framework 4.7 and ignores best practices concerning HttpClient usage but is seems to do the trick for me in my test suite. An explicit property like the one mentioned above would be preferable but does not seem to be available. Since the last reply here is already a few years old there might be better ways to handle this, but I couldn't think of one.
I'm developing a consumer app for a publically avalible rest webservice.
I'm having 2 problems: My proxy and the service authentication.
i cant seem to get past my proxy, actually i do have a valid credential to get by it, but i dont know where or how to provide it!
And second, i also dont know how to responde the basic authentication challenge issued by the web-service...
I do can use it via browser, but i cant get it working on my c# app. Heres the code so far:
HttpClient cli = new HttpClient();
cli.BaseAddress = new Uri("http://myserver.com/");
HttpResponseMessage response = cli.GetAsync("api/service1").Result;
textBox1.Text = response.Content.ReadAsStringAsync().Result;
the result in textBox1 so far is always a 407 error... Can anyone help?
Edit1: Authentication on the webservice is of the type BASIC!
Edit2: clientHandler.Credentials = new NetworkCredential("user", "P#ssw0rd"); does not work... server returns "This request requires HTTP authentication"
Proxy information needs to be configured on the HttpClientHandler object which can be passed into the HttpClient constructor.
var clientHandler = new HttpClientHandler();
clientHandler.Proxy = new WebProxy("http://proxyserver:80/",true);
var httpClient = new HttpClient(clientHandler);
For credentials I do something like this...
var clientHandler = new HttpClientHandler() {PreAuthenticate = true};
var credentialCache = new CredentialCache();
credentialCache.Add(new Uri(Host), "Basic", new NetworkCredential(userName, password));
clientHandler.Credentials = credentialCache;
By setting this up this way, whenever you make a request to any URI that is below the "Host" URI, HttpClientHandler will automatically set the correct authorization header.
Also, be aware there is an alternative handler called WebRequestHandler that can be used instead of HttpClientHandler that add in extra stuff that is only available on the Windows OS like WinINet proxy and Pipelining.
I'm attempting to connect through a SOCKS5 proxy to a webservice. Currently, my app.config is as follows:
<system.net>
<defaultProxy enabled="true" >
<proxy proxyaddress="http://xxx.xxx.xxx.xxx:yyyy" bypassonlocal="True" />
</defaultProxy>
</system.net>
This works fine for http proxies, however it just doesn't connect to SOCKS5 ones. I've googled far and wide for an answer and just can't find anyone that's solved this problem, even though I've found the question a couple of times. Anyone have any idea on how I can do this?
Apparently Microsoft did not build SOCKS proxy support into their web classes (see http://windows-tech.info/13/f17246e7dd4a8a2b.php)
I too am in a situation where I'd like to use the built-in .NET web functionality with a SOCKS5 proxy, and have not found any satisfactory solution. I guess my next step is to obtain the source of the .NET classes either through Reflector or from the Mono codebase and refactor it to use the ProxySocket class mentioned in the above link.
This is more work than I wanted, but seems to be the only way.
In .NET 6 you can do it easily as follows:
var proxy = new WebProxy
{
Address = new Uri("socks5://localhost:8080")
};
//proxy.Credentials = new NetworkCredential(); //Used to set Proxy logins.
var handler = new HttpClientHandler
{
Proxy = proxy
};
var httpClient = new HttpClient(handler);
or when you can inject IHttpClientFactory (recommended way):
In your ConfigureServices method:
Services.AddHttpClient("WithProxy")
.ConfigurePrimaryHttpMessageHandler(() =>
{
var proxy = new WebProxy
{
Address = new Uri("socks5://localhost:8080")
};
return new HttpClientHandler
{
Proxy = proxy
};
});
Then in your controller or anywhere you can inject IHttpClientFactory object, create a new httpClient (with configured proxy) instance as follows:
httpClient = httpClientFactory.CreateClient("WithProxy");
Here is a more detailed description:
https://dotnetcoretutorials.com/2021/07/11/socks-proxy-support-in-net/
If you are able to use HttpClient, you can use SocksSharp.
I just followed the example and it worked for me.
var settings = new ProxySettings()
{
Host = "xxx.xxx.xxx.xxx",
Port = yyyy
};
using (var proxyClientHandler = new ProxyClientHandler<Socks5>(settings))
{
using (var httpClient = new HttpClient(proxyClientHandler))
{
var response = await httpClient.GetAsync("http://example.com/");
}
}
In theory you can inherit the webservice class and override GetWebRequest and GetWebResponse. Then you may be able to use this c# socks class http://www.codeproject.com/Articles/5954/C-class-for-connecting-via-a-SOCKS-Proxy-Server and convert the webrequest to sockets and back again.
The quickest solution I have been able to find is to set a system wide socks proxy using WideCap. This routes all traffic on the machine via the socks proxy server.