We have a background operation (Window service) that we want to use through a proxy server.
Basically, we're doing this:
public WebClient GetWebClient(){
var webClient = new WebClient();
webClient.proxy = new WebProxy(Configuration.ProxyHost, Configuration.ProxyPort);
// add a bunch of headers to the WebClient (sessionids, etc.)
return webClient;
}
The proxy is one that we have configured ourselves using FreeProxy.
I've enabled logging and on the machine I'm testing with, and can confirm that requests are being made to the proxy when using it in Firefox.
No authentication is required for the proxy server, except that the IP has to be within our office (which from the Firefox evidence, I assume is not the problem).
However, within our background process, I don't seem to be using the proxy when I use the webclient:
using(var wc = GetWebClient())
using(var s = wc.OpenRead("someurl"))
using(var sr = new StreamReader(s)){
return sr.ReadToEnd();
}
I receive no errors from the proxy however, it seems like we're just going along without it even though the proxy has explicitly been set.
The information seems to return fine, just not through our proxy.
Is there something I'm missing when using a proxy with a WebClient?
edit: more details. If we disable the proxy service on the server, then we get an exception that we can't connect. So it seems like the webclient is attempting to reach out to the proxy, but that traffic is not actually flowing through the proxy.
Inner Exception: SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
It turns out that FreeProxy wasn't accepting HTTPS traffic.
I guess the proxy must return the types of traffic it can route, and if it cannot, the webclient does nothing.
Switched to using the Burp suite as our proxy since it can accept HTTPS.
http://portswigger.net/burp/
You either have to configure windows for using the proxy by default, or set the proxy manually in your code, see: http://msdn.microsoft.com/en-us/library/system.net.webclient.proxy(v=VS.100).aspx
You are using the WebClient class correctly as far as I can tell. I am able to see the following request...
using(var client = new WebClient())
{
client.Proxy = new WebProxy("localhost", 8888);
Console.WriteLine(client.DownloadString("http://www.google.com"));
}
in Fiddler running on my local box. Now if I shut Fiddler down, I get the WebException:
Unable to connect to the remote server
With an inner SocketException of:
No connection could be made because the target machine actively refused it 127.0.0.1:8888
So, with that said, my guess is that your proxy is working as intened but it is just not logging the outgoing HTTP request.
Related
I am new to web stuff in general. Not sure what the issue is here.
I have a machine with IIS running a ASP NET Core REST API on https.
I can confirm the GET is working via Google Chrome on my machine by doing either
https://test-machine/api/Example (works)
https://10.0.0.21/api/Example (invalid certificate, N.B. this is as expected as the certificate is for "test-machine")
I confirmed the POST action is working using Postman (again ignoring invalid certificate).
Everything is fine till I try and write a client application.
I have the following code.
var ip = Dns.GetHostAddresses("test-machine");
// ip contains the correct IP
using (var hc = new HttpClient())
{
hc.GetAsync(#"https://test-machine/api/Example").Wait();
}
This code fails. I get an System.AggregateException: 'One or more errors occurred.' with 4 exceptions stacked up on top of one another.
HttpRequestException: An error occurred while sending the request.
WebException: The underlying connection was closed: An unexpected error occurred on a send.
IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
SocketException: An existing connection was forcibly closed by the remote host
If I change the code to use the IP instead:
using (var hc = new HttpClient())
{
hc.GetAsync(#"https://10.0.0.21/api/Example").Wait();
}
Then the request works as expected I.e. as with Chrome, I get a certificate problem:
AuthenticationException: The remote certificate is invalid according to the validation procedure.
What am I doing wrong? The Dns is clearly able to get the correct IP, I've seen it in the IPAddress[] returned by GetHostAddresses. Why is the HttpClient not resolving the IP - or is it resolving the IP with another problem?
I have tried
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
and
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
but this makes no difference. Using the
ServicePointManager.ServerCertificateValidationCallback
Also doesn't work as it never reaches the point of calling the callback. It's failing before then.
I suspect the problem is that as I am not familiar with any of this stuff I have missed something important somewhere.
I should note also - this is a .net 4.5.2 Console app. I tried moving to 4.6.1 and this still fails.
I was looking completely in the wrong place. The problem lay in the IIS setup.
The site binding was for some reason explicitly set to 10.0.0.21:433. Switching this to "All Unassigned" - seems to have made this work.
I am probably out of my depth, and could be wrong, but I suspect the reason for this is that according to Wireshark, the response to the MDNS request to find the hostname IP, was returning the ipv6 address of the server machine, which of course isn't 10.0.0.21. Again, I could be wrong.
The reason I am so unsure is that chrome seems to have ploughed right through and found the correct thing when I put in https://test-machine/api/Example and I cannot explain why.
So I implemented an interface to communicate with a rest web service using the HttpClient class to make requests.
The code works perfectly locally, but when I deploy to Azure my application can't fire the request, it crashes on this line:
using (var response = await HttpClient.PostAsync(uri, content)) { ... }
// uri = https://api-rest.zenvia360.com.br/services/send-sms
The exact exception message is this:
A connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because
connected host has failed to respond 200.203.125.26:443
The web service provider states that "if you use firewall or proxy you must add and exception for our IPs. For HTTPS, use port 443:
200.203.125.24
200.203.125.25
200.203.125.26 (the ip of the exception message)
200.203.125.27
200.203.125.28
200.203.125.29"
I looked everywhere in Azure looking for a firewall or something and I got nothing. Also this exception message is pretty cryptc. I tested the same code with another url (fired a post to www.gooogle.com) and it worked.
Any ideas?
The problem turned out to be on the web service side. The service I'm using blocks international requests by default. I asked them to whitelist the azure outbound IPs and now it works.
I have written a code to hit a particular URL and download data using a WebClient, and have set a proxy which I have purchased from MyPrivateProxies.net to this webclient with credentials and other needed parameters through which it hits the mentioned URL.
But I get this exception every time at the last line.
System.Net.WebException: Unable to connect to the remote server
System.Net.Sockets.SocketException: A connection attempt failed
because the connected party did not properly respond after a period of
time, or established connection failed because connected host has
failed to respond
below is my snippet of code:
WebClient client = new WebClient();
WebProxy webProxy = new WebProxy("my_proxy_with_port");
NetworkCredential netcredit = new NetworkCredential(username, password, domain);
client.Credentials = netcredit;
webProxy.Credentials = netcredit;
webProxy.UseDefaultCredentials = false;
webProxy.BypassProxyOnLocal = false;
client.Proxy = webProxy;
client.Encoding = Encoding.UTF8;
String strURL = "url_to_be_hit";
String jsonString = client.DownloadString(strURL);
Do I need some other settings on my windows 7 system or any config changes in app.config?
Are there other configuration for a console app to use proxies?
I think you just need to assign the proxy credentials to the WebProxy object, not to the WebClient.
I dont know myprivateproxies.net but you have to make sure that you are using a http proxy, not a socks proxy or something else.
got it resolved wasn't my code issue or URI points or any of my strings which I was passing...the port was not opened at my end on local system
I've been trying get my app to connect to a WCF service within the corporate network.
We use a VPN to keep everything somewhat hidden and secure.
I've tried several different methods and I'm unable to get the phone to connect to the service.
I first added the service as a service reference, and built the client out with the correct URL, but this just did nothing and then failed with a nondescript error message after around 50sec.
I then switched to System.Net.HttpClient. This again failed after around 50sec, but this time it threw a System.Net.WebException with the message "A server with the specified hostname could not be found".
I finally tried the ModernHttpClient as well, and this had the same result as before.
(Note that I tried the first two in a test console app project on my local machine first to make sure that they would complete a request successfully)
To make sure I wasn't going crazy and that the VPN was correctly resolving the name, I created a test page within the app which solely has a WebView on it, with the source set to the service url.
I opened the app and navigated to the test page and it loaded the service definition page without a problem.
for reference, this is the current code I have using ModernHttpClient:
using (var client = new HttpClient(new NativeMessageHandler()))
{
const string soap = "<soap msg>";
client.DefaultRequestHeaders.Add("SOAPAction", "<service namespace>");
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("text/xml"));
using (var response = await client.PostAsync("http://<url>/Service.svc", new StringContent(soap, Encoding.UTF8, "text/xml")))
{
using (var content = response.Content)
{
// parse the xml result
}
}
}
Is there something special that I should be doing to tell the HttpClient that it has to use the VPN? I thought that this was handled automatically.
I looked into the VPN configuration to make sure it was correct.
I looked into the DNS configuration to make sure it resolves correctly.
I looked into the phone's configuration to make sure it was correct.
I eventually found the issue is a problem with Airwatch's Tunnel VPN.
The VPN only works the first time the app is launched.
If the VPN remains open and is not closed before the app is restarted, then it will not allow connections to pass through.
Expecting a patch form airwatch soon that should (hopefully) rectify the issue!
I have made written a c# program which is successfully connecting to the remote host with and without proxy. We work in two different networks, home and office networks, which uses proxy. Here is the code snippet.
while(true) {
Thread.sleep(5000);
using (var client = new WebClient()) {
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.ContentType] = "application/json";
string result = client.UploadString(Event.GetInsertURL(), "POST", json);
if (result.Contains("SUCCESS")) {
// Console.WriteLine("SUCCESS");
}
}
}
The above code runs in a loop to keep making the request to the same api. It is working in both the networks if the program is started in those networks. However, if I start the program in home and hibernate or sleep the computer and restart it in office, I'm getting the following exception.
System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 74.125.130.141:443
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
The reason is that the connection made for first time is reused in subsequent requests. Is there any way for me to force creation of connection when I get this exception?
P.S:
Code of Event.GetInsertURL()
public static string GetInsertURL(){
return "https://my-app.appspot.com/_ah/api/"eventendpoint/v1/insertEvents";
}
The code already creates a new client session for each connection in new WebClient().
It may be that the sleep occurs during the course of one of the sessions and then triggers the fault.
In general, any of the network methods could throw under unexpected conditions. The only real solution is to wrap the code in a try/catch block and retry under the correct conditions a few times before reporting a permanent failure.
Based on experience more than knowledge:
use HttpWebRequest instead of webClient , it's just more robust.
my examples are too messy but that's the base:
var httpWebRequest = (HttpWebRequest)System.Net.WebRequest.Create(URI);