I have a web application that is polling a web service on another server. The server is located on the same network, and is referenced by an internal IP, running on port 8080.
Every 15 secs, a request is sent out, which receives an xml response with job information. 95% of the time, this works well, however at random times, the request to the server is null, and reports a "response forcibly closed by remote host."
Researching this issue, others have set KeepAlive = false. This has not solved the issue. The web server is running .NET 3.5 SP1.
Uri serverPath = new Uri(_Url);
// create the request and set the login credentials
_Req = (HttpWebRequest)WebRequest.Create(serverPath);
_Req.KeepAlive = false;
_Req.Credentials = new NetworkCredential(username, password);
_Req.Method = this._Method;
Call to the response:
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
_ResponseStream = response.GetResponseStream();
The method for this is GET. I tried changing the timeout, but the default is large enough to take this into account.
The other request we perform is a POST to post data to the server, and we are getting the same issue randomly as well. There are no firewalls affecting this, and we ruled out the Virus scanner. Any ideas to help solving this is greatly appreciated!
Are you closing the response stream and disposing of the response itself? That's the most frequent cause of "hangs" with WebRequest - there's a limit to how many connections you can open to the same machine at the same time. The GC will finalize the connections eventually, but if you dispose them properly it's not a problem.
I wouldn't rule out network issues as a possible reason for problems. Have you run a ping to your server to see if you get dropped packets that correspond to the same times as your failed requests?
Set the timeout property of FtpWebRequest object to maximum i tried it with 4 GB File and it's working great.
Related
I am having a problem connecting a Windows service to an FTP site.
I inherited a Windows service from another developer. The service connects to a 3rd party server, downloads a csv file and then processes it. For some reason, the service stopped working (well over a year ago, before I was given the project).
So I went back to basics, created a console app and tried the connection/ file download function only in that app. I have tried many different methods to connect to the FTP, but all of them return the same error to my application:
The remote server returned an error: 227 Entering Passive Mode ()
This is one of the many methods I've tried:
FtpWebRequest request = (FtpWebRequest)WebRequest.Create("ftp://ftpaddress/filename.csv");
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.Credentials = new NetworkCredential("username", "password");
request.UsePassive = true;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader reader = new StreamReader(responseStream);
Console.WriteLine(reader.ReadToEnd());
Console.WriteLine("Download Complete, status {0}", response.StatusDescription);
reader.Close();
response.Close();
But it falls down on this part:
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
I read in several forums that setting the UsePassive property to False fixes these errors, but all that happened to me was that I got a syntax error instead, as below:
The remote server returned an error: (500) Syntax error, command unrecognized.
The file is hosted on a 3rd party FTP server I have no control over. I can paste the URL into a browser, and I am prompted for a username and password, which then allows me through and I can download the file.
To eliminate our firewall as the cause of the problem, I ran the app on both the internal network and the WiFi (which isn't behind the firewall), and it makes no difference. I also connected through FileZilla in Default, Active and Passive modes, and it worked every time. So no problem there.
So then I ran Wireshark. Here is an image of the wire capture using Filezilla (i.e. a successful one), in Passive mode:
And here is the capture when connecting (and failing) using the app, with passive set to true:
So as you can see in the failed connection above, I can log in to the server just fine. Then for whatever reason an extra request is sent, namely "TYPE I", which prompts the response of "Switching to binary mode." The below that, I get the following:
500 oops: vsf_sysutil_recv_peek: no data
In addition, I also ran it again after setting the Passive property to false, and this is what I got that time:
So my question is twofold;
1, if I somehow get past the UsePassive issue and set that property to false, will that solve my problem?
2, ignoring the UsePassive property, why can't I download the file from the app, but can from everywhere else?
The issue is now resolved. It turned out to be Kaspersky's built-in firewall that was blocking the connection. It's annoying that it didn't present me with a warning when I tried to connect, but reassuring to know my PC is safe.
The clue was in the detail of the 227 return:
10051 – A socket operation was attempted to an unreachable network
Also, for anyone reaching this via Google etc, the remote server was configured to only allow Passive connections, which is why I was getting the 500 syntax error. Studying a Wire capture when downloading a file revealed that Filezilla actually reverts to Passive mode automatically if Active is selected but fails.
The code in my original post works fine now.
I've written a windows service in C# that interfaces with a third party via external web service calls (SOAP). Some of the calls respond quickly and some slowly
(for example, quick = retrieving a list of currencies; slow = retrieving a historical value of currencies per product over many years)
The external service works fine when I run it on my local machine - the quick calls runs for about 20 seconds; the slow calls runs for about 30 minutes. I can't do anything about the speed of the 3rd party service and I don't mind the time it takes to return an answer..
My problem is that when I deploy my service to my Azure Virtual Machine: The quick call works perfectly like it does locally; the slow ones just never returns anything. I have tried exception handling, logging to files, logging to eventLog.
There is no clear indication of what goes wrong - it just seems that for whatever reason - the long running web service calls, never return successfully to Azure.
I've read somewhere that there is some sort of connection-recycling happening every 4 minutes, which I suspect is somehow causing the external web service response to land up somewhere in a void with the load balancer or whatever not knowing any longer whom requested the content.
I start by creating the request with the relevant SOAP envelope, like this:
HttpWebRequest tRequest = (HttpWebRequest)WebRequest.Create(endpoint);
Then i set the stuff, like this:
tRequest.ClientCertificates.Add(clientCertificate);
tRequest.PreAuthenticate = true;
tRequest.KeepAlive = true;
tRequest.Credentials = CredentialCache.DefaultCredentials;
tRequest.ContentLength = byteArray.Length;
tRequest.ContentType = #"text/xml; charset=utf-8";
tRequest.Headers.Add("SOAPAction", #"http://schemas.xmlsoap.org/soap/envelope/");
tRequest.Method = "POST";
tRequest.Timeout = 3600000;
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; }; //the SSL certificate is bad
Stream requestStream = tRequest.GetRequestStream();
requestStream.Write(byteArray, 0, byteArray.Length);
requestStream.Close();
requestStream.Dispose(); //works fine up to this point.
WebResponse webResponse = tRequest.GetResponse(); //The slow calls never make it past this. Fast one does
Anyone else experienced something similar and any suggestions how to solve it please?
Many thanks
http://www.fourtimesfour.co.za
When you deploy to Azure (Cloud Service, Virtual Machine) there is always the Azure Load Balancer, which sits between your VMs and the Internet.
In order to keep resources equally available to all the cloud users, Azure Load Balancer will kill idle connections. What idle for Azure LB is - default is 4 minutes of no communication sent over established channel. So if you call a web service and there is absolutely no response on the pip for 4 minutes - your connection will get terminated.
You can configure this timeout, but I would really argue the need for keeping connection open that long. Yes, you can do nothing about it, besides looking for a service which has better design (i.e. either returns responses faster, or implements asynchronous calls where the first call to the service will just give you a task id, using which you can poll periodically to get a result)
Here is a good article on how to configure the Azure Load Balancer timeout. Be aware, the maximum timeout for Azure LB is 30 minutes.
I have a set up with a custom HTTP server using sockets, and a client that uses HttpWebRequest to connect to that server. The client is multi-threaded, using this code:
ServicePointManager.UseNagleAlgorithm = false;
ServicePointManager.Expect100Continue = false;
ServicePointManager.DefaultConnectionLimit = 48;
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = verb;
request.Timeout = timeout;
request.AutomaticDecompression = DecompressionMethods.GZip;
request.Proxy = null;
request.SendChunked = false;
/*offending lines*/
var writer = new StreamWriter(request.GetRequestStream(),Encoding.UTF8);
writer.Write(stringData);
writer.Close();
/*end offending lines*/
var response = (HttpWebResponse)request.GetResponse();
I have been running thousands of requests through this code, and it works very well most of the time. The average response time is about 200ms, including some work done on the server.
The problem is that about 0.2% of the requests are very slow. When measuring I notice that the slow requests are pausing for 3 seconds (+- 20ms) when getting and writing to the request stream. GetResponse() is still fast, and nothing on the server takes extra time.
The server is implemented as a listener Socket listening asyncronously on a port, using Listen() and BeginAccept(). After a connection is accepted, BeginRecieve is called on the new Socket (aquired from EndAccept()) and BeginAccept is called again on the listener socket. From what I've read, this is the way to do it.
So my question is: what happens during GetRequestStream() and writing to it? Is data actually being sent here or is it flushed on GetResponse()? How come some of the requests take 3 seconds longer without any apparent reason? Is my server implementation flawed in some way? The weird thing is that it is always 3 seconds, except sometimes it can be 9 seconds. It seems to be a multiple of 3, for some reason.
A few possible scenarios:
The connection is initiated on GetRequestStream() and the server does not accept it right away. How can I test if this is the case?
Something is going on behind the scenes on GetRequestStream() or writing to it.
Waiting for network hardware?
Waiting for DNS resolution?
Something else?
Any help is very appreciated.
EDIT: I've tried connecting through a web proxy, and then the delay moves from GetRequestStream() to GetResponse(), indicating that the problem indeed lies in the web server. It seems like it is not accepting the connection. I realize that there might be some delay between EndAccept() and BeginAccept(), but it should not amount to a 3 second delay, right?
I've also tried running my client single-threaded, and the problem seems to disappear. This suggests that the connection delay only occurs when multiple requests are made at the same time.
I guess I'm a bit too late, but I've encountered similar problem and found and article, which describes what seems to be pretty low-level reason for such an issue - http://www.percona.com/blog/2011/04/19/mysql-connection-timeouts/
Maybe its because you don't dispose the request stream. Try to replace your offending lines with the following:
var stream = request.GetRequestStream();
var buffer = Encoding.UTF8.GetBytes(stringData);
stream.Write(buffer, 0, buffer.Length);
stream.Dispose();
Preliminary testing suggests that the issue was indeed connection crowding on the server. With some restructuring of Socket code, I managed to get rid of the slow connections. I will run more testing tonight to make sure, but it looks promising!
I am using C# to upload some file to a ftp server. If the file already existed the FtpWebRequest timed out, so I thought to deleate it first.
However the WebRequestMethods.Ftp.DeleteFile also always times out. Am I doing something wrong?
Here is my code:
FtpWebRequest request = (FtpWebRequest)FtpWebRequest.Create(address);
request.Credentials = new NetworkCredential(Username, Password);
request.KeepAlive = false;
request.Method = WebRequestMethods.Ftp.DeleteFile;
try
{
FtpWebResponse resp = (FtpWebResponse)request.GetResponse();
}
catch (Exception e)
{
...
}
EDIT: Oh and it doesn't matter witch file I am trying to delete. As long as the file exists the request will always time out. If the file doesn't exist a different exception is thrown.
Nothing is wrong with credentials, I can do other operations (upload/download without a problem). Also it's not a server problem, if I connect to it with a client (FileZilla) with the same username / pass everything works as it should.
Thank you for your help.
The thing I have found using this Ftp via FtpWebRequest, is it is inherently a lot slower (since it is using the HTTP protocol over port 80), and it drives me crazy because FileZilla can do it a lot quicker (obviously using FTP protocol over port 20/21). There is a open source ftp component found here, I do not know if it will work for you, but worth a shot.
I know this is a subjective answer that will get downvoted, but personally, using ftp over port 80 is bound to be a lot slower especially on file operations like what you are trying to achieve.
Do you have access to the logs of the FTP server? If you do have a look at what commands the FTPWebRequest is making. It could be that it is trying to list the directory before deleting it.
Another issue maybe that the server is in passive mode, I believe FileZilla may automagicly detect this, check the connection in filezilla to see.
Knowing what commands are sent between client and FTP server could help find out what is causing the timeout. Would it be possible to use a packet analyzer such as Ethereal to capture the communication log?
Alternative approach could be using a third party FTP component and enabling logging in it. Following code uses our Rebex FTP:
// create client
Ftp client = new Ftp();
// enable logging
client.LogWriter = new Rebex.FileLogWriter(#"c:\temp\log.txt", Rebex.LogLevel.Debug);
// connect
client.Connect("ftp.example.org");
client.Login("username", "password");
// browse directories, transfer files
client.DeleteFile("file.txt");
// disconnect
client.Disconnect();
I am using an open source library to connect to my webserver. I was concerned that the webserver was going extremely slow and then I tried doing a simple test in Ruby and I got these results
Ruby program: 2.11seconds for 10 HTTP
GETs
Ruby program: 18.13seconds for 100 HTTP
GETs
C# library: 20.81seconds for 10 HTTP
GETs
C# library: 36847.46seconds for 100 HTTP
GETs
I have profiled and found the problem to be this function:
private HttpWebResponse GetRawResponse(HttpWebRequest request) {
HttpWebResponse raw = null;
try {
raw = (HttpWebResponse)request.GetResponse(); //This line!
}
catch (WebException ex) {
if (ex.Response is HttpWebResponse) {
raw = ex.Response as HttpWebResponse;
}
}
return raw;
}
The marked line is takes over 1 second to complete by itself while the ruby program making 1 request takes .3 seconds. I am also doing all of these tests on 127.0.0.1, so network bandwidth is not an issue.
What could be causing this huge slow down?
UPDATE
Check out the changed benchmark results. I actually tested with 10 GETs and not 100, I updated the results.
What I have found to be the main culprit with slow web requests is the proxy property. If you set this property to null before you call the GetResponse method the query will skip the proxy autodetect step:
request.Proxy = null;
using (var response = (HttpWebResponse)request.GetResponse())
{
}
The proxy autodetect was taking up to 7 seconds to query before returning the response. It is a little annoying that this property is set on by default for the HttpWebRequest object.
It may have to do with the fact that you are opening several connections at once. By default the Maximum amount of open HTTP connections is set to two. Try adding this to your .config file and see if it helps:
<system.net>
.......
<connectionManagement>
<add address="*" maxconnection="20"/>
</connectionManagement>
</system.net>
I was having a similar issue with a VB.Net MVC project.
Locally on my pc (Windows 7) it was taking under 1 second to hit the page requests, but on the server (Windows Server 2008 R2) it was taking 20+ seconds for each page request.
I tried a combination of setting the proxy to null
System.Net.WebRequest.DefaultWebProxy = Nothing
request.Proxy = System.Net.WebRequest.DefaultWebProxy
And changing the config file by adding
<system.net>
.......
<connectionManagement>
<add address="*" maxconnection="20"/>
</connectionManagement>
</system.net>
This still did not reduce the slow page request times on the server. In the end the solution was to uncheck the “Automatically detect settings” option in the IE options on the server itself. (Under Tools -> Internet Options select the Connections tab. Press the LAN Settings button)
Immediately after I unchecked this browser option on the server all the page request times dropped from 20+ seconds to under 1 second.
I started observing a slow down similar to the OP in this area which got a little better when increasing the MaxConnections.
ServicePointManager.DefaultConnectionLimit = 4;
But after building this number of WebRequests the delays came back.
The problem, in my case, was that I was calling a POST and not bothered about the response so wasn't picking up or doing anything with it. Unfortunately this left the WebRequest floating around until they timed out.
The fix was to pick up the Response and just close it.
WebRequest webRequest = WebRequest.Create(sURL);
webRequest.Method = "POST";
webRequest.ContentLength = byteDataGZ.Length;
webRequest.Proxy = null;
using (var requestStream = webRequest.GetRequestStream())
{
requestStream.WriteTimeout = 500;
requestStream.Write(byteDataGZ, 0, byteDataGZ.Length);
requestStream.Close();
}
// Get the response so that we don't leave this request hanging around
WebResponse response = webRequest.GetResponse();
response.Close();
Use a computer other than localhost, then use WireShark to see what's really going over the wire.
Like others have said, it can be a number of things. Looking at things on the TCP level should give a clear picture.
I don't know how exactly I've reach to this workaround, I didn't have time to do some research yet, so it's up to you guys. There's a parameter and I've used it like this (at the constructor of my class, before instantiating the HTTPWebRequest object):
System.Net.ServicePointManager.Expect100Continue = false;
I don't know why exactly but now my calls look quite faster.
I tried all the solutions described here with no luck, the call took about 5 minutes.
What was the issue:
I needed the same session and obviously the same cookies (request made on the same server), so I recreated the cookies from Request.Cookies into WebRequest.CookieContainer. The response time was about 5 minutes.
My solution:
Commented out the cookie-related code and bam! Call took less then one second.
I know this is some old thread, but I have lost whole day with slow HttpWebRequest, tried every provided solution with no luck. Every request for any address was more than one minute.
Eventually, problem was with my Antivirus Firewall (Eset). I'm using firewall with interactive mode, but Eset was somehow turned off completely. That caused request to last forever. After turning ON Eset, and executing request, prompt firewall message is shown, and after confirmation, request executing for less than one second.
For me, using HttpWebRequest to call an API locally averaged 40ms, and to call an API on a server averaged 270ms. But calling them via Postman averaged 40ms on both environments. None of the solutions in this thread made any difference for me.
Then I found this article which mentioned the Nagle algorithm:
The Nagle algorithm increases network efficiency by decreasing the number of packets sent across the network. It accomplishes this by instituting a delay on the client of up to 200 milliseconds when small amounts of data are written to the network. The delay is a wait period for additional data that might be written. New data is added to the same packet.
Setting ServicePoint.UseNagleAlgorithm to false was the magic I needed, it made a huge difference and the performance on the server is now almost identical to local.
var webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.Method = "POST";
webRequest.ServicePoint.Expect100Continue = false;
webRequest.ServicePoint.UseNagleAlgorithm = false; // <<<this is the important bit
Note that this worked for me with small amounts of data, however if your request involves large data then it might be worth making this flag conditional depending on the size of the request/expected size of the response.
We encountered a similar issue at work where we had two REST APIs communicating locally with each other. In our case all HttpWebRequest took more than 2 seconds even though the request url was http://localhost:5000/whatever which was way to slow.
However upon investigating this issue with Wireshark we found this:
It turned out the framework was trying to establish an IPv6 connection to localhost first, however as our services were listening on 0.0.0.0 (IPv4) the connection attempt failed and after 500 ms timeout it retried until after 4 failed attempts (4 * 500 ms = 2 seconds) it eventually gave up and used IPv4 as a fallback (which obviously succeeded almost immediately).
The solution for us was to change the request URI to http://127.0.0.1/5000/whatever (or to also listen on IPv6 which we deemed unnecessary for our local callbacks).
For me, the problem was that I had installed LogMeIn Hamachi--ironically to remotely debug the same program that then started exhibiting this extreme slowness.
FYI, disabling the Hamachi network adapter was not enough because it seems that its Windows service re-enables the adapter.
Also, re-connecting to my Hamachi network did not solve the problem. Only disabling the adapter (by way of disabling the LogMeIn Hamachi Windows service) or, presumably, uninstalling Hamachi, fixed the problem for me.
Is it possible to ask HttpWebRequest to go out through a specific network adapter?
This worked for me:
<configuration>
<system.net>
<defaultProxy enabled="false"/>
</system.net>
</configuration>
Credit: Slow HTTPWebRequest the first time the program starts
In my case add AspxAutoDetectCookieSupport=1 to the code and the problem was solved
Uri target = new Uri("http://payroll");
string responseContent;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(target);
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(new Cookie("AspxAutoDetectCookieSupport", "1") { Domain = target.Host });
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (StreamReader sr = new StreamReader(responseStream))
responseContent = sr.ReadToEnd();
}
}
I was experiencing a 15 second or so delay upon creating a session to an api via httpwebrequest. The delay was around waiting for the getrequeststream() . After scrounging for answers, I found this article and the solution was just changing a local windows policy regarding ssl:
https://www.generacodice.com/en/articolo/1296839/httpwebrequest-15-second-delay-performance-issue
We had the same problem on web app. We waited on response 5 seconds. When we change user on applicationPool on IIS to networkService, the response began to arrive less than 1 second