We have for some time been using Cloudant NoSQL from the IBM Cloud and we have been extremely happy with the speed, simplicity and reliability. BUT a few weeks ago our front-end server which stores data in Cloudant database startet periodically to log exceptions: "The remote name could not be resolved: '[A unique ID]-bluemix.cloudant.com" at System.Net.HttpWebRequest.EndGetRequestStream.
I added a DNS lookup when the error occurs which logs: "This is usually a temporary error during hostname resolution and means that the local server did not receive a response from an authoritative server" at System.Net.Dns.GetAddrInfo(String name).
This relaxed error message indicates it is not harmful but for us it is.
We see the error for 1-3 minutes every 30-120 minutes on servers but not while debugging locally (this could be lack of patience and/or traffic).
Below is one method of seven which fails
using (HttpClientHandler handler = new HttpClientHandler())
{
handler.Credentials = new NetworkCredential(Configuration.Cloudant.ApiKey, Configuration.Cloudant.ApiPassword);
using (var client = new HttpClient(handler))
{
var uri = new Uri(Configuration.Cloudant.Url); //{https://[A unique ID]-bluemix.cloudant.com/[Our Product]/_find}
var stringContent = new StringContent(QueryFromResource(),
UnicodeEncoding.UTF8,
"application/json");
var task = TaskEx.Run(async () => await client.PostAsync(uri, stringContent));
task.Wait(); // <------ Exception here
if (task.Result.StatusCode == HttpStatusCode.OK)
{
// Handle response deleted
}
}
}
We have updated our .Net framework, experimented with DnsRefreshTimeout, refactored code, extended caching but we keep seeing the issue.
We also added a DNS lookup to Google when the error occurs and this is consistently successful.
Initially we thought load might we an issue but we see the issue even when there is no traffic.
Suggestions are much appreciated!
I am making some calls to an api that syncs and saves appointments to/from my application and mail agenda which is most commonly Outlook mails. The calls to the api are made from a web application and it is working fine for some time but then immediately after a few hours, the calls are failed. This continues for some time and starts to work again after some time.
The DefaultConnectionLimit was set to 100 in the beginning and during this scenario, the process stopped working after some time (Say 30 mins to 1 hour). Then DefaultConnectionLimit was set to 20 and in this case, it worked fine for 3-4 hours and stopped working after that. I don't know if this is the actual cause but just mentioning.
The code where the call to the api is made from the web application is mentioned below :
public bool SyncNewAppToExch(string encryptedAppSyncDetails)
{
try
{
string result = string.Empty;
JArray paramList = new JArray();
paramList.Add(encryptedAppSyncDetails);
var emailApiUrl = ConfigurationManager.AppSettings["emailApiUrl"].ToString();
Uri baseAddress = new Uri(emailApiUrl);
var url = "/api/EmailProcess/ManualSaveAppointments";
HttpClient client = new HttpClient();
client.BaseAddress = baseAddress;
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
Task.Run(() => client.PostAsJsonAsync(url, paramList)).Wait(5000);
var responseMessage = true;
if (responseMessage)
return true;
else
return false;
}
catch (Exception ex)
{
return false;
}
}
The exception that follows the api call failure reads :
Exception : System.AggregateException: One or more errors occurred. System.Threading.Tasks.TaskCanceledException: A task was canceled. --- End of inner exception stack trace --- at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) at System.Threading.Tasks.Task1.get_Result()
If you are making a new HttpClient object each time you want to make a call, you are not using it appropriately. HttpClient may continue to use sockets even if it is no longer being used and this will likely cause socket exhaustion under load. You can see more on this topic here
You should create a single instance and reuse it. If you are using .Net core it would be a better idea to use the HttpClientFactory.
In addition, using Task.Run to call an async method is not wise. You are setting yourself up for a deadlock. It would be much better to mark your method as async and await, if possible. Since you are effectively doing fire and forget, the way you have done this if your App Domain shuts down this work will be lost.
If you need to do this, you should at least consider registering your task as such:
HostingEnvironment.QueueBackgroundWorkItem(ct => SendMailAsync(user.Email));
The short answer is that it is difficult to say what your issue is, it could be one of several problems.
I have a Windows Phone 8 app containing code as follows:
using (var client = new HttpClient())
{
var httpRequest = new HttpRequestMessage(method, uri);
try
{
var response = client.SendAsync(httpRequest);
var httpResponse = await response;
if (httpResponse.IsSuccessStatusCode)
{
var result = await httpResponse.Content.ReadAsStringAsync();
return result;
}
else
{
HandleError(httpResponse);
return null;
}
}
catch (Exception ex)
{
throw;
}
}
If the client successfully connects to the server, I will get the expected results, including all the appropriate HTTP status codes and reason phrases.
If the client is unable to contact the server (e.g. incorrect domain/IP/port in the URL), the awaited task completes after some delay, with a 404 status and no reason phrase. The task never throws an exception. The task (the response variable in the code snippet) has the status of "ran to completion". Nothing about the result is indicative of the actual problem - be it networking failure, unreachable server, etc.
How can I capture more meaningful errors in the case where the URL points to a non-existent or unreachable server, socket connection refused, etc.? Shouldn't SendAsync be throwing specific exceptions for such cases?
FWIW, the client code is built into a PCL library using VS 2013 update 3 and running in the Windows Phone emulator, using System.Net.Http 2.2.28 from NuGet
Windows Phone's implementation of the .NET HttpClient is different from Desktop's implementation (wininet vs. custom stack). This is one of the incompatibilities that you need to be aware of, if there is a semantic difference for you.
In my simple test I do get a ReasonPhrase of "Not Found" on the phone emulator. Also you can see that the Headers collection is empty whereas if the server was found and actually returned a real 404 then there would be headers.
i'm testing some code here and came across a really weird issue. The HttpClient class works like a charm if my tablet is on but whenever the battery safe state kicks in and the screen is locked the HttpClient class throw an exception and the app suddendly exits giving me small chances to log the exception. If i'm not mistaken the error seems to be a send request error but that's weird since i have full access to the internet while the tablet is on.
Here the code i'm using and the one that crash on that state(System.Net.Http namespace ):
private static async Task<string> HttpGet(string uri)
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("User-Agent", BotUserAgent);
var response = await client.GetAsync(uri);
return await response.Content.ReadAsStringAsync();
}
}
Alternatively i've used the client of the web.http namespace with the same result:
using(var client = new Windows.Web.Http.HttpClient())
{
client.DefaultRequestHeaders.Add("User-Agent", BotUserAgent);
var response = await client.GetAsync(new Uri(uri));
return await response.Content.ReadAsStringAsync();
}
This is the error i get(after adding an exception logger):
An error occurred while sending the request.
Any ideas?
The problem is that your WinRT-device sends your app to the background when the battery safe state kiks in.
whenever the user moves your app into the background, your app can be suspended or frozen, essentially stopping any downloads dead in their tracks. In some scenarios, the app might
even be terminated, forcing you to create a new instance of the class in an attempt to start the download again. Fortunately, WinRT provides a way to handle this specific scenario using a background task.
This is a quote from http://www.informit.com/articles/article.aspx?p=2229233&seqNum=6
On this site is an easy tutorial to follow on how to use BackgroundTransfers in the Windows.Networking.BackgroundTransfer-Namespace.
When using the System.Net.WebClient.DownloadData() method I'm getting an unreasonably slow response time.
When fetching an url using the WebClient class in .NET it takes around 10 sec before I get a response, while the same page is fetched by my browser in under 1 sec.
And this is with data that's 0.5kB or smaller in size.
The request involves POST/GET parameters and a user agent header if perhaps that could cause problems.
I haven't (yet) tried if other ways to download data in .NET gives me the same problems, but I'm suspecting I might get similar results. (I've always had a feeling web requests in .NET are unusually slow...)
What could be the cause of this?
Edit:
I tried doing the exact thing using System.Net.HttpWebRequest instead, using the following method, and all requests finish in under 1 sec.
public static string DownloadText(string url)
var request = (HttpWebRequest)WebRequest.Create(url);
var response = (HttpWebResponse)request.GetResponse();
using (var reader = new StreamReader(response.GetResponseStream()))
{
return reader.ReadToEnd();
}
}
While this (old) method using System.Net.WebClient takes 15-30s for each request to finish:
public static string DownloadText(string url)
{
var client = new WebClient();
byte[] data = client.DownloadData(url);
return client.Encoding.GetString(data);
}
I had that problem with WebRequest. Try setting Proxy = null;
WebClient wc = new WebClient();
wc.Proxy = null;
By default WebClient, WebRequest try to determine what proxy to use from IE settings, sometimes it results in like 5 sec delay before the actual request is sent.
This applies to all classes that use WebRequest, including WCF services with HTTP binding.
In general you can use this static code at application startup:
WebRequest.DefaultWebProxy = null;
Download Wireshark here http://www.wireshark.org/
Capture the network packets and filter the "http" packets.
It should give you the answer right away.
There is nothing inherently slow about .NET web requests; that code should be fine. I regularly use WebClient and it works very quickly.
How big is the payload in each direction? Silly question maybe, but is it simply bandwidth limitations?
IMO the most likely thing is that your web-site has spun down, and when you hit the URL the web-site is slow to respond. This is then not the fault of the client. It is also possible that DNS is slow for some reason (in which case you could hard-code the IP into your "hosts" file), or that some proxy server in the middle is slow.
If the web-site isn't yours, it is also possible that they are detecting atypical usage and deliberately injecting a delay to annoy scrapers.
I would grab Fiddler (a free, simple web inspector) and look at the timings.
WebClient may be slow on some workstations when Automatic Proxy Settings in checked in the IE settings (Connections tab - LAN Settings).
Setting WebRequest.DefaultWebProxy = null; or client.Proxy = null didn't do anything for me, using Xamarin on iOS.
I did two things to fix this:
I wrote a downloadString function which does not use WebRequest and System.Net:
public static async Task<string> FnDownloadStringWithoutWebRequest(string url)
{
using (var client = new HttpClient())
{
//Define Headers
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var response = await client.GetAsync(url);
if (response.IsSuccessStatusCode)
{
string responseContent = await response.Content.ReadAsStringAsync();
//dynamic json = Newtonsoft.Json.JsonConvert.DeserializeObject(responseContent);
return responseContent;
}
Logger.DefaultLogger.LogError(LogLevel.NORMAL, "GoogleLoginManager.FnDownloadString", "error fetching string, code: " + response.StatusCode);
return "";
}
}
This is however still slow with Managed HttpClient.
So secondly, in Visual Studio Community for Mac, right click on your Project in the Solution -> Options -> set HttpClient implementation to NSUrlSession, instead of Managed.
Screenshot: Set HttpClient implementation to NSUrlSession instead of Managed
Managed is not fully integrated into iOS, doesn't support TLS 1.2, and thus does not support the ATS standards set as default in iOS9+, see here:
https://learn.microsoft.com/en-us/xamarin/ios/app-fundamentals/ats
With both these changes, string downloads are always very fast (<<1s).
Without both of these changes, on every second or third try, downloadString took over a minute.
Just FYI, there's one more thing you could try, though it shouldn't be necessary anymore:
//var authgoogle = new OAuth2Authenticator(...);
//authgoogle.Completed...
if (authgoogle.IsUsingNativeUI)
{
// Step 2.1 Creating Login UI
// In order to access SFSafariViewController API the cast is neccessary
SafariServices.SFSafariViewController c = null;
c = (SafariServices.SFSafariViewController)ui_object;
PresentViewController(c, true, null);
}
else
{
PresentViewController(ui_object, true, null);
}
Though in my experience, you probably don't need the SafariController.
Another alternative (also free) to Wireshark is Microsoft Network Monitor.
What browser are you using to test?
Try using the default IE install. System.Net.WebClient uses the local IE settings, proxy etc. Maybe that has been mangled?
Another cause for extremely slow WebClient downloads is the destination media to which you are downloading. If it is a slow device like a USB key, this can massively impact download speed. To my HDD I could download at 6MB/s, to my USB key, only 700kb/s, even though I can copy files to this USB at 5MB/s from another drive. wget shows the same behavior. This is also reported here:
https://superuser.com/questions/413750/why-is-downloading-over-usb-so-slow
So if this is your scenario, an alternative solution is to download to HDD first and then copy files to the slow medium after download completes.