We Keep Coding

.NET Core 2.1 new HttpClient based on .NET sockets and Span<T>. seems to have a problem

I have a strange issue I'm trying to triage having to do with the new HttpClient on .NET Core 2.1. From this article here (https://blogs.msdn.microsoft.com/dotnet/2018/04/11/announcing-net-core-2-1-preview-2/) I know that the HttpClient has been completely re-written to use a different low level library for handling HTTP requests. I'm wondering if anyone has seen any issues with the new implementation.
What I'm seeing is a strange case where my application (.NET Core 2.1) which sends a POST request to some API periodically (every 10 seconds) a few of times every 15 min it will throw an exception with the error: An error occurred while sending the request.; The server returned an invalid or unrecognized response.
No other details are available, it's just an exception when I make a call like this:
using (var res = await _httpClient.PostAsync(uriBuilder.Uri, new StringContent(serializedRequestBody, Encoding.UTF8, "application/json")))
{
//Do something here
}
The exception caught is a System.Net.Http.HttpRequestException and it has some inner exception with the above error message.
So as I mentioned this does NOT happen all time, it happens seemingly at random, or at least I can not discern any particular pattern. All I can say is these POST requests are made once every 10 seconds 24/7 and anywhere between 5% and 10% of the POST requests fail with the above exception.
So used tcdump and piped it into wireshark to examine the requests to see what's actually happening when the requests fail and what i see is the following:
On a good POST I see: my app sends the request to server, server sends response back, my app sends ACK to server and server responds with FIN,ACK. Done. Good Stuff.
On POST which gets the above exception I see the following: my app sends the request to server, and almost immediately after (like a few milliseconds after) my application sends FIN, ACK to server.
This seems consistent with what I see in my application logs, which show that the request duration is 0 before the exception is thrown.
So what it looks like to me is, my application sends the request and then immediately after closes the connection for some reason. However, I don't understand why this happens. I tried comparing the raw HTTP requests (good POST vs bad POST) to see any differences and I can not see any difference.
One last thing to mention, is that I ONLY see this in applications running on .NET Core 2.1. When I run my application on .NET 2.0 I do not see this problem. Also when I use the same library (where the HTTP call is being made) in the .NET 4.5.1 application (I use multi-targeting to compile the library targeting .net standard and net451) I also do NOT see this problem. So it seems to affect only .NET Core 2.1
Any ideas of where I can go from here? Is there something else I should look for ? How would someone go about trying to triage this type of issue ?
[EDIT] I added a screenshot of the wireshark output which shows the last POST request the server never does not respond before the client sends FIN,ACK
[EDIT]
#Svek Pointed out something in the comments about the sequence of ACKs. I think there maybe something here, because (in the screenshot) after the very last POST there is a FIN, ACK and it shows Ack=7187, so I look back I see the previous FIN,ACK had sequence=7186. Now, I'm by far not an expect in TCP or networking so I maybe saying something completely dumb, but does that mean that the last FIN,ACK (which comes from my host to server) is essentially my host FIN,ACK'ing the previous FIN,ACK (from server to my host) and essentially closing the connection.
So since the next POST is made to the same host:port, using the same connection and yet the connection is closed (via that last FIN,ACK) that's why I never get a response back?

Poor HTTP request performance unless using Fiddler

Edit: after talking it over with a couple IT guys, I've realized it's only the POLL requests that are having issues. I'm fetching the images via GET requests that go through quickly and as expected, whether or not the POLL messages are having issues.
I'm working on a client to interface with an IP camera in C#.
It's all working dandy except that I can get really poor http request performance when I'm not using Fiddler (a web traffic inspection proxy).
I'm using an httpclient to send my requests, this is my code that actually initiates the poll request:
public async Task<bool> SetPoll(int whichpreset)
{
string action = "set";
string resource = presetnames[whichpreset];
string value = presetvalues[whichpreset];
int requestlen = 24 + action.Length + resource.Length + value.Length;
var request = new HttpRequestMessage
{
RequestUri = new Uri("http://" + ipadd + "/res.php"),
Method = HttpMethod.Post,
Content = new FormUrlEncodedContent(new[]{
new KeyValuePair<string,string>("action",action),
new KeyValuePair<string,string>("resource",resource),
new KeyValuePair<string,string>("value",value)
}),
Version = new System.Version("1.1"),
};
HttpResponseMessage mess = await client.SendAsync(request);
if (mess.IsSuccessStatusCode)
{
return true;
}
else
{
return false;
}
}
When Fiddler is up, all my http requests go through quickly, and without a hitch (I'm making about 20 post requests upon connecting). Without it, they only go through as expected ~1/5 of the time, and the rest of the time they're never completed, which is a big issue. Additionally, the initial connection request often takes 1+ minutes when not using Fiddler, and consistently only takes a few seconds when I am, so it doesn't seem to be a timing issue of sending requests too soon after connecting.
This leads me to think that the request, as written, is fairly poorly behaved, and perhaps Fiddler's requests behave better. I'm a newbie to HTTP, so I'm not sure exactly why this would be. My questions:
does Fiddler modify HTTP requests (E.G. different headers, etc.)
as they are sent to the server?
even if it doesn't modify the requests, are Fiddler's requests in
some way better behaved than I'd be getting out of .net 4.0 in C# in
VS2013?
is there a way to improve the behavior of my requests to emulate
whatever Fiddler is doing? Ideally while still working within the
stock HTTP namespace, but I'm open to using others if necessary.
I'll happily furnish more code if helpful (though tomorrow).

Inserting
await Task.Delay(50);
between all requests fixed the problem (I haven't yet tested at different delays). Because Fiddler smoothed the problem out, I suspect it's an issue the camera has with requests sent in too quick of a succession, and fiddler sent them at a more tolerable rate. Because it's an async await, there is no noticeable performance impact, other than it taking a little while to get through all ~20 (30 now) requests through on startup, which is not an issue for my app.

Fiddler installs itself as a system proxy. It is possible that the Fiddler process has better access to the network than your application's process.
Fiddler might be configured to bypass your normal system proxy (check the gateway tab under options) and perhaps the normal system proxy has issues.
Fiddler might be running as a different user with a different network profile, e.g. could be using a different user cert store or different proxy settings such as exclusion list.
Fiddler might be configured to override your hosts file and your hosts file may contain errors.
Your machine might be timing out trying to reach the servers necessary to check for certificate revocation. Fiddler has CRL checking disabled by default (check the HTTPS tab).
Fiddler has a ton of options and the above are just some guesses.
My recommendation would be to check and/or toggle the above options to see if any of them apply. If you can't get anywhere, you may have to forget Fiddler exists and troubleshoot your network problems independently, e.g. by using NSLOOKUP, PING, TRACERT, and possibly TELNET to isolate the problem.
There is nothing in your code sample that suggests a code flaw that could cause intermittent network failures of the kind you are describing. In fact it is hard to imagine any code flaw that would cause that sort of behavior.

How To Prevent HttpWebRequest with long response from timing-out Azure Web App

We are using an HttpWebRequest in C# to get data from an internet resource in our Azure Web App. The problem is that Azure has a limitation on how long it keeps the connection alive (around 240 seconds). Due to the nature of our application, the response will sometimes take longer than 240 seconds. When this happens, the webpage will go white, and the "View Source" will show zero source code (which has made this issue difficult to debug).
Here's some sample code to illustrate:
webRequest = WebRequest.Create(PAGE_URL) as HttpWebRequest;
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.CookieContainer = cookies;
webRequest.Timeout = Timeout.Infinite;
webRequest.KeepAlive = true;
StreamWriter requestWriter2 = new
StreamWriter(webRequest.GetRequestStream());
requestWriter2.Write(postString);
requestWriter2.Close();
WebResponse response = webRequest.GetResponse();
Stream stream = response.GetResponseStream();
Adding webRequest.Timeout and webRequest.KeepAlive did not solve the issue.
jbq on this thread mentioned he had a workaround by sending a "newline character every 5 seconds", however did not explain how to accomplish this exactly. He was answering a question about an Azure VM, but I think an Azure Web App would have similar behaviors with respect to what I believe are the load balancers that are responsible for the timeout.
The Question:
How can I send one HttpWebRequest, and then send another HttpWebRequest while the previous one is running with a blank line to maintain the connection and prevent the Azure load balancer(?) from timing out the azure application? Would a new session variable need to be used? Perhaps an asynchronous method? Do I need to send the "pinging" request before the main request? If so, how would this look in implementation? Or is it something else entirely? Please provide some source code as an example :)
Note: you do not need to use an HttpWebRequest to replicate this issue. Attach a debugger from within Visual Studio to a live Azure Web App. Place a breakpoint within Visual Studio at any piece of code. When that breakpoint is hit, after roughly 4 minutes you'll see the page in your browser stop loading and go white with an empty source. So, it's not specifically related to HttpWebRequest, but that is an operation that would typically cause this sort of issue since some responses take longer than 4 minutes.
*EDIT: I think what I am looking for is an implementation of Asynchronous methods. I will update this post as I find a satisfactory implementation.

If you are making an HttpWebRequest to an Azure Website then you use ServicePointManager.SetTcpKeepAlive on your client code which uses HttpWebRequest.
https://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.settcpkeepalive(v=vs.110).aspx
The 4 minute timeout that you are talking about is an IDLE timeout over the TCP layer and setting this will ensure that your client (who is using HttpWebRequest) sends ACK packet over TCP so that the connection doesn't get idle.
If your web application is making a HttpWebRequest to some other service, you can still use this function but that will just ensure that the Idle timeout is not hit when calling the remote service. Your actual HTTP request to the Azure Webapp may still hit the 4 minute time and if the client to your Azure web app is not HttpWebRequest, then again the 4 minute idle timeout will bite you...
The best thing to do here is to change the code a bit to implement a JOB Model kind of pattern where-in you make a server call which returns a JOBID. The client then queries the server using this JOBID in a polling fashion and when the job completes on the server the status of this JOBID can be set to COMPLETED in which case the client can then retrieve the data. You can use Webjobs in Azure Webapps to achieve something like this.
Hope this helps...

MonoDroid HttpWebRequest and WebClient unreliable?

I am having a lot of trouble using Webrequests in MonoDroid and getting timeouts at random. My code works fine then sometimes all requests just timeout and don't work.
I have verified the webservices used in my requests are not the problem.
Here is an example of some code that I may use to request some data from a webservice using MonoDroid:
bool bolOk = false;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create ("http://www.website.com/service/");
request.Timeout = 20000;
request.Credentials = gv_objCredentials;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse ()) {
bolOk = response.StatusCode == HttpStatusCode.OK;
}
As you can see it is basic stuff. I use code like the above always on another thread to the UI using ThreadPool.QueueUserWorkItem or TaskFactory.
What I have noticed is that if the requests start timing out from my app and I plug it in to my computer then debug the app from MonoDevelop the requests work without timing out. I am not sure if this means anything. This is similar to testing the webservices from my computer using a browser on the same network as the phone. The webservices always work without any issues.
What is the best way to make Webrequests from MonoDroid?
How can I ensure my requests are always successful and won't timeout if the webservice is operating correctly?

I had the Issue on Xamarin 4.2.6 and 4.2.8.
Thanks to Xamarin support, they identified the issue and suggested I targeted my build to armeabi-v7a rather than armeabi in my project properties (some multi-core processor issue described here)
Depending on whether you plan to support multi-core processors or not, you should check our this post and may need to manually edit your .csproj file.

There's a new version of Mono for Android (4.2.5) that fixes a number of bugs with the WebRequest and webRequestStream. You can check the release notes here: http://docs.xamarin.com/android/Releases/Mono_For_Android_4/Mono_for_Android_4.2.5
I suggest downloading the latest bits and check if it works. If not, please file a bug and they will surely fix it in the next version of the product.

Why is WebClient (System.Net) getting from URL twice?

I've got a method like this:
private string getFromURL(string url)
{
WebClient myClient = new WebClient();
return myClient.DownloadString(url);
}
using WebClient from System.Net. It appears to be hitting the url twice (I'm also watching the log of the web server in question and it records two hits). Any idea why this might be?
EDIT: the answer was in fact programmer error. I no longer have any reason to think this is behaving strangely. Thanks for the answers.

Or if the URL is subtly different in the two cases it could be responding to a HTTP redirect request.

My guess is that it's doing a HEAD before the GET. Does your log show the HTTP method being used?

check out tcpmon:
https://tcpmon.dev.java.net/
it's a java tool - but you can run it easy w/out being a "java" guy
Chances are there's a redirect or something to itself, so you should be able to see if the http requests are identical or slightly different.
Also, check out curl (cygwin) - you can test sending the requests from there and see if there's a redirect or something.