We Keep Coding

.NET Core 2.1 new HttpClient based on .NET sockets and Span<T>. seems to have a problem

I have a strange issue I'm trying to triage having to do with the new HttpClient on .NET Core 2.1. From this article here (https://blogs.msdn.microsoft.com/dotnet/2018/04/11/announcing-net-core-2-1-preview-2/) I know that the HttpClient has been completely re-written to use a different low level library for handling HTTP requests. I'm wondering if anyone has seen any issues with the new implementation.
What I'm seeing is a strange case where my application (.NET Core 2.1) which sends a POST request to some API periodically (every 10 seconds) a few of times every 15 min it will throw an exception with the error: An error occurred while sending the request.; The server returned an invalid or unrecognized response.
No other details are available, it's just an exception when I make a call like this:
using (var res = await _httpClient.PostAsync(uriBuilder.Uri, new StringContent(serializedRequestBody, Encoding.UTF8, "application/json")))
{
//Do something here
}
The exception caught is a System.Net.Http.HttpRequestException and it has some inner exception with the above error message.
So as I mentioned this does NOT happen all time, it happens seemingly at random, or at least I can not discern any particular pattern. All I can say is these POST requests are made once every 10 seconds 24/7 and anywhere between 5% and 10% of the POST requests fail with the above exception.
So used tcdump and piped it into wireshark to examine the requests to see what's actually happening when the requests fail and what i see is the following:
On a good POST I see: my app sends the request to server, server sends response back, my app sends ACK to server and server responds with FIN,ACK. Done. Good Stuff.
On POST which gets the above exception I see the following: my app sends the request to server, and almost immediately after (like a few milliseconds after) my application sends FIN, ACK to server.
This seems consistent with what I see in my application logs, which show that the request duration is 0 before the exception is thrown.
So what it looks like to me is, my application sends the request and then immediately after closes the connection for some reason. However, I don't understand why this happens. I tried comparing the raw HTTP requests (good POST vs bad POST) to see any differences and I can not see any difference.
One last thing to mention, is that I ONLY see this in applications running on .NET Core 2.1. When I run my application on .NET 2.0 I do not see this problem. Also when I use the same library (where the HTTP call is being made) in the .NET 4.5.1 application (I use multi-targeting to compile the library targeting .net standard and net451) I also do NOT see this problem. So it seems to affect only .NET Core 2.1
Any ideas of where I can go from here? Is there something else I should look for ? How would someone go about trying to triage this type of issue ?
[EDIT] I added a screenshot of the wireshark output which shows the last POST request the server never does not respond before the client sends FIN,ACK
[EDIT]
#Svek Pointed out something in the comments about the sequence of ACKs. I think there maybe something here, because (in the screenshot) after the very last POST there is a FIN, ACK and it shows Ack=7187, so I look back I see the previous FIN,ACK had sequence=7186. Now, I'm by far not an expect in TCP or networking so I maybe saying something completely dumb, but does that mean that the last FIN,ACK (which comes from my host to server) is essentially my host FIN,ACK'ing the previous FIN,ACK (from server to my host) and essentially closing the connection.
So since the next POST is made to the same host:port, using the same connection and yet the connection is closed (via that last FIN,ACK) that's why I never get a response back?

In Signalr .net core - where should validation take place?

If I have a hub method that accepts parameters
e.g.
public IObservable<MyStreamItem> StreamData(SomeRequestData request)
{}
How do I propogate validation errors in the request?
An actual http request is only made when the socket connection is established.
So subsequent calls to Hub methods dont pass through any middleware. They are just frames/messages in the open websocket.
I've had a look at this package which is for the previous version of Signalr (for the full .net framework)
https://github.com/AGiorgetti/SignalR.Validation
This uses a HubPipelineModule which doesn't seem to exist in the new .net core Signalr.
Is there an appropriate place in the pipeline that I can tap into to do the validation?
Or should it be done in the hub method itself? And if so, how would you conditionally return a structured set of errors, as opposed to what the actual return type is meant to be?
thanks

There are currently no HubPipelineModules in SignalR alpha but we're looking at an equivalent for preview 2. Today, you'd need to do it in the method and potentially throw an error to get it back to the client.

Should I use response codes, exceptions or textual messages for my service responses?

I am building a WCF based service application in .Net. I am currently designing the contracts.
Should I use response codes, exceptions or textual messages for my service responses to report service result status?
They will be consumed by web applications and other systems.

You should take a look at FaultContracts. See http://msdn.microsoft.com/en-us/library/system.servicemodel.faultcontractattribute.aspx
Your Fault Contract can include a (string based) error code for client side processing, and / or a textual message for display to users.
If your service, or rather you as a service designer, don't know what a (future) client application will want to do with an error message (display or process), include both.

In my opinion best is to use response enum (code) that make sense to client. Apart from it returning message may increase the dataload on the WCF service.
E.g.
throw new FaultException<InvalidArgumentException>(new InvalidArgumentException(),Constants.MaxLengthFields.PhoneNumber)), Response.OrderIdMissing);
[DataContract, Serializable]
enum Response
{
[EnumMember]
OrderIdMissing,
[EnumMember]
ProductCodeInvalid,
}
There are several articles available on the web. Please go though them for more concrete information.
http://blogs.msdn.com/b/brajens/archive/2007/04/23/exception-handling-in-wcf-web-service.aspx
http://blogit.create.pt/blogs/marcosilva/archive/2008/05/18/Developing-a-WCF-Service-%5F2D00%5F-Fault-Exceptions-AND-FAULT-Contracts.aspx
http://blogs.msdn.com/b/brajens/archive/2007/04/23/exception-handling-in-wcf-web-service.aspx

Returning results/erros from WCF services. Complex type or an exception?

A few days ago I asked a very similar question. I was about returning values/errors inside the application. I accept an answer to not return error as and object but throw an custom exceptions.
Now, I'm not exactly sure how to deal with it using WCF services. I wish the service will be as simple as it could be for client.
Speaking very generally, my project looks like that:
Client -> WCF Service application -> Database.
Inside WCF app I've got my own exception class, and it works fine. But let's say client want sth from db. My question is:
What should I return to the client outside my app?
A. Complex type: A value with meta data (information about a possible error[1]). And when really unexpected error occurs throw an exception.
B. Just a value (when possible error occurs - throw an exception)
[1] - invalid string format, can't connect to database or sth like that. I mean - If I know what the error exactly is - why throw an exception to client - I'm not sure about that.

The decision how to tackle this problem depends on the client. If you have to support Silverlight Clients, throwing exceptions is out of the picture because Silverlight Clients using the browser HTTP Stack cannot deal with them. For Silverlight Clients I generally recommend (and use myself) the approach described here.

By design, WCF hides exceptions from the client. It's up to the developer to determine what, if any, information is returned to the client when an exception occurs.
I would look at using FaultContract or FaultContract<T>, and return exceptions that you wish to via that mechanism.
Here's some articles to look at:
Fault Contract
WCF Tutorial Fault Contract
You might also want to look at IErrorHandler for WCF:
IErrorHandler Interface

WCF REST client with TransferMode=Streamed - logging entire ("raw") http requests/responses

This is on .Net 4, full framework.
I'm trying to make a simple winforms app that will make some simple WCF REST calls. It's using ChannelFactory and the service contract interface. Of the ~20 methods in the interface, 2 of them involve Stream (an upload and a download method) so the service side (and currently also the client side) using TransferMode=Streamed.
My goal is to include the full HTTP request and response (much like you would see in ethereal/wireshark, or fiddler, or whatever), with headers, in a textbox of the winforms app (just to show what went over the wire)
In trying to use the built-in diagnostics (via SvcConfigEditor) and my own (via implementing IClientMessageInspector and then IEndpointBehavior to add the inspector, then channelFactory.Endpoint.Behaviors.Add to add the behavior :), I'm having 2 issues:
When doing request.ToString() or reply.ToString() in BeforeSendRequest and AfterReceiveReply, it only gets the 'body' and not the headers. Digging around in the objects in the debugger it looks like the reply has them in reply.Properties["httpResponse"], but the request's request.Properties["httpRequest"] has an empty Headers property even though Fiddler shows headers for Content-Type, Host, Accept-Encoding, and Connection. It seems like there's likely a better way to get the 'raw' message that I'm missing (and if there's not, someone probably knows an existing chunk of code to 'reconstruct' the raw one from the Message)
Since the transfer mode is Streamed, the 'body' part just shows up as the string '... stream ...', both in SvcTraceViewer (and the 'raw' svclog - even with logEntireMessage=true) and when doing a ToString(). If the mode is Buffered instead, it shows the actual body fine. I tried making a copy with reply.CreateBufferedCopy(int.MaxValue); but that then caused the actual WCF call to fail with an InvalidOperationException: This message cannot support the operation because it has been copied.
One fallback would be to move the client to Buffered and just change to StreamedRequest for the one upload call and StreamedResponse for the download call (but I'd have to do that programmatically AFAICT, as it's set at the binding level in the config and I don't see anyway of doing it via attributes on the calls), which would take care of the 'body' part and leave me with just the "get the http request headers" (issue #1, specifically request.Properties["httpRequest"].Headers being empty) to deal with, but I'm hoping there's some way of logging the 'raw' messages without doing so, leaving the TransferMode as Streamed.
Thanks!

I can't find any reference right now, but it's a known fact that you cannot capture the contents of a streamed message to WCF tracing. When streaming is enabled, only the headers of the message will be traced.
Here's the source: Configuring Message Logging on MSDN
See towards the end of the page:
Service Level
Messages logged at this layer are
about to enter (on receiving) or leave
(on sending) user code. If filters
have been defined, only messages that
match the filters are logged.
Otherwise, all messages at the service
level are logged. Infrastructure
messages (transactions, peer channel,
and security) are also logged at this
level, except for Reliable Messaging
messages. On streamed messages, only
the headers are logged. In addition,
secure messages are logged decrypted
at this level.