We Keep Coding

.NET HttpClient do not persist authentication between reqeusts to IIS when using NTLM Negotiate

IIS site configured to use windows authentication with default options. The client is written in C# and uses single HttpClient instance to perform requests. Requests success, but every request triggers 401 Challenge:
Traffic captured with Wireshark. We performed loud test, and noticed, that with anonymous authentication client performs 5000 reqeusts per second, but with windows authentication - 800. So, looks like wireshark does not impact to authentication, performance slowdown indicates, that 401 Challenge also occurs without wireshark.
Wirehshark log: https://drive.google.com/file/d/1vDNZMjiKPDisFLq6ZDhASQZJJKuN2cpj/view?usp=sharing
Code of client is here:
var httpClientHandler = new HttpClientHandler();
httpClientHandler.UseDefaultCredentials = true;
var httpClient = new HttpClient(httpClientHandler);
while (working)
{
var response = await httpClient.GetAsync(textBoxAddress.Text + "/api/v1/cards/" + cardId);
var content = await response.Content.ReadAsStringAsync();
}
IIS site settings:
How to make HttpClient to persist authentication between requests, to prevent waste negotiate handshakes on every request?
UPD: Code of client: https://github.com/PFight/httpclientauthtest
Steps to reproduce:
Create folder with simple file index.html, create application 'testsite' in IIS for this folder. Enable anonymous authentication.
Run client (https://github.com/PFight/httpclientauthtest/blob/main/TestDv5/bin/Debug/TestDv5.exe), press start button - see count of requests per second. Press stop.
Disable anonymous atuhentication, enable windows authentication.
Press start button in client, see count of reqeusts per second.
On my computer I see ~1000 requests per second on anonymous, and ~180 on windows. Wireshark shows 401 challenges on every request for windows authentication. keep-alive header enabled in IIS.
IIS version: 10.0.18362.1 (windows 10)
Version of System.Net.Http.dll loaded to process: 4.8.3752.0

Firstly I tried to save the Authorization header for re-use it with every new request.
using System.Net.Http.Headers;
Requester requester = new Requester();
await requester.MakeRequest("http://localhost/test.txt");
await Task.Delay(100);
await requester.MakeRequest("http://localhost/test.txt");
class Requester
{
private readonly HttpClientHandler _httpClientHandler;
private readonly HttpClient _httpClient;
private AuthenticationHeaderValue _auth = null;
public Requester()
{
_httpClientHandler = new HttpClientHandler();
_httpClientHandler.UseDefaultCredentials = true;
_httpClient = new HttpClient(_httpClientHandler);
_httpClient.DefaultRequestHeaders.Add("User-Agent", Guid.NewGuid().ToString("D"));
}
public async Task<string> MakeRequest(string url)
{
HttpRequestMessage message = new HttpRequestMessage(HttpMethod.Get, url);
message.Headers.Authorization = _auth;
HttpResponseMessage resp = await _httpClient.SendAsync(message);
_auth = resp.RequestMessage?.Headers?.Authorization;
resp.EnsureSuccessStatusCode();
string responseText = await resp.Content.ReadAsStringAsync();
return responseText;
}
}
But it didn't work. Every time there was http code 401 asking for authentication despite of Authorization header.
The IIS logs is listed below.
2021-12-23 15:07:47 ::1 GET /test.txt - 80 - ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 401 2 5 127
2021-12-23 15:07:47 ::1 GET /test.txt - 80 MicrosoftAccount\account#domain.com ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 200 0 0 4
2021-12-23 15:07:47 ::1 GET /test.txt - 80 - ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 401 1 2148074248 0
2021-12-23 15:07:47 ::1 GET /test.txt - 80 MicrosoftAccount\account#domain.com ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 200 0 0 0
IIS's Failed Requests Tracing reports the following when receiving re-used Authentication Header:
Property
Value
ModuleName
WindowsAuthenticationModule
Notification
AUTHENTICATE_REQUEST
HttpStatus
401
HttpReason
Unauthorized
HttpSubStatus
1
ErrorCode
The token supplied to the function is invalid (0x80090308)
I've made a research and I can say that this is not possible without alive connection.
Every time the connection is closed there will be new handshake.
According this and this answers NTLM authenticates a connection, so you need to keep your connection open.
NTLM over http is using HTTP persistent connection or http keep-alive.
A single connection is created and then kept open for the rest of the session.
If using the same authenticated connection, it is not necessary to send the authentication headers anymore.
This is also the reason why NTLM doesn't work with certain proxy servers that don't support keep-alive connections.
UPDATE:
I found the key point using your example.
First: You must enable keep-alive at your IIS
Second: You must set authPersistSingleRequest flag to false. Setting this flag to True specifies that authentication persists only for a single request on a connection. IIS resets the authentication at the end of each request, and forces re-authentication on the next request of the session. The default value is False.
Third: You can force the HttpClient to send keep-alive headers:
httpClient.DefaultRequestHeaders.Add("Connection", "keep-alive");
httpClient.DefaultRequestHeaders.Add("Keep-Alive", "600");
Using this three key points I have achieved only one NTLM handshake during connection lifetime.
Also it's important which version of .NET \ .NET Framework do you use.
Because HttpClient hides different realizations dependent on framework version.
Framework
Realization of HttpClient
.Net Framework
Wrapper around WebRequest
.Net Core < 2.1
Native handlers (WinHttpHandler / CurlHandler)
.Net Core >= 2.1
SocketsHttpHandler
I tried it on .NET 6 and it works great, but it didn't work on .Net Framework as I can see, so here is the question: which platform do you use?
UPDATE 2:
Found the solution for .Net Framework.
CredentialCache myCache = new CredentialCache();
WebRequestHandler handler = new WebRequestHandler()
{
UseDefaultCredentials = true,
AllowAutoRedirect = true,
UnsafeAuthenticatedConnectionSharing = true,
Credentials = myCache,
};
var httpClient = new HttpClient(handler);
httpClient.DefaultRequestHeaders.Add("Connection", "keep-alive");
httpClient.DefaultRequestHeaders.Add("Keep-Alive", "600");
var from = DateTime.Now;
var countPerSecond = 0;
working = true;
while (working)
{
var response = await httpClient.GetAsync(textBoxAddress.Text);
var content = await response.Content.ReadAsStringAsync();
countPerSecond++;
if ((DateTime.Now - from).TotalSeconds >= 1)
{
this.labelRPS.Text = countPerSecond.ToString();
countPerSecond = 0;
from = DateTime.Now;
}
Application.DoEvents();
}
The key point is to use WebRequestHandler with UnsafeAuthenticatedConnectionSharing enabled option and use credential cache.
If this property is set to true, the connection used to retrieve the response remains open after the authentication has been performed. In this case, other requests that have this property set to true may use the connection without re-authenticating. In other words, if a connection has been authenticated for user A, user B may reuse A's connection; user B's request is fulfilled based on the credentials of user A.
Caution
Because it is possible for an application to use the connection without being authenticated, you need to be sure that there is no administrative vulnerability in your system when setting this property to true. If your application sends requests for multiple users (impersonates multiple user accounts) and relies on authentication to protect resources, do not set this property to true unless you use connection groups as described below.
Big thanks to this article for solution.

Add CorrelationId to remote calls in ASP.NET MVC5

I use Delgating handler to add correlation ids to inward requests to my application.
But, my MVC app makes frequent calls to remote domain https://remoteservice.net/xyz via an webapi client provided by them. The remote service API allows users to set correlation id via request header, but the webapi client doesn't provide a way to expose that.
So I am now thinking to set correlation id via owin pipeline for all outgoing remote calls from my MVC app. Is this possible? Any help?

You could extend an HttpClientHandler and add a correlationId header there:
public class CorrelatingHandler : HttpClientHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
request.Headers.Add("currelationId", Guid.NewGuid());
return base.SendAsync(request, cancellationToken);
}
}
using(var client = new HttpClient(new CorrelatingHandler())){
.....
}
source: https://forums.asp.net/post/6025521.aspx

How to set "Connection: keep-alive" header in lower case through HttpClient?

I have a task where I need to be able to send Connection: keep-alive header the same way as it is done by the Firefox browser (notice that keep-alive has to be all lower-case):
"Connection: keep-alive"
However, I had no luck in achieving it using HttpClient. No matter what I try, the request always have
"Connection: Keep-Alive"
Here is an example code:
var client = new HttpClient();
var request = new HttpRequestMessage()
{
RequestUri = new Uri("http://www.someURI.com"),
Method = HttpMethod.Get,
};
request.Headers.Connection.Clear(); // No need to do it as it is empty anyway
request.Headers.Connection.Add("keep-alive"); // Still results in "Keep-Alive"
var task = client.SendAsync(request);
Another attempt:
var client = new HttpClient();
client.DefaultRequestHeaders.Clear();
client.DefaultRequestHeaders.Connection.Add("keep-alive"); // Still results in "Keep-Alive"
string result = await client.GetStringAsync("http://www.someURI.com");
There is an answer about how it can be done in HttpWebRequest: How to send lower-case Keep-Alive header through HttpWebRequest
Would something similar be possible in HttpClient?

Did you try this :
var client = new HttpClient();
client.DefaultRequestHeaders.Add("Connection", "keep-alive");
It gives this :

This problem is only present when you run code on Windows platform. It is present in .Net and .Net Core 2.0. The underlying reason for this lies in win32 library "winhttp.dll" which contains hardcoded string for "Keep-Alive", so if you need to get it working on windows, you will need to do something about this dll.
The good news is that this issue does not affect Linux platforms running Mono or .NET Core 2.0

It does not matter for HTTP Server whether "keep-alive" is upper case or not.
Just add the text "keep-alive" or "Keep-Alive" or "KEEP_ALIVE" then should be treated same.
hope this helps.

Azure Mobile Service Filter - Removed?

I've been reading about custom auth methods using Azure, and stumbled upon 12 Days of ZUMO. It's an excellent write up, but I can't seem to find anything related to IServiceFilter in the current .Net Client SDK for Azure.
I'm developing a Xamarin app in C#.
Has this functionality been removed or relocated? I have all Azure namespaces referenced and intellisense can't find IServiceFilter or anything else related to service filters.
Thanks for reading.

I currently develop an app for Android on Java (I don't think that library for C# is very different).
And I can access ServiceFilter. But I don't see IServiceFilter.
Namespace - com.microsoft.windowsazure.mobileservices.http.ServiceFilter
Library version - 2.0.2-beta2

In Xamarin you need to use a DelegateHandler instead. This is the standard .net way to filter http traffic.
public class MyLogFilter : DelegatingHandler
{
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request, CancellationToken cancellationToken)
{
// Do any pre-request requirements here
Log.Warning("http request", request.RequestUri.ToString() );
// Request happens here
var response = await base.SendAsync(request, cancellationToken);
// Do any post-request requirements here
Log.Warning("http response", response.StatusCode + " " + response.Content.ReadAsStringAsync().Result);
return response;
}
}
Once you've done that, you just chain the handler when initializing your MobileServiceClient, changing this:
Client = new MobileServiceClient (API_URL, new MyExistingHandler());
to:
Client = new MobileServiceClient (API_URL, new MyLogFilter(), new MyExistingHandler());

ASP.NET Web API: RC to RTM change. Sending content using HttpContext ends in a NoContent status

Since ASP.NET Web API RC I was using some approach based on declaring void API controller's operations.
I was customizing a response object DTO (instead of using HttpResponseMessage) using AOP and PostSharp, and finally this was sent to the client using HttpContext.Response.Write(...) serializing the DTO into a JSON string.
When I upgraded my solution to ASP.NET Web API RTM, this approach didn't work anymore.
Whenever I send a response from the Web API and I receive it in the client-side, I find that the response is sent with a 204 status (NoContent) while I was setting a 200 status (OK) for the response itself.
Because this approach was working in the RC version of WebAPI I suspect that's an unknown breaking change when WebAPI development team transitioned to RTM version.
Am I wrong?

As far as I know, since RTM if a POST action does not return an HttpResponseMessage the default status code is 204 (and not 200 as was back in RC). There are two things, I know, we can do to keep clients from complaining about 204.
a) Change the response message from within your action:
[HttpPost]
public HttpResponseMessage DoWork(MyModel model)
{
// Do work
return new HttpResponseMessage(HttpStatusCode.OK) { Content = new ObjectContent<MyModel>(model, FormatterConfig.JsonFormatter) };
}
b) Change the response in a DelegatingHandler (dirty by generic way)
public class ResponseHandler : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var response = base.SendAsync(request, cancellationToken);
response.Result.StatusCode = response.Result.IsSuccessStatusCode ? System.Net.HttpStatusCode.OK : response.Result.StatusCode;
return response;
}
}

I am not aware of such breaking change but I can confirm that this doesn't work in the RTM. Anyway, that's such a wrong approach of using the Web API that it's probably a good thing that it doesn't work. You are killing the whole point of the Web API if you are going to manually write the response to the client. If you have some existing code that does this that you cannot modify then I would recommend you using a Generic ASHX handler until you are ready to upgrade.