We Keep Coding

C# HttpClient authorization header removed after send to server

I wanna send request to external API through the HttpClient from my own API in c#.
I am using dot net 5 and basic authentication.
Here is my code:
var client = new HttpClient
{
BaseAddress = new Uri(baseUrl)
};
HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Put, "apiUrl");
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var param = JsonConvert.SerializeObject(new
{
param1="",
param2=""
});
requestMessage.Content = new StringContent(param, Encoding.UTF8, "application/json");
requestMessage.Headers.Authorization = new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(Encoding.ASCII.GetBytes($"{user}:{pass}")));
HttpResponseMessage response = await client.SendAsync(requestMessage);
Usually, I send http request like this.
but now I have some problem.
After line HttpResponseMessage response = await client.SendAsync(requestMessage); authorization header removed from my request header and I got UnAuthorized http error.
I know that when redirection occurs, authorization header removed for security reason. Also I'm not sure about redirect in external API.
I add the HttpClientHandler with AllowAutoRedirect = false to my HttpClient
var handler = new HttpClientHandler()
{
AllowAutoRedirect = false,
};
var client = new HttpClient (handler)
{
BaseAddress = new Uri(baseUrl)
};
Now I got Redirect error 301(Permanently moved).
I decided to test the code in Postman. By default, when I call API in Postman, I got http error 405 method not allowed and error detail like this:
{
"detail": "Method "GET" not allowed."}
External API method is PUT. but here I got GET Error.
I tried many options in postman and finally I find the option in Postman:
When I toggle it on, external API work's properly.
Also I test it with Insomnia and it's work properly.
Does it related to my code or dot net 5 or what other thing in my code or it's related to external API?
If it's related to my code, How can I solve the error?
If error related to external API, why Postman and Insomnia response is ok?
External API has Core Policy for specific domain and I send request from other domain.
All I Know is that the CORS policy applied in browser. not in Postman, Insomnia or C# Code.
What about CORS? Does it related to CORS? if Yes, what shall I do?
I will be grateful for your answer.
Update
I detect WWW-Authenticate: JWT realm="api" in the response header.
What exactly is it? and what shall I do?

I find out the problem. It's really Ridiculous.
When I use URL like www.abc.com/api/something the request gets 301 error and postman sends another request like www.abc.com/api/something/. The difference is just / at the end of request.
I tried new URL in postman and first request got ok.
Also I tried URL in my C# code and again its ok.
But i could not understand why.
Thanks a lot dear #pharaz-fadaei

You are right about the removal of authorization headers after redirects. But keep in mind that this behavior is part of the design of the HttpClient in C#. Postman and Insomnia may have different mechanisms to send the authorization headers on each consecutive request that is caused by redirects. The option that you enabled in Postman will make it use the original HTTP method you specified (PUT) as the HTTP method to send further requests based on the redirect messages (Postman uses GET method by default on requests instructed by redirect messages).
The fact that you see a 301 shows that a redirection is required. You can check Location header value in response.Headers to see the real location and send your requests with the authorization headers to that endpoint directly. If I were you I wouldn't use the new location directly because the original endpoint is what you were given by the authors of the API. Instead I would programmatically send the request to the original endpoint and resend the request on 301 codes to the new Location (use PUT method due to the behavior of Postman) until you get the result. This answer can give you some ideas: https://stackoverflow.com/a/42566541/1539231

Since you see WWW-Authenticate: JWT realm="api" header, the external API is required a JWT token authentication, not basic authentication. I think first you might need to check external api's documentation.

Is HttpClient automatically secure by just using an HTTPS URI?

I'm using the C# class HttpClient to make requests to a RESTful API. Are these requests secure by virtue of using an HTTPS URI? Or should I be doing more in the code to make sure the requests are secure?
Here is some sample code:
HttpClient client = new HttpClient();
client.BaseAddress = new Uri("https://www.example.com");
client.DefaultRequestHeaders.Add("custom-header-1", "value-1");
client.DefaultRequestHeaders.Add("custom-header-2", "value-2");
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue(type, credentials);
HttpResponseMessage response = client.PostAsync(URLParameters, content).Result;
return response.Content.ReadAsStringAsync().Result;
I've read some material online with conflicting ideas:
Some people say that the URL being HTTPS is good enough.
Others say you have to manually take care of certificates in the code.
I'm confused as to which school of though is correct.

Check url with NTLM on server side

I need to check if url exists and can be reached. In order to do it I send Get request and handle the status:
var httpClient = new HttpClient();
var response = httpClient.GetAsync(new Uri(pageUrl));
isPageAccessible = response.Result.StatusCode == HttpStatusCode.OK;
However, the server uses NTLM for authentication. As I found it here, there are several steps (requests) before I get success OK status. For the first request I get 401 Unauthorized status and can't go to further steps.
All in all, how can I check url on the server with NTML upon completion of all requests?

If you are accessing an authenticated server, you should provide credentials. Credentials of running process for NTLM can be provided with HttpClient as below:
var handler = new HttpClientHandler {
Credentials = System.Net.CredentialCache.DefaultCredentials
};
var httpClient = new HttpClient(handler);
var response = httpClient.GetAsync(new Uri(pageUrl));

You're setting yourself up for failure since there are dozens of reasons why a request may not return a 200 OK response. One may be that the response has no content 204 No Content. Another may be that the request only accepts POST or PUT requests. Another, as you've discovered, may be that it has an authentication system in front of it 401 Not Authorized. Another may be just that the response is a redirect 301 Moved Permanently or 302 Found. Or it could be behind a proxy 305, 306, etc.
The only way you can determine if a URL really exists is to request that the other end prove it. Google does it, Facebook does it, Pinterest does it, etc. The way they do it is they ask the sender to set an MX record in their DNS or a meta tag on their index.html with a custom token they generate. If the token exists, then they're who they say they are.
Anything else is unreliable.

HttpClient strange behavior when trying to access WCF Rest service with Windows Auth

I am creating a client-server application using WCF REST services (.Net 4.0) and I see really strange error there. To access the service I use HttpClient class, the code looks like:
public TResponse Post<TRequest, TResponse>(Uri uri, TRequest request)
using (HttpClient client = new HttpClient())
{
client.TransportSettings.Credentials = CredentialCache.DefaultCredentials;
client.TransportSettings.ConnectionTimeout = TimeSpan.FromSeconds(200);
using (HttpResponseMessage response = client.Post(uri, HttpContentExtensions.CreateDataContract(request, null)))
{
response.EnsureStatusIsSuccessful();
return response.Content.ReadAsDataContract<TResponse>();
}
}
}
I host my service on IIS7 and when anonymous auth is enabled everything works well, but when I turn on windows auth some random requests (in average 1 of 10) get failed with 400-Bad request status and error message is : Bad Request - Invalid Verb. HTTP Error 400. The request verb is invalid.
What could it be?
And a little update: if fiddler is turned on everything works perfect.

I just ran into the same thing here. Quite a mystery really. I don't understand why having fiddler running removes the problem.
I eventually found that the issue was caused the by the "expect continue" header being added to the post. I removed that and the post succeeds instead of the 400 bad request.
Here's a good entry about it: http://blogs.msdn.com/b/jaskis/archive/2009/04/19/asmx-post-request-fails-with-http-400-error-when-content-length-size-increases.aspx
In my case this was apache HTTP Client v4.0 so,
httpClient.getParams().setBooleanParameter(
CoreProtocolPNames.USE_EXPECT_CONTINUE, false);
did the trick.

Parallel HTTPRequest using HttpClient in Windows 8 app

I am using HttpClient for sending HTTP requests and receiving HTTP responses in my Windows 8 app. I have few questions on the same:
1) Can I send multiple/parallel HTTP requests using a single HttpClient object? Is there a recommended way to use HttpClient object efficiently?
2) What is the difference when I create HttpClient object every time and when I re-use the same object for each new request?
3) I am tracking the requests and responses using Fiddler. What I found out is that the response time in Fiddler is different than the response time I am calculating manually inside my App. The response time for a request in Fiddler is always lower than the calculated response time in my app. Can anybody please tell me why it is like that?
4) One more thing I came across is that for every request it is doing HTTPS handshake. Instead it should do it only first time. I checked it using Fiddler and it is clearly visible there. Is there any property I need to set in HttpClient object to stop this from doing it every time.
5) Whether HttpClient is thread-safe?

1 & 5:
HttpClient manual:
The following methods are thread safe:
CancelPendingRequests
DeleteAsync
GetAsync
GetByteArrayAsync
GetStreamAsync
GetStringAsync
PostAsync
PutAsync
SendAsync
2 & 4:
HttpClient manual:
The HttpClient class instance acts as a session to send HTTP requests.
3:
Fiddler acts as a proxy. Your browser sends the request to Fiddler, which forwards it to the origin server. This adds upon the request time.

Make sure that you use the same HttpClient object for each async HttpRequest which will prevent it from overlapping the requests