Below codes worked when the Uri = "https://someRestApi", but reports 401 Unauthorized when Uri = "http://SomeRestApi", why is it happening? The server is hosted in Azure App Service, with
TLS/SSL settings . Sending the http request with Postman does work.
public static void Main(string[] args)
{
var uri = new Uri("http://someTestApiUri.com");
var token = GetTestToken();
HttpWebRequest httpRequest = HttpWebRequest.Create(uri) as HttpWebRequest;
httpRequest.Headers["Authorization"] = "Bearer " + token.AccessToken;
using (HttpWebResponse response = httpRequest.GetResponse() as HttpWebResponse)
{
//Some logic here...
}
}
This maybe because the developer of the API may have specified that endpoint allows only https either through the CORS and origin specification or Attribute Filter. by decorating the ApiController to allow only https.
kindly read some of the articles below.
https://jonathancrozier.com/blog/how-to-configure-your-asp-net-web-apps-and-apis-to-require-https
https://www.c-sharpcorner.com/article/how-to-enable-https-in-asp-net-web-api/
there are many articles that will give you a clue but these are just few so it is not exhaustive.
so if the backend or whoever developed the endpoint specifies that only https endpoint should be hit, when you attempt to hit the http he may return the message or http statusCode 401 . again read more about http status code and what they mean.
Also, it may work in postman because you disabled a security settings on your postman. please try and use a secure channel when making API calls.
Thanks.
Regards
Authentication and authorization in Azure App Service and Azure Functions in the section about disabling requireHttps clearly states:
However, we do recommend sticking with HTTPS, and you should ensure no security tokens ever get transmitted over non-secure HTTP connections..
Related
I wanna send request to external API through the HttpClient from my own API in c#.
I am using dot net 5 and basic authentication.
Here is my code:
var client = new HttpClient
{
BaseAddress = new Uri(baseUrl)
};
HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Put, "apiUrl");
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var param = JsonConvert.SerializeObject(new
{
param1="",
param2=""
});
requestMessage.Content = new StringContent(param, Encoding.UTF8, "application/json");
requestMessage.Headers.Authorization = new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(Encoding.ASCII.GetBytes($"{user}:{pass}")));
HttpResponseMessage response = await client.SendAsync(requestMessage);
Usually, I send http request like this.
but now I have some problem.
After line HttpResponseMessage response = await client.SendAsync(requestMessage); authorization header removed from my request header and I got UnAuthorized http error.
I know that when redirection occurs, authorization header removed for security reason. Also I'm not sure about redirect in external API.
I add the HttpClientHandler with AllowAutoRedirect = false to my HttpClient
var handler = new HttpClientHandler()
{
AllowAutoRedirect = false,
};
var client = new HttpClient (handler)
{
BaseAddress = new Uri(baseUrl)
};
Now I got Redirect error 301(Permanently moved).
I decided to test the code in Postman. By default, when I call API in Postman, I got http error 405 method not allowed and error detail like this:
{
"detail": "Method "GET" not allowed."}
External API method is PUT. but here I got GET Error.
I tried many options in postman and finally I find the option in Postman:
When I toggle it on, external API work's properly.
Also I test it with Insomnia and it's work properly.
Does it related to my code or dot net 5 or what other thing in my code or it's related to external API?
If it's related to my code, How can I solve the error?
If error related to external API, why Postman and Insomnia response is ok?
External API has Core Policy for specific domain and I send request from other domain.
All I Know is that the CORS policy applied in browser. not in Postman, Insomnia or C# Code.
What about CORS? Does it related to CORS? if Yes, what shall I do?
I will be grateful for your answer.
Update
I detect WWW-Authenticate: JWT realm="api" in the response header.
What exactly is it? and what shall I do?
I find out the problem. It's really Ridiculous.
When I use URL like www.abc.com/api/something the request gets 301 error and postman sends another request like www.abc.com/api/something/. The difference is just / at the end of request.
I tried new URL in postman and first request got ok.
Also I tried URL in my C# code and again its ok.
But i could not understand why.
Thanks a lot dear #pharaz-fadaei
You are right about the removal of authorization headers after redirects. But keep in mind that this behavior is part of the design of the HttpClient in C#. Postman and Insomnia may have different mechanisms to send the authorization headers on each consecutive request that is caused by redirects. The option that you enabled in Postman will make it use the original HTTP method you specified (PUT) as the HTTP method to send further requests based on the redirect messages (Postman uses GET method by default on requests instructed by redirect messages).
The fact that you see a 301 shows that a redirection is required. You can check Location header value in response.Headers to see the real location and send your requests with the authorization headers to that endpoint directly. If I were you I wouldn't use the new location directly because the original endpoint is what you were given by the authors of the API. Instead I would programmatically send the request to the original endpoint and resend the request on 301 codes to the new Location (use PUT method due to the behavior of Postman) until you get the result. This answer can give you some ideas: https://stackoverflow.com/a/42566541/1539231
Since you see WWW-Authenticate: JWT realm="api" header, the external API is required a JWT token authentication, not basic authentication. I think first you might need to check external api's documentation.
I have this proprietary code I am working on for my job.
I am writing test cases for it because the code was changed and the test cases are now broken.
It is a C# web Api MVC .Net Framework app
I have a method that I enter a string url in
Then this code executes
HttpResponseMessage response = await Client.GetAsync(url).ConfigureAwait(true);
System.Uri uri = new System.Uri(url); // convert string to Uri
var cert = System.Net.ServicePointManager.FindServicePoint(uri).Certificate;
response.EnsureSuccessStatusCode();
when it gets to response.EnsureSuccessStatusCode() , it gives a 401 unauthorized and then throws an exception not allowing my test to pass
When I try the same thing with http://www.google.com which is not an https, then
it gives a 200. So something is going on with security stuff
What are the things I need to do to get a https to give a 200? Does it need username and password credentials or something or some other token of some sort?
Also, when I test it using Rest Client DHC with the secure https link that was failing above it gives me a 200. However, I had to refresh the bearer token for it to give a 200. If I used an old token it would give a 401.
Furthermoore, when I test a different link like https://www.facebook.com (which is not the one I want to test in my application, just troubleshooting) which is secure, it works giving me a 200 both in my application above and Rest Client DHC even with an old bearer token.
If you're trying to makes CORS requests to web API, you'll need to configure It to accept cross origins requests. If you dont configure your web api to accept cross origin requests, it'll throw these type of errors when calling It. And keep in mind that for web api access, https://www.domain.so and http://www.domain.so are completely diff clients.
Look at this link:
https://learn.microsoft.com/en-us/aspnet/web-api/overview/security/enabling-cross-origin-requests-in-web-api
I need to programmatically download a file from a SharePoint server.
When I download the file with Firefox it looks like a single request, but Httpfox shows that the HTTPS conversation is actually 4 requests:
REQ1: GET https://mycorp.raxsp.com/_windows/default.aspx?ReturnUrl=/personal/mycorp_user1/_vti_bin/cmis/rest?getRepositories
RESP1: 401 Unauthorized, WWW-Authenticate NTLM
REQ2: Authorization NTLM TlRMTVNTUAABAAAAB4IIAAAAAAAAAAAAAAAAAAAAAAA=
RESP2: 401, WWW-Authenticate NTLM TlRMTVNTUAACAAAACgAKADgAAAAFgokC+[...]
REQ3: Authorization NTLM TlRMTVNTUAADAAAAGAAYAIAAAAAYA[...]
RESP3: 302 Found, Set-Cookie FedAuth=77u/PD94bW[...], Location /personal/mycorp_user1/_vti_bin/cmis/rest?getRepositories
REQ4: GET /personal/mycorp_user1/_vti_bin/cmis/rest?getRepositories
RESP4: 200 OK, <download begins>
I tried downloading the file with a simple HttpWebRequest with user/password, but as expected I just get the error 401. I am considering implementing the whole 4 requests, computing challenges with the NTLM over HTTP authentication algorithm (spec), but that sounds very error-prone...
Is there a client-side library or a code snippet that does NTLM over HTTP authentication?
It is for an Open Source project, so must be Open Source, and preferably using HttpWebRequest.
No Kerberos/SSO/domains involved.
We download files from SharePoint all the time with this code using System.Net.WebClient
public static byte [] downloadSharepointFile (string url){
using (var client = new WebClient { Credentials = new NetworkCredential("username", "password", "domain") })
{
client.Headers.Add("Accept: application/json");
return client.DownloadData(url);
}
}
I have some basic questions on HTTP authentication
1) How does client know about Server's authentication type (Basic/Digest/NTLM) of HTTP ? is this configurable at HTTP Server side?
My Answer: Server will be set with which authentication type it has to perform with the client.
So our client API(in C# HttpWebRequest API) will automatically take care of it.
Best use of Wireshrk with applying HTTP filter; you will get source and Destination IP at Internet Protocol layer. And src and dest port at Transmission control protocol and athentication type at http layer.
2) If i place squid linux proxy in between client and Server; is there any need from my client code should know about authentication type of proxy also? or authentication type is only related to the end HTTP server?
My Answer: If squid proxy is placed in between Client and Server; it wont use HTTP authentication. It may use a) DB: Uses a SQL database b) LDAP: Uses the Lightweight Directory Access Protocol. c) RADIUS: Uses a RADIUS server for login validation. and etc..
So we have to mention proxy authentication credentials in HTTP Headers.
3) Using WireShark found that there are three request from Browser to Server to fulfill single request.
a) Browser sends a request without any authentication credentials; So server responded with 401 along with relam and nonce.
WWW-Authenticate: Digest realm="realm", qop="auth", nonce="MTM1OTYyMzkyNDU4MzpiOWM0OWY0NmMzMzZlMThkMDJhMzRhYmU5NjgwNjkxYQ=="\r\n <BR>
b) The second time Browser sends request with credentials, relam, nonce, cnonce; but still server responded with 401;
WWW-Authenticate: Digest realm="realm", qop="auth", nonce="MTM1OTYyMzk0OTAyMTo3Njk3MDNhZTllZDQyYzQ5MGUxYzI5MWY2MGU5ZDg0Yw==", stale="true"\r\n
c) The third time Browser send the same request with same credentials, relam, nonce, cnonce. This time Server sends 200 ok.
My Question is in the second and third time Browser send the same request; why Server failed at second time and success at the third time. Is this because of my server implementation ? (I am having REST Java server with SPRING Security filter).
I have C# HTTP client;
where in the first time HttpWebRequest is sent without credentials though the System.Net.NetworkCredentials is set; so clinet got 407 with relam, and nonce. The second time HttpWebRequest is scuccess. There is no third request from this client like browser.
Why this difference between the browser and C# client?
My Answer: I am still don't know what is happening here: Q3.
4) Now the real issue i am facing is When SQUID LINUX PROXY came in between our Client and HTTP Server, Browser did the same three request authentication and succeeded. However C# HttpWebRequest is failed (401) at the second request and reached the cache(exception){} block and not tried for the third time.
Could you please anyone clarify me how to resolve this issue in C# client when PROXY SERVER is in between?
Below code is doing the GET Request.
HttpWebRequest request = WebRequest.Create("url") as HttpWebRequest;
request.Credentials = new NetworkCredential(loginUserName, password);
WebResponse response = request.GetResponse();
Note that our request to proxy is send via TCP protocol not with HTTP protocol. Then from PROXY to SERVER is communicated with HTTP protocol.
But HTTP request from Proxy has the info about our client ip in the HTTP header X-Forwarded-For.
Below are the possible solutions
These solutions only required if your proxy requires any authentication else ignore it.
Solution 1: working for me
request.Proxy.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;
Solution 2:
IWebProxy proxy = WebRequest.GetSystemWebProxy();
proxy.Credentials = new NetworkCredential(UserName, UserPassword, UserDomain);
request.Proxy = proxy;
Solution 3 given by Martin:
var proxy = new WebProxy ("http://localhost:3128/");
proxy.Credentials = new NetworkCredential (UserName, UserPassword, UserDomain);
request.Proxy = proxy;
Learn more about proxy authentication at
http://wiki.squid-cache.org/Features/Authentication
It's a challenge/response protocol. Usually, the client makes an initial request without any authentication headers (you may set request.PreAuthenticate = true to send the credentials with the first request).
Then, the server responds with a list of authentication methods that it supports.
If the user did not explicitly specify an authentication method using CredentialsCache, the runtime will try them all, from strongest to weakest. Some protocols (NTLM, for instance) require multiple requests from client to server. In theory, Digest should work with a single one, no idea why it's sending the request twice.
Regarding your proxy question, there are two different kinds of authentication:
The target web server's authentication, a proxy simply passes this though and you don't need any special code in your client.
In addition to that, the proxy itself may also require authentication - which may be different from the one of target web server.
You specify these using
var proxy = new WebProxy ("http://localhost:3128/");
proxy.Credentials = new NetworkCredential ("username", "password");
and then
WebRequest.DefaultWebProxy = proxy;
or
request.Proxy = proxy;
Don't set any credentials on the WebProxy if your proxy server doesn't use any authentication.
If you can't get authentication working while using a proxy server, look at the actual requests that are being sent between the three parties (web server, proxy, client) with Wireshark.
I am creating a client-server application using WCF REST services (.Net 4.0) and I see really strange error there. To access the service I use HttpClient class, the code looks like:
public TResponse Post<TRequest, TResponse>(Uri uri, TRequest request)
using (HttpClient client = new HttpClient())
{
client.TransportSettings.Credentials = CredentialCache.DefaultCredentials;
client.TransportSettings.ConnectionTimeout = TimeSpan.FromSeconds(200);
using (HttpResponseMessage response = client.Post(uri, HttpContentExtensions.CreateDataContract(request, null)))
{
response.EnsureStatusIsSuccessful();
return response.Content.ReadAsDataContract<TResponse>();
}
}
}
I host my service on IIS7 and when anonymous auth is enabled everything works well, but when I turn on windows auth some random requests (in average 1 of 10) get failed with 400-Bad request status and error message is : Bad Request - Invalid Verb. HTTP Error 400. The request verb is invalid.
What could it be?
And a little update: if fiddler is turned on everything works perfect.
I just ran into the same thing here. Quite a mystery really. I don't understand why having fiddler running removes the problem.
I eventually found that the issue was caused the by the "expect continue" header being added to the post. I removed that and the post succeeds instead of the 400 bad request.
Here's a good entry about it: http://blogs.msdn.com/b/jaskis/archive/2009/04/19/asmx-post-request-fails-with-http-400-error-when-content-length-size-increases.aspx
In my case this was apache HTTP Client v4.0 so,
httpClient.getParams().setBooleanParameter(
CoreProtocolPNames.USE_EXPECT_CONTINUE, false);
did the trick.