I am making a simple HttpClient call like this:
Uri basePath = new Uri("https://my-host.com/");
string path = "api/my-path";
using (HttpClient httpClient = new HttpClient())
{
httpClient.BaseAddress = basePath;
HttpResponseMessage response = await httpClient.GetAsync(path);
if (response.IsSuccessStatusCode)
{
response.Headers.Select(header => header.Key).Dump();
}
}
I am looking for a custom auth-header on the response but it is missing when I iterate over Headers collection.
However, same Http request caught by Fiddler shows the header.
If I make same request on Postman or any browser, I see the header.
Wondering what am I missing here.
[Update]
Raw Headers captured on Fiddler by executing same code above:
HTTP/1.1 302 Found
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Location: <redacted>
Server: Microsoft-IIS/8.0
WWW-Authenticate-Test: <redacted>
X-AspNet-Version: 4.0.30319
Set-Cookie: <redacted>
X-UA-Compatible: IE=edge
Date: Sat, 11 Nov 2017 23:15:19 GMT
Content-Length: 0
Headers printed by above code:
Pragma
Strict-Transport-Security
X-Content-Type-Options
X-Frame-Options
x-ms-request-id
Cache-Control
P3P
Set-Cookie
Server
X-Powered-By
Date
I would like to capture the header WWW-Authenticate-Test which is somehow filtered out while it goes through the HttpClient magic.
It's probably a content header:
response.Content.Headers.Select(header => header.Key).Dump();
Related
I am trying to send a get request to a third-party API using HttpClient. However, I keep getting 400. The request succeeds with curl. Is this something related to headers?
using (var client = new HttpClient())
{
string endpoint = "https://api.exchange.coinbase.com/products/BTC-USDT/trades/?limit=100";
var response = await client.GetAsync(endpoint);
Console.WriteLine(response); //Bad Request
....
}
The response I am getting:
StatusCode: 400, ReasonPhrase: 'Bad Request', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
{
Date: Mon, 06 Feb 2023 19:10:26 GMT
Connection: keep-alive
CF-Cache-Status: DYNAMIC
Server: cloudflare
CF-RAY: 79562f95dcce0531-OTP
Content-Type: application/json; charset=utf-8
Content-Length: 44
}
With the command below, I get a valid response:
curl -X "GET" "https://api.exchange.coinbase.com/products/BTC-USDT/trades/?limit=100"
I am trying to getting data from this github api link https://api.github.com/users/arif2009 . It works fine using posman get request.
But if i try to get data using .net HttpClient then it says Forbidden.
C# code:
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("https://api.github.com/");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var response = await client.GetAsync("users/arif2009");
if (response.IsSuccessStatusCode)
{
var data = response.Content.ReadAsAsync<GithubUser>();
}
}
Response:
Can anybody tell me where i made the mistake?
I took a look at your requests using Telerik's Fiddler and found the following.
Request with your code:
GET https://api.github.com/users/arif2009 HTTP/1.1
Accept: application/json
Host: api.github.com
Request from Postman:
GET https://api.github.com/users/arif2009 HTTP/1.1
Host: api.github.com
Connection: keep-alive
Accept: application/json
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
Postman-Token: d560ee28-b1e8-ece5-2612-87371ddcb295
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,ja-JP;q=0.8,ja;q=0.7,en-US;q=0.6
The obvious missing header seemed to be "User-Agent", so I added this:
client.DefaultRequestHeaders.UserAgent.Add(new ProductInfoHeaderValue("product", "1")); // set your own values here
Which produced the following request:
GET https://api.github.com/users/arif2009 HTTP/1.1
Accept: application/json
User-Agent: product/1
Host: api.github.com
And returned the following response:
HTTP/1.1 200 OK
Date: Wed, 17 Jul 2019 15:19:35 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 1249
Server: GitHub.com
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 52
X-RateLimit-Reset: 1563380375
Cache-Control: public, max-age=60, s-maxage=60
Vary: Accept
ETag: "1df3e0be6e824ca684f27963806533da"
Last-Modified: Tue, 16 Jul 2019 05:58:59 GMT
X-GitHub-Media-Type: github.v3
Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Content-Security-Policy: default-src 'none'
Vary: Accept-Encoding
X-GitHub-Request-Id: D5E0:8D63:8BE92C:A799AE:5D2F3C86
{"login":"arif2009","id":6396346,"node_id":"MDQ6VXNlcjYzOTYzNDY=","avatar_url":"https://avatars0.githubusercontent.com/u/6396346?v=4","gravatar_id":"","url":"https://api.github.com/users/arif2009","html_url":"https://github.com/arif2009","followers_url":"https://api.github.com/users/arif2009/followers","following_url":"https://api.github.com/users/arif2009/following{/other_user}","gists_url":"https://api.github.com/users/arif2009/gists{/gist_id}","starred_url":"https://api.github.com/users/arif2009/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arif2009/subscriptions","organizations_url":"https://api.github.com/users/arif2009/orgs","repos_url":"https://api.github.com/users/arif2009/repos","events_url":"https://api.github.com/users/arif2009/events{/privacy}","received_events_url":"https://api.github.com/users/arif2009/received_events","type":"User","site_admin":false,"name":"Arif","company":"#BrainStation-23 ","blog":"https://arif2009.github.io/","location":"Bangladesh","email":null,"hireable":true,"bio":"Software Engineer | Full Stack | Web Developer | Technical Writer","public_repos":15,"public_gists":2,"followers":9,"following":7,"created_at":"2014-01-14T05:03:47Z","updated_at":"2019-07-16T05:58:59Z"}
this API reference from github says that 'User-Agent' is a required header:
All API requests MUST include a valid User-Agent header. Requests with no User-Agent header will be rejected.
Postman automatically adds its own User-Agent to the call when this is not provided by the user. (This is nicely demonstrated by #John's answer).
simply adding this header will resolve your issue:
client.DefaultRequestHeaders.UserAgent.Add(new ProductInfoHeaderValue("yourAppName", "yourVersionNumber"));
You may try to put the complete URI into the GetAsync instead using the BaseAddress, as you do not give the GetAsync an URI, but only a string.
When sending a HttpRequestMessage with a relative Uri, the message Uri will be added to the BaseAddress property to create an absolute Uri.
https://learn.microsoft.com/de-de/dotnet/api/system.net.http.httpclient.baseaddress?view=netframework-4.8
I'm trying to implement a third party service on my project and i need to do some authentication to consume their API.
To do so i need to execute a POST request on their auth url, the problem is when i execute the request their server sends a response with a 302 and the Location header, now the weird part:
I need to get this location link and read the query string to get the information i need.
I have already tried everything i could think of, even Postman wont work with it.
This is my code:
var client = new RestClient(APIBase + "/portal/rest/oauth2/login");
var request = new RestRequest(Method.POST);
request.AddHeader("content-type", "application/x-www-form-urlencoded");
request.AddHeader("cache-control", "no-cache");
request.AddParameter("application/x-www-form-urlencoded", $"user={User}&password={Password}&client_id={ClientId}", ParameterType.RequestBody);
var content = client.Execute(request);
This is the response headers i managed to get using curl on the command prompt:
HTTP/1.1 302 Moved Temporarily
Date: Wed, 26 Jul 2017 17:59:32 GMT
Server: Apache-Coyote/1.1
Location: LOCATION-LINK
Content-Length: 0
HTTP/1.1 200 OK
Date: Wed, 26 Jul 2017 17:59:32 GMT
Server: Apache-Coyote/1.1
Accept-Ranges: bytes
ETag: W/"20095-1497998126000"
Last-Modified: Tue, 20 Jun 2017 22:35:26 GMT
Content-Type: text/html
Content-Length: 20095
Vary: Accept-Encoding
Is it possible ?
You can disable RestSharp redirects with:
client.FollowRedirects = false;
And then just check that response status code is 302 and read Location header.
For new versions of RestSharp:
var client = new RestClient(new RestClientOptions(url) { FollowRedirects = false });
Using ASP.NET Web API 2, if I want to return HttpResponseMessage with Unauthorized status code, I'll get different response headers - with or without WWW-Authenticate header field - depending on whether this response message has content or not.
WWW-Authenticate header field is a required field for Unauthorized response according to Status Code Definitions.
And the lack of WWW-Authenticate header field in the response causes an error for the next request.
To see the problem you can create a new Web API project and to add a simple test controller:
public class TestController : ApiController
{
public HttpResponseMessage Get()
{
var responseMessage = new HttpResponseMessage(HttpStatusCode.Unauthorized)
{
// Content = new StringContent("You are not authorized"),
};
return responseMessage;
}
}
If the response message doesn't have a content we'll get normal 401 responses for our calls:
With responses:
HTTP/1.1 401 Unauthorized
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
WWW-Authenticate: Bearer
X-SourceFiles: =?UTF-8?B?RDpcUHJvamVjdHNcVGVzdEFwaVxUZXN0QXBpXGFwaVxUZXN0?=
X-Powered-By: ASP.NET
Date: Fri, 01 May 2015 05:31:20 GMT
Content-Length: 0
If we add content to the response message (uncomment the content line), each second response will be not 401, but 500
The responses with 401 will have the following headers:
HTTP/1.1 401 Unauthorized
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 22
Content-Type: text/plain; charset=utf-8
Expires: -1
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-SourceFiles: =?UTF-8?B?RDpcUHJvamVjdHNcVGVzdEFwaVxUZXN0QXBpXGFwaVxUZXN0?=
X-Powered-By: ASP.NET
Date: Fri, 01 May 2015 05:34:38 GMT
You are not authorized
And the responses with 500 will say
Server cannot append header after HTTP headers have been sent.
Description: an unhandled exception occurred during the execution of the current web request.Please review the stack trace for more information about the error and where it originated in the code..... Exception Details: System.Web.HttpException: Server cannot append header after HTTP headers have been sent.
So it looks like it happens because previous responses are illegal - they don't contain WWW-Authenticate header attribute.
But it doesn't help even if I try to add WWW-Authenticate manually
public class TestController : ApiController
{
public HttpResponseMessage Get()
{
var responseMessage = new HttpResponseMessage(HttpStatusCode.Unauthorized)
{
Content = new StringContent("You are not authorized"),
};
responseMessage.Headers.Add("WWW-Authenticate", "Bearer");
return responseMessage;
}
}
Now it's in the response
HTTP/1.1 401 Unauthorized
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 22
Content-Type: text/plain; charset=utf-8
Expires: -1
Server: Microsoft-IIS/8.0
WWW-Authenticate: Bearer
X-AspNet-Version: 4.0.30319
X-SourceFiles: =?UTF-8?B?RDpcUHJvamVjdHNcVGVzdEFwaVxUZXN0QXBpXGFwaVxUZXN0?=
X-Powered-By: ASP.NET
Date: Fri, 01 May 2015 05:43:51 GMT
You are not authorized
but I still have every second request 500 instead of 401.
Can anyone clarify what's going on here and how to make it work properly?
The Server cannot append header after HTTP headers have been sent. error message from the response with 500 is caused by a module trying to write http headers into the response stream once your action has executed - since your action has already written both the headers and the body.
The usual culprits are FormsAuthentication and Owin cookie authentication kicking in and trying to replace 401 with 302 (redirect to login page).
Scenario:
1. your action executed - added header 401 Unauthorized to the headers collection plus wrote the header into the response stream and then wrote the body (You are not authorized) to the response stream.
2. FormsAuthenticationModule sees 401 Unauthorized header in the response headers collection and attempts to change that to 302 Redirect to login page, plus attempts to write the corresponding header to the response stream - which fails with Server cannot append header after HTTP headers have been sent., since the headers were already written earlier.
Possible fixes:
1) if FormsAuthenticationModule is in your request pipeline
consider disabling it if you are not using it
in .net 4.5 you you can suppress forms authentication redirection by setting HttpResponse.SuppressFormsAuthenticationRedirect property to false
if not on .net 4.5, consider using HttpModule which will suppress this redirection
Phil Haack blogged on all the above options.
2) if using OWIN cookie authentication - consider not setting LoginPath property
I am trying to access google drive from my app for WP7. But when i try to get access token in exchange for Authorization code, I get BAD REQUEST from server.
My POST request as seen in Fidler:
POST https://accounts.google.com/o/oauth2/token HTTP/1.1
Accept: */*
Referer: file:///Applications/Install/7128457C-3AF4-41C4-A606-742068B1463F/Install/
Content-Length: 240
Accept-Encoding: identity
Content-Type: application/x-www-form-urlencoded
User-Agent: NativeHost
Host: accounts.google.com
Connection: Keep-Alive
Cache-Control: no-cache
code=<*Authorization_Code*>&
client_id=<*My_Client_Id*>&
client_secret=<*My_Client_Secret*>&
redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&
grant_type=authorization_code
Response from server:
HTTP/1.1 400 Bad Request
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Date: Sat, 07 Sep 2013 14:05:35 GMT
Content-Type: application/json
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Alternate-Protocol: 443:quic
Transfer-Encoding: chunked
21
{
"error" : "invalid_request"
}
0
My Code:
StringBuilder postData = new StringBuilder();
postData.AppendFormat("{0}={1}", "code", HttpUtility.UrlEncode(AuthorizationCode));
postData.AppendFormat("&\n{0}={1}", "client_id", HttpUtility.UrlEncode(ClientId));
postData.AppendFormat("&\n{0}={1}", "client_secret", HttpUtility.UrlEncode(ClientSecret));
postData.AppendFormat("&\n{0}={1}", "redirect_uri", HttpUtility.UrlEncode("urn:ietf:wg:oauth:2.0:oob"));
postData.AppendFormat("&\n{0}={1}", "grant_type", HttpUtility.UrlEncode("authorization_code"));
WebClient client = new WebClient();
client.UploadStringCompleted += TokenResponse;
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
client.UploadStringAsync(new Uri("https://accounts.google.com/o/oauth2/token",UriKind.Absolute), "POST", postData.ToString());
I get this result both on emulator as well as Lumia 820. I also tried without using HttpUtility in POST request but didn't work. Any help?
Its likely due to the fact that you are adding a new line via \n between all the param/value pairs.
I do it without that without it and it works - https://github.com/entaq/GoogleAppsScript/blob/master/IO2013/YouTubeAnalytics/oauth2.gs#L25