HttpWebRequest.Headers are missing some headers - c#

I iam working on a tool that let users check their API. One of the features is to show the actual send request headers.
Iam having trouble getting these headers though as the Headers property doesnt seem to include them all. I tried looking at tracelisteners but these seem to be more oriented to debugging and the config is global so it applies to all webrequests send by the app which is not what I want.
When I run this code on net48 (in core I seem to get 0 headers back):
// Create a new 'HttpWebRequest' Object to the mentioned URL.
HttpWebRequest myHttpWebRequest=(HttpWebRequest)WebRequest.Create("http://www.contoso.com");
// Assign the response object of 'HttpWebRequest' to a 'HttpWebResponse' variable.
HttpWebResponse myHttpWebResponse=(HttpWebResponse)myHttpWebRequest.GetResponse();
Console.WriteLine("\nThe HttpHeaders are \n\n\tName\t\tValue\n{0}",myHttpWebRequest.Headers);
I get the following output
The HttpHeaders are
Name Value
Host: www.microsoft.com
However in fiddler and with trace listeners I see these headers:
Host: www.contoso.com
Connection: Keep-Alive
Why can't I see the Connection header?

Now I see there is some redirecting going on. WebRequest seems to only show the headers send in the LAST request which didn't had the Connection header.

Related

Flurl get request with "Content-Type" header?

The API I am tasked with consuming requests a "Content-Type" with value "application/json" along with method/verb "GET". When I attempt using Flurl, I get response "ProtocolViolationException: Cannot send a content-body with this verb-type.".
Is there a way to do this? I was also trying with HttpClient (see related SO post). (btw.. using .NET Framework 4.5).
The reason why in you've got this exception is that the Content-Type header is meant to be used for the POST/PUT request payloads.
From MDN:
The Content-Type entity header is used to indicate the media type of the resource.
HTTP GET method has no payload, and there is no sense to have Content-Type in the request.
In Furl, if you use Content-Type: application/json with POST or PUT, make sure to provide actual payload: null is not an option.

c# httpclient PostAsJson sending GET request instead of POST

I am using HttpClient to make a post request. I get back 405 method not allowed. When capturing a trace in fiddler, it goes out as GET instead of POST!
using (var client = new HttpClient())
{
var url = AppSettingsUtil.GetString("url");
var response = client.PostAsJsonAsync(url, transaction).Result;
}
I am aware of the async/await issues. This is a simplified sample to show the issue.
Is there some sort of web.config or machine.config setting that could be affecting this? Other requests (sent through RestSharp) send Posts correctly
Here is what fiddler captures. Rerunning the trace in fiddler also returns the 405 (as expected). Manually switching it to POST and running works from fiddler.
Also, perhaps because the method was switched to GET, there is no body captured in fiddler, I had to manually paste in the JSON
GET /*URL*/ HTTP/1.1
Content-Type: application/json; charset=utf-8
Host: /*host*/
Connection: Keep-Alive
The problem appears to be that someone changed the URL without telling us, and they put a redirect in place. HttpClient is responding to the redirect, but ends up actually sending the request to the final destination as a Get.
This seems like a bug in HttpClient to me, that it should either send the ultimate request as a Post, or throw an exception saying it can't do what I asked it to.
See Forwarding a response from another server using JAX-RS

HttpClient with WebRequestHandler returns invalid cached response

We have stumble unto a problem while using WebRequestHandler and HttpRequestCachePolicy where the cached entry seems to get corrupted after a precise sequence. The scenario goes as follow:
request returns a 200 response with Cache-Control: public, must-revalidate, max-age=0 and Last-Modified.
a second request is made with Last-Modified-Since that returns a 500 response with Cache-Control: private. In our case, it is caused by the IIS server when an error occurs during the request.
a third request is sent. It contains a Last-Modified-Since header which causes the server to respond with 304.
The HttpClient result resolves to the 500 response.
My assumption is that the Last-Modified-Since header in the 3rd request should not have been present and is a bug.
The test was made in a sample console application in .NET 4.5 while using the HttpRequestCacheLevel.Default cache policy. Trying the same test in a browser doesn't seem to reproduce the problem when navigating to the URL (I verified that max-age=0 was not present in the request).
What I am trying to understand is if it's a bug in the .NET framework, if we didn't use the API correctly or if we didn't use the response headers correctly.

Error 406 Not Acceptable JSON

http://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API#ArtifactoryRESTAPI-CreateorReplaceRepositoryConfiguration
I am using the Create or Replace Repository Configuration call. However I am getting a 406 Error: Not Acceptable. Other PUT calls are working but do not return JSON. I believe JSON is the source of the error but have not been able to resolve or prove this.
I have added the code as below
RestClient Client = new RestClient(uriString);
RestRequest Request = new RestRequest(requestType);
Request.AddHeader("Authorization", "Basic " + credentials);
Request.AddHeader("Accept", "application/json");
I've seen threads where adding the header to accept JSON resolves the error but this has not worked for me.
A 406 HTTP status means that if a web server detects that the data it wants to return is not acceptable to the client, it returns a header containing the 406 error code.
The client can define the characteristics of the data it will accept back from the web server by using the accept headers.
In this case you declare the you would like to accept application/json:
Request.AddHeader("Accept", "application/json");
however the REST API method you are invoking is returning text/plain.
You should change the code to accept text/plain:
Request.AddHeader("Accept", "text/plain");
Wanted to add this for for future users stuck like me. I was having the same issue and tried the request with Postman and saw that the Content-Type was "application/hal+json" I was trying it with application/json without luck.
So running a test in postman I was able to figure out what the server needed exactly.
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/hal+json"));
I faced the same 406 Error: Not Acceptable when trying to get JSON on another site. In my case I could see correct JSON when typed url in my browser address field. But downloading it from the same url via my C# code have been producing 406 Error.
None of the answers in this topic solved my problem directly. But at least they pointed out to me that's the point is HTTP headers.
So I googled that page:
https://www.whatismybrowser.com/detect/what-http-headers-is-my-browser-sending
and added all browser headers to my code, and voila! It started to work.
In my case it was enough to fill some data in user-agent header.
First, the Accept header states what the client is ready to get back, not what the client sends.
The header that states what the client sends is Content-Type.
Also, this method does not accept application/json. As clearly stated in the docs, it accepts one of the following:
application/vnd.org.jfrog.artifactory.repositories.LocalRepositoryConfiguration+json
application/vnd.org.jfrog.artifactory.repositories.RemoteRepositoryConfiguration+json
application/vnd.org.jfrog.artifactory.repositories.VirtualRepositoryConfiguration+json

Passing multiple request headers to libcurl.net

I have recently started using libcurl.net with one of my projects as a replacement to the HttpWebRequest and HttpWebResponse classes. The reason I chose to use libcurl.net instead of the managed classes is that libcurl.net mimics the behavior of cURL from PHP and I was porting over some code from PHP. I attempted to use the built-in managed classes, but the CookieContainer class was not capturing all of the cookies correctly from the website that I was trying to capture cookies from. I may end up going back to the managed classes if I can figure out how to capture the cookies correctly.
My PHP script works perfectly fine in capturing cookies so I ported most of the cURL functionality using libcurl.net to my C# project. The problem I'm having is when I have to send more than one request header with the CURLOPT_HTTPHEADER cURL option and I have to use an Slist datatype to pass in more than one header like so:
Slist headers = new Slist();
headers.Append("Content-Type: application/x-www-form-urlencoded");
headers.Append("X-Requested-With: XMLHttpRequest");
easy.SetOpt(CURLoption.CURLOPT_HTTPHEADER, headers);
I sometimes have to fake an AJAX request but it does not seem to pass the X-Requested-With: XMLHttpRequest header with the request as the website I'm scraping does not return any results for these "fake" AJAX requests. If I set the CURLOPT_HTTPHEADER do I need to set the Content-Type header or is that always defaulted to Content-Type: application/x-www-form-urlencoded?
It turns out that I was adding multiple headers correctly. I simply made an Slist object and added my headers to the request using the CURLOPT_HTTPHEADER option. In this way, one can "fake" AJAX requests or any other type of request sent by a web browser. The problem was that I wasn't sending the correct POST data with my request.

Categories