I'm trying to send HTTP requests in C# that look like HTTP requests from a certain software. I wanted to use System.Net.HttpWebRequest but it doesn't give me the control I need over its headers: their letter-casing can't be changed (e.g. I want the Connection header to be keep-alive and not Keep-Alive), I don't have full control over the headers ordering, etc.
I tried using HttpClient from CodeScales library. Unfortunately, it doesn't decompress responses automatically (see HttpWebRequest.AutomaticDecompression). I decompressed it myself with System.IO.Compression.GZipStream and DeflateStream, but it didn't work when the response had the header Transfer-Encoding: chunked.
System.Net.Http.HttpRequestHeaders seems to give more control over headers than HttpWebRequest, but still not enough.
How can it be done?
Edit: I know that HTTP accepts those headers as valid anyway, but I'm working with a server that validates the headers and refuses to respond if they're not exactly what it expects.
To set some headers in the HTTPWebRequest class, you have to either use an attribute from the class (for example HttpWebRequest.KeepAlive = true), or you have to add the custom header to the request by calling the add method to the request headers.
Something important is that is you try to add the header (in a custom way) while it's already an attribute of the request, it'll send you an error.
objRequest.Headers.Add("Accept", "some data");
is incorrect. You'd rather say.
objRequest.Accept = "some data";
In your case you can :
objRequest.KeepAlive = true;
Don't worry to much for the letter-casing, it doesn't matter as far as you're sending the appropriate headers to the server.
Related
I received a Postman json collection from an API vendor that works perfectly, but has something mystifying to me: The request is in a GET format, yet there is an x-www-form-urlencoded body.
URL: https://login.microsoftonline.com/d1e<secret>9563/oauth2/token
And when I look at the postman-generated c# code, the mystery continues:
var client = new RestClient("https://login.microsoftonline.com/d1e...d3/oauth2/token");
client.Timeout = -1;
var request = new RestRequest(Method.GET);
request.AddHeader("Content-Type", "application/x-www-form-urlencoded");
request.AddParameter("grant_type", "client_credentials");
request.AddParameter("client_id", "c06bb...79");
request.AddParameter("client_secret", "7~u...D");
request.AddParameter("resource", "https://vault.azure.net");
IRestResponse response = client.Execute(request);
Console.WriteLine(response.Content);
Note the AddParameter constructions for a GET call. To me, this must be a slight-of-hand for merely adding those values to the querystring. But when I look at the postman console I see:
In the postman console I would have expected to see those params appended to the url as a querystring, and then everything would have made sense. But you can see that it's a bonafide Request Body.
When I make GET calls in my c# code I like to use the simple yet solid WebClient object to call the DownloadString() method. But this method is only for GETs and there's no way to send a form-post style body, understandably.
Is postman truly sending a GET with all those values being appended to the url as a querystring? And should I do the same in my DownloadString() call? Or is there something else going on here? Should I instead, in my c#, be calling the UploadString() method and sending a form post BODY as a GET??
Http protocol supports adding a body to a request, but the WebClient class you use doesn't. Presumably because it isn't considered the norm.
I'm sure there's good reasons for Microsoft using it in the OAuth flow though. Those guys normally do things right!
HTTP GET with request body
API is just an abstraction , you can send what ever you want to the API . It depends on the implementation , how the server handles these data.
Some services considers only what it requires and ignores other information
some services considers the entire requests and validates that it has only the allowed data. what should be allowed depends on the service
Postman is just a client that sends data to server , its upto you to decide what all information it should send . If you dont need any body then keep it as none. if you need some thing then add it.
I am trying to add Content-Type header to HttpClient GET request, here my code:
HttpClient client=new ....
bool added = client.DefaultRequestHeaders.TryAddWithoutValidation("Content-Type", "application/x-www-form-urlencoded");
var response = await client.GetAsync(...
but the added variable is false, i.e it failed to add the header.
How can I add this header?
NOTE:
This post deals with POST request, I asked about GET
If you look at the Http/1.1 specification:
A sender that generates a message containing a payload body SHOULD
generate a Content-Type header field in that message unless the
intended media type of the enclosed representation is unknown to the
sender. If a Content-Type header field is not present, the recipient
MAY either assume a media type of "application/octet-stream"
([RFC2046], Section 4.5.1) or examine the data to determine its type.
Check also the MDN on get requests
The HTTP GET method requests a representation of the specified resource. Requests using GET should only retrieve data.
Sending body/payload in a GET request may cause some existing implementations to reject the request — while not prohibited by the specification, the semantics are undefined. It is better to just avoid sending payloads in GET requests.
Effectively, that means that wether you send or not the header, it's going to be ignored and/or rejected.
When setting the content type, it's better to set it from the content itself: How do you set the Content-Type header for an HttpClient request?
Im currently working on a project, where I call an api using a POST request.
This might help in your case. Its how its done in an official Microsoft Documentation.
using (var content = new ByteArrayContent(byteData))
{
// This example uses the "application/octet-stream" content type.
// The other content types you can use are "application/json"
// and "multipart/form-data".
content.Headers.ContentType = new mediaTypeHeaderValue("application/octet-stream");
response = await client.PostAsync(uriBase, content);
}
I'm constructing an NSUrlSession as follows:
NSUrlSessionConfiguration sessionCfg = NSUrlSessionConfiguration.CreateBackgroundSessionConfiguration("mySpecialSessionName");
NSUrlSessionDelegate sessionDelegate = new MySessionDelegate();
urlSession = NSUrlSession.FromConfiguration(sessionCfg, sessionDelegate, NSOperationQueue.MainQueue);
And invoking background downloads with custom HTTP headers:
NSMutableUrlRequest mutableRequest = new NSMutableUrlRequest();
mutableRequest.HttpMethod = "POST";
mutableRequest.Url = NSUrl.FromString(someEndpoint);
mutableRequest["MyCustomHeader"] = someStringWithUnicodeChars;
mutableRequest.Body = NSData.FromString(somePostBody);
NSUrlSessionDownloadTask downloadTask = m_UrlSession.CreateDownloadTask(mutableRequest);
downloadTask.Resume();
However, the header value string seems to get truncated at the first character above 255. For example, the header value:
SupeЯ Σario Bros
is received by the server as
Supe
When instead using .NET HttpClient on xamarin, unicode header strings successfully make it to the server unmodified. However, I'd like to make use of NSUrlSession's background downloading feature.
(I realize that support of unicode in HTTP headers is hit-and-miss, but since the HTTP server in this case is a particular custom server that doesn't currently support things like base64 encoding, passing the raw string is desired)
I don't know whether you'll be able to make that work, but two things come to mind:
What you have here is equivalent to calling setValue:forKey: on the URL request. I don't think that will do what you're expecting. Try calling the setValue:forHTTPHeaderField: method instead.
Try specifying the encoding before you specify your custom header value, e.g. [theRequest setValue:#"...; charset=UTF-8" forHTTPHeaderField:#"Content-Type"];
If neither of those helps, you'll probably have to encode the data in some way. I would suggest using URL encoding, because that's a lot simpler to implement on the server side than Base64. For the iOS side, see this link for info on how to URL-encode a string:
https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/URLLoadingSystem/WorkingwithURLEncoding/WorkingwithURLEncoding.html
I am a bit confused about PUSH PROMISE http/2 header handling in .NET4.6.
When I look HttpResponse.PushPromise there are two overloads:
One that accepts path to resource public void PushPromise(string path) - am assuming resource is then read and binary sent across to client.
Second public void PushPromise(string path, string method, NameValueCollection headers) that accepts sting method and NameValueCollection headers which I am failing to understand.
Why would I want to pass method (assuming HttpMethod like GET, POST, etc) and collection of headers inside PUSH PROMISE header?
From reading the HTTP/2 spec (Section 8.2), here is what I gather:
Passing the method
PUSH_PROMISE frames are required to be cacheable and safe. You have the option of using GET and HEAD, as those are the only two http methods that are defined as both safe and cacheable.
Passing headers
Since PUSH_PROMISE frames are required to be cacheable, this could be used to add specific Cache-Control directives to the promise. Section 8.2.2 of the spec states that a client has the option to download the promised stream and can refuse it, which I imagine a client would do if it found that it had an up-to-date version of the resource in its cache.
Controlling caching is the most obvious reason I can see for why you might pass headers, but there may be other reasons as well. If you're writing a custom client, you may use certain X-Headers to provide other hints (that aren't related to caching) to the client so it can decide whether or not it wants to accept the promised stream.
You'll want to pass headers for anything that will cause your response to vary (i.e. anything in your Vary response header). The biggest one I've found is compression.
Read those headers from the original client request and include them with your push promise, e.g.:
var headers = new NameValueCollection { { "accept-encoding", this.Request.Headers["accept-encoding"] } };
this.Response.PushPromise("~/Scripts/jquery.js", "GET", headers);`
I have the following code with which I download a web-page into a byte array and then print it with Response.Write:
WebClient client = new WebClient();
byte[] data = client.DownloadData(requestUri);
/*********** Init response headers ********/
WebHeaderCollection responseHeaders = client.ResponseHeaders;
for (int i = 0; i < responseHeaders.Count; i++)
{
Response.Headers.Add(responseHeaders.GetKey(i), responseHeaders[i]);
}
/***************************************************/
Besides of the response headers, I need to add request headers as well. I try to do it with the following code:
/*********** Init request headers ********/
NameValueCollection requestHeaders = Request.Headers;
foreach (string key in requestHeaders)
{
client.Headers.Add(key, requestHeaders[key]);
}
/***************************************************/
However it does not work and I get the following exception:
This header must be modified using the appropriate property.Parameter name: name
Could anybody help me with this? What's the correct way of adding request headers with WebClient?
Thank you.
The headers collection "protects" some of the possible headers as described on the msdn page here: http://msdn.microsoft.com/en-us/library/system.net.webclient.headers.aspx
That page seems to give all the answer you need but to quote the important part:
Some common headers are considered restricted and are protected by the
system and cannot be set or changed in a WebHeaderCollection object.
Any attempt to set one of these restricted headers in the
WebHeaderCollection object associated with a WebClient object will
throw an exception later when attempting to send the WebClient
request.
Restricted headers protected by the system include, but are not
limited to the following:
Date
Host
In addition, some other headers are also restricted when using a
WebClient object. These restricted headers include, but are not
limited to the following:
Accept
Connection
Content-Length
Expect (when the value is set to "100-continue"
If-Modified-Since
Range
Transfer-Encoding
The HttpWebRequest class has properties for setting some of the above
headers. If it is important for an application to set these headers,
then the HttpWebRequest class should be used instead of the WebRequest
class.
I suspect the reason for this is that many of the headers such as Date and host must be set differently on a different request. You should not be copying them. Indeed I would personally probably suggest that you should not be copying any of them. Put in your own user agent - If the page you are getting relies on a certain value then I'd think you want to make sure you always send a valid value rather than relying on the original user to give you that information.
Essentially work out what you need to do rather than finding something that works and doing that without fully understanding what you are doing.
Looks like you're trying to set some header which is must be set using one of the WebClient properties (CachePolicy, ContentLength or ContentType)
Moreover, it's not very good to blindly copy all the headers, you need to get just those you really need.