We Keep Coding

.net httpclient Charset encoding is different than Fiddler - c#

I'm currently working with an api that requires me to send a json in UTF-32 encoding format.
I am using HttpClient lib but it does not encode my json the way Fiddler does so the server returns error. I've googled for hours but without success, anyway here's my raw request (Fiddler):
POST http://example.com/ HTTP/1.1
Content-Type: application/json; charset=utf-32
Host: example.com
Content-Length: 92
Expect: 100-continue
Connection: Keep-Alive
{"jsonKey":"JsonValue"}
The encoded content looks like this:
and here's my simple C# request:
using(HttpClient client = new HttpClient(new HttpClientHandler() { Proxy = new WebProxy("127.0.0.1:8080") }))
{
_ = await client.PostAsync("http://example.com", new StringContent("{\"jsonKey\":\"JsonValue\"}", Encoding.UTF32, "application/json")).ConfigureAwait(false);
}
C#'s content looks like this:

Related

I'm trying to "repurpose" a third-party API used by a desktop application. I've found that the below code gets me very close to matching the packets sent by the app:
var formData = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>(JsonConvert.SerializeObject(myPayload), "")
});
var response = Client.PostAsync(myURL, formData).Result;
var json = response.Content.ReadAsStringAsync().Result;
This gets me almost exactly the same payload sent by the application, except it encodes the data (I know, "encoded" is right there in the name). I need to get the exact same request but without the data being encoded, but I can't quite find the right object(s) to pull it off. How do I keep this payload from being URL encoded?
Edit:
This is a login request I pulled from Wireshark emanating from the application:
POST /Login HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: 1.1.1.1
Content-Length: 161
Expect: 100-continue
Connection: Close
{"username":"myuser","auth-id":"0a0a140f81a2ce0c303386e93cec41bf04660c22a881be9a"}
This is what the above will generate:
POST /Login HTTP/1.1
Expect: 100-continue
Connection: Close
Content-Type: application/x-www-form-urlencoded
Content-Length: 221
Host: 1.1.1.1
%7B%22user-name%22%3A%22myuser%22%2C%22auth-id%22%3A%220a0a140f81a2ce0c303386e93cec41bf04660c22a881be9a%22%7D=
I've edited them for brevity so the Content-Length is wrong. I realize it might not be the best way to send this data, but I have no control over how it's consumed.

Since you're actually trying to send JSON, I think you need to wrap the JSON in a StringContent object rather than a FormUrlEncoded object. Form-encoded data and JSON data are two different ways of formatting a payload (another commonly used format would be XML, for example). Using them both together doesn't make any sense.
I think something like this should work:
var content = new StringContent(JsonConvert.SerializeObject(myPayload), Encoding.UTF8, "application/json");
var response = Client.PostAsync(myURL, content).Result;
var json = response.Content.ReadAsStringAsync().Result;
(P.S. the Content-Type: application/x-www-form-urlencoded header sent by the application appears to be misleading, since the request body clearly contains JSON. Presumably the receiving server is tolerant of this nonsense, or just ignores it because it's always expecting JSON.)

I receive the following response when trying to consume text/plain:
{
"type": "https://tools.ietf.org/html/rfc7231#section-6.5.13",
"title": "Unsupported Media Type",
"status": 415,
"traceId": "|b28d0526-4ca38d2ff7036964."
}
Controller definition:
[HttpPost]
[Consumes("text/plain")]
public async Task<IActionResult> PostTrace([FromBody]string body)
{ ... }
HTTP message:
POST /api/traces HTTP/1.1
Content-Type: text/plain
User-Agent: PostmanRuntime/7.19.0
Accept: */*
Cache-Control: no-cache
Postman-Token: 37d27eb6-92a0-4a6a-8b39-adf2c93955ee
Host: 0.0.0.0:6677
Accept-Encoding: gzip, deflate
Content-Length: 3
Connection: keep-alive
I am able to consume JSON or XML just fine. What am I missing?

Reference: Accepting Raw Request Body Content in ASP.NET Core API Controllers:
Unfortunately ASP.NET Core doesn't let you just capture 'raw' data in any meaningful way just by way of method parameters. One way or another you need to do some custom processing of the Request.Body to get the raw data out and then deserialize it.
You can capture the raw Request.Body and read the raw buffer out of that which is pretty straight forward.
The easiest and least intrusive, but not so obvious way to do this is to have a method that accepts POST or PUT data without parameters and then read the raw data from Request.Body:
[HttpPost]
[Route("api/BodyTypes/ReadStringDataManual")]
public async Task<string> ReadStringDataManual()
{
using (StreamReader reader = new StreamReader(Request.Body, Encoding.UTF8))
{
return await reader.ReadToEndAsync();
}
}
Request:
POST http://localhost:5000/api/BodyTypes/ReadStringDataManual HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/plain
Host: localhost:5000
Content-Length: 37
Expect: 100-continue
Connection: Keep-Alive
Windy Rivers with Waves are the best!

I'm trying to POST a request with one lonely parameter, as such:
var client = new RestClient("http://www.fluff.com");
var request = new RestRequest("whatever", Method.POST);
request.AddParameter("param", "Оксана");
client.Execute(request);
This results in the following request, notice the bunch of encoded question marks:
POST http://www.fluff.com/whatever HTTP/1.1
Accept: application/json, application/xml, text/json, text/x-json, text/javascript, text/xml
User-Agent: RestSharp/105.0.1.0
Content-Type: application/x-www-form-urlencoded
Host: www.fluff.com
Content-Length: 24
Accept-Encoding: gzip, deflate
param=%3F%3F%3F%3F%3F%3F
Imagine the sadness when the receiver gets those question marks..
How do I make RestSharp properly encode the body as UTF-8, or how do I send the request in a RestSharp friendly way so that she doesn't garble the data?

Christer, that's standard encoding for Content-Type: application/x-www-form-urlencoded, which uses ISO-8859-1 as a default. If you specifically want to tell the server to expect UTF-8, you can add ; charset=UTF-8 at the end Content-Type: application/x-www-form-urlencoded ; charset=UTF-8. But then it's your responsibility to make sure the data you post is really encoded in UTF-8.
Or if you want to do it in "application/json", you can set the content type in this way (I got this from http://itanex.blogspot.com/2012/02/restsharp-and-advanced-post-requests.html):
request.AddHeader("Accept", "application/json");
request.Parameters.Clear();
request.AddParameter("application/json", strJSONContent, ParameterType.RequestBody);
You could also use multipart/form-data: <form action="YOUR_ACTION_NAME_HERE" method="post" enctype="multipart/form-data">

I am trying to call a REST service from a C# ASP.NET 4.0 application using RestSharp.
It's a fairly straightforward POST call to a https:// address; my code is something like this (CheckStatusRequest is a plain simple DTO with about four or five string and int properties - nothing fancy):
public CheckStatusResponse CheckStatus(CheckStatusRequest request) {
// set up RestClient
RestClient client = new RestClient();
string uri = "https://.......";
// create the request (see below)
IRestRequest restRequest = CreateRequestWithHeaders(url, Method.POST);
// add the body to the request
restRequest.AddBody(request);
// execute call
var restResponse = _restClient.Execute<CheckStatusResponse>(restRequest);
}
// set up request
private IRestRequest CreateRequestWithHeaders(string uri, Method method) {
// define request
RestRequest request = new RestRequest(uri, method);
// add two required HTTP headers
request.AddHeader("Accept", "application/json");
request.AddHeader("Content-Type", "application/json");
// define JSON as my format
request.RequestFormat = DataFormat.Json;
// attach the JSON.NET serializer for RestSharp
request.JsonSerializer = new RestSharpJsonNetSerializer();
return request;
}
The problem I'm having when I send these requests through Fiddler to see what's going on is that my request suddenly gets a third and unwanted HTTP header:
POST https://-some-url- HTTP/1.1
Accept: application/json
User-Agent: RestSharp/104.4.0.0
Content-Type: application/json
Host: **********.com
Content-Length: 226
Accept-Encoding: gzip, deflate <<<=== This one here is UNWANTED!
Connection: Keep-Alive
I suddenly have that Accept-Encoding HTTP header, which I never specified (and which I don't want to have in there). And now my response is no longer proper JSON (which I'm able to parse), but suddenly I get back gzipped binary data instead (which doesn't do real well when trying to JSON-deserialize)....
How can I get rid of that third unwanted HTTP header?
I tried to set it to something else - whatever I enter just gets appended to those settings
I tried to somehow "clear" that HTTP header - without any success
I tried finding a property on the RestClient or the RestRequest classes to specify "do not use GZip"

Looking at the sources (Http.Sync.cs and Http.Async.cs) of RestSharp you can see that these values are hardcoded:
webRequest.AutomaticDecompression =
DecompressionMethods.Deflate | DecompressionMethods.GZip | DecompressionMethods.None;
There is also an open issue that describes this problem. It was opened August 2014 but still not solved. I think you can leave a comment there and maybe they will pay attention.

I have a HTTP handler which receives a HTTP POST. HTTP POST message with header is -
POST /ibe/example.ashx HTTP/1.1
Content-Length: 41
Content-Type: application/x-www-form-urlencoded; text/html; charset=GBK
Host: 202.177.46.142
User-Agent: Mozilla/4.0
param1=value1&param2=value2&param3=value3
Handler in my code is -
var V1 = context.Request["param1"];
var V2 = context.Request["param2"];
But values returning are null

if it's a post I think you'd want to use a stream reader to get the body, then deserialize if needed.