UTF-8 URL Encode - c#

I am having issues in encoding my query params using HttpUtility.UrlEncode() it is not getting converted to UTF-8.
query["agent"] = HttpUtility.UrlEncode("{\"mbox\":\"mailto: UserName#company.com\"}");
I tried using the overload method and passed utf encoding but still it is not working.
expected result:
?agent=%7B%22mbox%22%3A%22mailto%3AUserName%40company.com%22%7D
Actual Result:
?agent=%257b%2522mbox%2522%253a%2522mailto%253aUserName%2540company.com%2522%257d
public StatementService(HttpClient client, IConfiguration conf)
{
configuration = conf;
var BaseAddress = "https://someurl.com/statements?";
client.BaseAddress = new Uri(BaseAddress);
client.DefaultRequestHeaders.Add("Custom-Header",
"customheadervalue");
Client = client;
}
public async Task<Object> GetStatements(){
var query = HttpUtility.ParseQueryString(Client.BaseAddress.Query);
query["agent"] = HttpUtility.UrlEncode( "{\"mbox\":\"mailto:UserName#company.com\"}");
var longuri = new Uri(Client.BaseAddress + query.ToString());
var response = await Client.GetAsync(longuri);
response.EnsureSuccessStatusCode();
using var responseStream = await response.Content.ReadAsStreamAsync();
dynamic statement = JsonSerializer.DeserializeAsync<object>(responseStream);
//Convert stream reader to string
StreamReader JsonStream = new StreamReader(statement);
string JsonString = JsonStream.ReadToEnd();
//convert Json String to Object.
JObject JsonLinq = JObject.Parse(JsonString);
// Linq to Json
dynamic res = JsonLinq["statements"][0].Select(res => res).FirstOrDefault();
return await res;
}

The method HttpUtility.ParseQueryString internally returns a HttpValueCollection. HttpValueCollection.ToString() already performs url encoding, so you don't need to do that yourself. If you do it yourself, it is performed twice and you get the wrong result that you see.
I don't see the relation to UTF-8. The value you use ({"mbox":"mailto: UserName#company.com"}) doesn't contain any characters that would look different in UTF-8.
References:
HttpValueCollection and NameValueCollection
ParseQueryString source
HttpValueCollection source

I strongly suggest you this other approach, using Uri.EscapeDataString method. This method is inside System.Net instead of System.Web that is a heavy dll. In addition HttpUtility.UrlEncode encode characters are in uppercase this would be an issue in certain cases while implementing HTTP protocols.
Uri.EscapeDataString("{\"mbox\":\"mailto: UserName#company.com\"}")
"%7B%22mbox%22%3A%22mailto%3A%20UserName%40company.com%22%7D"

Related

Rewriting a Java HttpURLConnection method using C# HttpClient

I've got a working Java method that uses java.net.HttpURLConnection that I should re-implement in C# using the .NET HttpClient.
Java method:
public static String getMyThingAPIToken() throws IOException{
URL apiURL = new URL("https://myThingAPI/token");
HttpURLConnection apiConnection = (HttpURLConnection) apiURL.openConnection();
apiConnection.setRequestMethod("POST");
apiConnection.setDoOutput(true);
String apiBodyString = "myThingAPI login id and secret key";
byte[] apiBody = apiBodyString.getBytes(StandardCharsets.UTF_8);
OutputStream apiBodyStream = apiConnection.getOutputStream();
apiBodyStream.write(apiBody);
StringBuffer apiResponseBuffer;
try (BufferedReader in = new BufferedReader(new InputStreamReader(apiConnection.getInputStream()))){
String inputline;
apiResponseBuffer = new StringBuffer();
while((inputline = in.readLine()) != null) {
apiResponseBuffer.append(inputline);
}
}
}
So far, my C# looks like below, and you'll notice that this early form of my implementation does not interpret the response. Nor does it have a string return type required for the token string.
This is because when I test it, the response has:
StatusCode: 400
ReasonPhrase: 'Bad Request'
So something in my apiBody byte array or use of PostAsync must be different to what the Java method does, but I cannot work out what it could be.
public async static Task<HttpResponseMessage> getMyThingAPIToken(HttpClient client)
{
var apiURI = new Uri("https://myThingAPI/token");
string apiBodystring = "myThingAPI login id and secret key";
byte[] apiBody = System.Text.Encoding.UTF8.GetBytes(apiBodystring);
var response = await client.PostAsync(apiURI, new ByteArrayContent(apiBody));
return response;
}
The Java code doesn't specify a type which means that by default the request uses application/x-www-form-urlencoded. This is used for FORM POST requests.
The default content type for ByteArrayContent on the other hand is application/octet-stream while for StringContent it's text/plain.
FORM content is used through the FormUrlEncoodedContent class which can accept any Dictionary<string,string> as payload.
The input in the question is not in a x-www-form-urlencoded form so either it's not the real content or the API is misusing content types.
Assuming the API accepts proper x-www-form-urlencoded content, the following should work:
var data=new Dictionary<string,string>{
["login"]=....,
["secret"]=.....,
["someOtherField"]=....
};
var content= new FormUrlEncodedContent(data);
var response=await client.PostAsync(apiURI,content);
To send any text using application/x-www-form-urlencoded, we need to specify the content type in StringContent's constructor:
var contentType="application/x-www-form-urlencoded";
var content= new StringContent(apiBodyString, Encoding.UTF8,contentType);
var response=await client.PostAsync(apiURI,content);
Can you try using following code:
client.BaseAddress = new Uri("https://myThingAPI/");
var message = new HttpRequestMessage(HttpMethod.Post, "/token");
// Add your login id and secret key here with the format you want to send
message.Content = new StringContent(string.Format("userName={0}&password={1}", UserName, Password));
var result = await client.SendAsync(message);
return result;

Decoding Http Response Content from Russian (Cyrillic)

I send a request to an API and the server sends some portion of its response in Russian. I url decode the response using code page 1251 encoding but still don't get the result I want.
How can I convert the response back to plain english? What encoding do I use?
If you just need to convert Russian letters (Cyrillic) to Latin ones you can use Dictionary structure with Cyrillic-Latin relationship.
var map = new Dictionary<char, string>
{
{ 'Ж', "G" },
{ 'е', "e" },
{ 'ф', "f" },
{ 'Й', "Y" },
...
}
var result = string.Concat("Россия".Select(c => map[c]));
Not sure if I understood your intention correctly, but in case of HttpClient you can work with Windows-1251 (or another encoding) like this:
using (var httpClient = new HttpClient())
{
var httpResponse = await httpClient.GetAsync("requestUri");
var httpContent = await httpResponse.Content.ReadAsByteArrayAsync();
string responseString = Encoding.GetEncoding(1251).GetString(httpContent, 0, httpContent.Length);
// - check status code
// (int)httpResponse.StatusCode
// - and here's your response
// responseString
}
If responseString still contains some gibberish, then I would assume that this server uses not Windows-1251 but some other encoding, so first you'll need to establish which one exactly.
P.S. For Encoding.GetEncoding(1251) to work you might need to install System.Text.Encoding.CodePages NuGet package and register encoding provider:
Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);

How to pass long string in HttpClient.PostAsync request

Trying to send a rather long string to a REST web api (youtrack). I get the following exception:
Invalid URI: The Uri string is too long.
My code:
var encodedMessage = HttpUtility.UrlEncode(message);
var requestUri = string.Format("{0}{1}issue/{2}/execute?comment={3}", url, YoutrackRestUrl, issue.Id, encodedMessage);
var response = await httpClient.PostAsync(requestUri, null).ConfigureAwait(false);
So I took my chances with a FormUrlEncodedContent
var requestUri = string.Format("{0}{1}issue/{2}/execute", url, YoutrackRestUrl, issue.Id);
var postData = new List<KeyValuePair<string, string>>();
postData.Add(new KeyValuePair<string, string>("comment", message));
var content = new FormUrlEncodedContent(postData);
var response = await httpClient.PostAsync(requestUri, content).ConfigureAwait(false);
Which results in the exact same issue.
The string (comment) I am sending, is the changed file set of a commit into SVN. Which can be really long, so I don't really have a way to get around that. Is there a way to post content without the string length restriction?
Read the following topics, but didn't find an answer there:
.NET HttpClient. How to POST string value?
How do I set up HttpContent for my HttpClient PostAsync second parameter?
https://psycodedeveloper.wordpress.com/2014/06/30/how-to-call-httpclient-postasync-with-a-query-string/
http://forums.asp.net/t/2057125.aspx?Invalid+URI+The+Uri+string+is+too+long+HttpClient
Well the short answer to it - just put it into the Body, instead of trying to push all the data via the URL
But as the work on the ticket showed - the answer was here How to set large string inside HttpContent when using HttpClient?
The actual problem beeing in the FormUrlEncodedContent
Try this..Will be helpful for uwp..
Uri uri = new Uri("your uri string");
Windows.Web.Http.HttpClient client = new Windows.Web.Http.HttpClient();
var value1 = new System.Collections.Generic.List<System.Collections.Generic.KeyValuePair<string,string>>
{
// your key value pairs
};
var response = await client.PostAsync(uri,new HttpFormUrlEncodedContent(value1));
var result = await response.Content.ReadAsStringAsync();

ISO-8859 encode post request content C#

I am trying to send a POST request in C# with a parameter encoded to ISO-8859. I am using this code:
using (var wb = new WebClient())
{
var encoding = System.Text.Encoding.GetEncoding("ISO-8859-1");
var encodedText = System.Web.HttpUtility.UrlEncode("åæ ÆÆ øØ ø", encoding);
wb.Encoding = encoding;
wb.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
var data = new NameValueCollection();
data["TXT"] = encodedText;
var response = wb.UploadValues(_url, "POST", data);
}
I have figured out that the correctly encoded string for "åæ ÆÆ øØ ø" is %E5%E6+%C6%C6++%F8%D8+%F8, and I can see when debugging that encodedText actually is this string. However when inspecting the raw request in fiddler, I can see that the string looks like this: TXT=%25e5%25e6%2B%25c6%25c6%2B%25f8%25d8%2B%25f8. I am guessing some kind of extra encoding is being done to the string after or during the call to UploadValues().
Thank you so much in advance.
I checked Google for this. According to another question here on SO at UTF32 for WebClient.UploadValues? (second answer), Webclient.UploadValues() indeed does encoding itself. However, it does ASCII encoding. Youll have to use another method to upload this, like HttpWebRequest.

Windows 8: Download string with encoding (WinRT)

I use this code to download string from the Internet
public static async Task<string> DownloadPageAsync(string url)
{
HttpClientHandler handler = new HttpClientHandler {UseDefaultCredentials = true, AllowAutoRedirect = true};
HttpClient client = new HttpClient(handler);
client.MaxResponseContentBufferSize = 196608;
HttpResponseMessage response = await client.GetAsync(url);
response.EnsureSuccessStatusCode();
string responseBody = await response.Content.ReadAsStringAsync();
return responseBody;
}
but it only works for UTF8 documents. Where do I set the Encoding?
Change ReadAsStringAsync to ReadAsBufferAsync and parse result with required encoding
var buffer = await response.Content.ReadAsBufferAsync();
byte [] rawBytes = new byte[buffer.Length];
using (var reader = DataReader.FromBuffer(buffer))
{
reader.ReadBytes(rawBytes);
}
var res = Encoding.UTF8.GetString(rawBytes, 0, rawBytes.Length);
In WinRT the HttpContent reads Enconding from the Headers property. If the HTTP response from server doesn't set the Content-Type header with encoding, it tries to find BOM mark in the stream and if there's no BOM it will default to the UTF-8 encoding.
If the server is not sending the right Content-Type header you use the HttpContent.ReadAsStreamAsync() method and use your own instance of the Encoding class to correctly decode data.
Set the "ContentEncoding" property of your HttpResponse object:
http://msdn.microsoft.com/en-us/library/system.web.httpresponse.contentencoding%28v=vs.71%29.aspx
Values include:
http://msdn.microsoft.com/en-us/library/system.text.encoding%28v=vs.71%29.aspx
System.Text.ASCIIEncoding
System.Text.UnicodeEncoding
System.Text.UTF7Encoding
System.Text.UTF8Encoding
PS:
This really isn't "Metro" per se - just C#/.Net (albeit .Net 4.x)

Categories