Is it required to encode values when using UploadString with C# WebClient? - c#

I have code like the following to do a POST to a server:
string URI = "http://mydomain.com/foo";
string myParameters =
"&token=1234" +
"&text=" + HttpUtility.UrlEncode(someVariable);
using (WebClient wc = new WebClient())
{
wc.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
string HtmlResult = wc.UploadString(URI, myParameters);
}
Is it required to UrlEncode the parameters like I'm doing or does UploadString handle that automatically under the covers? I don't want to risk any type of double encoding.

Yes, it is required to encode them if you use the UploadString method.
But you could use a more intelligent overload for your case (UploadValues):
string URI = "http://mydomain.com/foo";
var values = new NameValueCollection
{
{ "token", "1234" },
{ "text", someVariable },
};
using (var wc = new WebClient())
{
byte[] result = wc.UploadValues(URI, values);
string htmlResult = Encoding.UTF8.GetString(result);
}
Now you no longer need to worry about any encodings. The WebClient will take them into account when sending the request. Also you will notice that I have removed the application/x-www-form-urlencoded you were adding because when you use the UploadValues method this Content-Type header will automatically be added to the request.

Related

issue with post with string containg & in c#

i am trying to make a post to url which should look like below image in body of fiddler.
correct post from fiddler
ISSUE HAPPENING : when i post it spilt from & (ampersand) and create separate entries for key value in body of fiddler.
my post in fiddler
what should i do to appear in single key value pair?
i am trying to post following data :
responseData = "paymentid=100131806470125606&result=NOT+CAPTURED&auth=138601&ref=806420053674&tranid=131806429899386&postdate=0305&trackid=100001182618838&udf1=1&udf2=2&udf3=3&udf4=4&udf5=5&amt=5190.0&authRespCode=51"
using (WebClient wc = new WebClient())
{
wc.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
string HtmlResult = wc.UploadString("https://example.com/test.php", "authRespCode=1&responseData = " + responseData);
}
You can use HttpUtility.ParseQueryString:
var res = HttpUtility.ParseQueryString(responseData);
then you can access you parameters like so:
res["paymentid"]
etc...
adding this line solved my problem :
responseData = responseData.Replace("&", "&");
responseData = HttpUtility.UrlEncode(responseData);
Complete code :
string responseData = Utilitycl.GetHtmlInputValue(getString, "responseData", Utilitycl.ELEMENTTYPE.INPUT);
responseData = responseData.Replace("&", "&");
responseData = HttpUtility.UrlEncode(responseData);
using (WebClient wc = new WebClient())
{
wc.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
string HtmlResult = wc.UploadString("https://example.com/test.php", "authRespCode=1&responseData = " + responseData);
}
You will need to escape your URI string:
string HtmlResult = wc.UploadString("https://example.com/test.php", "authRespCode=1&responseData = " + Uri.EscapeUriString(responseData));

Get original filename when downloading with WebClient

Is there any way to know the original name of a file you download using the WebClient when the Uri doesn't contain the name?
This happens for example in sites where the download originates from a dynamic page where the name isn't known beforehand.
Using my browser, the file gets the orrect name. But how can this be done using the WebClient?
E.g.
WebClient wc= new WebClient();
var data= wc.DownloadData(#"www.sometime.com\getfile?id=123");
Using DownloadFile() isn't a solution since this method needs a filename in advance.
You need to examine the response headers and see if there is a content-disposition header present which includes the actual filename.
WebClient wc = new WebClient();
var data= wc.DownloadData(#"www.sometime.com\getfile?id=123");
string fileName = "";
// Try to extract the filename from the Content-Disposition header
if (!String.IsNullOrEmpty(wc.ResponseHeaders["Content-Disposition"]))
{
fileName = wc.ResponseHeaders["Content-Disposition"].Substring(wc.ResponseHeaders["Content-Disposition"].IndexOf("filename=") + 9).Replace("\"", "");
}
Read the Response Header "Content-Disposition" with WebClient.ResponseHeaders
It should be:
Content-Disposition: attachment; filename="fname.ext"
your code should look like:
string header = wc.ResponseHeaders["Content-Disposition"]??string.Empty;
const string filename="filename=";
int index = header.LastIndexOf(filename,StringComparison.OrdinalIgnoreCase);
if (index > -1)
{
fileName = header.Substring(index+filename.Length);
}
To get the filename without downloading the file:
public string GetFilenameFromWebServer(string url)
{
string result = "";
var req = System.Net.WebRequest.Create(url);
req.Method = "HEAD";
using (System.Net.WebResponse resp = req.GetResponse())
{
// Try to extract the filename from the Content-Disposition header
if (!string.IsNullOrEmpty(resp.Headers["Content-Disposition"]))
{
result = resp.Headers["Content-Disposition"].Substring(resp.Headers["Content-Disposition"].IndexOf("filename=") + 9).Replace("\"", "");
}
}
return result;
}
If you, like me, have to deal with a Content-Disposition header that is not formatted correctly or cannot be parsed automatically by the ContentDisposition class for some reason, here's my solution :
string fileName = null;
// Getting file name
var request = WebRequest.Create(url);
request.Method = "HEAD";
using (var response = request.GetResponse())
{
// Headers are not correct... So we need to parse manually
var contentDisposition = response.Headers["Content-Disposition"];
// We delete everything up to and including 'Filename="'
var fileNameMarker= "filename=\"";
var beginIndex = contentDisposition.ToLower().IndexOf(fileNameMarker);
contentDisposition = contentDisposition.Substring(beginIndex + fileNameMarker.Length);
//We only get the string until the next double quote
var fileNameLength = contentDisposition.ToLower().IndexOf("\"");
fileName = contentDisposition.Substring(0, fileNameLength);
}

Encode in webclient unexpected result

I try use webclient to translate word 'Banana' into rus
private void button1_Click(object sender, EventArgs e)
{
Navigate("http://translate.google.ru/translate_a/t?client=x&text=Banana&hl=en&sl=en&tl=ru");
}
private void Navigate(String address)
{
WebClient client = new WebClient();
client.Proxy = WebRequest.DefaultWebProxy;
client.Credentials = new NetworkCredential("user", "password", "domain");
client.Proxy.Credentials = new NetworkCredential("user", "password", "domain");
string _stranslate = client.DownloadString(new Uri(address));
}
And I want to see in "_stranslate "
{"sentences":[{"trans":"Банан","orig":"Banana#","translit":"Banan #","src_translit":""}],"src":"en","server_time":0}
but got this
{"sentences":[{"trans":"вБОБО","orig":"Banana#","translit":"Banan #","src_translit":""}],"src":"en","server_time":0}
Can some one help me?
Try checking the response headers, the content types tells you what encoding you should use.
Content-Type => text/javascript; charset=KOI8-R
So just add this line.
client.Encoding = Encoding.GetEncoding(20866);
or
client.Encoding = Encoding.GetEncoding("KOI8-R");
A complete list for encodings can be found in the documentation for the Encoding Class
Another way would be to just use System.Net.Mime.ContentType to get the charset.
Like this:
byte[] data = client.DownloadData(url);
ContentType contentType = new System.Net.Mime.ContentType(client.ResponseHeaders[HttpResponseHeader.ContentType]);
string _stranslate = Encoding.GetEncoding(contentType.CharSet).GetString(data);
Add this before your client.DownloadString():
client.Encoding = System.Text.Encoding.UTF8;
Your encoding is likely getting messed up when you read the string.
Using this HTTP header viewer and putting in your URL, I see the following in the headers:
Content-Type: text/javascript; charset=UTF-8
Content-Language: ru
Basically, you need to find out what encoding they are sending back and set your encoding to match.
It is very important to set the encoding before you call DownloadString().
IMHO better solution: add URI query parameter
oe=UTF-8
and use UTF-8 everywhere
var nameValueCollection = new NameValueCollection
{
{"client", "x"},
{"text", HttpUtility.UrlEncode(text)},
{"hl", "en"},
{"sl", fromLanguage},
{"tl", toLanguage},
{"ie", "UTF-8"},
{"oe", "UTF-8"}
};
string htmlResult;
using (var wc = new WebClient())
{
wc.Encoding = Encoding.UTF8;
wc.QueryString = nameValueCollection;
htmlResult = wc.DownloadString(GoogleAddress);
}

How to make an HTTP get request with parameters

Is it possible to pass parameters with an HTTP get request? If so, how should I then do it? I have found an HTTP post requst (link). In that example the string postData is sent to a webserver. I would like to do the same using get instead. Google found this example on HTTP get here. However no parameters are sent to the web server.
My preferred way is this. It handles the escaping and parsing for you.
WebClient webClient = new WebClient();
webClient.QueryString.Add("param1", "value1");
webClient.QueryString.Add("param2", "value2");
string result = webClient.DownloadString("http://theurl.com");
First WebClient is easier to use; GET arguments are specified on the query-string - the only trick is to remember to escape any values:
string address = string.Format(
"http://foobar/somepage?arg1={0}&arg2={1}",
Uri.EscapeDataString("escape me"),
Uri.EscapeDataString("& me !!"));
string text;
using (WebClient client = new WebClient())
{
text = client.DownloadString(address);
}
In a GET request, you pass parameters as part of the query string.
string url = "http://somesite.com?var=12345";
The WebRequest object seems like too much work for me. I prefer to use the WebClient control.
To use this function you just need to create two NameValueCollections holding your parameters and request headers.
Consider the following function:
private static string DoGET(string URL,NameValueCollection QueryStringParameters = null, NameValueCollection RequestHeaders = null)
{
string ResponseText = null;
using (WebClient client = new WebClient())
{
try
{
if (RequestHeaders != null)
{
if (RequestHeaders.Count > 0)
{
foreach (string header in RequestHeaders.AllKeys)
client.Headers.Add(header, RequestHeaders[header]);
}
}
if (QueryStringParameters != null)
{
if (QueryStringParameters.Count > 0)
{
foreach (string parm in QueryStringParameters.AllKeys)
client.QueryString.Add(parm, QueryStringParameters[parm]);
}
}
byte[] ResponseBytes = client.DownloadData(URL);
ResponseText = Encoding.UTF8.GetString(ResponseBytes);
}
catch (WebException exception)
{
if (exception.Response != null)
{
var responseStream = exception.Response.GetResponseStream();
if (responseStream != null)
{
using (var reader = new StreamReader(responseStream))
{
Response.Write(reader.ReadToEnd());
}
}
}
}
}
return ResponseText;
}
Add your querystring parameters (if required) as a NameValueCollection like so.
NameValueCollection QueryStringParameters = new NameValueCollection();
QueryStringParameters.Add("id", "123");
QueryStringParameters.Add("category", "A");
Add your http headers (if required) as a NameValueCollection like so.
NameValueCollection RequestHttpHeaders = new NameValueCollection();
RequestHttpHeaders.Add("Authorization", "Basic bGF3c2912XBANzg5ITppc2ltCzEF");
GET request with multiple params:
curl --request GET --url
http://localhost:8080/todos/?limit=10&offset=2 --header
'content-type:application/json'
You can also pass value directly via URL.
If you want to call method
public static void calling(string name){....}
then you should call usingHttpWebRequest webrequest = (HttpWebRequest)WebRequest.Create("http://localhost:****/Report/calling?name=Priya);
webrequest.Method = "GET";
webrequest.ContentType = "application/text";
Just make sure you are using ?Object = value in URL

How to fill forms and submit with Webclient in C#

I'm new at using the the libraries WebClient, HttpResponse and HttpRequest in C#, so bear with me, if my question is confusing to read.
I need to build a WinForm based on C# which can open a URL, which is secured with the basic authorization. I did this with adding this to the header, like this:
using (WebClient wc = new WebClient())
{
wc.Headers.Add(HttpRequestHeader.Authorization, "Basic " +
Convert.ToBase64String(
Encoding.ASCII.GetBytes(username + ":" + password)));
}
So far, so good! Now I would like to fill a form with a number, and I find the source-code from the site, and discover that the name is "number". So I write this:
NameValueCollection formData = new NameValueCollection();
formData["number"] = number
byte[] responseBytes = wc.UploadValues(theurl, "POST", formData);
string response = Encoding.ASCII.GetString(responseBytes);
textBox_HTML.Text = response;
But how do I submit this? I will like to receive my "search-results"...
You should probably be using HttpWebRequest for this. Here's a simple example:
var strId = UserId_TextBox.Text;
var strName = Name_TextBox.Text;
var encoding=new ASCIIEncoding();
var postData="userid="+strId;
postData += ("&username="+strName);
byte[] data = encoding.GetBytes(postData);
var myRequest =
(HttpWebRequest)WebRequest.Create("http://localhost/MyIdentity/Default.aspx");
myRequest.Method = "POST";
myRequest.ContentType="application/x-www-form-urlencoded";
myRequest.ContentLength = data.Length;
var newStream=myRequest.GetRequestStream();
newStream.Write(data,0,data.Length);
newStream.Close();
var response = myRequest.GetResponse();
var responseStream = response.GetResponseStream();
var responseReader = new StreamReader(responseStream);
var result = responseReader.ReadToEnd();
responseReader.Close();
response.Close();
Try this:
using System.Net;
using System.Collections.Specialized;
NameValueCollection values = new NameValueCollection();
values.Add("TextBox1", "value1");
values.Add("TextBox2", "value2");
values.Add("TextBox3", "value3");
string Url = urlvalue.ToLower();
using (WebClient client = new WebClient())
{
client.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
byte[] result = client.UploadValues(Url, "POST", values);
string ResultAuthTicket = System.Text.Encoding.UTF8.GetString(result);
}
I have found a solution to my problem. First of all I was confused about some of the basics in http-communication. This was caused by a python-script i wrote, which have a different approach with the communication.
I solved it with generating the POST-data from scratch and open the uri, which was contained in the form-action.
BFree's answer works great. One thing I would note, though, is that the data concatenation should really be url-encoded, otherwise you'd have trouble with things like "=" and "&" signs within the data.
The VB.NET version, urlencoded and with UTF-8 support, is below (note that url-encoding requires a reference to System.Web.dll, which only worked for me after I switched from .NET 4 Compact Framework to the regular .NET 4 Framework).
Imports System.Web
Imports System.Net
Imports System.IO
Public Class WebFormSubmitter
Public Shared Function submit(ByVal address As String,
ByVal values As Dictionary(Of String, String)) As String
Dim encoding As New UTF8Encoding
Dim postData As String = getPostData(values:=values)
Dim data() As Byte = encoding.GetBytes(postData)
Dim request = CType(WebRequest.Create(address), HttpWebRequest)
request.Method = "POST"
request.ContentType = "application/x-www-form-urlencoded"
request.ContentLength = data.Length
Dim newStream = request.GetRequestStream()
newStream.Write(data, 0, data.Length)
newStream.Close()
Dim response = request.GetResponse()
Dim responseStream = response.GetResponseStream()
Dim responseReader = New StreamReader(responseStream)
Return responseReader.ReadToEnd()
End Function
Private Shared Function getPostData(ByVal values As Dictionary(Of String, String)) As String
Dim postDataPairList As New List(Of String)
For Each anEntry In values
postDataPairList.Add(anEntry.Key & "=" & HttpUtility.UrlEncode(anEntry.Value))
Next
Return String.Join(separator:="&", values:=postDataPairList)
End Function
End Class
Posting a form with System.Net.Http.HttpClient and reading the response as string:
var formData = new Dictionary<string, string>();
formData.Add("number", number);
var content = new FormUrlEncodedContent(formData);
using (var httpClient = new HttpClient())
{
var httpResponse = await httpClient.PostAsync(theurl, content);
var responseString = await httpResponse.Content.ReadAsStringAsync();
}
You submitted it already with UploadValues. The question is: what is your "result-search"? What does the page return? HTML? If so - the HTML Agility Pack is the easiest way to parse html.

Categories