This code
using (WebClient webClient = new WebClient())
{
string address = "https://www.any.com/name.htm";
byte[] postData = Encoding.ASCII.GetBytes("login=123");
webClient.Headers[HttpRequestHeader.Accept] = "text/html, application/xhtml+xml, */*";
webClient.Headers[HttpRequestHeader.AcceptLanguage] = "ru-RU";
webClient.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)";
webClient.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
webClient.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
webClient.UploadData(address, postData);
}
produces this request (postData doubles):
POST /name.htm HTTP/1.1
Accept-Language: ru-RU
Accept-Encoding: gzip, deflate
Accept: text/html, application/xhtml+xml, */*
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)
Host: [url]www.any.com[/url]
Content-Length: 9
Connection: Keep-Alive
login=123login=123
If I change
string address = "https://www.any.com/name.htm"
to
string address = "http://www.any.com/name.htm"
then data don't doubles.
How avoid data doubling when send request by HTTPS ?
It is bug of HttpAnalizer.
Many thanks to Hardrada.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 days ago.
Improve this question
I want to do this request with C#
POST /ImageDownloadPro HTTP/1.1
Host: access.imgdownloader.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/109.0
Accept: application/json, text/plain, */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: application/json;charset=utf-8
Content-Length: 58
Origin: https://imgdownloader.com
DNT: 1
Connection: keep-alive
Referer: https://imgdownloader.com/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-site
{"url":"https://i.ytimg.com/vi/yMDfF_dNctE/hqdefault.jpg"}
output with firefox
{"code":200,"data":[{"originalurl":"https://access.imgdownloader.com/original_20230213020654271378/imgdownloader.com-20230213020654303205.jpg","thumburl":"https://access.imgdownloader.com/original_20230213020654271378/imgdownloader.com-20230213020654303205.jpg"}],"msg":null}
I tested this
System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
byte[] bytes = Encoding.ASCII.GetBytes($"{{"url":"https://i.ytimg.com/vi/yMDfF_dNctE/hqdefault.jpg"}}");
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create("https://test.com");
httpWebRequest.Method = "POST";
httpWebRequest.Host = "access.test.com";
httpWebRequest.KeepAlive = true;
httpWebRequest.AutomaticDecompression = DecompressionMethods.GZip;
httpWebRequest.Headers.Add("Accept-Language", "en-US,en;q=0.5");
httpWebRequest.Headers.Add("sec-fetch-dest", "document");
httpWebRequest.Headers.Add("sec-fetch-mode", "navigate");
httpWebRequest.Headers.Add("sec-fetch-site", "cross-site");
httpWebRequest.ContentType = "application/json;charset=utf-8";
httpWebRequest.ContentLength = bytes.LongLength;
httpWebRequest.Referer = "https://test.com/TestTe";
httpWebRequest.Accept = "\ttext/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8";
httpWebRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/109.0";
using (Stream requestStream = httpWebRequest.GetRequestStream())
{
requestStream.Write(bytes, 0, bytes.Length);
}
string responseText = null;
using (WebResponse response = httpWebRequest.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (StreamReader streamReader = new StreamReader(responseStream))
{
responseText = streamReader.ReadToEnd();
}
}
}
return responseText;
But the result is very big and it seems to download the source of the site
I am trying to download a pdf file from a web server over https in c# and getting HTTP OK response but file stream is not coming with response. What am I doing wrong?
Here is my code behind:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
ServicePointManager.ServerCertificateValidationCallback = (s, cert, ch, sec) => { return true; };
HttpWebRequest myRequest = (HttpWebRequest)HttpWebRequest.Create("https://services.dps.ohio.gov/CrashOnline/CrashRetrieval/OhioCrashReportRetrieval/GetReport");
myRequest.Method = "POST";
myRequest.CookieContainer = cookies;
myRequest.ContentType = "application/x-www-form-urlencoded";
myRequest.Accept = myRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
myRequest.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip, deflate, br");
myRequest.Headers.Add(HttpRequestHeader.AcceptLanguage, "en-US,en;q=0.9");
myRequest.Referer = "https://services.dps.ohio.gov/CrashOnline/CrashRetrieval/OhioCrashReportRetrieval/Search";
myRequest.UserAgent = "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36";
byte[] bytes = Encoding.Default.GetBytes(content);
myRequest.ContentLength = bytes.Length;
Stream dataStream = myRequest.GetRequestStream();
dataStream.Write(bytes, 0, bytes.Length);
dataStream.Close();
byte[] result = null;
byte[] buffer = new byte[2048];
using (HttpWebResponse response = (HttpWebResponse)myRequest.GetResponse())
Downloading file from chrome is working fine and here is fiddler RAW http packet from browser:
POST https://services.dps.ohio.gov/CrashOnline/CrashRetrieval/OhioCrashReportRetrieval/GetReport HTTP/1.1
Host: services.dps.ohio.gov
Connection: keep-alive
Content-Length: 177
Cache-Control: max-age=0
Origin: https://services.dps.ohio.gov
Upgrade-Insecure-Requests: 1
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Referer: https://services.dps.ohio.gov/CrashOnline/CrashRetrieval/OhioCrashReportRetrieval/Search
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: ASP.NET_SessionId=t1nddo51g0jotl51cqev15rp; __RequestVerificationToken_L0NyYXNoT25saW5l0=iMy0lJ3OviGs0iXame8ebp8IdS6K9wpc8tFinYd0nhNnySTLlii-yHYU5tDJjv49Fqi29M_31mPiPdKWWtJJPWSmp4qGGvna9lW2rkDHUAs1
__RequestVerificationToken=eTJN31araRPeB-bhQaeU3awBDsCwR9BpkIYNqpsakTPRT1w1nXtbGFcfVRZS776ZW0NGv1aKahx3WJv5PNQ199iM7GF9Y9f7ws5KTG4gI3I1&id=NjgzMTIwOQ%3D%3D&btn_NjgzMTIwOQ%3D%3D=
This is my application's http packet:
POST https://services.dps.ohio.gov/CrashOnline/CrashRetrieval/OhioCrashReportRetrieval/GetReport HTTP/1.1
Accept: text/html, application/xhtml+xml, image/jxr, */*
Referer: https://services.dps.ohio.gov/CrashOnline/CrashRetrieval/OhioCrashReportRetrieval/Search
Accept-Language: en-GB,es-ES;q=0.8,es;q=0.6,en-US;q=0.4,en;q=0.2
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
Host: services.dps.ohio.gov
Content-Length: 177
Connection: Keep-Alive
Cache-Control: no-cache
__RequestVerificationToken=eTJN31araRPeB-bhQaeU3awBDsCwR9BpkIYNqpsakTPRT1w1nXtbGFcfVRZS776ZW0NGv1aKahx3WJv5PNQ199iM7GF9Y9f7ws5KTG4gI3I1&id=NjgzMTIwOQ%3D%3D&btn_NjgzMTIwOQ%3D%3D=
i'am using burp suit to check the requests and i m trying to convertthis to c# code
POST /sso HTTP/1.1
Host: account.ankama.com
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
Referer: http://www.dofus.com/fr
Cookie: LANG=fr; _ga=GA1.1.1197518596.1489526959; SID=452EDCF3C4BD32057F9F08254BE40001
Connection: close
Upgrade-Insecure-Requests: 1
Content-Type: application/x-www-form-urlencoded
Content-Length: 102
action=login&from=http%3A%2F%2Fwww.dofus.com%2Ffr&login=user123&password=password1232F&remember=1
So i tried to :
HttpWebRequest Request = (HttpWebRequest)WebRequest.Create("https://account.ankama.com/sso?action=login&from=https%3A%2F%2Faccount.ankama.com%2Ffr%2Fsecurite%2Fmode-restreint%3Ff%3Dhttps%3A%2F%2Faccount.ankama.com%2Ffr%2Fidentification%3Ff%3Dhttps%3A%2F%2Faccount.ankama.com%2Ffr%2Fcompte%2Finformations&login=user111&password=password1472F");
Request.ContentType = "application/x-www-form-urlencoded";
Request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
Request.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0";
Request.Host = "account.ankama.com";
Request.Referer = "https://account.ankama.com/fr/votre-compte/profil";
Request.Method = "POST";
Request.AllowAutoRedirect = true;
Request.CookieContainer = new CookieContainer();
//quest.Credentials = new NetworkCredential("user123", "passowrd123");
using (HttpWebResponse response = (HttpWebResponse)Request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
StreamReader reader = new StreamReader(stream);
StreamWriter writer = new StreamWriter("odm.html");
writer.Write(reader.ReadToEnd());
writer.Close();
reader.Close();
Console.WriteLine("Done");
}
}
Console.ReadKey();
in the file odm.html I m checking if the html code contain "My account" that shown when the user is actually logged in .
but this doesnt seems to be working for some reasons that i still don't know .
i made some research to about HTTP status code but in my brup suit after trying to login in with an actual exisiting account and a none valid account it gives the same http code 302 with a different Content Length .
EDIT:
the issue is i don't find 'my account' in the html file , i only find the page where the user is going to login
You are trying to send request body in query string, you are setting the request method as POST but you are not sending the body. The request url should be:
https://account.ankama.com/sso
And you need to set request body before sending the request:
var bytes = Encoding.UTF8.GetBytes("action=login&from=http%3A%2F%2Fwww.dofus.com%2Ffr&login=user123&password=password1232F&remember=1");
request.ContentLength = bytes.Length;
using (var stream = request.GetRequestStream())
{
stream.Write(bytes, 0, bytes.Length);
}
I have realy strange problem with Headers collection in WebClient class.
Here is my example:
WebClient client = new WebClient();
client.Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0");
client.Headers.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
client.Headers.Add("Accept-Language", "pl,en-us;q=0.7,en;q=0.3");
Console.WriteLine("Before request:");
foreach (string key in client.Headers)
{
Console.WriteLine(key + ": " + client.Headers[key]);
}
client.DownloadString("http://www.google.com");
Console.WriteLine();
Console.WriteLine("After request:");
foreach (string key in client.Headers)
{
Console.WriteLine(key + ": " + client.Headers[key]);
}
Console.ReadLine();
My result of running this simple program:
Before request:
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: pl,en-us;q=0.7,en;q=0.3
After request:
Accept-Language: pl,en-us;q=0.7,en;q=0.3
Why my headers are disappearing?
because headers are sent and webclient did its job. if you want same headers for next request, you should add them again.
I'm setting up some HTTP request headers like this:
this.Url = new Uri(u);
HttpWebRequest http = (HttpWebRequest)WebRequest.Create(Url);
WebResponse response = http.GetResponse();
//headers
http.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0\r\n";
http.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n";
http.Headers.Add("Accept-Encoding", "gzip,deflate,sdch'r'n");
http.Headers.Add("Accept-Language", "en-US,en;q=0.9\r\n");
http.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.3\r\n");
I am capturing them like this:
for (count = 0; count < http.Headers.Keys.Count; count++)
{
headerKey = http.Headers.Keys[count];
headerValue = http.Headers[headerKey];
if (headerValue != null)
{
if (headerKey == null)
{
requestbuffer.Append(headerValue);
requestbuffer.Append(Newline);
}
else
{
requestbuffer.Append(headerKey + ": " + headerValue);
requestbuffer.Append(Newline);
}
}
}
When I run the testing tool everything seems good:
Host: domain.com
Connection: Keep-Alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0)Gecko/20100101 Firefox/17.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Encoding: gzip,deflate,sdch Accept-Language: en-US,en;q=0.9
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
However in Wireshark and Fiddler only the following header is sent:
GET / HTTP/1.1
Host: domain.com
Any idea why that may be?
You're trying to add the headers after you've called http.GetResponse(). That's after the request has been sent. Change it to this:
this.Url = new Uri(u);
HttpWebRequest http = (HttpWebRequest)WebRequest.Create(Url);
//headers
http.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0\r\n";
http.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n";
http.Headers.Add("Accept-Encoding", "gzip,deflate,sdch'r'n");
http.Headers.Add("Accept-Language", "en-US,en;q=0.9\r\n");
http.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.3\r\n");
using (WebResponse response = http.GetResponse())
{
// Do whatever
}
(Note that you really should be disposing of the response.)
Yes, you are setting the header after you have sent the request.
Try this:
//headers
http.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0\r\n";
http.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n";
http.Headers.Add("Accept-Encoding", "gzip,deflate,sdch'r'n");
http.Headers.Add("Accept-Language", "en-US,en;q=0.9\r\n");
http.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.3\r\n");
//Added diesposal of response too
using (WebResponse response = http.GetResponse())
{
}