We Keep Coding

HttpWebRequest Auth Post Request in c#

i'am using burp suit to check the requests and i m trying to convertthis to c# code
POST /sso HTTP/1.1
Host: account.ankama.com
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
Referer: http://www.dofus.com/fr
Cookie: LANG=fr; _ga=GA1.1.1197518596.1489526959; SID=452EDCF3C4BD32057F9F08254BE40001
Connection: close
Upgrade-Insecure-Requests: 1
Content-Type: application/x-www-form-urlencoded
Content-Length: 102
action=login&from=http%3A%2F%2Fwww.dofus.com%2Ffr&login=user123&password=password1232F&remember=1
So i tried to :
HttpWebRequest Request = (HttpWebRequest)WebRequest.Create("https://account.ankama.com/sso?action=login&from=https%3A%2F%2Faccount.ankama.com%2Ffr%2Fsecurite%2Fmode-restreint%3Ff%3Dhttps%3A%2F%2Faccount.ankama.com%2Ffr%2Fidentification%3Ff%3Dhttps%3A%2F%2Faccount.ankama.com%2Ffr%2Fcompte%2Finformations&login=user111&password=password1472F");
Request.ContentType = "application/x-www-form-urlencoded";
Request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
Request.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0";
Request.Host = "account.ankama.com";
Request.Referer = "https://account.ankama.com/fr/votre-compte/profil";
Request.Method = "POST";
Request.AllowAutoRedirect = true;
Request.CookieContainer = new CookieContainer();
//quest.Credentials = new NetworkCredential("user123", "passowrd123");
using (HttpWebResponse response = (HttpWebResponse)Request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
StreamReader reader = new StreamReader(stream);
StreamWriter writer = new StreamWriter("odm.html");
writer.Write(reader.ReadToEnd());
writer.Close();
reader.Close();
Console.WriteLine("Done");
}
}
Console.ReadKey();
in the file odm.html I m checking if the html code contain "My account" that shown when the user is actually logged in .
but this doesnt seems to be working for some reasons that i still don't know .
i made some research to about HTTP status code but in my brup suit after trying to login in with an actual exisiting account and a none valid account it gives the same http code 302 with a different Content Length .
EDIT:
the issue is i don't find 'my account' in the html file , i only find the page where the user is going to login

You are trying to send request body in query string, you are setting the request method as POST but you are not sending the body. The request url should be:
https://account.ankama.com/sso
And you need to set request body before sending the request:
var bytes = Encoding.UTF8.GetBytes("action=login&from=http%3A%2F%2Fwww.dofus.com%2Ffr&login=user123&password=password1232F&remember=1");
request.ContentLength = bytes.Length;
using (var stream = request.GetRequestStream())
{
stream.Write(bytes, 0, bytes.Length);
}

C# - Sending and receiving HTTP Requests to REST service

So I'm writing the client side user authentication code for my Web API ReST service.
This is how the request and response should look like and I've written this code to register a user and it works fine.
Http Request:
POST https://localhost:44305/api/Account/Register HTTP/1.1
Host: localhost:44305
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0
Accept: */*
Content-Type: application/json; charset=utf-8
X-Requested-With: XMLHttpRequest
Referer: https://localhost:44305/
Content-Length: 84
{"Email":"alice#example.com","Password":"Password1!","ConfirmPassword":"Password1!"}
Http Response:
HTTP/1.1 200 OK
Server: Microsoft-IIS/8.0
Date: Wed, 01 Oct 2014 00:57:58 GMT
Content-Length: 0
Code to accomplish this:
byte[] data = Encoding.UTF8.GetBytes("{\"Email\":\"alssice#example.com\"," + "\"Password\":\"Password1!\"," + "\"ConfirmPassword\":\"Password1!\"}");
WebRequest request = WebRequest.Create("http://localhost:8091/api/Account/Register");
request.Method = "POST";
request.ContentType = "application/json; charset=utf-8";
request.ContentLength = data.Length;
using (Stream stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
string responseContent = null;
using (WebResponse response = request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
using (StreamReader sr99 = new StreamReader(stream))
{
responseContent = sr99.ReadToEnd();
}
}
}
Console.WriteLine(responseContent);
This works fine and the user account is created in the database but I don't get the response.
The next problem is sending sending the login details to get a bearer token? How can I do this?
This is what the request and response should look like
HTTP REQUEST FOR AUTHENTICATION:
POST https://localhost:44305/Token HTTP/1.1
Host: localhost:44305
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0
Accept: */*
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: https://localhost:44305/
Content-Length: 68
grant_type=password&username=alice%40example.com&password=Password1!
HTTP RESPONSE FOR AUTHENTICATION:
HTTP/1.1 200 OK
Content-Length: 669
Content-Type: application/json;charset=UTF-8
Server: Microsoft-IIS/8.0
Date: Wed, 01 Oct 2014 01:22:36 GMT
{
"access_token":"imSXTs2OqSrGWzsFQhIXziFCO3rF...",
"token_type":"bearer",
"expires_in":1209599,
"userName":"alice#example.com",
".issued":"Wed, 01 Oct 2014 01:22:33 GMT",
".expires":"Wed, 15 Oct 2014 01:22:33 GMT"
}
SO how can I send a request for a token?

I'd kindly like to suggest you using HttpClient if you're going for your own http rest client in your app.
Solution using HttpClient and Json.net:
public class TokenResponse
{
public string access_token { get; set; }
public string token_type { get; set; }
public int expires_in { get; set; }
public string userName { get; set; }
public string issued { get; set; }
public string expires { get; set; }
}
private async Task Login()
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("https://localhost:44305");
var content = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("grant_type", "password"),
new KeyValuePair<string, string>("username", "alice#example.com"),
new KeyValuePair<string, string>("password", "password1")
});
var result = await client.PostAsync("/token", content);
string resultContent = await result.Content.ReadAsStringAsync();
var tokenResponse = JsonConvert.DeserializeObject<TokenResponse>(resultContent);
}
}
And if you need to use HttpWebRequest:
If this is the case, your solution might look something like the following.
private void Login()
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("https://localhost:44305/Token");
request.Method = "POST";
request.AllowAutoRedirect = false;
request.Accept = "*/*";
request.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
StringBuilder postData = new StringBuilder();
postData.Append("grant_type=" + HttpUtility.UrlEncode("password") + "&");
postData.Append("username=" + HttpUtility.UrlEncode("alice#example.com") + "&");
postData.Append("password=" + HttpUtility.UrlEncode("password"));
using (StreamWriter stOut = new StreamWriter(request.GetRequestStream(), Encoding.UTF8))
{
stOut.Write(postData);
stOut.Close();
}
}
Update
Now Im not sure where and how you are trying this. If you're invoking this from a console application main method, make sure the following so you don't have to come back to the same asynchronous context and this should always be invoked from an async method itself:
var result = await client.PostAsync("https://localhost:44305/token", content).ConfigureAwait(false);
I updated my solution in a github gist. It's working for me as it's trying to post the request right away.

Access website with C# POST

I am trying to access the data on a website using some C# code. From a browser, a user would go to http://www.hkexnews.hk/sdw/search/search_sdw.asp, put in a date (17 May 2016) and a stock code ("00001") and be taken to a table with the relevant data.
I have used a Firefox add-on (LiveHTTPHeaders) to see that this process produces the following POST request:
http://www.hkexnews.hk/sdw/search/search_sdw.asp
POST /sdw/search/search_sdw.asp HTTP/1.1
Host: www.hkexnews.hk
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://www.hkexnews.hk/sdw/search/search_sdw.asp
Cookie: ASPSESSIONIDCASQQTQQ=FOLELNMCLAOEPAAEMIICCPAN; TS0161f2e5=017038eb4956ab204b9c8ba8c23ae9307c7879ba5fccd4664528a1407b7fab58353e89b27052a3d4d4df3b4b47034b5a31085f2c11
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 341
txt_today_d=20&txt_today_m=5&txt_today_y=2016&current_page=1&stock_market=HKEX&IsExist_Slt_Stock_Id=False&IsExist_Slt_Part_Id=False&rdo_SelectSortBy=Shareholding&sessionToken=1458.568&sel_ShareholdingDate_d=17&sel_ShareholdingDate_m=05&sel_ShareholdingDate_y=2016&txt_stock_code=00001&txt_stock_name=&txt_ParticipantID=&txt_Participant_name=
HTTP/1.1 200 OK
Cache-Control: private
Pragma: No-Cache
Content-Length: 300802
Content-Type: text/html
Expires: 0
X-Powered-By: ASP.NET
Date: Fri, 20 May 2016 04:58:57 GMT
My code:
string PostData = "txt_today_d=20&txt_today_m=5&txt_today_y=2016&current_page=1&stock_market=HKEX&IsExist_Slt_Stock_Id=False&IsExist_Slt_Part_Id=False&rdo_SelectSortBy=Shareholding&sessionToken=1458.568&sel_ShareholdingDate_d=17&sel_ShareholdingDate_m=05&sel_ShareholdingDate_y=2016&txt_stock_code=00001&txt_stock_name=&txt_ParticipantID=&txt_Participant_name=";
byte[] ByteArray = Encoding.UTF8.GetBytes(PostData);
HttpWebRequest Request = (HttpWebRequest)WebRequest.Create("http://www.hkexnews.hk/sdw/search/search_sdw.asp");
Uri Target = new Uri("http://www.hkexnews.hk/sdw/search/search_sdw.asp");
Request.Method = "POST";
Request.Host = "www.hkexnews.hk";
Request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0";
Request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
WebHeaderCollection Headers = Request.Headers;
Headers.Add("Accept-Language", "en-US,en;q=0.5");
Headers.Add("Accept-Encoding", "gzip, deflate");
Request.Referer = "http://www.hkexnews.hk/sdw/search/search_sdw.asp";
Request.CookieContainer = new CookieContainer();
Request.CookieContainer.Add(new Cookie("ASPSESSIONIDCASQQTQQ", "FOLELNMCLAOEPAAEMIICCPAN") { Domain = Target.Host });
Request.CookieContainer.Add(new Cookie("TS0161f2e5", "017038eb49a575136721aebc3f3ef14ec9c810d2e25d5e909be6fe49639ec38eec624237bf9759851675d091f3a4ed0ebc3d8bb3d2") { Domain = Target.Host });
Request.KeepAlive = true;
Request.ContentType = "Content-Type: application/x-www-form-urlencoded";
Request.ContentLength = ByteArray.Length;
Stream dataStream = Request.GetRequestStream();
// Write the data to the request stream.
dataStream.Write(ByteArray, 0, ByteArray.Length);
// Close the Stream object.
dataStream.Close();
// Get the response.
WebResponse response = Request.GetResponse();
// Display the status.
Console.WriteLine(((HttpWebResponse)response).StatusDescription);
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
// Display the content.
Console.WriteLine(responseFromServer);
// Clean up the streams.
reader.Close();
dataStream.Close();
response.Close();
What this code appears to do is simply spit out the Html of the original search screen instead of taking me to the results.
Edit: I also tried doing the same with the HttpClient class, as below:
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://www.hkexnews.hk/sdw/search/search_sdw.asp");
var content = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("txt_today_d", "20"),
new KeyValuePair<string, string>("txt_today_m", "5"),
new KeyValuePair<string, string>("txt_today_y", "2016"),
new KeyValuePair<string, string>("current_page", "1"),
new KeyValuePair<string, string>("stock_market", "HKEX"),
new KeyValuePair<string, string>("IsExist_Slt_Stock_Id", "False"),
new KeyValuePair<string, string>("IsExist_Slt_Part_Id", "False"),
new KeyValuePair<string, string>("rdo_SelectSortBy", "Shareholding"),
new KeyValuePair<string, string>("sessionToken", "1458.568"),
new KeyValuePair<string, string>("sel_ShareholdingDate_d", "17"),
new KeyValuePair<string, string>("sel_ShareholdingDate_m", "05"),
new KeyValuePair<string, string>("sel_ShareholdingDate_y", "2016"),
new KeyValuePair<string, string>("txt_stock_code", "00001"),
new KeyValuePair<string, string>("txt_stock_name", ""),
new KeyValuePair<string, string>("txt_ParticipantID", ""),
new KeyValuePair<string, string>("txt_Participant_name", "")
});
var result = client.PostAsync("/sdw/search/search_sdw.asp", content).Result;
string resultContent = result.Content.ReadAsStringAsync().Result;
Console.WriteLine(resultContent);
This produces the same result (Html of original form)

Because something wrong in your request(maybe cookie is not correct) and your request was redirected to original website.
HTTP/1.1 302 Object moved
Cache-Control: private
Pragma: No-Cache
Content-Length: 135
Content-Type: text/html
Expires: 0
Location: search_sdw.asp
X-Powered-By: ASP.NET
Date: Fri, 20 May 2016 07:58:52 GMT
<head><title>Object moved</title></head>
<body><h1>Object Moved</h1>This object may be found here.</body>

HttpWebRequest.GetResponse Operation has timed out

I'm trying to get simple gzip encoded html response from a website and it keeps getting time out, following is my code:
HttpWebRequest httpClient = (HttpWebRequest)WebRequest.Create(url);
httpClient.Method = "GET";
httpClient.Accept = "text/html, application/xhtml+xml, */*";
httpClient.Headers.Add("Accept-Encoding: gzip, deflate");
httpClient.Headers.Add("Accept-Language: en-US");
httpClient.Headers.Add("DNT: 1");
httpClient.ProtocolVersion = HttpVersion.Version10;
httpClient.KeepAlive = true;
httpClient.Timeout = System.Threading.Timeout.Infinite;
httpClient.CookieContainer = cookieJar;
String responseAsText;
using (HttpWebResponse response = (HttpWebResponse)httpClient.GetResponse())
{
System.IO.StreamReader sr;
if (response.ContentEncoding.Equals("gzip"))
{
sr = new StreamReader(new GZipStream(response.GetResponseStream(), CompressionMode.Decompress));
}
else
{
sr = new System.IO.StreamReader(response.GetResponseStream());
}
responseAsText = sr.ReadToEnd();
}
The url I'm trying to hit is "https client.schwab.com/Login/SignOn/CustomerCenterLogin.aspx"
This works perfectly fine in the Browser, using Fiddler I viewed the browser's Request header and since its Transfer-Encoding: chunked, I have used HttpVersion10
I have also tried setting httpClient.Timeout = System.Threading.Timeout.Infinite, but it never gets back with a response, however in browser the response gets in few seconds.
Please someone help me in achieving this.

probably you can try setting Agent property, so it doesn't recognize you as a bot.

I think Nero has answered your question ..
Try adding these Lines in your code..
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0";

Truncated response received using HttpWebRequest

I'm using the following code to make HttpWebRequests to a web site:
public static HttpWebResponse SendGETRequest(string url, string agent, CookieContainer cookieContainer)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.UserAgent = agent;
request.Method = "GET";
request.ContentType = "text/html";
request.CookieContainer = cookieContainer;
return (HttpWebResponse)request.GetResponse();
}
Everything worked fine with several web pages until I tried with a new one and only received the last part of the page. This is the received response:
<tr>
<td colspan="2" height="5"><spacer type="block" width="100%" height="5"></td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>
The header is correct and says that only the received data is sent. The following are the headers of the request and response:
Request:
GET /Broker/Ops/FichaContratoJS.asp?nc=815044&IP=5&YY=2012&M=6&St=0&CC=FESX201206 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.168 Safari/535.19
Content-Type: text/html
Host: www.xxxx.com
Cookie: ASPSESSIONIDACBDCDBT=MGDNMNABOANDMILHBNCIDFCH;Autenticacion=Sid=230fae3d%2De0e2%2D4df1%2D8aa8%2D000fb352eaef&IdUsuarioWeb=xxxx; ASPSESSIONIDACBCCDAT=AFDJMNABAFJDDHABLOLAINDK; ASPSESSIONIDCADCBCAT=CEBJGNABLCALPJLDJFPBMLDE
Response:
HTTP/1.1 200 OK
Date: Wed, 09 May 2012 07:25:03 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Pragma: no-cache
**Content-Length: 155**
Content-Type: text/html
Expires: Wed, 09 May 2012 07:24:02 GMT
Set-Cookie: Autenticacion=Sid=230fae3d%2De0e2%2D4df1%2D8aa8%2D000fb352eaef&IdUsuarioWeb=xxxx; path=/
Cache-control: no-cache
Doing the same with a web browser works fine and returns a content length of about 4000 bytes.
Any ideas?
PD: Just in case it matters I'm doing several calls to the SendGETRequest from different threads to the same site but as there are no shared variables I think it shouldn't make a difference.
EDIT: This is the extension I use to extract the text from the Stream:
public static string ReadTextResponse(this Stream stream)
{
int count;
Encoding enconding = System.Text.Encoding.GetEncoding(1252);
System.Text.StringBuilder stringBuilder = new StringBuilder();
byte[] buffer = new byte[1023];
do
{
count = stream.Read(buffer, 0, buffer.Length);
if (count != 0)
{
string tempString = enconding.GetString(buffer, 0, count);
stringBuilder.Append(tempString);
}
}
while (count > 0);
return stringBuilder.ToString();
}
As far as I know it's correct. Also, note that the response header from the server contains the length of the truncated data

I think that you are not using right the HttpWebResponse object.
Maybe you are not closing the request or reading all the response strem.
http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.getresponse.aspx
Your method should be:
public static string SendGETRequest(string url, string agent, CookieContainer cookieContainer)
{
var request = (HttpWebRequest)WebRequest.Create(url);
request.UserAgent = agent;
request.Method = "GET";
request.ContentType = "text/html";
request.CookieContainer = cookieContainer;
string result;
using (var myResponse = (HttpWebResponse) request.GetResponse())
{
using (var stream = myResponse.GetResponseStream())
{
result = null;
if (stream != null)
{
using (var sr = new StreamReader(stream, System.Text.Encoding.UTF8))
{
result = sr.ReadToEnd();
sr.Close();
}
stream.Close();
}
}
myResponse.Close();
}
return result;
}

Incredible ... I was sending the URL /Broker/Ops/FichaContratoJS.asp?nc=815044&IP=5&YY=2012&M=6 and the browser was sending /Broker/Ops/FichaContratoJS.asp?nc=815044&IP=5&YY=2012&M=06& (note the extra 0 on the M parameter (it's a month). Putting there that 0 returned the full page. Sounds like a defect to me

I have run into a similar situation and found that copying the response stream into a MemoryStream seemed to fix my problems.
public static string SendGETRequest(string url, string agent, CookieContainer cookieContainer)
{
var request = (HttpWebRequest)WebRequest.Create(url);
request.UserAgent = agent;
request.Method = "GET";
request.ContentType = "text/html";
request.CookieContainer = cookieContainer;
string result;
using (var myResponse = (HttpWebResponse) request.GetResponse())
{
using (var stream = myResponse.GetResponseStream())
{
result = null;
if (stream != null)
{
MemoryStream memStream = new MemoryStream();
stream.CopyTo(memStream);
memStream.Flush();
stream.Close();
using (var sr = new StreamReader(memStream, System.Text.Encoding.UTF8))
{
result = sr.ReadToEnd();
sr.Close();
}
memStream.Close();
}
}
myResponse.Close();
}
return result;
}