We Keep Coding

Cannot get an httponly cookie with HttpWebResponse on windows phone 7 app

I'm developing an app using C# for a project and I need to get a cookie after a POST request on a website. I'm using HttpWebResponse to get the result of my request. My problem is that the CookieCollection is empty and I don't know why. Is it possible that the cookie doesn't appear because it's an HTTPOnly cookie ?
Here is my code for the entire POST request :
private void RequestPOST(string uri)
{
Uri myUri = new Uri(uri);
HttpWebRequest myRequest = (HttpWebRequest)HttpWebRequest.Create(myUri);
myRequest.Method = "POST";
myRequest.ContentType = "application/x-www-form-urlencoded";
Debug.WriteLine("RequestStream : BEGIN");
myRequest.BeginGetRequestStream(new AsyncCallback(GetRequestStreamCallback), myRequest);
}
private void GetRequestStreamCallback(IAsyncResult callbackResult)
{
HttpWebRequest myRequest = (HttpWebRequest)callbackResult.AsyncState;
Stream postStream = myRequest.EndGetRequestStream(callbackResult);
byte[] byteArray = Encoding.UTF8.GetBytes(this._postData);
postStream.Write(byteArray, 0, byteArray.Length);
postStream.Close();
myRequest.BeginGetResponse(new AsyncCallback(GetResponsetStreamCallback), myRequest);
}
private void GetResponsetStreamCallback(IAsyncResult callbackResult)
{
HttpWebRequest request = (HttpWebRequest)callbackResult.AsyncState;
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(callbackResult);
CookieCollection cookies = response.Cookies;
using (StreamReader httpWebStreamReader = new StreamReader(response.GetResponseStream()))
{
this._retourPost = httpWebStreamReader.ReadToEnd();
Debug.WriteLine(cookies.Count);//My problem appears here, cookieCount throws a NullException
foreach (Cookie cook in response.Cookies)
{
Debug.WriteLine(cook.Name + " : "+ cook.Value);
}
}
Debug.WriteLine("END");
}
I know that there already are some similar questions but I still can't make my application works.
I hope my question is clear.
Thank you.

I finally found why it doesn't work : I forgot to declare the cookiecontainer of the HttpWebRequest. So I use a local field to save the CookieContainer and reuse it for each call to RequestPOST() I do.
private CookieContainer cookiecontainer = new CookieContainer();
private void RequestPOST(string uri)
{
Uri myUri = new Uri(uri);
HttpWebRequest myRequest = (HttpWebRequest)HttpWebRequest.Create(myUri);
myRequest.Method = "POST";
myRequest.ContentType = "application/x-www-form-urlencoded";
myRequest.CookieContainer = this.cookiecontainer;
Debug.WriteLine("RequestStream : BEGIN");
myRequest.BeginGetRequestStream(new AsyncCallback(GetRequestStreamCallback), myRequest);
}
Actually I don't have to read the HTTPOnly cookie, I just need to have it. The CookieContainer still show no cookies because HTTPOnly cookies are not referenced in the container but they are in it anyway.
I hope it will help.

C# submit a POST form and retrieve the GET result

I've seen several posts speaking of this but can't manage to get a working solution...
So basically I want to login to a website by submitting the POST html form then (using the session cookie I guess) retrieve the GET result content once I'm logged in.
To prepare this I used WireShark to sniff the activity and sure enough I find the POST request with the form in index.php info then the GET request in user.php
So to reproduce this in C# :
var request = (HttpWebRequest) WebRequest.Create("xxx/index.php");
var postData = "login_name=xxx";
postData += "&login_pw=xxx";
postData += "&login_submit=Connexion";
postData += "&login_server=1";
var data = Encoding.ASCII.GetBytes(postData);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
using (var stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
var response = (HttpWebResponse) request.GetResponse();
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
First problem is that in WireShqrk when I get to the GET request I see a cookie but in C# in the response the cookies list is empty
I still try to get the resulting content :
request = (HttpWebRequest)WebRequest.Create("xxx/user.php");
response = (HttpWebResponse)request.GetResponse();
responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
But this response is basically the html code for the original login page.
There you go, any advice will be greatly appreciated
thanks

You need to create a new CookieContainer on the request, and share that container on subsequent requests.
var cookieContainer = new CookieContainer();
var request = (HttpWebRequest) WebRequest.Create("xxx/index.php");
request.CookieContainer = cookieContainer;
//do the post...
var getRequest = (HttpWebRequest) WebRequest.Create("xxx/index.php");
getRequest.CookieContainer = cookieContainer;
//do the get...

solution was to not allow auto redirect and get the cookie info from
cookieHeader = resp.Headers["Set-Cookie"];

How to add cookies to WebRequest?

I am trying to unit test some code, and I need to to replace this:
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create( uri );
httpWebRequest.CookieContainer = new CookieContainer();
with
WebRequest webRequest = WebRequest.Create( uri );
webRequest.CookieContainer = new CookieContainer();
Basically, how do I get cookies into the request without using a HttpWebRequest?

Based on your comments, you might consider writing an extension method:
public static bool TryAddCookie(this WebRequest webRequest, Cookie cookie)
{
HttpWebRequest httpRequest = webRequest as HttpWebRequest;
if (httpRequest == null)
{
return false;
}
if (httpRequest.CookieContainer == null)
{
httpRequest.CookieContainer = new CookieContainer();
}
httpRequest.CookieContainer.Add(cookie);
return true;
}
Then you can have code like:
WebRequest webRequest = WebRequest.Create( uri );
webRequest.TryAddCookie(new Cookie("someName","someValue"));

Try with something like this:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.contoso.com/default.html");
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(new Cookie("ConstoCookie", "Chocolate Flavour"));

WebRequest is an abstract class that does not have a CookieContainer property. In addition you can't use the Headers collection (not implemented exception) so any attempt like webRequest.Headers.Add("Cookie", "...") will fail.
Sorry, but you have no chance to use cookies with WebRequest.
Stick on HttpWebRequest and add/edit as many cookies you like using its Headers collection!

dlev's answer ended up working, but I had problems implementing the solution ("The parameter '{0}' cannot be an empty string."), so I decided to write the full code in case anybody else has similar problems.
My goal was to get the html as a string, but I needed to add the cookies to the web request. This is the function that downloads the string using the cookies:
public static string DownloadString(string url, Encoding encoding, IDictionary<string, string> cookieNameValues)
{
using (var webClient = new WebClient())
{
var uri = new Uri(url);
var webRequest = WebRequest.Create(uri);
foreach(var nameValue in cookieNameValues)
{
webRequest.TryAddCookie(new Cookie(nameValue.Key, nameValue.Value, "/", uri.Host));
}
var response = webRequest.GetResponse();
var receiveStream = response.GetResponseStream();
var readStream = new StreamReader(receiveStream, encoding);
var htmlCode = readStream.ReadToEnd();
return htmlCode;
}
}
We are using the code from dlev's answer:
public static bool TryAddCookie(this WebRequest webRequest, Cookie cookie)
{
HttpWebRequest httpRequest = webRequest as HttpWebRequest;
if (httpRequest == null)
{
return false;
}
if (httpRequest.CookieContainer == null)
{
httpRequest.CookieContainer = new CookieContainer();
}
httpRequest.CookieContainer.Add(cookie);
return true;
}
This is how you use the full code:
var cookieNameValues = new Dictionary<string, string>();
cookieNameValues.Add("varName", "varValue");
var htmlResult = DownloadString(url, Encoding.UTF8, cookieNameValues);

Login to Https Website with WebRequest and WebResponse Not Working C#

I am working on a program that will log into a website and get certian data. However I am having trouble posting the login parameters and dealing with the cookies, as each time I get a page saying "You have logged out or Session has expired." So clearly I'm doing something wrong with posting the parameters or dealing with the cookies, but don't know which. I have been working on this for a while and just can't get my head around why this is not working correctly.
void Login2(string username, string password)
{
string pageSource;
string formUrl = "https://forUrl.com";
string formParams = string.Format("login={0}&sslProt={1}&pwd={2}&gru={3}", username, "", password, "115237091");
string cookieHeader;
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(formUrl);
req.AllowAutoRedirect = false;
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string getUrl = "https://Urlbehindform.com";
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.Method = "GET";
getRequest.AllowAutoRedirect = false;
getRequest.Headers.Add("Cookie", cookieHeader);
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
Response.Redirect(getUrl);
}
I am getting the cookie when I do the POST and sending it back when I do the GET, but for some reason this doesn't seem to work. At first I thought it was the parameters, but after looking at the issue further using Tamper Data with Firefox the login parameters seem to be working fine. Any help would be great, as I have been working on this for a while and can't wrap my head around it. Thanks!
UPDATE:
After trying out a few suggestions I still can't get this to work. However Upon looking deeper into Data Tamper, It appears that there is a POST with the login parameters, then a GET to a different page and then finally the GET to the page after the login page (The one I'm trying to get to). After some further debugging I actually discovered that my login POST is not working as I thought, As the response header location is showing "/cv/scripts/A028/eng/logErr.asp". Meaning the rest of my code could have been fine all a long, it was that the POST wasn't giving me a valid login. Any Sugguestions as to why I am always getting the login error page? As always thanks for the help.
UPDATE:
After playing around further with Tamper Data is appears that the reason I am unable to get a successful login is that in order have a successful POST of the parameters there needs to be a cookie already obtained. How do I go about doing this?

Use a single CookieContainer for both requests. Then you don't have to copy cookies manually.
I [BMW1] added in a CookieContainer called cookies, but it still not working, Im not sure if im using the CookieContainer the right way. Here is an updated version of my code.
And edited by me [Hans Kesting], see comments with [HK]
void Login2(string username, string password)
{
string pageSource;
string formUrl = "https://server/cv/scripts/A028/eng/logProc.asp?ntry=0&dbg=";
string formParams = string.Format("login={0}&sslProt={1}&pwd={2}&gru={3}", username, "", password, "115237091");
// [HK] create a container for the cookies, where they are added automatically
CookieContainer cookies = new CookieContainer();
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(formUrl);
req.CookieContainer = cookies;
req.AllowAutoRedirect = false;
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
// [HK] no need to add cookies "by hand", that will happen automatically
//cookies.Add(resp.Cookies);
string getUrl = "https://server/cv/scripts/A028/eng/home.asp";
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
// [HK] use the same cookiecontainer as on the first request - correct
getRequest.CookieContainer = cookies;
getRequest.Method = "GET";
getRequest.AllowAutoRedirect = false;
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
// [HK] no need to add cookies, they should be there already
//cookies.Add(getResponse.Cookies);
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
// [HK] no need to add cookies, they should be there already
// cookies.Add(getResponse.Cookies);
Response.Redirect(getUrl);
}

You could use a Cookie aware web client,
public class CookieAwareWebClient : WebClient
{
public CookieContainer CookieContainer { get; set; }
public Uri Uri { get; set; }
public CookieAwareWebClient() : this (new CookieContainer())
{
}
public CookieAwareWebClient(CookieContainer cookies)
{
this.CookieContainer = cookies;
}
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
if (request is HttpWebRequest)
{
(request as HttpWebRequest).CookieContainer = this.CookieContainer;
}
HttpWebRequest httpRequest = (HttpWebRequest) request;
httpRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
return httpRequest;
}
protected override WebResponse GetWebResponse(WebRequest request)
{
WebResponse response = base.GetWebResponse(request);
String setCookieHeader = response.Headers[HttpResponseHeader.SetCookie];
if (setCookieHeader != null)
{
//do something if needed to parse out the cookie.
if (setCookieHeader != null)
{
Cookie cookie = new Cookie(); //create cookie
this.CookieContainer.Add(cookie);
}
}
return response;
}
}
Example usage:
var wc = new CookieAwareWebClient ();
wc.Headers["Content-type"] = "application/x-www-form-urlencoded";
string HtmlResult = wc.UploadString(URI, myParameters);

c# Post data cookies not working

Alright, so I've got a function I wrote that should allow me to post data with cookies. The problem is is I'm testing it out on an Amazon login page, and it keeps responding saying I need cookies enabled. Here's the code
public string DoPost(String url, PostData data, CookieContainer cookies)
{
HttpWebRequest objWebRequest = (HttpWebRequest)WebRequest.Create(url);
objWebRequest.CookieContainer = cookies;
objWebRequest.AllowAutoRedirect = true;
objWebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";
if(data != null)
{
String post = data.ToString();
objWebRequest.Method = "POST";
objWebRequest.ContentLength = post.Length;
objWebRequest.ContentType = "application/x-www-form-urlencoded";
// Post to the login form.
using(StreamWriter swRequestWriter = new StreamWriter(objWebRequest.GetRequestStream()))
{
swRequestWriter.Write(post);
}
}
// Get the response.
HttpWebResponse objWebResponse =
(HttpWebResponse)objWebRequest.GetResponse();
// Read the response
using(StreamReader srResponseReader = new StreamReader(objWebResponse.GetResponseStream()))
{
string strResponseData = srResponseReader.ReadToEnd();
return strResponseData;
}
}
And I call it like this
String action = "https://www.amazon.com/gp/flex/sign-in/select.html";
String s = DoPost(action, null, Cookies);
Cookies is created in my class constructer like this
CookieContainer Cookies;
public Constructz0r()
{
Cookies = new CookieContainer();
}
The thing is, I'm not even posting any post data, I'm just going to the page, and it's saying my cookies aren't enabled, though I feel I've done it write in DoPost.
I've even tried using this implementation of WebClient
public class CookieWebClient : WebClient
{
private CookieContainer _cookieContainer = new CookieContainer();
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
if (request is HttpWebRequest)
{
(request as HttpWebRequest).CookieContainer = _cookieContainer;
}
return request;
}
}
And calling it like this
using(CookieWebClient ck = new CookieWebClient())
{
String s = ck.DownloadString(action);
}
And it still tells me the cookies aren't enabled.

Amazon have an API to access their services (SOAP). So instead of trying to do some scraping I would strongly recommend you using their API.