I asked a question on here earlier and got some fantastic responses. I've since been diddling Visual C# and ran into a bit of a problem.
Here I made a simple page that sets a cookie.
If you go to it and then refresh, it'll see if there's a cookie present and change the output html.
Now, I want my C# program to fetch a page, get a cookie and then re-visit the page again with the cookie that is set, so that my page presents me the "updated" message. I accomplished phase one via:
private void button1_Click(object sender, RoutedEventArgs e)
{
WebRequest request = WebRequest.Create("http://www.binarywatch.biz/forms/cookietest.php");
request.Credentials = CredentialCache.DefaultCredentials;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
MessageBox.Show(responseFromServer, "Derp");
reader.Close();
dataStream.Close();
response.Close();
}
So at this point I have the page html but I'm a bit lost as to how to go about getting a cookie (Something to do with CookieContainer() ? ) and then making the page know that I have it (by adding it to the httpwebrequest somehow?)
I tried googling it of course but a LOT of the answers I find are about ASP.NET / web programming and that's not what I need.
PS. What's the difference between WebRequest and HttpWebRequest?
I hope this isn't too noobish, I'm a bit stumped.
according to MSDN you'd first create an instance of the CookieContainer before calling getResponse. After that you should be able to get cookie data out of the CookieContainer you created.
(request as HttpWebRequest).CookieContainer = new CookieContainer();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
CookieCollection cookies = (request as HttpWebRequest).CookieContainer.GetCookies("www.binarywatch.biz");
string myValue cookies["myCookie"].Value
You should be able to re-use the same CookieContainer object to make sure the server keeps getting access to the cookies.
The GetCookies(domain) is needed as a single CookieContainer is able to store separate cookies safely for multiple domains.
Taken from "how to use cookies in httpwebrequest?"
Yes, use CookieContainer.
CookieContainer cookieContainer = new CookieContainer();
httpWebRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
httpWebRequest.CookieContainer = cookieContainer;
From that answer:
"After the GetResponse call the
cookieContainer will contain the
cookies sent back from the requested
Url."
I haven't tested this, but it was the accepted answer so it must work. Hope it works for you.
Related
Before everyone gets upset that this has been answered. I have scoured the web looking for how to do this and have tried a number of methods. Login to website, via C# and How to programmatically log in to a website to screenscape? Both of these were helpful but I cannot figure out why I cannot get past the login page. Here is my code:
string url = "https://www.advocare.com/login.aspx";
string url2 = "https://url.after.login";
HttpWebRequest wReq = WebRequest.Create(url) as HttpWebRequest;
wReq.KeepAlive = true;
wReq.Method = "POST";
wReq.AllowAutoRedirect = false;
wReq.ContentType = "application/x-www-form-urlencoded";
string postData = "ctl00$cphContent$txtUserName=Username&ctl00$cphContent$txtPassword=Password";
byte[] dataBytes = UTF8Encoding.UTF8.GetBytes(postData);
wReq.ContentLength = dataBytes.Length;
using (Stream postStream = wReq.GetRequestStream())
{
postStream.Write(dataBytes, 0, dataBytes.Length);
}
HttpWebResponse wResp = wReq.GetResponse() as HttpWebResponse;
string pageSource;
wReq = WebRequest.Create(url2) as HttpWebRequest;
wReq.CookieContainer = new CookieContainer();
wReq.CookieContainer.Add(wResp.Cookies);
HttpWebResponse wResp2 = wReq.GetResponse() as HttpWebResponse;
using (StreamReader sr = new StreamReader(wResp2.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
Everytime I look at pageSource it is the HTML for the login.aspx page. I must be missing something here. Maybe it's not taking the cookie, I don't know. One question I have aside from, why doesn't this work, is in the string postData = "". Are those suppose to be the name or id portion of the html tag? Any help on this is greatly appreciated as I am stumped and will have to find a different way. I would like to continue with the WebRequest and WebResponse instead of using WebBrowser. If I can't, oh well. Thanks again for any help!
What are you trying to do besides login? If its like QAing a site programically, i would suggest using selenium andcreate a c# app based off of that. If u want i can post a link to a base project for a selenium based project.
Don't necessarily view the page source, but look at the actual HTTP POST. Install a HTTP proxy such as Fiddler and then re-visit the page you are trying to emulate. Complete the HTTP POST request, and check out the results produced in the proxy. From there you'll be able to see the actual parameters, cookies, headers, etc. that are being passed and you can then attempt to replicate this in your code. It's often easy to miss something when simply viewing the HTML source but monitoring the network traffic is pretty straight forward.
Is there a way to get the .ASPXAUTH value programmatically.
Example I login to a website with my own credentials (POST) and then read the response...it does not return the .APSXAUTH in the CookieContainer that I use to track the session.
Anyone has a clue how can I get it and send it with the subsequent gets and posts?
[EDIT] Here's what I do to be more specific:
send a HTTP GET to a page. read values like _VIEWSTATE etc.
send a HTTP POST to the Login page. It includes the login information.
The server sends a 302 response (redirect) to some Default page. The forms authentication cookie is supposed to be included but it's not.
So I was thinking that there might be a better way than this to track session:
CookieContainer _cookieJar = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(_url);
request.CookieContainer = _cookieJar;
So the summarize the answer:
If you're trying to login programatically on a Forms based authentication website trough your own application make sure you follow the steps you take that track the cookies.
First create a initial GET request, and then do the subsequential POST requests that will do the postback.The request and the responses should be formulated in this way:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(_url);
request.CookieContainer = _cookieJar;
HttpWebResponse httpsResponse = (HttpWebResponse)request.GetResponse();
The CookieContainer class handles the cookies as expected.
And if your response is encoded with Gzip just include the following line:
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
before you call request.GetResponse()
Hope this helps someone out there.
I'm trying to log in to my eBay account using the following code:
string signInURL = "https://signin.ebay.com/ws/eBayISAPI.dll?co_partnerid=2&siteid=0&UsingSSL=1";
string postData = String.Format("MfcISAPICommand=SignInWelcome&userid={0}&pass={1}", "username", "password");
string contentType = "application/x-www-form-urlencoded";
string method = "POST";
string userAgent = "Mozilla/5.0 (Windows; U; MSIE 7.0; Windows NT 6.0; en-US)";
CookieContainer cookieContainer = new CookieContainer();
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(signInURL);
req.CookieContainer = cookieContainer;
req.Method = method;
req.ContentType = contentType;
req.UserAgent = userAgent;
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] loginDataBytes = encoding.GetBytes(postData);
req.ContentLength = loginDataBytes.Length;
Stream stream = req.GetRequestStream();
stream.Write(loginDataBytes, 0, loginDataBytes.Length);
stream.Close();
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
StreamReader xsr = new StreamReader(res.GetResponseStream());
String responseText = xsr.ReadToEnd();
Obviously substituting my real username and password. When I look at the string responseText, I see that part of the response from eBay is
The browser you are using is rejecting cookies.
Any ideas what I'm doing wrong?
P.S. And yes, I am also using the eBay API, but this is for something slightly different than what I want to do with the API.
You're doing a direct http request. The Ebay site has functionality to talk to a browser (probably to store the session cookie). Unless you make the request code smart enough to use cookies correctly it won't work. You'll probably have to use the internet explorer object instead.
Before doing the POST you need to download the page with the form that you are submitting in your code, take the cookie they give you, put it in your CookieContainer (making sure you get the path right) and post it back up in your request.
To clarify, while you might be POSTing the correct data, you are not sending the cookie that needs to go with it. You will get this cookie from the login page.
You need to intercept the http traffic to see what exactly what had happened. I use Fiddler2. It is the good tools for debugging http. So I can know whos wrong, my application or the remote web server.
Using fiddler, you can see the request header, response header with its cookies as well as response content. It used in the middle of your app and the Ebay.
Based on my experience. I think it is because Ebay cookie sent to you is not send back to Ebay server. Fiddler will prove it whether yes or not.
Another thing, the response cookie you receive should be send back to next request by using the same CookieContainer.
You should notice that CookieContainer has a bug on .Add(Cookie) and .GetCookies(uri) method. You may not using it, but internal codes might use it.
See the details and fix here:
http://dot-net-expertise.blogspot.com/2009/10/cookiecontainer-domain-handling-bug-fix.html
CallMeLaNN
I'm trying to create an HttpWebRequest/HttpWebResponse session with an ASP.NET website to later parse an HTML form through url params (this part I know how to do), but I do not understand how to parse and set a cookie such as the session id. In Fiddler, it shows that the ASP.NET Session ID is returned through Set-Cookie in the response to the request to the / path of the url, but how can I extract this session id and set it as a cookie for the next HttpWebRequest? I understand that this Set-Cookie header would be found in HttpWebResponse.Headers.Keys, but is there a direct path to parsing it? Thanks!
The .NET framework will manage cookies for you. You don't have to concern yourself with parsing the cookie information out of the headers or adding a cookie header to your requests.
To store and send your session ID, use the Cookie and CookieContainer classes to store them and then make sure you send your cookies with every request.
The following example shows how to do this. The CookieContainer, 'cookieJar' can be shared across multiple domains and requests. Once you add it to a request object, the reference to it will also be added to the response object when the response is returned.
CookieContainer cookieJar = new CookieContainer();
var request = (HttpWebRequest)HttpWebRequest.Create("http://www.google.com");
request.CookieContainer = cookieJar;
var response = request.GetResponse();
foreach (Cookie c in cookieJar.GetCookies(request.RequestUri))
{
Console.WriteLine("Cookie['" + c.Name + "']: " + c.Value);
}
The output of this code will be:
Cookie['PREF']: ID=59e9a22a8cac2435:TM=1246226400:LM=1246226400:S=tvWTnbBhK4N7Tlpu
The answer from Dan Herbert helped me really. I appreciate your help.
Just want to post my usage - hope it helps some one at some point of time. My requirement is that I need to send back cookies from first http post response to second http post request.
1st:
CookieContainer cookieJar = new CookieContainer();
request.CookieContainer = cookieJar;
....
CookieCollection setCookies = cookieJar.GetCookies(request.RequestUri);
2nd:
CookieContainer cc = new CookieContainer();
cc.Add(setCookies);
request.CookieContainer = cc;
I have the same problem (with amazon)
I use the following regexp:
string regexp = "(?<name>[^=]+)=(?<val>[^;]+)[^,]+,?";);MatchCollection myMatchCollection = Regex.Matches(cookiesStr, regexp);foreach (Match myMatch in myMatchCollection) { string cookieName = myMatch.Groups["name"].ToString(); string cookieVal = myMatch.Groups["val"].ToString(); Cookie cookie = new Cookie(cookieName, cookieVal); cookies.Add(cookie); }
Note that I only care about the cookie name/value...
good luck
Elia
hum I may be wrong but from what I am observing lately
Cookies from a first response, don't include the 'set cookie' as cookies that come in the header (for example some session id...) in the case of a 302 (redirect) status
If the autofollowredirect is set to true, then the set cookie are processed, and the subsequent request which is done automatically, will include those cookies defined by set cookie on the first call
If autofollowredirect is set to false then the first request doesn't get the cookies positionned by the set cookie, and I guess and this is also my queston if anyone know, that the only way to subsequently have those cookies in next request, is parse the set cookies ?
Is there any way to automatically handle cookies in .NET with the HttpWebRequest/HttpWebResponse objects? I'm preferably looking for an equivalent to LWP::UserAgent and its behaviour (perl), only in a .NET environment.
Any suggestions or advice?
I think what you're looking for is the CookieContainer class. If I understand what you're trying to do correctly, you have separate objects for request & response, and you want to transfer the response cookie collection into the next request cookie collection automatically. Try using this code:
CookieContainer cookieJar = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://www.google.com");
request.CookieContainer = cookieJar;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
int cookieCount = cookieJar.Count;
Once you create a cookieJar and set it to the request's CookieContainer, it will store any cookies that come from the response, so in the example above, the cookie jar's count will be 1 once it visits Google.com. The cookie container properties of the request & response above will store a pointer to the cookieJar, so the cookies are automatically handled and shared between the objects.