I am using HttpWebRequest with HttpWebResponse, the latter named response22 in my code, so here is a snippet from my code:
HttpWebResponse response22 = request22.GetResponse() as HttpWebResponse;
CookieCollection cookiezzz = new CookieCollection();
cookiezzz.Add(response22.Cookies);
foreach (System.Net.Cookie cookie in cookiezzz)
{
MessageBox.Show(cookie.Name);
}
Strangely enough, Fiddler shows 5 cookies in the response, but when I iterate through the cookies, I get only four.
Also, my request is set to:
equest22.AllowAutoRedirect = false;
Target framework is .Net 4.5, using WinForms
And using CookieContaner did not help at all as it "picks" only 2 of these cookies, but I don't want to worry about that right now, just want to figure out how to get all five cookies.
Instead of trying to retrieve them from the response, you have to supply the cookie container to the request. That will force the container and the response cookies to be filled:
var cookiezzz = new CookieContainer();
request22.CookieContainer = cookiezzz;
HttpWebResponse response22 = request22.GetResponse() as HttpWebResponse;
foreach (System.Net.Cookie cookie in cookiezzz)
{
MessageBox.Show(cookie.Name);
}
Related
I have been trying to log in to a server to grab the authentication cookie (a session cookie), which I can then use for further calls to the server. Every example I have read follows the same pattern:
HttpWebRequest request = WebRequest.Create(loginURL) as HttpWebRequest;
var response = request.GetResponse() as HttpWebResponse;
var cookies = response.Cookies;
This didn't work for me, as the cookies variable ended up empty, and a debug analysis showed response.Cookies was empty. The server is mine, and I can see, through debugging, the cookie is being set. I can also see the cookie in Firefox if I log in to my site with it. So I know the cookie is being set.
After some messing around, I discovered the cookie was being set in the request, not the response. So the code below worked. My question is: Why? Why is the request being populated, but not the response? Is it something to do with being a post, not a get? I am totally baffled.
private void Login()
{
string userName = UserNameText.Text;
string password = PasswordText.Password;
string baseURL = URLText.Text;
string loginURL = baseURL + "/Authentication/LoginAction";
HttpWebRequest request = WebRequest.Create(loginURL) as HttpWebRequest;
request.Method = "POST";
string formContent =
"UserName=" + userName +
"&Password=" + password;
var byteArray = Encoding.UTF8.GetBytes(formContent);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = byteArray.Length;
request.CookieContainer = new CookieContainer();
try
{
using (var dataStream = request.GetRequestStream())
{
dataStream.Write(byteArray, 0, byteArray.Length);
using (var response = request.GetResponse() as HttpWebResponse)
{
var cookies = request.CookieContainer;
if (cookies.Count != 0)
{
cookies_ = cookies;
}
}
}
}
catch(Exception ex)
{
// don't bother too much
Debug.WriteLine(ex.Message);
}
}
The CookieContainer should be considered similar to a browser's cookie cache for a particular site. The idea is that you supply the container as part of the request, and then it's populated by the cookies you receive and you can reuse that container for subsequent requests. When you make a request, the cookies in the container are sent with the request (just like the browser would with stored cookies).
So, for example, if you have a page that uses cookies to store an authentication token, you can pass the cookie container with the login request, and then pass it with subsequent requests which require an authenticated cookie.
As to why you can't simply extract it from the request, I guess Microsoft just didn't want to duplicate things when you can pass in a reference to a mutable cookie container in the request.
The following code:
string url = "https://accounts.spotify.com/en/login";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.CookieContainer = new CookieContainer();
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
using (StreamReader sr = new StreamReader(resp.GetResponseStream()))
{
string source = sr.ReadToEnd();
MessageBox.Show(source);
MessageBox.Show(resp.Cookies.Count.ToString());
foreach (Cookie cookie in resp.Cookies)
{
MessageBox.Show(cookie.Name, cookie.Value);
}
}
Is meant to make a Message Box for each cookie with the name and value.
But it doesn't! not a single value gets returned.
Instead it does the following:
Shows the Source, the amount of Cookies (Shows 0) and then nothing else just a brief 1-2 seconds before the form loads.
It should respond with csrf_token=<...>
As you can see, even in the Fiddler response you can see the cookies.
I am not sure why resp.Cookies no having any cookies in it. But if you observe resp.Headers has one value Set-Cookie. You can use that to retrieve cookies coming in the response.
Use following line of code to get the cookie header.
var cookieHeader = resp.Headers["Set-Cookie"];
This will give you value like
csrf_token={sometokenvalue};Version=1;Domain=accounts.spotify.com;Path=/;Secure
Ok, I've been racking my brain on this one solo for too long. I've been unable to crack it even with hours spent on this and many other sites.
Essentially, I'm trying to strip some data from a webpage behind a LogIn page using WebRequest/Response. (I have gotten this to work using a WebBrowser control with some layered events which navigate to the different web pages but it's causing some problems when trying to refactor - not to mention it's been stated that using a hidden form to do the work is 'bad practice'.)
This is what I have:
string formParams = string.Format("j_username={0}&j_password={1}", userName, userPass);
string cookieHeader;
WebRequest request = WebRequest.Create(_signInPage);
request.ContentType = "text/plain";
request.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
request.ContentLength = bytes.Length;
using (Stream os = request.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse response = request.GetResponse();
cookieHeader = response.Headers["Set-Cookie"];
WebRequest getRequest = WebRequest.Create(sessionHistoryPage);
getRequest.Method = "GET";
getRequest.Headers.Add("Cookie", cookieHeader);
WebResponse getResponse = getRequest.GetResponse();
try
{
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
textBox1.AppendText(sr.ReadToEnd());
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
throw;
}
So far, I'm able to get to the proper page from the first link but when I go to the second, it sends me back to the login page as if I didn't log in.
The problem may lie in cookies not getting captured correctly but I'm a novice so maybe I'm just doing it wrong. It captures the cookies sent back from the POST: JSESSIONID and S2V however, when we go to the "GET", using FireFox WebConsole, the browser shows that it sends JSESSIONID, S2V and a SPRING_SECURITY_REMEMBER_ME_COOKIE, which I believe is the cookie used when I click the "Remember Me" box on the login form.
I've tried many different ways of doing this using the resources of SO but I have yet to get to the page I need. So, for the sake of the hair I have left, I've decided to ask for help on good ole SO. (This is one of those things I don't want to let up on - stubborn like that sometimes)
If someone wants the actual address of the site I'm trying to log into, I'd be more than happy to send it to a couple people in a private message.
Code that I have to reflect a suggested answer by Wal:
var request = (HttpWebRequest)WebRequest.Create(sessionHistoryPage);
request.Credentials = new NetworkCredential(userName, userPass);
request.CookieContainer = new CookieContainer();
request.PreAuthenticate = true;
WebResponse getResponse = request.GetResponse();
try
{
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
textBox1.AppendText(sr.ReadToEnd());
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
throw;
}
This suggestion, at least the way I implemented it, didn't work.
As Krizz suggested, I changed the code to use CookieContainer and transferring the cookies from one request to the other however, the response just gives me back the original login page as if I didn't login.
Are there certain sites that just WILL NOT allow this type of behavior?
Final Solution
The final solution was proposed by Adrian Iftode where he stated that the website I'm trying to log in might not allow to have an authentication without a valid session so adding a GET to the beginning of the process allowed me to get that cookie.
Thanks for all your help guys!
I was doing some sort of cookie transfer for a website written with PHP.
Clearly you are passing the cookie, but maybe is like in that situation
var phpsessid = response.Headers["Set-Cookie"].Replace("; path=/", String.Empty);
The Set-Cookie header contains other related info about the cookie and possible other instructions for other cookies. I had one cookie with its info (Path), the session id which I needed to sent back to the server so the server would know that I am the same client which did the GET request.
The new request had to include this cookie
request.Headers["Cookie"] = phpsessid;
You already do this, but make sure the cookies you receive, you sent back to the server.
Considering session and authentication, there are two cookies, one for session, one for authentication and some servers/application might not allow to have an authentication without a valid session. What I want to say is that you might need to pass the session cookie too. So the steps would be:
Do first a GET request to obtain the session cookie.
Next do the POST request to authenticate and get the auth cookie.
Use both cookies to navigate to the protected pages.
Also check this question, it doesn't show the entire class, but the idea is to keep the CookieContainer in the same class, add the new cookies from POST/GET requests and assign them to the each new request, like #Krizz answered.
Try using CookieContainer which is a class to keep cookies context between several requests. You simply create an instance of it and assign it to each WebRequest.
Therefore, modifying your code:
string formParams = string.Format("j_username={0}&j_password={1}", userName, userPass);
string cookieHeader;
var cookies = new CookieContainer(); // added this line
var request = WebRequest.Create(_signInPage) as HttpWebRequest; // modified line
request.CookieContainer = cookies; // added this line
request.ContentType = "text/plain";
request.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
request.ContentLength = bytes.Length;
using (Stream os = request.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
request.GetResponse(); // removed some code here, no need to read response manually
var getRequest = WebRequest.Create(sessionHistoryPage) as HttpWebRequest; //modified line
getRequest.CookieContainer = cookies; // added this line
getRequest.Method = "GET";
WebResponse getResponse = getRequest.GetResponse();
try
{
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
textBox1.AppendText(sr.ReadToEnd());
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
throw;
}
I know this question has been asked quite a lot of times which is how I have got to where I am at with the code below however I just can't get it to work on the particular website I am trying to access. At the site I am trying to access I need to retrieve certain values from the page however things like price and availability only come up after logging in so I am trying to submit my login information and then go to the product page to get the information I need using HTML Agility Pack.
At the moment it seems to attempt the login however the website is either not accepting it or the cookies are not present on the next page load to actually keep me logged in.
If someone could help me with this I would be very grateful as I am not a programmer but have been assigned this task as part of a software installation.
protected void Button5_Click(object sender, System.EventArgs e)
{
string LOGIN_URL = "http://www.videor.com/quicklogin/1/0/0/0/index.html";
string SECRET_PAGE_URL = "http://www.videor.com/item/47/32/0/703/index.html?scriptMode=&CUSTOMERNO=xxx&USERNAME=xxx&activeTabId=0";
// have a cookie container ready to receive the forms auth cookie
CookieContainer cookies = new CookieContainer();
// first, request the login form to get the viewstate value
HttpWebRequest webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
webRequest.CookieContainer = cookies;
StreamReader responseReader = new StreamReader(
webRequest.GetResponse().GetResponseStream()
);
string responseData = responseReader.ReadToEnd();
responseReader.Close();
string postData = "CUSTOMERNO=xxxx&USERNAME=xxxxx&PASSWORD=xxxxx";
// now post to the login form
webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.CookieContainer = cookies;
// write the form values into the request message
StreamWriter requestWriter = new StreamWriter(webRequest.GetRequestStream());
requestWriter.Write(postData);
requestWriter.Close();
// we don't need the contents of the response, just the cookie it issues
webRequest.GetResponse().Close();
// now we can send out cookie along with a request for the protected page
webRequest = WebRequest.Create(SECRET_PAGE_URL) as HttpWebRequest;
webRequest.CookieContainer = cookies;
responseReader = new StreamReader(webRequest.GetResponse().GetResponseStream());
// and read the response
responseData = responseReader.ReadToEnd();
responseReader.Close();
Response.Write(responseData);
}
This isn't a direct answer since I'm not sure what's wrong with your code (from a cursory glance it looks ok), but another approach is to use browser automation using Selenium . The following code will actually load the page using Chrome (you can swap out Firefox or IE) and is simpler to code against. It also won't break if they add javascript or something.
var driver = new ChromeDriver();
driver.Navigate().GoToUrl(LOGON_URL);
driver.FindElement(By.Id("UserName")).SendKeys("myuser");
driver.FindElement(By.Id("Password")).SendKeys("mypassword");
driver.FindElement(By.TagName("Form")).Submit();
driver.Navigate().GoToUrl(SECRET_PAGE_URL);
// And now the html can be found as driver.PageSource. You can also look for
// different elements and get their inner text and stuff as well.
I am running in a stupid problem. I have an method which returns initialized request resposible for loggin in on some external web site.
protected HttpWebRequest GetLoginRequest()
{
const string url = "https://someurl.com/login";
var queryParams = new ArrayList
{
String.Format("{0}={1}", "email", Email),
String.Format("{0}={1}", "password", DecryptedPassword)
};
var parameters = String.Join("&", (String[])queryParams.ToArray(typeof(String)));
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = parameters.Length;
request.Timeout = 10000;
var streamWriter = new StreamWriter(request.GetRequestStream());
streamWriter.Write(parameters);
streamWriter.Close();
return request;
}
I'm calling this method from two places in my code. First call looks like that:
var request = GetLoginRequest();
var response = (HttpWebResponse)request.GetResponse();
And the second one has CookieContainer assigned to request:
var cookieContainer = new CookieContainer();
var request = GetLoginRequest();
request.CookieContainer = cookieContainer;
var response = (HttpWebResponse)request.GetResponse();
because I need to store CookieContainer.
The thing is that the logon is performed only in second case. In the first case i'm getting response from the login page. I've checked all the cases and both resulting requests seem identic. I would suggest that it is target site secific, but still I don't see any reason for that.
Can you please explain what is the reason, because this behavior seems pretty unobvious to me.
When you set the CookieContainer property on your request, the response is populating that CookieContainer instance with the cookies received from the executed request.
Most login mechanisms use a cookie to store the state related to the established login. I.e. in the case of Forms authentication the cookie is the container for the forms authentication ticket. The ticket is passed as the value of the forms authentication cookie with each request and is used by forms authentication, on the server, to identify an authenticated user.
In short you need a CookieContainer for each request after you login, and it needs to contain the forms authentication cookie that you received when you logged in.
Edit to clarify comment - from MSDN:
CookieContainer is null by default.
You must assign a CookieContainer
object to the property to have cookies
returned in the Cookies property of
the HttpWebResponse returned by the
GetResponse method.
For security reasons, cookies are disabled by default. If you want to
use cookies, use the CookieContainer
property to enable cookies.