After a long day in searching and attempts I could not find a solution for my problem - getting multiple "Set-Cookie" headers.
I was trying to get them via Headers["Set-Cookie"] but it contains the only the first cookie.
Then my next try was to get them via Cookies (which is always with Count 0 and the purpose because I think they are malformed).
Here is the code to the last sample:
var request = WebRequest.Create(resourceUrl) as HttpWebRequest;
request.Method = "GET";
var response = request.GetResponse() as HttpWebResponse;
var cookiesCount = response.Cookies.Count;
And these are the cookies:
Set-Cookie:vjfmrii=67ea0de93a423ab17d168ee8327617b0
Set-Cookie:alpocjengi=dcf10w329x5d7e503ffb9f28123c7492f1c2deb4
Set-Cookie:vjfmrii=4abf7b9e97fff1a61fbcf5e11899ce71
Before I began, I thought that this would be one of the easiest parts, but unfortunately I was wrong. These cookies which have the same names and have several... have left me devastated.
I appreciate any help. Thanks in advance.
You need to add a CookieContainer to your request when you make it or the cookies will not be added to the response. See http://msdn.microsoft.com/en-us/library/dd920298(v=vs.95).aspx.
var request = WebRequest.Create(resourceUrl) as HttpWebRequest;
request.Method = "GET";
request.CookieContainer = new CookieContainer(); // <-- Add this
var response = request.GetResponse() as HttpWebResponse;
var cookiesCount = response.Cookies.Count;
Related
The following code:
string url = "https://accounts.spotify.com/en/login";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.CookieContainer = new CookieContainer();
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
using (StreamReader sr = new StreamReader(resp.GetResponseStream()))
{
string source = sr.ReadToEnd();
MessageBox.Show(source);
MessageBox.Show(resp.Cookies.Count.ToString());
foreach (Cookie cookie in resp.Cookies)
{
MessageBox.Show(cookie.Name, cookie.Value);
}
}
Is meant to make a Message Box for each cookie with the name and value.
But it doesn't! not a single value gets returned.
Instead it does the following:
Shows the Source, the amount of Cookies (Shows 0) and then nothing else just a brief 1-2 seconds before the form loads.
It should respond with csrf_token=<...>
As you can see, even in the Fiddler response you can see the cookies.
I am not sure why resp.Cookies no having any cookies in it. But if you observe resp.Headers has one value Set-Cookie. You can use that to retrieve cookies coming in the response.
Use following line of code to get the cookie header.
var cookieHeader = resp.Headers["Set-Cookie"];
This will give you value like
csrf_token={sometokenvalue};Version=1;Domain=accounts.spotify.com;Path=/;Secure
I have a web role in which I have the following code snippet:
var webRequest = HttpWebRequest.Create(path);
webRequest.Timeout = 5000;
webRequest.Method = "GET";
It's requesting a resource from Azure CDN, so path is something like the following:
https://<uniqueString>.vo.msecnd.net/<container>/path/image.png
I noticed that the response I am getting is cached, because I checked in Fiddler, going directly to the resource, that I have different headers from the following webResponse.
var webResponse = (HttpWebResponse)webRequest.GetResponse();
So HttpRequest enforces some kind of CachePolicy here?
I tried replacing the first snippet with the following code but it didn't work:
WebRequest.DefaultCachePolicy = new RequestCachePolicy(RequestCacheLevel.NoCacheNoStore);
var webRequest = HttpWebRequest.Create(path);
webRequest.Timeout = 5000;
webRequest.Method = "GET";
Only when I did set the headers it did work, so the following code gives me the correct response:
var webRequest = HttpWebRequest.Create(path);
webRequest.Timeout = 5000;
webRequest.Method = "GET";
webRequest.Headers.Set(HttpRequestHeader.CacheControl, "max-age=0, no-cache, no-store");
Why HttpWebRequest was enforcing cache in the response and why the DefaultCachePolicy did not work after I tried to set it, except only explicitly setting the headers worked?
Note: The response was only caching when requesting the resource over HTTPS
Update
To be sure I've just added a query string parameter to the requested resource URI, putting the time ticks. Is this enough for me to not have cached responses?
I am using HttpWebRequest with HttpWebResponse, the latter named response22 in my code, so here is a snippet from my code:
HttpWebResponse response22 = request22.GetResponse() as HttpWebResponse;
CookieCollection cookiezzz = new CookieCollection();
cookiezzz.Add(response22.Cookies);
foreach (System.Net.Cookie cookie in cookiezzz)
{
MessageBox.Show(cookie.Name);
}
Strangely enough, Fiddler shows 5 cookies in the response, but when I iterate through the cookies, I get only four.
Also, my request is set to:
equest22.AllowAutoRedirect = false;
Target framework is .Net 4.5, using WinForms
And using CookieContaner did not help at all as it "picks" only 2 of these cookies, but I don't want to worry about that right now, just want to figure out how to get all five cookies.
Instead of trying to retrieve them from the response, you have to supply the cookie container to the request. That will force the container and the response cookies to be filled:
var cookiezzz = new CookieContainer();
request22.CookieContainer = cookiezzz;
HttpWebResponse response22 = request22.GetResponse() as HttpWebResponse;
foreach (System.Net.Cookie cookie in cookiezzz)
{
MessageBox.Show(cookie.Name);
}
I let my program (c#) log in to website, i get correct buffered cookie information. Yet, i get a 401, session timed out, when i want to retrieve the correct data behind my login.
So i figured, the website must not be able to retrieve that cookie info. Can't figure out how to store it so that the website can retrieve it.
WebRequest req = WebRequest.Create(Url);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(Gegevens);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
cookieHeader, contains the correct info. Thanks in advance.
You need to assign a CookieContainer to your web request and use this very same CookieContainer for the following requests, too.
See MSDN for reference.
You could (if you want to persist the cookies upon closing your application) get the list of Cookies from the CookieContainer and serialize these. Upon opening the application you could deserialize and rebuild the CookieContainer.
From the comment you provided, I'm going to hazard a guess and say you aren't properly adding your login cookies to your next WebRequest. Cookie handling with a WebRequest object is a bit difficult, so I recommend using HttpWebRequest and HttpWebResponse which has built-in cookie parsing. You only have to change a couple lines of code here and there:
Building a request (using the same example in your question)
CookieContainer cookies = new CookieContainer();
// When using HttpWebRequest or HttpWebResponse, you need to cast for it to work properly
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.CookieContainer = cookies;
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(Gegevens);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
// Cast here as well
using (HttpWebResponse resp = (HttpWebResponse)req.GetResponse())
{
// Code related to the web response goes in here
}
Right now, your cookie information is saved in the CookieContainer object. It can be reused later in your code to validate your login. You don't need to write this cookie information to the disk if you don't need to.
Building a request with cookie information
(Pretty much the same as above, but you're not adding POST data and you're using a GET request)
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.CookieContainer = cookies; // This is where you add your login cookies to the current request
req.Method = "GET";
using (HttpWebResponse resp = (HttpWebResponse)req.GetResponse())
{
// Code related to the web response goes here
}
Hopefully this will get you on the right track :)
My web request with the method "Head" keeps returning the body of my webpage (on localhost).
Here is how it is basically created:
HttpWebRequest webrequest = WebRequest.Create(url.ToString()) as HttpWebRequest;
webrequest.Method = "HEAD";
WebResponse response = webrequest.GetResponse();
As I put a breakpoint in my aspx.cs page, I step into the OnInit() method and also the Page_Load() method where I believe I'm not supposed to step in with a Head method request (am I wrong?).
In my Page_Load() I execute some code that I do not want to be executed when I call with the Head method, but later when I call with the Get method (once I got the headers).
Am I missing something? (not too familiar with Http requests and responses yet... :/)
Thank you for your help!
Try this sample code approach ....
for (int i = 0; i < ParsedLinks.Count; i++)
{
Thread.Sleep(500);
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(ParsedLinks[i]);
req.Method = "HEAD";
req.Credentials = CredentialCache.DefaultCredentials;
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
bool b_Result = int.TryParse(resp.Headers.Get("Content-Length"), out i_ContentLength);
int i_Size = (int)(i_ContentLength / 1024);
req.Abort();
resp.Close();
}
hope it helps
http://forums.asp.net/t/1412824.aspx/1
The difference is, if you try to read the response you get nothing in case of HEAD. Where as you can see the response content in case of GET
var response = webrequest.GetResponse().GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
StreamReader readStream = new StreamReader(response, encode);
Console.WriteLine(readStream.ReadToEnd().Length) // you should see 0
readStream.Close();