We Keep Coding

Why HttpWebResponse response was cached and setting the DefaultCachePolicy for HttpWebRequest did not work?

I have a web role in which I have the following code snippet:
var webRequest = HttpWebRequest.Create(path);
webRequest.Timeout = 5000;
webRequest.Method = "GET";
It's requesting a resource from Azure CDN, so path is something like the following:
https://<uniqueString>.vo.msecnd.net/<container>/path/image.png
I noticed that the response I am getting is cached, because I checked in Fiddler, going directly to the resource, that I have different headers from the following webResponse.
var webResponse = (HttpWebResponse)webRequest.GetResponse();
So HttpRequest enforces some kind of CachePolicy here?
I tried replacing the first snippet with the following code but it didn't work:
WebRequest.DefaultCachePolicy = new RequestCachePolicy(RequestCacheLevel.NoCacheNoStore);
var webRequest = HttpWebRequest.Create(path);
webRequest.Timeout = 5000;
webRequest.Method = "GET";
Only when I did set the headers it did work, so the following code gives me the correct response:
var webRequest = HttpWebRequest.Create(path);
webRequest.Timeout = 5000;
webRequest.Method = "GET";
webRequest.Headers.Set(HttpRequestHeader.CacheControl, "max-age=0, no-cache, no-store");
Why HttpWebRequest was enforcing cache in the response and why the DefaultCachePolicy did not work after I tried to set it, except only explicitly setting the headers worked?
Note: The response was only caching when requesting the resource over HTTPS
Update
To be sure I've just added a query string parameter to the requested resource URI, putting the time ticks. Is this enough for me to not have cached responses?

C# WebRequest POST returns 500 internal server error

I want to submit xml via postbody to a web service, but am getting a 500 Error from the GetResponse() method.
Here is my snippet:
WebRequest request = WebRequest.Create(URL);
request.Method = "POST";
request.ContentType = "application/xml";
request.Credentials = new NetworkCredential("UserName", "Pass", "Domain");
byte[] body= Encoding.UTF8.GetBytes(myXML);
request.ContentLength = body.Length;
request.GetRequestStream().Write(body, 0, body.Length);
WebResponse response = request.GetResponse(); //500 response
Does anyone know why the server is responding with this error? Am I providing all the needed headers? I have confirmed that the service works through a REST Client Console, but can't seem to POST to it via code.

I had to set request.ContentType = "application/x-www-form-urlencoded" instead of request.ContentType = "application/xml" for it to correctly POST the xml data in the request. If anyone out there has a the technical explanation as to why application/x-www-form-urlencoded worked instead of application/xml, I would love to hear it.

Getting multiple malformed cookies via c#

After a long day in searching and attempts I could not find a solution for my problem - getting multiple "Set-Cookie" headers.
I was trying to get them via Headers["Set-Cookie"] but it contains the only the first cookie.
Then my next try was to get them via Cookies (which is always with Count 0 and the purpose because I think they are malformed).
Here is the code to the last sample:
var request = WebRequest.Create(resourceUrl) as HttpWebRequest;
request.Method = "GET";
var response = request.GetResponse() as HttpWebResponse;
var cookiesCount = response.Cookies.Count;
And these are the cookies:
Set-Cookie:vjfmrii=67ea0de93a423ab17d168ee8327617b0
Set-Cookie:alpocjengi=dcf10w329x5d7e503ffb9f28123c7492f1c2deb4
Set-Cookie:vjfmrii=4abf7b9e97fff1a61fbcf5e11899ce71
Before I began, I thought that this would be one of the easiest parts, but unfortunately I was wrong. These cookies which have the same names and have several... have left me devastated.
I appreciate any help. Thanks in advance.

You need to add a CookieContainer to your request when you make it or the cookies will not be added to the response. See http://msdn.microsoft.com/en-us/library/dd920298(v=vs.95).aspx.
var request = WebRequest.Create(resourceUrl) as HttpWebRequest;
request.Method = "GET";
request.CookieContainer = new CookieContainer(); // <-- Add this
var response = request.GetResponse() as HttpWebResponse;
var cookiesCount = response.Cookies.Count;

Writing a cookie locally

I let my program (c#) log in to website, i get correct buffered cookie information. Yet, i get a 401, session timed out, when i want to retrieve the correct data behind my login.
So i figured, the website must not be able to retrieve that cookie info. Can't figure out how to store it so that the website can retrieve it.
WebRequest req = WebRequest.Create(Url);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(Gegevens);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
cookieHeader, contains the correct info. Thanks in advance.

You need to assign a CookieContainer to your web request and use this very same CookieContainer for the following requests, too.
See MSDN for reference.
You could (if you want to persist the cookies upon closing your application) get the list of Cookies from the CookieContainer and serialize these. Upon opening the application you could deserialize and rebuild the CookieContainer.

From the comment you provided, I'm going to hazard a guess and say you aren't properly adding your login cookies to your next WebRequest. Cookie handling with a WebRequest object is a bit difficult, so I recommend using HttpWebRequest and HttpWebResponse which has built-in cookie parsing. You only have to change a couple lines of code here and there:
Building a request (using the same example in your question)
CookieContainer cookies = new CookieContainer();
// When using HttpWebRequest or HttpWebResponse, you need to cast for it to work properly
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.CookieContainer = cookies;
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(Gegevens);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
// Cast here as well
using (HttpWebResponse resp = (HttpWebResponse)req.GetResponse())
{
// Code related to the web response goes in here
}
Right now, your cookie information is saved in the CookieContainer object. It can be reused later in your code to validate your login. You don't need to write this cookie information to the disk if you don't need to.
Building a request with cookie information
(Pretty much the same as above, but you're not adding POST data and you're using a GET request)
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.CookieContainer = cookies; // This is where you add your login cookies to the current request
req.Method = "GET";
using (HttpWebResponse resp = (HttpWebResponse)req.GetResponse())
{
// Code related to the web response goes here
}
Hopefully this will get you on the right track :)

HttpWebRequest method HEAD returns body

My web request with the method "Head" keeps returning the body of my webpage (on localhost).
Here is how it is basically created:
HttpWebRequest webrequest = WebRequest.Create(url.ToString()) as HttpWebRequest;
webrequest.Method = "HEAD";
WebResponse response = webrequest.GetResponse();
As I put a breakpoint in my aspx.cs page, I step into the OnInit() method and also the Page_Load() method where I believe I'm not supposed to step in with a Head method request (am I wrong?).
In my Page_Load() I execute some code that I do not want to be executed when I call with the Head method, but later when I call with the Get method (once I got the headers).
Am I missing something? (not too familiar with Http requests and responses yet... :/)
Thank you for your help!

Try this sample code approach ....
for (int i = 0; i < ParsedLinks.Count; i++)
{
Thread.Sleep(500);
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(ParsedLinks[i]);
req.Method = "HEAD";
req.Credentials = CredentialCache.DefaultCredentials;
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
bool b_Result = int.TryParse(resp.Headers.Get("Content-Length"), out i_ContentLength);
int i_Size = (int)(i_ContentLength / 1024);
req.Abort();
resp.Close();
}
hope it helps
http://forums.asp.net/t/1412824.aspx/1

The difference is, if you try to read the response you get nothing in case of HEAD. Where as you can see the response content in case of GET
var response = webrequest.GetResponse().GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
StreamReader readStream = new StreamReader(response, encode);
Console.WriteLine(readStream.ReadToEnd().Length) // you should see 0
readStream.Close();