We Keep Coding

Why HttpWebResponse response was cached and setting the DefaultCachePolicy for HttpWebRequest did not work?

I have a web role in which I have the following code snippet:
var webRequest = HttpWebRequest.Create(path);
webRequest.Timeout = 5000;
webRequest.Method = "GET";
It's requesting a resource from Azure CDN, so path is something like the following:
https://<uniqueString>.vo.msecnd.net/<container>/path/image.png
I noticed that the response I am getting is cached, because I checked in Fiddler, going directly to the resource, that I have different headers from the following webResponse.
var webResponse = (HttpWebResponse)webRequest.GetResponse();
So HttpRequest enforces some kind of CachePolicy here?
I tried replacing the first snippet with the following code but it didn't work:
WebRequest.DefaultCachePolicy = new RequestCachePolicy(RequestCacheLevel.NoCacheNoStore);
var webRequest = HttpWebRequest.Create(path);
webRequest.Timeout = 5000;
webRequest.Method = "GET";
Only when I did set the headers it did work, so the following code gives me the correct response:
var webRequest = HttpWebRequest.Create(path);
webRequest.Timeout = 5000;
webRequest.Method = "GET";
webRequest.Headers.Set(HttpRequestHeader.CacheControl, "max-age=0, no-cache, no-store");
Why HttpWebRequest was enforcing cache in the response and why the DefaultCachePolicy did not work after I tried to set it, except only explicitly setting the headers worked?
Note: The response was only caching when requesting the resource over HTTPS
Update
To be sure I've just added a query string parameter to the requested resource URI, putting the time ticks. Is this enough for me to not have cached responses?

How to register the device via SOAP?

I'm executing the following code in order to register a randomized user name and a password for login using live id.
String regUrl = "https://login.live.com/ppsecure/DeviceAddCredential.srf";
WebRequest request = WebRequest.Create(regUrl);
request.ContentType = "application/soap+xml; charset=UTF-8";
request.Method = "POST";
request.Timeout = 60000;
StreamReader reader = new StreamReader(request.GetRequestStream());
String contents = reader.ReadToEnd();
I'm following the code in this article and it's not going quick nor easy. I get the error telling me that stream was not readable.
What am I missing and what can be done about it?

You're trying to read from a request stream, when you should be writing to it. The response stream contains the server's reply.

Writing a cookie locally

I let my program (c#) log in to website, i get correct buffered cookie information. Yet, i get a 401, session timed out, when i want to retrieve the correct data behind my login.
So i figured, the website must not be able to retrieve that cookie info. Can't figure out how to store it so that the website can retrieve it.
WebRequest req = WebRequest.Create(Url);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(Gegevens);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
cookieHeader, contains the correct info. Thanks in advance.

You need to assign a CookieContainer to your web request and use this very same CookieContainer for the following requests, too.
See MSDN for reference.
You could (if you want to persist the cookies upon closing your application) get the list of Cookies from the CookieContainer and serialize these. Upon opening the application you could deserialize and rebuild the CookieContainer.

From the comment you provided, I'm going to hazard a guess and say you aren't properly adding your login cookies to your next WebRequest. Cookie handling with a WebRequest object is a bit difficult, so I recommend using HttpWebRequest and HttpWebResponse which has built-in cookie parsing. You only have to change a couple lines of code here and there:
Building a request (using the same example in your question)
CookieContainer cookies = new CookieContainer();
// When using HttpWebRequest or HttpWebResponse, you need to cast for it to work properly
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.CookieContainer = cookies;
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(Gegevens);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
// Cast here as well
using (HttpWebResponse resp = (HttpWebResponse)req.GetResponse())
{
// Code related to the web response goes in here
}
Right now, your cookie information is saved in the CookieContainer object. It can be reused later in your code to validate your login. You don't need to write this cookie information to the disk if you don't need to.
Building a request with cookie information
(Pretty much the same as above, but you're not adding POST data and you're using a GET request)
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.CookieContainer = cookies; // This is where you add your login cookies to the current request
req.Method = "GET";
using (HttpWebResponse resp = (HttpWebResponse)req.GetResponse())
{
// Code related to the web response goes here
}
Hopefully this will get you on the right track :)

Formatting Request Header Values

Trying to figure out if I need to escape characters in my header value. Much like the example belowif I'm sending a header authorization over for OAuth to a server resource:
“PUT /api/v1/articles/6.json HTTP/1.1\r\nAccept: /\r\nUser-Agent: OAuth gem v0.4.5\r\nContent-Length: 9\r\nContent-Type: application/x-www-form-urlencoded\r\nAuthorization: OAuth oauth_consumer_key=\”nMu4u9pLRfDrxhPVK5yn\“, oauth_nonce=\”5346IG1e5bV3ytQwdFqkP8Rgr0VJiA9Xb4FE0\“, oauth_signature=\”64545G%2Byp%2F2BsqJ%2BUUgbjIIIV9E%3D\“, oauth_signature_method=\”HMAC-SHA1\“, oauth_timestamp=\”1330022891\“, oauth_token=\”ivouGxpsJbyIU5viPKOO\“, oauth_version=\”1.0\“\r\nConnection: close\r\nHost: someHostNameHere\r\n\r\n”
Right now I'm sending it (the value portion for my Authorization collection) over as one big string like this for my value for the authorization key:
"OAuth oauth_consumer_key=Mu4u9pLRfDrxhPVK5y, oauth_nonce=5346IG1e5bV3ytQwdFqkP8Rgr0VJiA9Xb4FE0, oauth_signature=64545G%2Byp%2F2BsqJ%2BUUgbjIIIV9E%3D, oauth_signature_method=HMAC-SHA1, oauth_timestamp=1330022891, oauth_token=ivouGxpsJbyIU5viPKOO, oauth_version=1.0"

Here's how I would do it:
var url = "https://somedomain.com/resource/v1/";
var request = WebRequest.Create(url) as HttpWebRequest;
request.Method = "POST";
request.ContentType = "text/xml";
var oAuthHeader = "OAuth oauth_consumer_key=Mu4u9pLRfDrxhPVK5y, oauth_nonce=5346IG1e5bV3ytQwdFqkP8Rgr0VJiA9Xb4FE0, oauth_signature=64545G%2Byp%2F2BsqJ%2BUUgbjIIIV9E%3D, oauth_signature_method=HMAC-SHA1, oauth_timestamp=1330022891, oauth_token=ivouGxpsJbyIU5viPKOO, oauth_version=1.0";
request.Headers.Add("Authorization", oAuthHeader);
var response = request.GetResponse();
Not sure why you would want to construct your own raw HttpRequest. Obviously you would call some function to generate your OAuth header values, this is demonstrative only.

Help with translating a PHP method to C#

Can anyone help? I have a PHP method that sends an http post:
<?php
// API URL
$api_url = "url_of_the_API ";
// Apache auth
$api_user = "user";
$api_pass = "pass";
// Request
$xml = "";
$context = stream_context_create(
array (
'http' => array(
'method' => 'POST',
'header' => sprintf("Authorization: Basic %s\r\n",
base64_encode($api_user.':'.$api_pass))
."Content-type: application/x-www-form-urlencoded\r\n",
'content' => http_build_query(array('xml' => $xml)),
'timeout' => 5,
),
)
);
$response = file_get_contents($api_url,false,$context);
header ("Content-Type:text/xml");
echo $response;
?>
I need to translate this to C# and I am getting a reply back from the webservice saying "POST call with XML required". I am being told that:
"the XML is not being sent via POST in the 'xml' (name) variable? This is a similar process to submitting a webform with one variable called 'xml'."
Below is my C#
// this is where we will send it
const string uri = "theUri";
// create a request
var request = (HttpWebRequest)
WebRequest.Create(uri); request.KeepAlive = false;
request.ProtocolVersion = HttpVersion.Version10;
request.Method = "POST";
request.Credentials = new NetworkCredential("theUserName", "thePassword");
// turn our request string into a byte stream
var postBytes = Encoding.ASCII.GetBytes(postData);
// this is important - make sure you specify type this way
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postBytes.Length;
var requestStream = request.GetRequestStream();
// now send it
requestStream.Write(postBytes, 0, postBytes.Length);
requestStream.Close();
// grab te response and print it out to the console along with the status code
var response = (HttpWebResponse)request.GetResponse();
Console.WriteLine(new StreamReader(response.GetResponseStream()).ReadToEnd());
Console.WriteLine(response.StatusCode);

Just as the error message says, you need to prepend xml= to your postData (just before converting it to bytes). In other words, your C# code skips this step:
$postdata = http_build_query(array('xml' => $xml))
and does this instead:
$postdata = $xml;
(In such cases, it has always helped me to set up Wireshark/tshark/tcpdump on either of the connection endpoints (in this case, probably on your server) and then compare the HTTP requests made by the two scripts. It's easy to notice the differences that way.)