We Keep Coding

Cookies and C# HttpWebRequest

I have been trying to log in to a server to grab the authentication cookie (a session cookie), which I can then use for further calls to the server. Every example I have read follows the same pattern:
HttpWebRequest request = WebRequest.Create(loginURL) as HttpWebRequest;
var response = request.GetResponse() as HttpWebResponse;
var cookies = response.Cookies;
This didn't work for me, as the cookies variable ended up empty, and a debug analysis showed response.Cookies was empty. The server is mine, and I can see, through debugging, the cookie is being set. I can also see the cookie in Firefox if I log in to my site with it. So I know the cookie is being set.
After some messing around, I discovered the cookie was being set in the request, not the response. So the code below worked. My question is: Why? Why is the request being populated, but not the response? Is it something to do with being a post, not a get? I am totally baffled.
private void Login()
{
string userName = UserNameText.Text;
string password = PasswordText.Password;
string baseURL = URLText.Text;
string loginURL = baseURL + "/Authentication/LoginAction";
HttpWebRequest request = WebRequest.Create(loginURL) as HttpWebRequest;
request.Method = "POST";
string formContent =
"UserName=" + userName +
"&Password=" + password;
var byteArray = Encoding.UTF8.GetBytes(formContent);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = byteArray.Length;
request.CookieContainer = new CookieContainer();
try
{
using (var dataStream = request.GetRequestStream())
{
dataStream.Write(byteArray, 0, byteArray.Length);
using (var response = request.GetResponse() as HttpWebResponse)
{
var cookies = request.CookieContainer;
if (cookies.Count != 0)
{
cookies_ = cookies;
}
}
}
}
catch(Exception ex)
{
// don't bother too much
Debug.WriteLine(ex.Message);
}
}

The CookieContainer should be considered similar to a browser's cookie cache for a particular site. The idea is that you supply the container as part of the request, and then it's populated by the cookies you receive and you can reuse that container for subsequent requests. When you make a request, the cookies in the container are sent with the request (just like the browser would with stored cookies).
So, for example, if you have a page that uses cookies to store an authentication token, you can pass the cookie container with the login request, and then pass it with subsequent requests which require an authenticated cookie.
As to why you can't simply extract it from the request, I guess Microsoft just didn't want to duplicate things when you can pass in a reference to a mutable cookie container in the request.

Login to website which requires Cookies using c#

I'm trying to write a script in c# capable to retrieve some information from a website. These information are protected so I need to login before I can read them. That's what I think should be my procedure:
First of all use a POST request to login into the website. Here my first problem: the page where I find the login form is this https://idp.kk-abcdefg.com/idp/Authn/UserPassword . Should I submit the POST request to this page or should I use a different address?
I've tested the headers using some tools of Firefox or Chrome but I can't understand which is the right procedure. I have noticed that If I open this login page I receive some cookies. If I delete these cookies and try to login by inserting user and password (via browser) I get an error as a response from the website .. it says that I need to activate cookies to be able to login. So it seems like when I open the login page for the first time I receive some cookies and then I need to send them together with the first POST request for login. Does it make sense for any of you?
That's the code I'm using right now:
string formUrl = "https://idp.kk-abcdefg.de/idp/Authn/UserPassword";
string formParams = string.Format("j_username=MyUserName&j_password=MyPassword”);
string cookieHeader;
string pageSource;
WebRequest req = WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
using (StreamReader sr = new StreamReader(resp.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
And I believe this is working because the result of pageSource changes if I use correct user / pass or I write them wrong. But when user / pass are correct, I'm still not able to login because I get the following error message: "This application requires cookies. Please make sure cookies are enabled in the settings of your browser. Please reload the login page and try logging in again".
This is the same error that I get if I disable cookies in my browser or if I delete cookies that I get when I load the login page for the first time.
Can you help me with all that? My idea is that I need to save the cookies received when I open the login page for the first time and then send them together with following requests but I don't know how to do ..
Thanks a lot!

In Web Application, once the user is logged in successfully a cookie is sent back to the browser to track the user session and to determine if the user is logged in or not during further requests. Furthermore login process of your application requires cookies to be sent from client along with username and password. So when you are trying to perform login without browser it complains about missing cookies.
If you know what cookies and their values need to be sent along with username and password for login, you can send them using cookieContainer in WebRequest as following.
string formUrl = "https://idp.kk-abcdefg.de/idp/Authn/UserPassword";
string formParams = string.Format("j_username=MyUserName&j_password=MyPassword");
string cookieHeader;
string pageSource;
CookieContainer cookieContainer = new CookieContainer();
Cookie cookie1 = new Cookie("<<cookiename>>", "<<cookievalue>>","/", "<<yourdomainname>>");
Cookie cookie2 = new Cookie("<<cookiename>>", "<<cookievalue>>", "/", "<<yourdomainname>>");
cookieContainer.Add(cookie1);
cookieContainer.Add(cookie2);
// You can keep adding all required cookies this way.
var req = (HttpWebRequest)WebRequest.Create(formUrl);
req.CookieContainer = cookieContainer;
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
// You can access the cookies coming as part of response as following.
HttpWebResponse response = resp as HttpWebResponse;
if(response != null)
{
var cookiesCollections = response.Cookies;
}
using (StreamReader sr = new StreamReader(resp.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
If you don't know the cookies and you need to get the cookies first by requesting login page before posting username and password then use following.
var loginPageUrl = "<<Your Login Page url>>";
CookieContainer cookieContainer = new CookieContainer();
var req = (HttpWebRequest)WebRequest.Create(loginPageUrl);
req.CookieContainer = cookieContainer;
req.Method = "GET";
WebResponse resp = req.GetResponse();
HttpWebResponse response = resp as HttpWebResponse;
CookieCollection cookies;
if (response != null)
{
cookies = response.Cookies; //Use this cookies in above code to send with username and password.
}

You send the request login twice and use single CookieContainer object!

C# setting cookie using HttpWebRequest

I'm using a test automation platform called Ranorex. The code is C#. I would like to set a cookie to the server using HttpWebRequest before I open a browser to begin the test.
Below is the code. Everything executes with no problem. When I view the cookies using the browser - mine is not there (there are 54 other cookies) - when I iterate the response as shown below - I only have three (3) cookies.
Your help is appreciated.
This method will execute the test
void ITestModule.Run()
{
SummaryHelper.KillAllInternetExplorerProcesses();
uri = this.createURI();
// Using HttpWebRequest to set a cookie to the session
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.CookieContainer = new CookieContainer();
Cookie myCookie = new Cookie("mockFlagForTesting", "true", "/", "safeqa.thomson.com");
request.CookieContainer.Add(myCookie);
// Create the processStartInfo obejct to open the IE Browser
// I expect the cookie to be loaded into the session
ProcessStartInfo processStartInfo = new ProcessStartInfo(
#"C:\Program Files\Internet Explorer\iexplore.exe");
processStartInfo.WindowStyle = ProcessWindowStyle.Maximized;
processStartInfo.Arguments = uri;
SummaryBase.process = Process.Start(processStartInfo);
// Create and set a session cookie.
setHTTPCookie();
}
private void setHTTPCookie()
{
// We will attempt to set the cookie here
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.CookieContainer = new CookieContainer();
Cookie myCookie = new Cookie("mockFlagForTesting", "true", "/", "safeqa.thomson.com");
// Add the cookie
request.CookieContainer.Add(myCookie);
// Do we need to use POST here to write to the server ??
// Set the request.Method using WebRequestMethods.Http.Get
request.Method = WebRequestMethods.Http.Get;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
// Iterate the cookies
// We only display three (3) ??
foreach (Cookie cook in response.Cookies)
{
Report.Info("-------------------------------------------------------------");
Report.Info("cook.Name", cook.Name);
Report.Info("cook.Value", cook.Value);
Report.Info("Domain: ", cook.Domain);
Report.Info("Path: ", cook.Path);
}
response.Close();
}
Thanks
Chris

You need to set cookie in browser, not on some random web request.
You can either push cookies via script running on the page OR inject cookies to request if you can intercept requests (i.e. using Fiddler/ Fiddler Core).

How to set cookie in request header for REST service

I am developing webservice which calls REST service and as per requirement I need to set cookie in the request header. EVen if I set everything , it gives me "401 Unauthorized message" as response.
var request = (HttpWebRequest)HttpWebRequest.Create("https://TESTRestAPI/service/1/sub");
request.Headers.Add("Authorization", "Basic" + Encoded);
request.Method = "GET";
request.ContentType = "application/json";
var response = (WebResponse)request.GetResponse();
So far I tried this :-
1. request.Headers["Cookie"] = "BasicAuth=fromTest";
2. request.CookieContainer = new CookieContainer();
Uri target = new Uri("https://TESTRestAPI/service/1/sub"););
Cookie ck = new Cookie("BasicAuth","fromTest") { Domain = target.Host };
This is first time I am calling REST service, any help appreciated.

this code is to authenticate an http request by using basic authentication, with some Username and Password
byte[] credentialBuffer = new UTF8Encoding().GetBytes(username + ":" + password);
string authToken = Convert.ToBase64String(credentialBuffer);
string authHeaderValue = string.Format(System.Globalization.CultureInfo.InvariantCulture, "Basic {0}", authToken);
HttpWebRequest request = (HttpWebRequest)System.Net.WebRequest.Create("https://TESTRestAPI/service/1/sub");
request.Headers.Add(HttpRequestHeader.Authorization, authHeaderValue);
Additionaly, if it is required that the request contains some cookie with some name and some value, Id' use your code number 2.
CookieContainer requestCookieContainer = new CookieContainer();
Cookie requiredCookie = new Cookie("BasicAuth", "fromTest");
requiredCookie.Domain = "https://TESTRestAPI/service/1/sub";
requiredCookieContainer.Add(requiredCookie);
request.CookieContainer = requestCookieContainer;
If your application runs on IIS, the config file may need some lines. I think basic authentication is disabled by default, as it isn't too much safe.
<system.webServer>
<security>
<authentication>
<basicAuthentication enabled="true"/>
</authentication>
</security>
</system.webServer>

Problem with http 'POST' request

I am running in a stupid problem. I have an method which returns initialized request resposible for loggin in on some external web site.
protected HttpWebRequest GetLoginRequest()
{
const string url = "https://someurl.com/login";
var queryParams = new ArrayList
{
String.Format("{0}={1}", "email", Email),
String.Format("{0}={1}", "password", DecryptedPassword)
};
var parameters = String.Join("&", (String[])queryParams.ToArray(typeof(String)));
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = parameters.Length;
request.Timeout = 10000;
var streamWriter = new StreamWriter(request.GetRequestStream());
streamWriter.Write(parameters);
streamWriter.Close();
return request;
}
I'm calling this method from two places in my code. First call looks like that:
var request = GetLoginRequest();
var response = (HttpWebResponse)request.GetResponse();
And the second one has CookieContainer assigned to request:
var cookieContainer = new CookieContainer();
var request = GetLoginRequest();
request.CookieContainer = cookieContainer;
var response = (HttpWebResponse)request.GetResponse();
because I need to store CookieContainer.
The thing is that the logon is performed only in second case. In the first case i'm getting response from the login page. I've checked all the cases and both resulting requests seem identic. I would suggest that it is target site secific, but still I don't see any reason for that.
Can you please explain what is the reason, because this behavior seems pretty unobvious to me.

When you set the CookieContainer property on your request, the response is populating that CookieContainer instance with the cookies received from the executed request.
Most login mechanisms use a cookie to store the state related to the established login. I.e. in the case of Forms authentication the cookie is the container for the forms authentication ticket. The ticket is passed as the value of the forms authentication cookie with each request and is used by forms authentication, on the server, to identify an authenticated user.
In short you need a CookieContainer for each request after you login, and it needs to contain the forms authentication cookie that you received when you logged in.
Edit to clarify comment - from MSDN:
CookieContainer is null by default.
You must assign a CookieContainer
object to the property to have cookies
returned in the Cookies property of
the HttpWebResponse returned by the
GetResponse method.
For security reasons, cookies are disabled by default. If you want to
use cookies, use the CookieContainer
property to enable cookies.