I know this question has been asked quite a lot of times which is how I have got to where I am at with the code below however I just can't get it to work on the particular website I am trying to access. At the site I am trying to access I need to retrieve certain values from the page however things like price and availability only come up after logging in so I am trying to submit my login information and then go to the product page to get the information I need using HTML Agility Pack.
At the moment it seems to attempt the login however the website is either not accepting it or the cookies are not present on the next page load to actually keep me logged in.
If someone could help me with this I would be very grateful as I am not a programmer but have been assigned this task as part of a software installation.
protected void Button5_Click(object sender, System.EventArgs e)
{
string LOGIN_URL = "http://www.videor.com/quicklogin/1/0/0/0/index.html";
string SECRET_PAGE_URL = "http://www.videor.com/item/47/32/0/703/index.html?scriptMode=&CUSTOMERNO=xxx&USERNAME=xxx&activeTabId=0";
// have a cookie container ready to receive the forms auth cookie
CookieContainer cookies = new CookieContainer();
// first, request the login form to get the viewstate value
HttpWebRequest webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
webRequest.CookieContainer = cookies;
StreamReader responseReader = new StreamReader(
webRequest.GetResponse().GetResponseStream()
);
string responseData = responseReader.ReadToEnd();
responseReader.Close();
string postData = "CUSTOMERNO=xxxx&USERNAME=xxxxx&PASSWORD=xxxxx";
// now post to the login form
webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.CookieContainer = cookies;
// write the form values into the request message
StreamWriter requestWriter = new StreamWriter(webRequest.GetRequestStream());
requestWriter.Write(postData);
requestWriter.Close();
// we don't need the contents of the response, just the cookie it issues
webRequest.GetResponse().Close();
// now we can send out cookie along with a request for the protected page
webRequest = WebRequest.Create(SECRET_PAGE_URL) as HttpWebRequest;
webRequest.CookieContainer = cookies;
responseReader = new StreamReader(webRequest.GetResponse().GetResponseStream());
// and read the response
responseData = responseReader.ReadToEnd();
responseReader.Close();
Response.Write(responseData);
}
This isn't a direct answer since I'm not sure what's wrong with your code (from a cursory glance it looks ok), but another approach is to use browser automation using Selenium . The following code will actually load the page using Chrome (you can swap out Firefox or IE) and is simpler to code against. It also won't break if they add javascript or something.
var driver = new ChromeDriver();
driver.Navigate().GoToUrl(LOGON_URL);
driver.FindElement(By.Id("UserName")).SendKeys("myuser");
driver.FindElement(By.Id("Password")).SendKeys("mypassword");
driver.FindElement(By.TagName("Form")).Submit();
driver.Navigate().GoToUrl(SECRET_PAGE_URL);
// And now the html can be found as driver.PageSource. You can also look for
// different elements and get their inner text and stuff as well.
Related
I'm trying to write a utility which will attempt to login to the Microsoft Online Admin website and report back as to whether it is reachable.
Using code mainly from this article, http://odetocode.com/articles/162.aspx and some screen scraping I have pieced together the following. Unfortunately it doesn't work, the final response shows that I am still looking at the login page rather than the target page.
Any help would be terrific. Thanks in advance.
private void LoginToSite()
{
const string LOGIN_URL = "https://admin.microsoftonline.com/Login.aspx";
const string USERNAME = "<username>";
const string PASSWORD = "<password>";
const string TARGET_PAGE_URL = "https://admin.noam.microsoftonline.com/Home/Home.aspx";
// first, request the login form to get the viewstate value
HttpWebRequest webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
StreamReader responseReader = new StreamReader(
webRequest.GetResponse().GetResponseStream()
);
string responseData = responseReader.ReadToEnd();
responseReader.Close();
// extract the viewstate value and build out POST data
string viewState = ExtractViewState(responseData);
string postData =
String.Format(
"__VIEWSTATE={0}&AdminCenterLoginControl$UserNameTextBox={1}&AdminCenterLoginControl$PasswordTextbox={2}&__EVENTTARGET=AdminCenterLoginControl_ActionButton",
viewState, USERNAME, PASSWORD
);
// have a cookie container ready to receive the forms auth cookie
CookieContainer cookies = new CookieContainer();
// now post to the login form
webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.CookieContainer = cookies;
// write the form values into the request message
StreamWriter requestWriter = new StreamWriter(webRequest.GetRequestStream());
requestWriter.Write(postData);
requestWriter.Close();
// we don't need the contents of the response, just the cookie it issues
webRequest.GetResponse().Close();
// now we can send out cookie along with a request for the protected page
webRequest = WebRequest.Create(TARGET_PAGE_URL) as HttpWebRequest;
webRequest.CookieContainer = cookies;
responseReader = new StreamReader(webRequest.GetResponse().GetResponseStream());
// and read the response
responseData = responseReader.ReadToEnd();
responseReader.Close();
MessageBox.Show(responseData);
}
private string ExtractViewState(string s)
{
string viewStateNameDelimiter = "__VIEWSTATE";
string valueDelimiter = "value=\"";
int viewStateNamePosition = s.IndexOf(viewStateNameDelimiter);
int viewStateValuePosition = s.IndexOf(
valueDelimiter, viewStateNamePosition
);
int viewStateStartPosition = viewStateValuePosition +
valueDelimiter.Length;
int viewStateEndPosition = s.IndexOf("\"", viewStateStartPosition);
return HttpUtility.UrlEncodeUnicode(
s.Substring(
viewStateStartPosition,
viewStateEndPosition - viewStateStartPosition
)
);
}
edit
private void LoginToSite()
{
const string LOGIN_URL = "https://admin.microsoftonline.com/login.aspx?ReturnUrl=%2fDefault.aspx";
const string USERNAME = "<username>";
const string PASSWORD = "<password>";
// Request the login form to get the viewstate value
HttpWebRequest webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
string response1 = new StreamReader(webRequest.GetResponse().GetResponseStream()).ReadToEnd();
// Extract the viewstate value and build our POST data
string viewState = ExtractViewState(response1);
string postData = String.Format(
"__VIEWSTATE={0}&AdminCenterLoginControl$UserNameTextBox={1}&AdminCenterLoginControl$PasswordTextbox={2}&__EVENTTARGET=AdminCenterLoginControl_ActionButton",
viewState, USERNAME, PASSWORD);
// Set up the Request properties
webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
CookieContainer cookies = new CookieContainer();
webRequest.CookieContainer = cookies;
// Post back to the form
using (StreamWriter requestWriter = new StreamWriter(webRequest.GetRequestStream()))
{
requestWriter.Write(postData);
}
// Read response
string response2 = new StreamReader(webRequest.GetResponse().GetResponseStream()).ReadToEnd();
MessageBox.Show(response2);
}
It would appear that MicrosoftOnline.com does not use Windows Live IDs (aka Passport) for login. This is a shame, since there are libraries available that make logging into LiveID pretty simple for client apps.
Your code hits the login page first, scraps cookies from the response, then attempts to navigate to the target page. This doesn't match the normal flow of user behavior. Normally, the user clicks on a link to go to a target page and the web site redirects the request to the login page if the user is not logged in. After logging in, the login page redirects back to the originally requested target page.
You can see this by looking at the login URL when you visit admin.microsoftonline.com in the browser. You are immediately redirected to the login page, but the full URL on the login page is: https://admin.microsoftonline.com/login.aspx?ReturnUrl=%2fDefault.aspx
Note the ReturnUrl query param at the end. This tells the login page what page to redirect back to when the login is completed.
I don't know if redirect is required by the login page, but since this is the primary path for actual end user interaction (that works) and not the path your code is taking, it's something to consider. Among other things, the redirect to login/ redirect back to target technique will take care of setting the browser cookies for the target domain automatically.
p.s. I notice also that the email administration portion of Microsoft Online services uses a different login URL. From this page (http://www.microsoft.com/online/signin.aspx) clicking on the Exchange Hosted Services Administrative Center link takes you to http:admin.messaging.microsoft.com, which immediately redirects to a login url of https://sts.messaging.microsoft.com/login.aspx?ReturnUrl=%2fDefault.aspx%3fwa%3dwsignin1.0%26wtrealm%3dhttps%253a%252f%252fadmin.messaging.microsoft.com%26wctx%3drm%253d0%2526id%253dpassive%2526ru%253d%25252f%26wct%3d2010-10-27T17%253a11%253a50Z&wa=wsignin1.0&wtrealm=https%3a%2f%2fadmin.messaging.microsoft.com&wctx=rm%3d0%26id%3dpassive%26ru%3d%252f&wct=2010-10-27T17%3a11%3a50Z
The domain name sts.messaging.microsoft.com suggests that the Exchange Hosted Services portion of Microsoft Online Services is using a Security Token Service, which suggests that this login system is capable of federated single sign on between different services. You might be able to connect to this using something like the Windows Identity Foundation (WIF) client components. Will that work with the rest of Microsoft Online Services? I don't know.
> // now we can send out cookie along with a request for the protected page
> webRequest = WebRequest.Create(TARGET_PAGE_URL) as
> HttpWebRequest;
> webRequest.CookieContainer = cookies;
> responseReader = new StreamReader(webRequest.GetResponse().GetResponseStream());
Aren't you setting WebRequest.CookieContainer equal to the blank cookie container that you generated earlier?
Shouldn't you be doing something like:
// we don't need the contents of the response, just the cookie it issues
WebResponse response = webRequest.GetResponse();
cookies = response.cookies;
response.Close();
Ok, I've been racking my brain on this one solo for too long. I've been unable to crack it even with hours spent on this and many other sites.
Essentially, I'm trying to strip some data from a webpage behind a LogIn page using WebRequest/Response. (I have gotten this to work using a WebBrowser control with some layered events which navigate to the different web pages but it's causing some problems when trying to refactor - not to mention it's been stated that using a hidden form to do the work is 'bad practice'.)
This is what I have:
string formParams = string.Format("j_username={0}&j_password={1}", userName, userPass);
string cookieHeader;
WebRequest request = WebRequest.Create(_signInPage);
request.ContentType = "text/plain";
request.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
request.ContentLength = bytes.Length;
using (Stream os = request.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse response = request.GetResponse();
cookieHeader = response.Headers["Set-Cookie"];
WebRequest getRequest = WebRequest.Create(sessionHistoryPage);
getRequest.Method = "GET";
getRequest.Headers.Add("Cookie", cookieHeader);
WebResponse getResponse = getRequest.GetResponse();
try
{
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
textBox1.AppendText(sr.ReadToEnd());
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
throw;
}
So far, I'm able to get to the proper page from the first link but when I go to the second, it sends me back to the login page as if I didn't log in.
The problem may lie in cookies not getting captured correctly but I'm a novice so maybe I'm just doing it wrong. It captures the cookies sent back from the POST: JSESSIONID and S2V however, when we go to the "GET", using FireFox WebConsole, the browser shows that it sends JSESSIONID, S2V and a SPRING_SECURITY_REMEMBER_ME_COOKIE, which I believe is the cookie used when I click the "Remember Me" box on the login form.
I've tried many different ways of doing this using the resources of SO but I have yet to get to the page I need. So, for the sake of the hair I have left, I've decided to ask for help on good ole SO. (This is one of those things I don't want to let up on - stubborn like that sometimes)
If someone wants the actual address of the site I'm trying to log into, I'd be more than happy to send it to a couple people in a private message.
Code that I have to reflect a suggested answer by Wal:
var request = (HttpWebRequest)WebRequest.Create(sessionHistoryPage);
request.Credentials = new NetworkCredential(userName, userPass);
request.CookieContainer = new CookieContainer();
request.PreAuthenticate = true;
WebResponse getResponse = request.GetResponse();
try
{
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
textBox1.AppendText(sr.ReadToEnd());
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
throw;
}
This suggestion, at least the way I implemented it, didn't work.
As Krizz suggested, I changed the code to use CookieContainer and transferring the cookies from one request to the other however, the response just gives me back the original login page as if I didn't login.
Are there certain sites that just WILL NOT allow this type of behavior?
Final Solution
The final solution was proposed by Adrian Iftode where he stated that the website I'm trying to log in might not allow to have an authentication without a valid session so adding a GET to the beginning of the process allowed me to get that cookie.
Thanks for all your help guys!
I was doing some sort of cookie transfer for a website written with PHP.
Clearly you are passing the cookie, but maybe is like in that situation
var phpsessid = response.Headers["Set-Cookie"].Replace("; path=/", String.Empty);
The Set-Cookie header contains other related info about the cookie and possible other instructions for other cookies. I had one cookie with its info (Path), the session id which I needed to sent back to the server so the server would know that I am the same client which did the GET request.
The new request had to include this cookie
request.Headers["Cookie"] = phpsessid;
You already do this, but make sure the cookies you receive, you sent back to the server.
Considering session and authentication, there are two cookies, one for session, one for authentication and some servers/application might not allow to have an authentication without a valid session. What I want to say is that you might need to pass the session cookie too. So the steps would be:
Do first a GET request to obtain the session cookie.
Next do the POST request to authenticate and get the auth cookie.
Use both cookies to navigate to the protected pages.
Also check this question, it doesn't show the entire class, but the idea is to keep the CookieContainer in the same class, add the new cookies from POST/GET requests and assign them to the each new request, like #Krizz answered.
Try using CookieContainer which is a class to keep cookies context between several requests. You simply create an instance of it and assign it to each WebRequest.
Therefore, modifying your code:
string formParams = string.Format("j_username={0}&j_password={1}", userName, userPass);
string cookieHeader;
var cookies = new CookieContainer(); // added this line
var request = WebRequest.Create(_signInPage) as HttpWebRequest; // modified line
request.CookieContainer = cookies; // added this line
request.ContentType = "text/plain";
request.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
request.ContentLength = bytes.Length;
using (Stream os = request.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
request.GetResponse(); // removed some code here, no need to read response manually
var getRequest = WebRequest.Create(sessionHistoryPage) as HttpWebRequest; //modified line
getRequest.CookieContainer = cookies; // added this line
getRequest.Method = "GET";
WebResponse getResponse = getRequest.GetResponse();
try
{
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
textBox1.AppendText(sr.ReadToEnd());
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
throw;
}
I took an example of filling in a form in C# from another question, but when I run the code I just get the page back again (as if i didn't submit anything). I tried manually going to the URL and added ?variable=value&variable2=value2 but that didn't seem to pre-populate the form, not sure if that is why this isn't working.
private void button2_Click(object sender, EventArgs e)
{
var encoding = new ASCIIEncoding();
var postData = "appid=001";
postData += ("&email=chris#test.com");
postData += ("&receipt=testing");
postData += ("&machineid=219481142226.1");
byte[] data = encoding.GetBytes(postData);
var myRequest =
(HttpWebRequest)WebRequest.Create("http://www.example.com/licensing/check.php");
myRequest.Method = "POST";
myRequest.ContentType = "application/x-www-form-urlencoded";
myRequest.ContentLength = data.Length;
var newStream = myRequest.GetRequestStream();
newStream.Write(data, 0, data.Length);
newStream.Close();
var response = myRequest.GetResponse();
var responseStream = response.GetResponseStream();
var responseReader = new StreamReader(responseStream);
label2.Text = responseReader.ReadToEnd();
}
To submit a form programmatically, the general idea is that you want to impersonate a browser. To do this, you'll need to find the right combination of URL, HTTP headers, and POST data to satisfy the server.
The easiest way to figure out what the server needs is to use Fiddler, FireBug, or another tool which lets you examine exactly what the browser is sending over the wire. Then you can experiment in your code by adding or removing headers, changing POST data, etc. until the server accepts the request.
Here's a few gotchas you may run into:
first, make sure you're submitting the form to the right target URL. Many forms don't post to themselves but will post to a different URL
next, the form might be checking for a session cookie or authentication cookie, meaning you'll need to make one request (to pick up the cookie) and then make a subsequent cookied request to submit the form.
the form may have hidden fields you forgot to fill in. use Fiddler or Firebug to look at the form fields submitted when you fill in the form manually in the browser, and make sure to include the same fields in your code
depending on the server implementation, you may need to encode the # character as %40
There may be other challenges too, but those above are the most likely. To see the full list, take a look at my answer to another screen-scraping question.
BTW, the code you're using to submit the form is much harder and verbose than needed. Instead you can use WebClient.UploadValues() and accomplish the same thing with less code and with the encoding done automatically for you. Like this:
NameValueCollection postData = new NameValueCollection();
postData.Add ("appid","001");
postData.Add ("email","chris#test.com");
postData.Add ("receipt","testing");
postData.Add ("machineid","219481142226.1");
postData.Add ("checkit","checkit");
WebClient wc = new WebClient();
byte[] results = wc.UploadValues (
"http://www.example.com/licensing/check.php",
postData);
label2.Text = Encoding.ASCII.GetString(results);
UPDATE:
Given our discussion in the comments, the problem you're running into is one of the causes I originally noted above:
the form might be checking for a session cookie or authentication
cookie, meaning you'll need to make one request (to pick up the
cookie) and then make a subsequent cookied request to submit the form.
On a server that uses cookies for session tracking or authentication, if a request shows up without a cookie attached, the server will usually redirect to the same URL. The redirect will contain a Set-Cookie header, meaning when the redirected URL is re-requested, it will have a cookie attached by the client. This approach breaks if the first request is a form POST, because the server and/or the client isn't handling redirection of the POST.
The fix is, as I originally described, make an initial GET request to pick up the cookie, then make your POST as a second request, passing back the cookie.
Like this:
using System;
public class CookieAwareWebClient : System.Net.WebClient
{
private System.Net.CookieContainer Cookies = new System.Net.CookieContainer();
protected override System.Net.WebRequest GetWebRequest(Uri address)
{
System.Net.WebRequest request = base.GetWebRequest(address);
if (request is System.Net.HttpWebRequest)
{
var hwr = request as System.Net.HttpWebRequest;
hwr.CookieContainer = Cookies;
}
return request;
}
}
class Program
{
static void Main(string[] args)
{
var postData = new System.Collections.Specialized.NameValueCollection();
postData.Add("appid", "001");
postData.Add("email", "chris#test.com");
postData.Add("receipt", "testing");
postData.Add("machineid", "219481142226.1");
postData.Add("checkit","checkit");
var wc = new CookieAwareWebClient();
string url = "http://www.example.com/licensing/check.php";
// visit the page once to get the cookie attached to this session.
// PHP will redirect the request to ensure that the cookie is attached
wc.DownloadString(url);
// now that we have a valid session cookie, upload the form data
byte[] results = wc.UploadValues(url, postData);
string text = System.Text.Encoding.ASCII.GetString(results);
Console.WriteLine(text);
}
}
private void button1_Click(object sender, EventArgs e)
{
string userName = textBox1.Text;
string password = textBox2.Text;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.youmint.com/LoginVerification.php?name="+userName+"&pass="+password+"&agreement=true&checkvalue=true");
request.Method = "GET";
request.KeepAlive = true;
request.Headers.Add("Keep-Alive: 300");
WebResponse response = request.GetResponse();
Stream dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
if (responseFromServer.Equals(""))
MessageBox.Show("Successfully logged in!!");
else if (responseFromServer.Equals("1"))
MessageBox.Show("Login failed!");
request = (HttpWebRequest)WebRequest.Create("http://www.youmint.com/FreeSms.html");
response = request.GetResponse();
dataStream = response.GetResponseStream();
reader = new StreamReader(dataStream);
responseFromServer = reader.ReadToEnd();
/*secret code :P */
reader.Close();
dataStream.Close();
response.Close();
}
So, that's my code... The first webrequest logs into the website. It works fine in the browser and returns 1 if the login is not correct. Then the second one is a normal webrequest to a webpage of the same website. But the login is already gone and the response I get is what I get if I'm not logged in! Can someone please tell me what I'm doing wrong? How do I keep it alive? Do I have to use an invisible webbroswer control or something like that?
This has nothing to do with "keep alive".
You need to preserve session cookie between requests. For that you first need to enable cookies for your login request (read HttpWebRequest docs - it is a bit unobvious). Then you need to pass that cookie with all the following reuests.
Also please make use of the using()
#liho1eye is correct. Here's some more info from the HttpWebRequest page:
For security reasons, cookies are
disabled by default. If you want to
use cookies, use the CookieContainer
property to enable cookies.
You'll need to reference the HttpWebResponse.Cookies property to get the initial session token cookie.
Edit:
Here's a quick and dirty sample of making a request to a page, and transferring response cookies to the next request. Didn't do much testing or validation (so beware!) - just to give you the idea of the approach.
//this only has login/password info, you may need other parameters to trigger the appropriate action:
const string Parameters = "Login1$username=pfadmin&Login1$Password=password";
System.Net.HttpWebRequest req = (HttpWebRequest)System.Net.WebRequest.Create("http://[WebApp]/Login.aspx");
req.Method = "GET";
req.CookieContainer = new CookieContainer();
System.Net.HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
//Create POST request and transfer session cookies from initial request
req = (HttpWebRequest)System.Net.WebRequest.Create("http://localhost/AdminWeb/Login.aspx");
req.CookieContainer = new CookieContainer();
foreach (Cookie c in resp.Cookies)
{
req.CookieContainer.Add(c);
}
req.ContentType = "application/x-www-form-urlencoded";
//...continue on with your form POST
I'm trying to write a utility which will attempt to login to the Microsoft Online Admin website and report back as to whether it is reachable.
Using code mainly from this article, http://odetocode.com/articles/162.aspx and some screen scraping I have pieced together the following. Unfortunately it doesn't work, the final response shows that I am still looking at the login page rather than the target page.
Any help would be terrific. Thanks in advance.
private void LoginToSite()
{
const string LOGIN_URL = "https://admin.microsoftonline.com/Login.aspx";
const string USERNAME = "<username>";
const string PASSWORD = "<password>";
const string TARGET_PAGE_URL = "https://admin.noam.microsoftonline.com/Home/Home.aspx";
// first, request the login form to get the viewstate value
HttpWebRequest webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
StreamReader responseReader = new StreamReader(
webRequest.GetResponse().GetResponseStream()
);
string responseData = responseReader.ReadToEnd();
responseReader.Close();
// extract the viewstate value and build out POST data
string viewState = ExtractViewState(responseData);
string postData =
String.Format(
"__VIEWSTATE={0}&AdminCenterLoginControl$UserNameTextBox={1}&AdminCenterLoginControl$PasswordTextbox={2}&__EVENTTARGET=AdminCenterLoginControl_ActionButton",
viewState, USERNAME, PASSWORD
);
// have a cookie container ready to receive the forms auth cookie
CookieContainer cookies = new CookieContainer();
// now post to the login form
webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.CookieContainer = cookies;
// write the form values into the request message
StreamWriter requestWriter = new StreamWriter(webRequest.GetRequestStream());
requestWriter.Write(postData);
requestWriter.Close();
// we don't need the contents of the response, just the cookie it issues
webRequest.GetResponse().Close();
// now we can send out cookie along with a request for the protected page
webRequest = WebRequest.Create(TARGET_PAGE_URL) as HttpWebRequest;
webRequest.CookieContainer = cookies;
responseReader = new StreamReader(webRequest.GetResponse().GetResponseStream());
// and read the response
responseData = responseReader.ReadToEnd();
responseReader.Close();
MessageBox.Show(responseData);
}
private string ExtractViewState(string s)
{
string viewStateNameDelimiter = "__VIEWSTATE";
string valueDelimiter = "value=\"";
int viewStateNamePosition = s.IndexOf(viewStateNameDelimiter);
int viewStateValuePosition = s.IndexOf(
valueDelimiter, viewStateNamePosition
);
int viewStateStartPosition = viewStateValuePosition +
valueDelimiter.Length;
int viewStateEndPosition = s.IndexOf("\"", viewStateStartPosition);
return HttpUtility.UrlEncodeUnicode(
s.Substring(
viewStateStartPosition,
viewStateEndPosition - viewStateStartPosition
)
);
}
edit
private void LoginToSite()
{
const string LOGIN_URL = "https://admin.microsoftonline.com/login.aspx?ReturnUrl=%2fDefault.aspx";
const string USERNAME = "<username>";
const string PASSWORD = "<password>";
// Request the login form to get the viewstate value
HttpWebRequest webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
string response1 = new StreamReader(webRequest.GetResponse().GetResponseStream()).ReadToEnd();
// Extract the viewstate value and build our POST data
string viewState = ExtractViewState(response1);
string postData = String.Format(
"__VIEWSTATE={0}&AdminCenterLoginControl$UserNameTextBox={1}&AdminCenterLoginControl$PasswordTextbox={2}&__EVENTTARGET=AdminCenterLoginControl_ActionButton",
viewState, USERNAME, PASSWORD);
// Set up the Request properties
webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
CookieContainer cookies = new CookieContainer();
webRequest.CookieContainer = cookies;
// Post back to the form
using (StreamWriter requestWriter = new StreamWriter(webRequest.GetRequestStream()))
{
requestWriter.Write(postData);
}
// Read response
string response2 = new StreamReader(webRequest.GetResponse().GetResponseStream()).ReadToEnd();
MessageBox.Show(response2);
}
It would appear that MicrosoftOnline.com does not use Windows Live IDs (aka Passport) for login. This is a shame, since there are libraries available that make logging into LiveID pretty simple for client apps.
Your code hits the login page first, scraps cookies from the response, then attempts to navigate to the target page. This doesn't match the normal flow of user behavior. Normally, the user clicks on a link to go to a target page and the web site redirects the request to the login page if the user is not logged in. After logging in, the login page redirects back to the originally requested target page.
You can see this by looking at the login URL when you visit admin.microsoftonline.com in the browser. You are immediately redirected to the login page, but the full URL on the login page is: https://admin.microsoftonline.com/login.aspx?ReturnUrl=%2fDefault.aspx
Note the ReturnUrl query param at the end. This tells the login page what page to redirect back to when the login is completed.
I don't know if redirect is required by the login page, but since this is the primary path for actual end user interaction (that works) and not the path your code is taking, it's something to consider. Among other things, the redirect to login/ redirect back to target technique will take care of setting the browser cookies for the target domain automatically.
p.s. I notice also that the email administration portion of Microsoft Online services uses a different login URL. From this page (http://www.microsoft.com/online/signin.aspx) clicking on the Exchange Hosted Services Administrative Center link takes you to http:admin.messaging.microsoft.com, which immediately redirects to a login url of https://sts.messaging.microsoft.com/login.aspx?ReturnUrl=%2fDefault.aspx%3fwa%3dwsignin1.0%26wtrealm%3dhttps%253a%252f%252fadmin.messaging.microsoft.com%26wctx%3drm%253d0%2526id%253dpassive%2526ru%253d%25252f%26wct%3d2010-10-27T17%253a11%253a50Z&wa=wsignin1.0&wtrealm=https%3a%2f%2fadmin.messaging.microsoft.com&wctx=rm%3d0%26id%3dpassive%26ru%3d%252f&wct=2010-10-27T17%3a11%3a50Z
The domain name sts.messaging.microsoft.com suggests that the Exchange Hosted Services portion of Microsoft Online Services is using a Security Token Service, which suggests that this login system is capable of federated single sign on between different services. You might be able to connect to this using something like the Windows Identity Foundation (WIF) client components. Will that work with the rest of Microsoft Online Services? I don't know.
> // now we can send out cookie along with a request for the protected page
> webRequest = WebRequest.Create(TARGET_PAGE_URL) as
> HttpWebRequest;
> webRequest.CookieContainer = cookies;
> responseReader = new StreamReader(webRequest.GetResponse().GetResponseStream());
Aren't you setting WebRequest.CookieContainer equal to the blank cookie container that you generated earlier?
Shouldn't you be doing something like:
// we don't need the contents of the response, just the cookie it issues
WebResponse response = webRequest.GetResponse();
cookies = response.cookies;
response.Close();