Screen scrape an ASP.NET Page not working - c#

I am trying to bring back the calendar events on the page at the following site: http://www.wphospital.org/News-Events/Calendar-of-Events.aspx
Notice that this site has a link called "Month" - I need to be able POST data requesting calendar events for a particular month. I cannot get this to work. Here is the code:
private static void GetData(ref string buf)
{
try
{
//First, request the search form to get the viewstate value
HttpWebRequest webRequest = default(HttpWebRequest);
webRequest = (HttpWebRequest)System.Net.WebRequest.Create("http://www.wphospital.org/News-Events/Calendar-of-Events.aspx");
StreamReader responseReader = new StreamReader(webRequest.GetResponse().GetResponseStream());
string responseData = responseReader.ReadToEnd();
responseReader.Close();
//Extract the viewstate value and build out POST data
string viewState = ExtractViewState(responseData);
string eventValidation = ExtractEventValidation(responseData);
string postData = null;
postData = String.Format("ctl00$manScript={0}&__EVENTTARGET=&__EVENTARGUMENT&__LASTFOCUS=&__VIEWSTATE={1}&lng={2}&__EVENTVALIDATION={3}&ctl00$searchbox1$txtWord={4}&textfield2={5}&ctl00$plcMain$lstbxCategory={6}&ctl00$plcMain$lstbxSubCategory={7}", "ctl00$plcMain$updMonthNav|ctl00$plcMain$btnNextMonth", viewState, "en-US", eventValidation, "Search", "your search here", 0, 0);
var encoding = new ASCIIEncoding();
byte[] data = encoding.GetBytes(postData);
//Now post to the search form
webRequest = (HttpWebRequest)System.Net.WebRequest.Create("http://www.wphospital.org/News-Events/Calendar-of-Events.aspx");
webRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)";
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.ContentLength = data.Length;
var newStream = webRequest.GetRequestStream();
newStream.Write(data, 0, data.Length);
newStream.Close();
responseReader = new StreamReader(webRequest.GetResponse().GetResponseStream());
//And read the response
responseData = responseReader.ReadToEnd();
responseReader.Close();
buf = responseData;
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.ProtocolError)
{
Console.Write("The server returned protocol error ");
// Get HttpWebResponse so that you can check the HTTP status code.
HttpWebResponse httpResponse = (HttpWebResponse)ex.Response;
int sc = (int)httpResponse.StatusCode;
string strsc = httpResponse.StatusCode.ToString();
}
}
}
private static string ExtractViewState(string s)
{
string viewStateNameDelimiter = "__VIEWSTATE";
string valueDelimiter = "value=\"";
int viewStateNamePosition = s.IndexOf(viewStateNameDelimiter);
int viewStateValuePosition = s.IndexOf(valueDelimiter, viewStateNamePosition);
int viewStateStartPosition = viewStateValuePosition + valueDelimiter.Length;
int viewStateEndPosition = s.IndexOf("\"", viewStateStartPosition);
return HttpUtility.UrlEncodeUnicode(s.Substring(viewStateStartPosition, viewStateEndPosition - viewStateStartPosition));
}
Can anyone point me in the right direction?

This may or may not solve your problem because I don't know exactly what the problem is when you say it's not working. But as "Al W" noted - the response from an async postback is not going to look like a straight HTML stream. So if your problem is parsing it afterwards then this might help.
I had the "opportunity" to discover this recently because I needed to rewrite that output. I'm working on a C# jQuery port and found that I was breaking WebForms pages when I tried to re-render the output stream during an async postback. I went through the client script that parses the response and figured out the format of the response.
Each panel that is updated will return a block of data that is formatted like:
"Length|Type|ID|Content"
There could be any number of these strung together. Type is "updatePanel" for UpdatePanels. ID is the UniqueID of the control, and Content is the actual HTML data. Length is equal to the number of bytes in Content, and you need to use that to parse each block, because the separator character may appear inside Content itself. So if you decided you wanted to rewrite this data before sending it back to an ASP.NET page (like I did) you need to update Length to reflect the final length of your content.
The code I used to parse and rewrite it is in Server/CsQueryHttpContext.

For POST operations, you want it to be UTF-8 encoded, so just re-do the one line
//var encoding = new ASCIIEncoding();
//byte[] data = encoding.GetBytes(postData);
//do this instead.....
byte[] data = Encoding.UTF8.GetBytes(postData);
and see if this helps you out

Below is the network trace I get in chrome when clicking the monthly button. Notice the __EVENTTARGET:ctl00$plcMain$monthBtn asp.net has a javascript framework in there that is calling a javascript:postback() method when that link is clicked, which sets the event target. That's basically how ASP.NET webforms knows which event to fire on a postback. One tricky thing here is that the web page is using an update panel so you might not get a true html response. If you can get your request to look something like this, then you should get a successful response. Hope this helps.
Request URL:http://www.wphospital.org/News-Events/Calendar-of-Events.aspx
Request Method:POST
Status Code:200 OK
Request Headers
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Content-Length:9718
Content-Type:application/x-www-form-urlencoded
Cookie:CMSPreferredCulture=en-US; ASP.NET_SessionId=h2nval45vq0q5yb0cp233huc; __utma=101137351.234148951.1312486481.1312486481.1312486481.1; __utmb=101137351.1.10.1312486481; __utmc=101137351; __utmz=101137351.1312486481.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __unam=ef169fe-131964a5f2a-24ec879b-1
Host:www.wphospital.org
Origin:http://www.wphospital.org
Proxy-Connection:keep-alive
Referer:http://www.wphospital.org/News-Events/Calendar-of-Events.aspx
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.124 Safari/534.30
X-MicrosoftAjax:Delta=true
Form Dataview URL encoded
ctl00$manScript:ctl00$plcMain$updTab|ctl00$plcMain$monthBtn
__EVENTTARGET:ctl00$plcMain$monthBtn
__EVENTARGUMENT:
__LASTFOCUS:
__VIEWSTATE:<removed for brevity>
lng:en-US
__EVENTVALIDATION:/wEWEgLbj/nSDgKt983zDgKWlOLbAQKr3LqFAwKL3uqpBwK9kfRnArDHltMCAuTk0eAHAsfniK0DAteIosMPAsiIosMPAsmIosMPAsuIosMPAoD0ookDApCbiOcPAo biOcPAombiOcPAoubiOcPyfqRx8FdqYzlnnkXcJEJZzzopJY=
ctl00$searchbox1$txtWord:Search
textfield2:Enter your search here
ctl00$plcMain$lstbxCategory:0
ctl00$plcMain$lstbxSubCategory:0
ctl00$plcMain$hdnEventCount:2

Related

I get HttpWebResponse with unclear characters when use a class in web application

I created a class in Windows Form application that uses HttpWebResponse.
This class gets the response from some page before login and after login and I analyze this pages.
Everything is good working at all time.
BUT
I used this class in a web application and uploaded project files in server.
I get good response from all page EXCEPT login page. Login page response contains unclear characters like this:
� �Z�N#G�v��CM�'��X؎2�LvV�e�H{��݅����t�
(ڋa���z.��2�͒H��膇
�+�9U�v�n��$pW��S�|��s�)=z����ߗ����0*T��~JuF5y�Ǟ���D��
and it happens often, that means I get sometimes normal response.
For example when extract my zip file again in server, it may work fine.
I am confused, I can't tell regular rule.
URL = "http://www.nonsense-website.com/login";
var request2 = (HttpWebRequest)WebRequest.Create(URL);
string username1 = jandoe;
string password1 = pass1234;
string postData = "hidlogin=1&username=" + username1 + "&password=" + password1;
var data = Encoding.UTF8.GetBytes(postData);
request2.Method = "POST";
request2.ContentType = "application/x-www-form-urlencoded";
request2.ContentLength = data.Length;
request2.UserAgent = "Mozilla/5.0 (Windows NT 10.0; rv:55.0) Gecko/20100101 Firefox/55.0";
request2.Headers.Add("Cookie:" + cookie);
request2.Headers.Add("Accept-Language: en-US,en;q=0.5");
request2.Headers.Add("Accept-Encoding: gzip, deflate");
request2.KeepAlive = true;
request2.Headers.Add("Upgrade-Insecure-Requests: 1");
using (var stream = request2.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
try
{
using (var response2 = (HttpWebResponse)request2.GetResponse())
{
responseString = new StreamReader(response2.GetResponseStream(),Encoding.UTF8).ReadToEnd();
AnalyzeResponse(responseString); // other function that analyze response data
}
}
catch (Exception ex)
{
WriteData("Exception in Login" + Environment.NewLine + ex.Message); // write in file
}
and my exception:
Exception in Login
Error in Parse the HTML
Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
You're accepting gzip and deflate compressed responses (via Accept-Encoding), but you aren't checking whether the response is compressed. Turn on automatic decompression to have this done for you:
request2.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;

C#: Post login data to form and process website's response

I am trying to write a C# program to prolong the deadline for my books in my university's library. What I want to do is the following:
1.) Login to library website via WebRequest & POST method, with username & password entered in C# program
2.) Get the url to "View borrowed books" site containing the encrypted password & plain text username as GET parameters
3.) Download content of named page to display to user in the C# program
4.) If the user presses the corresponding button in the program, submit the prolongation form to the website to prolong all media at once.
Right now I'm stuck between 1 and 2, I seem to be able to connect to the website and to enter the userdata, but the WebResponse I get is again the login page (which is not the case if you login manually on the website).
This is the method I wrote to connect to the website:
// Login function, logs the user in, uses passed user number & password
public static Boolean userLogin(String unr, String pass)
{
// Login
// Cookie needed for maintaining php session
CookieContainer cContainer = new CookieContainer();
Console.WriteLine(unr+","+pass);
String postUrl = "https://universitylibrary.com/loan/DB=4/LNG=DU/USERINFO_LOGIN";
String formParams = String.Format("ACT={0}&HOST_NAME={1}&HOST_PORT={2}&HOST_SCRIPT={3}&LOGIN={4}&STATUS={5}&BOR_U={6}&BOR_PW={7}","UI_DATA","","","","KNOWNUSER","HML_OK", unr, pass);
String cookieHeader;
WebRequest wreq = WebRequest.Create(postUrl);
wreq.ContentType = "application/x-www-form-urlencoded";
wreq.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
wreq.ContentLength = bytes.Length;
using (Stream os = wreq.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = wreq.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
//Authentication trial
String PageSource;
String getUrl = "https://universitylibrary.com:443/loan/DB=4/USERINFO";
WebRequest getReq = WebRequest.Create(getUrl);
getReq.Headers.Add("Cookie",cookieHeader);
WebResponse getResp = getReq.GetResponse();
using (StreamReader sr = new StreamReader(getResp.GetResponseStream()))
{
PageSource = sr.ReadToEnd();
}
Console.Write(PageSource);
return true;
}
Can you see my mistake? I get the sourcecode and the params (username, password) output on the console, but the output is again the login page.
I would just look at the php page, but I don't have access to any of the internal system data, all I have is the HTML page.
Any suggestions would be highly appreciated!
EDIT:
I have rethought the whole thing, and rebuilt the HTTP request header completely as recorded by fiddler. That part of the function looks like this now:
// Login
// Cookie needed for maintaining php session
CookieContainer cContainer = new CookieContainer();
HttpCookie cookie = new HttpCookie("cookie", "PSC_4='xxxxxxx'; DB='n'");
CookieCollection cookieCol = new CookieCollection();
cookieCol.Add(cookieCol);
cContainer.Add(cookieCol);
Console.WriteLine(unr+","+pass);
String postUrl = "https://universitylibrary.com:443/loan/DB=4/USERINFO";
String formParams = String.Format("ACT={0}&HOST_NAME={1}&HOST_PORT={2}&HOST_SCRIPT={3}&LOGIN={4}&STATUS={5}&BOR_U={6}&BOR_PW={7}","UI_DATA","","","","KNOWNUSER","HML_OK", unr, pass);
String cookieHeader;
HttpWebRequest wreq = (HttpWebRequest) WebRequest.Create(postUrl);
wreq.Referer = "https://universitylibrary.com/loan/DB=4/LNG=DU/USERINFO_LOGIN";
wreq.KeepAlive = true;
wreq.ContentLength = 119;
wreq.Host = "universitylibrary.com";
wreq.ContentType = "application/x-www-form-urlencoded";
wreq.Method = "POST";
wreq.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0";
wreq.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
wreq.SendChunked = true;
wreq.TransferEncoding = "gzip, deflate";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
wreq.ContentLength = bytes.Length;
using (Stream os = wreq.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
cookieHeader = "";
try
{
HttpWebResponse resp = (HttpWebResponse) wreq.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
}
catch (WebException ex)
{
Console.WriteLine(ex.Status);
Console.WriteLine(ex.Response);
}
The whole thing still doesn't work though, same issue as before.
Is it possible that HttpWebRequest can't handle https or that something else is missing for https to work? (HTTP & HTTPS seem to be syntactically identical and the port is correctly set to 443, the real difference seems to lay in the additional SSL/TLS layer, maybe I need to add this somewhere?)
When you are returned the login page it probably means that you have not been correctly authenticated. There could be several reasons for this, but in the end it is because you not mirroring the HTTP communication from the manual login on the website correctly.
What I usually do is to use a monitor such as Fiddler to capture the full request/response pattern from the manual login, which I can then subsequently mirror in my code.
You can just modify your login page after checking login data and make the only output is "SuccessfullySignIn", for example.
If this is the data your receive,
$if (getReq == "SuccessfullySignIn")
{
//Do something
}
And try to use Redirect features in your webpage

programmatically clicking links in c#

I'am trying to make a script to programmatically.
I managed to get the hole html page to a string, Now I want to somehow click the elements that I have there. I'm kind of lost so any info could help.
I'v tried to get the document as a HtmlDocument but for some reason when I use the getElementById method it doesnt find the element.
Please, Any info would help.
Thanks.
Currently this is the code i'v got,
It brings me up to the point where i have string that is it's value is the html document, now I need to some how extract the relavent tag and click it programmatically.
Thanks for your inputs,
Still waiting for one that can help me.
string email = "someemail*";`enter code here`
string pw = "somepass";
string PostData = String.Format("email={0}&pass={1}", email, pw);
CookieContainer cookieContainer = new CookieContainer();
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("http://www.facebook.com/*******");
req.CookieContainer = cookieContainer;
req.Method = "POST";
req.ContentLength = PostData.Length;
req.ContentType = "application/x-www-form-urlencoded";
req.AllowAutoRedirect = true;
req.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] loginDataBytes = encoding.GetBytes(PostData);
req.ContentLength = loginDataBytes.Length;
Stream stream = req.GetRequestStream();
stream.Write(loginDataBytes, 0, loginDataBytes.Length);
HttpWebResponse webResp = (HttpWebResponse)req.GetResponse();
Stream datastream = webResp.GetResponseStream();
StreamReader reader = new StreamReader(datastream);
string sLine = "";
string json = "";
while (sLine != null)
{
sLine = reader.ReadLine();
json += sLine;
}
json.ToString();
perhaps you might want to look at WaitN, it allows you to do all this really really easily
A "click on a link" is the same thing as sending a HTTP request. If you can parse the URI from the document you have, you can create the HTTP request separately and send that.
Clicking on a link is done by issuing a HTTP-Get for the href of the link.
If there is JavaScript interactivity, then you need to take a webbrowser element, and inject a javascript, that on document.ready executes document.getElementById("whatever").click()
See
How do I programmatically click a link with javascript?
You can use HTML agility pack to parse a HTML document and extract the HREF argument.

Logging into a website via C#, navigating to another web page, and then output the source code as a string

I am relatively new to working with web pages in C#. What I am trying to do is log into a particular website ( https://www15.swalife.com/PortalWeb/portal/cwaLogon.jsp ) and allowing the page to be redirected to the default page, and then navigating from there to (https://www15.swalife.com/csswa/ea/plt/accessELITT.do) and downloading the source code and output it to a string.
I have figured out how to download the source code via HTTPWebRequest and HTTPWebResponse but am having trouble on coding the logging in function. I assume I will have to do something with POST? I have looked at http://www.dreamincode.net/forums/topic/152297-c%23-log-in-to-website-programmatically/ also.
Thanks in advance!!
EDIT:
The code supplied by jimmyjambles works flawlessly, except it doesn't quite get me the source code of the page I wanted. The code suggests that the log-in process failed, but I believe with a little bit of tweaking I could get it to work...also to everyone having issues with:
ServicePointManager.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback(AcceptAllCertifications);
Try changing your "public string" and "public bool" functions to "public static string" and "public static bool" respectively :)
EDIT 2:
Response HTML:
"<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">\n<HTML>\n<HEAD>\n\n\n\n\n\n\n<META http-equiv=\"Content-Type\" content=\"text/html; charset=ISO-8859-1\">\n<META name=\"GENERATOR\" content=\"IBM WebSphere Studio\">\n<TITLE>endSession.jsp</TITLE>\n<LINK rel=\"stylesheet\" href=\"eipPortletStyles/swalife.css\" type=\"text/css\">\n\t<script type=\"text/javascript\" language=\"JavaScript\" \n\t\tsrc=\"eipCommonJavaScript/eipGeneralFunctions.js\"/> </script>\n\t\t\n<script type=\"text/javascript\">\n\n\tfunction refreshParent()\n\t{\n\t if(window.parent)\n\t {\n\t if(window.parent.name == 'appMainFrame')\n\t window.parent.location = \"/csswa/ea/plt/logout.do\";\n\t // alert('Your session has expired. Please login again. ');\n\t }\n\t}\n\n</script>\n</HEAD>\n<BODY onload=\"refreshParent();\">\n \n\t \t<div class=\"eipErrors\">\n \t\t\t<div class=\"legendLabel\">Message</div>\n \t\t\t\n \t\t\t <div class=\"errorsHeader formTitle\">You Have Exited Out of Crew Web Access.<br> \t\t\t \n \t\t\t </div>\n \t\t\t \n \t\t\t<div class=\"errorsHeader formTitle\"> Please Close this Window and <font size=\"+1\">Log Out of SWALife</font> to Complete the Log Out Process. </div>\n \t\t<div class=\"errorsText\">\n \t\t \n \t\t\t\t\n \t\t</div>\n \t\t\n \t\t\t\n \t\t\n \t\t<div class=\"errorsFooter\">You will need to log back in before continuing.</div> \t\n \t\t\n \t</div>\n \n</BODY>\n</HTML>\n"
In order to use HttpWebRequest to access a secondary URL after login you will need to keep in mind a few things.
Firstly as Casperah has mentioned you will need to inspect the login form and identify the "name" attribute of the controls used to receive the login data.
Once you have done this you will need to format a post string accordingly and provide it to the WebRequest.
The last consideration is that once you have logged in, you will need to store and maintain the cookie assigned to you from the server that keeps you logged in.
I have taken a WebRequest snippet from this msdn article and modified it to perform a second page request after the login.
string loginurl = "http://www.gmail.com";
string secondurl = "http://mail.google.com/prefs";
string username = "bob#gmail.com";
string password = "12345";
GetSecondaryLoginPage(loginurl, secondurl, username, password);
public string GetSecondaryLoginPage(string loginurl, string secondurl, string username, string password, string cookieName = null)
{
// Create a request using a URL that can receive a post.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(loginurl);
// Set the Method property of the request to POST.
request.Method = "POST";
CookieContainer container = new CookieContainer();
if (cookieName != null)
container.Add(new Cookie(cookieName, username, "/", new Uri(loginurl).Host));
request.CookieContainer = container;
// Create POST data and convert it to a byte array. Modify this line accordingly
string postData = String.Format("username={0}&password={1}", username, password);
ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(AcceptAllCertifications);
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
// Set the ContentType property of the WebRequest.
request.ContentType = "application/x-www-form-urlencoded";
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
// Get the request stream.
Stream dataStream = request.GetRequestStream();
// Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
// Close the Stream object.
dataStream.Close();
// Get the response.
WebResponse response = request.GetResponse();
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
using (StreamWriter outfile =
new StreamWriter("output.html"))
{
outfile.Write(responseFromServer.ToString());
}
// Clean up the streams.
reader.Close();
dataStream.Close();
response.Close();
request = (HttpWebRequest)WebRequest.Create(secondurl);
request.CookieContainer = container;
response = request.GetResponse();
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
reader = new StreamReader(dataStream);
// Read the content.
responseFromServer = reader.ReadToEnd();
// Clean up the streams.
reader.Close();
dataStream.Close();
response.Close();
return responseFromServer;
}
public bool AcceptAllCertifications(object sender, System.Security.Cryptography.X509Certificates.X509Certificate certification, System.Security.Cryptography.X509Certificates.X509Chain chain, System.Net.Security.SslPolicyErrors sslPolicyErrors)
{
return true;
}
The only added lines are for the postData and cookies.
You will need to modify the line
string postData = String.Format("username={0}&password={1}", username, password);
based on your controls on the form, since you posted the site your trying to work with I can guess that you are probably looking for
string postData = String.Format("uid={0}&portalBase=cwa&password={1}", username, password);
Using a WebBrowser in a form is fairly easy. Start by navigating to the cwaLogon.jsp page and locate the input-controls and Invoke "click" on the submit button.
Either that or make the appropriate GET/POSTs using HTTPWebRequest and Response.
Using Fiddler2 to check what to post is a great start.

getting this error: "The remote server returned an error: (422) Unprocessable Entity." when doing post from C# to RoR

This code is for an outlook plugin. We're trying to POST to a page and are getting this error:
The remote server returned an error: (422) Unprocessable Entity.
The C# code is here:
webClient.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
ASCIIEncoding asciiEncoding = new System.Text.ASCIIEncoding();
Byte[] postData = asciiEncoding.GetBytes("email=e2#email.com&password=hunter2");
char[] resultHTML = asciiEncoding.GetChars(webClient.UploadData("http://url", "POST", postData));
string convertedResultHTML = new string(resultHTML);
Any idea what could be causing this?
POST data must be encoded prior to be sent out on the wire as ASCII, if you are sending character not in the ASCII range. You should try something like:
Byte[] postData = asciiEncoding.GetBytes(HttpUtility.UrlEncode("email=e2#email.com&password=hunter2"));
Because of its limited functionality, I avoid using WebClient and use WebRequest instead. The code below:
does not expect an HTTP 100 status code to be returned,
creates a CookieContainer to store any cookies we pick up,
sets the Content Length header, and
UrlEncodes each value in the post data.
Give the following a try and see if it works for you.
System.Net.ServicePointManager.Expect100Continue = false;
System.Net.CookieContainer cookies = new System.Net.CookieContainer();
// this first request just ensures we have a session cookie, if one exists
System.Net.WebRequest req = System.Net.WebRequest.Create("http://localhost/test.aspx");
((System.Net.HttpWebRequest)req).CookieContainer = cookies;
req.GetResponse().Close();
// this request submits the data to the server
req = System.Net.WebRequest.Create("http://localhost/test.aspx");
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
((System.Net.HttpWebRequest)req).CookieContainer = cookies;
string parms = string.Format("email={0}&password={1}",
System.Web.HttpUtility.UrlEncode("e2#email.com"),
System.Web.HttpUtility.UrlEncode("hunter2"));
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(parms);
req.ContentLength = bytes.Length;
// perform the POST
using (System.IO.Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
// read the response
string response;
using (System.Net.WebResponse resp = req.GetResponse())
{
if (resp == null) return;
using (System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream()))
{
response = sr.ReadToEnd().Trim();
}
}
// the variable response holds the results of the request...
Credits: Hanselman, Simon (SO Question)
This is the RoR application telling you that you have not formed a request that it can handle; the destination script exists (otherwise you'd see a 404), the request is being handled (otherwise you'd get a 400 error) and it's been encoded correctly (or you'd get a 415 error) but the actual instruction can't be carried out.
Looking at it, you seem to be loading some email information. The RoR application could be telling you that the username and password is wrong, or that the user doesn't exist, or something else. It's up to the RoR application itself.
I think the code itself is good; it's just that the app at the other end isn't happy about doing what you ask it. Are you missing something else in the request information, like a command? (eg command=getnetemails&email=e2#email.com&password=hunter2) Are you sure the email/password combination you are passing is good?
see here for more on the 422 error.
Add the below line above your code.
System.Net.ServicePointManager.Expect100Continue = false;
Are you trying to access an authentication required page?
it was solved by returning xml instead of just unstructured text on the RoR side

Categories