Before everyone gets upset that this has been answered. I have scoured the web looking for how to do this and have tried a number of methods. Login to website, via C# and How to programmatically log in to a website to screenscape? Both of these were helpful but I cannot figure out why I cannot get past the login page. Here is my code:
string url = "https://www.advocare.com/login.aspx";
string url2 = "https://url.after.login";
HttpWebRequest wReq = WebRequest.Create(url) as HttpWebRequest;
wReq.KeepAlive = true;
wReq.Method = "POST";
wReq.AllowAutoRedirect = false;
wReq.ContentType = "application/x-www-form-urlencoded";
string postData = "ctl00$cphContent$txtUserName=Username&ctl00$cphContent$txtPassword=Password";
byte[] dataBytes = UTF8Encoding.UTF8.GetBytes(postData);
wReq.ContentLength = dataBytes.Length;
using (Stream postStream = wReq.GetRequestStream())
{
postStream.Write(dataBytes, 0, dataBytes.Length);
}
HttpWebResponse wResp = wReq.GetResponse() as HttpWebResponse;
string pageSource;
wReq = WebRequest.Create(url2) as HttpWebRequest;
wReq.CookieContainer = new CookieContainer();
wReq.CookieContainer.Add(wResp.Cookies);
HttpWebResponse wResp2 = wReq.GetResponse() as HttpWebResponse;
using (StreamReader sr = new StreamReader(wResp2.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
Everytime I look at pageSource it is the HTML for the login.aspx page. I must be missing something here. Maybe it's not taking the cookie, I don't know. One question I have aside from, why doesn't this work, is in the string postData = "". Are those suppose to be the name or id portion of the html tag? Any help on this is greatly appreciated as I am stumped and will have to find a different way. I would like to continue with the WebRequest and WebResponse instead of using WebBrowser. If I can't, oh well. Thanks again for any help!
What are you trying to do besides login? If its like QAing a site programically, i would suggest using selenium andcreate a c# app based off of that. If u want i can post a link to a base project for a selenium based project.
Don't necessarily view the page source, but look at the actual HTTP POST. Install a HTTP proxy such as Fiddler and then re-visit the page you are trying to emulate. Complete the HTTP POST request, and check out the results produced in the proxy. From there you'll be able to see the actual parameters, cookies, headers, etc. that are being passed and you can then attempt to replicate this in your code. It's often easy to miss something when simply viewing the HTML source but monitoring the network traffic is pretty straight forward.
Related
I have a know a website that contains an open database of the result of an academic test.
http://nts.org.pk/NTSWeb/PPL_30Sep2012_Result/search.asp
I am expert with C# but newbie to web development.
Usually, using web browser, we can enter and roll number and server sends back the result. E.G. Use my Roll Num: 3912125
What I need to do is, use a C# application to communicate this roll null number and get anything, of my result. (any string is excepted, I will parse out my result from that string.)
How do I send query? when I don't know a list of possible query strings.
I tried this code:
string queryString = "RollNo=3912125";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(#"http://nts.org.pk/NTSWeb/PPL_30Sep2012_Result/search.asp");
request.UseDefaultCredentials = true;
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
byte[] requestBytes = Encoding.UTF8.GetBytes(queryString);
request.ContentLength = requestBytes.Length;
using (var requestStream = request.GetRequestStream())
{
requestStream.Write(requestBytes, 0, requestBytes.Length);
requestStream.Close();
}
WebResponse response = request.GetResponse();
textBox1.AppendText(((HttpWebResponse)response).StatusDescription);
Stream dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
textBox1.AppendText(responseFromServer);
reader.Close();
dataStream.Close();
response.Close();
You have to append the querystring to the url like this:
string queryString = "RollNo=3912125";
string url = String.Format(#"http://foo/search.asp?{0}", queryString);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
You should take a look at the code in my answer to C# https login and download file. It gives a good demonstration of how to perform a POST request. As far as knowing what's valid to use for the query-formatted string in your POST, it's simply a matter of looking for appropriate input elements in the page content. It looks like what you have (RollNo) is correct. You may, however, need to also add the submit button value to your request as well depending on how the server behaves, giving you something like. RollNo=3912125&submit=submit.
You're most of the way there. Your queryString should look like RollNo=3912125&Submit=+Search+. When you are calling WebRequest.Create, the Url should in fact be http://nts.org.pk/NTSWeb/PPL_30Sep2012_Result/result.asp.
The rest of your code should work, though the answer #JamieSee recommended to you has some very good advice about wrapping things in using blocks correctly
I'm hoping I've done something knuckle-headed here and there is an easy answer. I'm simply trying to retrieve the list of comments for a page on my site. I use the social plug-in and then retrieve the comment id via the edge event. Server side I send the page id back and do a simple request using a HttpWebRequest. Worked well back in October, but now I get an 'internal error' response from FB. I can use the same url string put it into a browser and get the comments back in the browser in json.
StringBuilder url = new StringBuilder();
url.Append("https://graph.facebook.com/comments/?ids=" + comment.page);
string requestString = url.ToString();
HttpWebRequest request = WebRequest.Create(requestString) as HttpWebRequest;
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
Ideas? Thanks much in advance.
Since you're using the Facebook C# SDK (per your tag), try:
var url = "{your url}";
var api = new Facebook.FacebookClient(appId,appSec);
dynamic commentsObj = api.Get("/comments/?ids=" + url);
dynamic arrayOfComments = commentsObj[url].data
I asked a question on here earlier and got some fantastic responses. I've since been diddling Visual C# and ran into a bit of a problem.
Here I made a simple page that sets a cookie.
If you go to it and then refresh, it'll see if there's a cookie present and change the output html.
Now, I want my C# program to fetch a page, get a cookie and then re-visit the page again with the cookie that is set, so that my page presents me the "updated" message. I accomplished phase one via:
private void button1_Click(object sender, RoutedEventArgs e)
{
WebRequest request = WebRequest.Create("http://www.binarywatch.biz/forms/cookietest.php");
request.Credentials = CredentialCache.DefaultCredentials;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
MessageBox.Show(responseFromServer, "Derp");
reader.Close();
dataStream.Close();
response.Close();
}
So at this point I have the page html but I'm a bit lost as to how to go about getting a cookie (Something to do with CookieContainer() ? ) and then making the page know that I have it (by adding it to the httpwebrequest somehow?)
I tried googling it of course but a LOT of the answers I find are about ASP.NET / web programming and that's not what I need.
PS. What's the difference between WebRequest and HttpWebRequest?
I hope this isn't too noobish, I'm a bit stumped.
according to MSDN you'd first create an instance of the CookieContainer before calling getResponse. After that you should be able to get cookie data out of the CookieContainer you created.
(request as HttpWebRequest).CookieContainer = new CookieContainer();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
CookieCollection cookies = (request as HttpWebRequest).CookieContainer.GetCookies("www.binarywatch.biz");
string myValue cookies["myCookie"].Value
You should be able to re-use the same CookieContainer object to make sure the server keeps getting access to the cookies.
The GetCookies(domain) is needed as a single CookieContainer is able to store separate cookies safely for multiple domains.
Taken from "how to use cookies in httpwebrequest?"
Yes, use CookieContainer.
CookieContainer cookieContainer = new CookieContainer();
httpWebRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
httpWebRequest.CookieContainer = cookieContainer;
From that answer:
"After the GetResponse call the
cookieContainer will contain the
cookies sent back from the requested
Url."
I haven't tested this, but it was the accepted answer so it must work. Hope it works for you.
I'm trying to log in to my eBay account using the following code:
string signInURL = "https://signin.ebay.com/ws/eBayISAPI.dll?co_partnerid=2&siteid=0&UsingSSL=1";
string postData = String.Format("MfcISAPICommand=SignInWelcome&userid={0}&pass={1}", "username", "password");
string contentType = "application/x-www-form-urlencoded";
string method = "POST";
string userAgent = "Mozilla/5.0 (Windows; U; MSIE 7.0; Windows NT 6.0; en-US)";
CookieContainer cookieContainer = new CookieContainer();
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(signInURL);
req.CookieContainer = cookieContainer;
req.Method = method;
req.ContentType = contentType;
req.UserAgent = userAgent;
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] loginDataBytes = encoding.GetBytes(postData);
req.ContentLength = loginDataBytes.Length;
Stream stream = req.GetRequestStream();
stream.Write(loginDataBytes, 0, loginDataBytes.Length);
stream.Close();
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
StreamReader xsr = new StreamReader(res.GetResponseStream());
String responseText = xsr.ReadToEnd();
Obviously substituting my real username and password. When I look at the string responseText, I see that part of the response from eBay is
The browser you are using is rejecting cookies.
Any ideas what I'm doing wrong?
P.S. And yes, I am also using the eBay API, but this is for something slightly different than what I want to do with the API.
You're doing a direct http request. The Ebay site has functionality to talk to a browser (probably to store the session cookie). Unless you make the request code smart enough to use cookies correctly it won't work. You'll probably have to use the internet explorer object instead.
Before doing the POST you need to download the page with the form that you are submitting in your code, take the cookie they give you, put it in your CookieContainer (making sure you get the path right) and post it back up in your request.
To clarify, while you might be POSTing the correct data, you are not sending the cookie that needs to go with it. You will get this cookie from the login page.
You need to intercept the http traffic to see what exactly what had happened. I use Fiddler2. It is the good tools for debugging http. So I can know whos wrong, my application or the remote web server.
Using fiddler, you can see the request header, response header with its cookies as well as response content. It used in the middle of your app and the Ebay.
Based on my experience. I think it is because Ebay cookie sent to you is not send back to Ebay server. Fiddler will prove it whether yes or not.
Another thing, the response cookie you receive should be send back to next request by using the same CookieContainer.
You should notice that CookieContainer has a bug on .Add(Cookie) and .GetCookies(uri) method. You may not using it, but internal codes might use it.
See the details and fix here:
http://dot-net-expertise.blogspot.com/2009/10/cookiecontainer-domain-handling-bug-fix.html
CallMeLaNN
This is driving me nuts and I can't figure out where I am dropping the ball. I've followed a few examples found via the googlemonsta to no avail. Any pointer to where I goofed would be greatly apperciated.
var writer = new StringWriter();
param = "location=" + Server.UrlEncode(param);
byte[] paramStream = Encoding.ASCII.GetBytes(param + "¶m2=value");
var URL = "http://www.somesite.com";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2";
request.ContentLength = paramStream.Length;
using( var stream = request.GetRequestStream())
{
stream.Write(paramStream, 0, paramStream.Length);
}
var response = request.GetResponse();
string result;
using (var sr = new StreamReader(response.GetResponseStream()))
{
result = sr.ReadToEnd();
}
Thanks!
EDIT: As far as I can tell its hitting the site (i'm getting html back) but the params aren't pushed over. I'm basically getting where the values would appear had it been successful. I've tried removing the first & but didnt get anywhere.
EDIT: Edited code to reflect changes.
Possibly get rid of the & from the start of the first parameter? Other than that it basically looks okay. (Check the parameter names in your real code - where you've got "paramater" in the sample it should almost certainly be "parameter" - but we don't know what your real code looks like or what the real site expects.)
Please give more information about what's actually happening. We know it doesn't work, but there are a lot of different possible failure modes :)
One extra thought occurs - you haven't specified the content length. I'm not sure whether this is filled in automatically by WebRequest. It would be worth using WireShark to check whether or not it's present in the outbound request.
Just as a general point of practice, you should dispose of the WebResponse, and you don't need to call Close if you've already got a using statement for the response stream:
string result;
using (WebResponse response = request.GetResponse())
{
using (var sr = new StreamReader(response.GetResponseStream()))
{
result = sr.ReadToEnd();
}
}
Are you sure you have all the values neccessary for the post? I once had a case where there was a hidden input field on the form that was something like:
<input name="action" type="hidden" id="action" value="login">
and I had to supply that as a param as:
&action=login
Make sure you're not missing anything from the form is what I'm saying...
EDIT: One more thing: I just looked at my code again where I've done this, and noticed that I also had this line in there:
request.ContentLength = bytes.Length;
Not sure if you need that, but I noticed that you weren't setting the length.
It is not something as simple as a carriage-return/new-line after the parameters is it? Looking at some docs on HTTP on the internets, you apparently need a blank line afterwards.
(I would suggest telneting to the web server manually and pasting your request.)