I am trying to login to a website via WebRequest. I get an exception at this point :
WebRequest req = WebRequest.Create(formUrl.Trim());
string url,string username,string password come from a text box. This is the full code:
public void LoginToUrl(string url,string username, string password )
{
formUrl = url;
formParams = string.Format("username={0}&password={1}", username,password);
WebRequest req = WebRequest.Create(formUrl.Trim());//
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
}
This is the POST Data:
Host=internetlogin1.cu.edu.ng
User-Agent=Mozilla/5.0 (Windows NT 10.0; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
Accept=text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language=en-US,en;q=0.5
Accept-Encoding=gzip, deflate
Refer this link
Connection=keep-alive
Content-Type=application/x-www-form-urlencoded
Content-Length=49
POSTDATA=dst=&popup=true&username=13ck015373&password=F3NB
You should pass a valid URL to create a WebRequest.
The error says that URL (that comes from textbox) dose not contains scheme ('http://' or 'https://') or it is invalid.
Enter this URL in text-box (don't forget http or https):
http://internetlogin1.cu.edu.ng or https://internetlogin1.cu.edu.ng
If there are the parameters of url-string then you need to add them through '?' and '&' chars
I had a schema uri invalid issue as well. Had to do the following for it to work. Not sure why.. but fyi
Works:
Uri serverUri = new Uri("http://url.com/sub/somethingService");
var webRequest = (HttpWebRequest)WebRequest.Create(serverUri);
What did not work (had this same error):
var webRequest = (HttpWebRequest)WebRequest.Create("http://url.com/sub/somethingService");
please add 'http://' or 'https://' to your url or check that not have ("")
for example "http://YourUrl:YourPort" is not correct and should replace to http://YourUrl:YourPort . you enter your url into notepad to you find problem
Related
You can see my code down there.This is going to show user id of a instagram user as the response but it is showing "�".
private void button18_Click(object sender, EventArgs e)
{
string username = ""; // your username
string password = ""; // your password
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create("https://i.instagram.com/api/v1/accounts/login/");
httpWebRequest.Headers.Add("X-IG-Connection-Type", "WiFi");
httpWebRequest.Method = "POST";
httpWebRequest.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
httpWebRequest.Headers.Add("X-IG-Capabilities", "AQ==");
httpWebRequest.Accept = "*/*";
httpWebRequest.UserAgent = "Instagram 10.9.0 Android (23/6.0.1; 944dpi; 915x1824; samsung; SM-T185; gts210velte; qcom; en_GB)";
httpWebRequest.Headers.Add("Accept-Encoding", "gzip, deflate");
httpWebRequest.Headers.Add("Accept-Language", "en;q=1, ru;q=0.9, ar;q=0.8");
httpWebRequest.Headers.Add("Cookie", "mid=qzejldb8ph9eipis9e6nrd1n457b;csrftoken=a7nd2ov9nbxgqy473aonahi58y21i8ee");
httpWebRequest.Host = "i.instagram.com";
httpWebRequest.KeepAlive = true;
byte[] bytes = new ASCIIEncoding().GetBytes("ig_sig_key_version=5&signed_body=5128c31533802ff7962073bb1ebfa9972cfe3fd9c5e3bd71fe68be1d02aa92c8.%7B%22username%22%3A%22"+ username + "%22%2C%22password%22%3A%22"+ password +"%22%2C%22_uuid%22%3A%22D26E6E86-BDB7-41BE-8688-1D60DE60DAF6%22%2C%22_uid%22%3A%22%22%2C%22device_id%22%3A%22android-a4d01b84202d6018%22%2C%22_csrftoken%22%3A%22a7nd2ov9nbxgqy473aonahi58y21i8ee%22%2C%22login_attempt_count%22%3A%220%22%7D");
httpWebRequest.ContentLength = (long)bytes.Length;
using (Stream requestStream = httpWebRequest.GetRequestStream())
{
requestStream.Write(bytes, 0, bytes.Length);
}
string result = new StreamReader(((HttpWebResponse)httpWebRequest.GetResponse()).GetResponseStream()).ReadToEnd();
textBox1.Text = result;
}
It's answered in a comment by #Dour, but I'm going to add a bit of detail that isn't mentioned.
This following line tells the server: Hey server! Please send me a compressed response (to reduce size).
httpWebRequest.Headers.Add("Accept-Encoding", "gzip, deflate");
so, the response you're getting is a compressed response.
Does that mean you just remove this line ?
Well, Removing it will fix your problem but that isn't the correct way to do that.
It's really better to get a reduced size response because that will need lower time to download it from the server.
What you really need to do is to make HttpWebRequest handles the decompression process for you.
You can do that by setting AutomaticDecompression property.
httpWebRequest.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
Note: With the previous line, you do NOT need to set Accept-Encoding yourself. HttpWebRequest will send it automatically for you. And when you call GetResponse, HttpWebRequest will handle the decompression process for you.
Besides of your problem:
I suggest that you use using statement for getting the response, because with your current code, I think it won't get disposed.
so I want to send a request to log in to a website, how can I do that
I have tried the code bellow:
string formUrl = "https://account.mojang.com/login";
string formParams = string.Format("email_address={0}&password={1}", "your email", "your password");
string cookieHeader;
WebRequest req = WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "GET";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
I get an error at line 9 : "Can not send a body of content with this type of verb."
I will answer this question in two parts: first the error message that you have received, and secondly how you should solve the problem.
Your error message resulted from line 9, evidently req.GetRequestStream() has failed for some reason:
using (Stream os = req.GetRequestStream())
Looking closer at the text of the message, it is evident what the problem is - you are trying to send a x-form-urlencoded message body with a GET request when this is not supported by the WebRequest implementation.
Can not send a body of content with this type of verb.
The function of GetRequestStream() is to get a stream to which you can write a request body (a "body of content"), in your case the login parameters. You are sending a GET request (GET and POST are referred to as "verbs" in the HTTP specification), and WebRequest does not support this.
Going back to the actual problem here, I believe that you did not mean to send a GET request and need to send a POST request intead. If that is the case then you simply need to change line 6 to read:
req.Method = "POST";
Most login services expect POST requests and GET requests do not work in any case. However, if you wanted to do a GET request with data using WebRequest you need to do it in a different way. The data must be encoded into the initial URL and there is no need to get a request stream or write to it. The fixed code in that case would read:
string formURL = string.Format("https://account.mojang.com/login?email_address={0}&password={1}", "your email", "your password");
string cookieHeader;
WebRequest req = WebRequest.Create(formUrl);
req.Method = "GET";
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
I am trying to write a C# program to prolong the deadline for my books in my university's library. What I want to do is the following:
1.) Login to library website via WebRequest & POST method, with username & password entered in C# program
2.) Get the url to "View borrowed books" site containing the encrypted password & plain text username as GET parameters
3.) Download content of named page to display to user in the C# program
4.) If the user presses the corresponding button in the program, submit the prolongation form to the website to prolong all media at once.
Right now I'm stuck between 1 and 2, I seem to be able to connect to the website and to enter the userdata, but the WebResponse I get is again the login page (which is not the case if you login manually on the website).
This is the method I wrote to connect to the website:
// Login function, logs the user in, uses passed user number & password
public static Boolean userLogin(String unr, String pass)
{
// Login
// Cookie needed for maintaining php session
CookieContainer cContainer = new CookieContainer();
Console.WriteLine(unr+","+pass);
String postUrl = "https://universitylibrary.com/loan/DB=4/LNG=DU/USERINFO_LOGIN";
String formParams = String.Format("ACT={0}&HOST_NAME={1}&HOST_PORT={2}&HOST_SCRIPT={3}&LOGIN={4}&STATUS={5}&BOR_U={6}&BOR_PW={7}","UI_DATA","","","","KNOWNUSER","HML_OK", unr, pass);
String cookieHeader;
WebRequest wreq = WebRequest.Create(postUrl);
wreq.ContentType = "application/x-www-form-urlencoded";
wreq.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
wreq.ContentLength = bytes.Length;
using (Stream os = wreq.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = wreq.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
//Authentication trial
String PageSource;
String getUrl = "https://universitylibrary.com:443/loan/DB=4/USERINFO";
WebRequest getReq = WebRequest.Create(getUrl);
getReq.Headers.Add("Cookie",cookieHeader);
WebResponse getResp = getReq.GetResponse();
using (StreamReader sr = new StreamReader(getResp.GetResponseStream()))
{
PageSource = sr.ReadToEnd();
}
Console.Write(PageSource);
return true;
}
Can you see my mistake? I get the sourcecode and the params (username, password) output on the console, but the output is again the login page.
I would just look at the php page, but I don't have access to any of the internal system data, all I have is the HTML page.
Any suggestions would be highly appreciated!
EDIT:
I have rethought the whole thing, and rebuilt the HTTP request header completely as recorded by fiddler. That part of the function looks like this now:
// Login
// Cookie needed for maintaining php session
CookieContainer cContainer = new CookieContainer();
HttpCookie cookie = new HttpCookie("cookie", "PSC_4='xxxxxxx'; DB='n'");
CookieCollection cookieCol = new CookieCollection();
cookieCol.Add(cookieCol);
cContainer.Add(cookieCol);
Console.WriteLine(unr+","+pass);
String postUrl = "https://universitylibrary.com:443/loan/DB=4/USERINFO";
String formParams = String.Format("ACT={0}&HOST_NAME={1}&HOST_PORT={2}&HOST_SCRIPT={3}&LOGIN={4}&STATUS={5}&BOR_U={6}&BOR_PW={7}","UI_DATA","","","","KNOWNUSER","HML_OK", unr, pass);
String cookieHeader;
HttpWebRequest wreq = (HttpWebRequest) WebRequest.Create(postUrl);
wreq.Referer = "https://universitylibrary.com/loan/DB=4/LNG=DU/USERINFO_LOGIN";
wreq.KeepAlive = true;
wreq.ContentLength = 119;
wreq.Host = "universitylibrary.com";
wreq.ContentType = "application/x-www-form-urlencoded";
wreq.Method = "POST";
wreq.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0";
wreq.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
wreq.SendChunked = true;
wreq.TransferEncoding = "gzip, deflate";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
wreq.ContentLength = bytes.Length;
using (Stream os = wreq.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
cookieHeader = "";
try
{
HttpWebResponse resp = (HttpWebResponse) wreq.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
}
catch (WebException ex)
{
Console.WriteLine(ex.Status);
Console.WriteLine(ex.Response);
}
The whole thing still doesn't work though, same issue as before.
Is it possible that HttpWebRequest can't handle https or that something else is missing for https to work? (HTTP & HTTPS seem to be syntactically identical and the port is correctly set to 443, the real difference seems to lay in the additional SSL/TLS layer, maybe I need to add this somewhere?)
When you are returned the login page it probably means that you have not been correctly authenticated. There could be several reasons for this, but in the end it is because you not mirroring the HTTP communication from the manual login on the website correctly.
What I usually do is to use a monitor such as Fiddler to capture the full request/response pattern from the manual login, which I can then subsequently mirror in my code.
You can just modify your login page after checking login data and make the only output is "SuccessfullySignIn", for example.
If this is the data your receive,
$if (getReq == "SuccessfullySignIn")
{
//Do something
}
And try to use Redirect features in your webpage
I am trying to scrape a website to get the Textarea information.
I'm using:
HtmlDocument doc = this.webBrowser1.Document;
When I look at the view source it shows <textarea name="message" class="profile">
But when I try to access this textarea with:
HtmlDocument doc = this.webBrowser1.Document;
doc.GetElementsByTagName("textarea")
.GetElementsByName("message")[0]
.SetAttribute("value", "Hello");
It shows the error:
Value of '0' is not valid for 'index'. 'index' should be between 0 and -1.
Parameter name: index
Any Help?
For your current need you can simply use this:
doc.GetElementsByTagName("textarea")[0].InnerText = "Hello";
For complex things you can use HtmlDocument class with MSHTML class.
I can entrust HtmlAgilityPack to you!
I'd like to think that you try to access a website that uses cookies to determine if a user is logged in (or not). If not, it will force you to register/log-in else you aren't allowed to see anything. Am I right?
Your browser stores that cookies, your C# does not! (broadly speaking)
You need to create a cookie container to solve that problem.
Your C#-App may log-in, request a cookie/session, may grab the Cookies from the responseheader and then you should be able to scrape the profiles or whatever you want.
Get the Post Data, which is send to server. You can use tools/addons like Fiddler, Tamper, ect..
E.g. PostdataString: user_name=TESTUSER&password=TESTPASSWORD&language=en&action%3Asubmit=Submit
Here is a snippet you can use.
//Create the PostData
string strPostData = "user_name=" + txtUser.Text + "&password=" + txtPass.Text + "&language=en&action%3Asubmit=Submit";
CookieContainer tempCookies = new CookieContainer();
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] data = encoding.GetBytes(strPostData);
//Create the Cookie
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.website.com/login.php");
request.Method = "POST";
request.KeepAlive = true;
request.AllowAutoRedirect = false;
request.Accept = "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.ContentType = "application/x-www-form-urlencoded";
request.Referer = "http://www.website.com/login.php";
request.UserAgent = "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20100101 Firefox/14.0.1";
request.ContentLength = data.Length;
Stream requestStream = request.GetRequestStream();
requestStream.Write(data, 0, data.Length);
HttpWebResponse response;
response = (HttpWebResponse)request.GetResponse();
string sRequestHeaderBuffer = Convert.ToString(response.Headers);
requestStream.Close();
//Stream(-output) of the new website
StreamReader postReqReader = new StreamReader(response.GetResponseStream());
//RichTextBox to see the new source.
richTextBox1.Text = postReqReader.ReadToEnd();
You will need to adjust the Cookie-parameters in between and add your current sessionid aswell to the code. This depends on the requested website you visit.
E.g.:
request.Headers.Add("Cookie", "language=en_US.UTF-8; StationID=" + sStationID + "; SessionID=" + sSessionID);
I'am trying to make a script to programmatically.
I managed to get the hole html page to a string, Now I want to somehow click the elements that I have there. I'm kind of lost so any info could help.
I'v tried to get the document as a HtmlDocument but for some reason when I use the getElementById method it doesnt find the element.
Please, Any info would help.
Thanks.
Currently this is the code i'v got,
It brings me up to the point where i have string that is it's value is the html document, now I need to some how extract the relavent tag and click it programmatically.
Thanks for your inputs,
Still waiting for one that can help me.
string email = "someemail*";`enter code here`
string pw = "somepass";
string PostData = String.Format("email={0}&pass={1}", email, pw);
CookieContainer cookieContainer = new CookieContainer();
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("http://www.facebook.com/*******");
req.CookieContainer = cookieContainer;
req.Method = "POST";
req.ContentLength = PostData.Length;
req.ContentType = "application/x-www-form-urlencoded";
req.AllowAutoRedirect = true;
req.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] loginDataBytes = encoding.GetBytes(PostData);
req.ContentLength = loginDataBytes.Length;
Stream stream = req.GetRequestStream();
stream.Write(loginDataBytes, 0, loginDataBytes.Length);
HttpWebResponse webResp = (HttpWebResponse)req.GetResponse();
Stream datastream = webResp.GetResponseStream();
StreamReader reader = new StreamReader(datastream);
string sLine = "";
string json = "";
while (sLine != null)
{
sLine = reader.ReadLine();
json += sLine;
}
json.ToString();
perhaps you might want to look at WaitN, it allows you to do all this really really easily
A "click on a link" is the same thing as sending a HTTP request. If you can parse the URI from the document you have, you can create the HTTP request separately and send that.
Clicking on a link is done by issuing a HTTP-Get for the href of the link.
If there is JavaScript interactivity, then you need to take a webbrowser element, and inject a javascript, that on document.ready executes document.getElementById("whatever").click()
See
How do I programmatically click a link with javascript?
You can use HTML agility pack to parse a HTML document and extract the HREF argument.