I have some code that logs me into Facebook and it works fine as far as returning me the page you see right after you log in.
But I'm trying to request my actual profile after I log in rather than just my news feed.
So I'd have to send my email and password to the login page, then request my profile.
How can I keep the login data around while requesting my profile?
Here is what I've got
public static string logIn()
{
//get the cookies before you try to log in
CookieCollection cookies = new CookieCollection();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.facebook.com");
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookies);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
cookies = response.Cookies;
response.Close();
//logging in
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create("https://www.facebook.com/login.php?login_attempt=1");
getRequest.CookieContainer = new CookieContainer();
getRequest.CookieContainer.Add(cookies);
getRequest.Method = WebRequestMethods.Http.Post;
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
//sending the email/password
byte[] byteArray = Encoding.ASCII.GetBytes("email=myemail#yahoo.com&pass=mypassword");
getRequest.ContentLength = byteArray.Length;
Stream newStream = getRequest.GetRequestStream();
newStream.Write(byteArray, 0, byteArray.Length);
newStream.Close();
//returns the source of the page after logging in
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
StreamReader sr = new StreamReader(getResponse.GetResponseStream());
string source = sr.ReadToEnd();
cookies.Add(getResponse.Cookies);
//tries to get my profile source
//everything works fine until here
getRequest = (HttpWebRequest)WebRequest.Create("http://www.facebook.com/myprofile");
getRequest.CookieContainer = new CookieContainer();
getRequest.CookieContainer.Add(cookies);
getRequest.Method = WebRequestMethods.Http.Get;
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
getResponse = (HttpWebResponse)getRequest.GetResponse();
sr = new StreamReader(getResponse.GetResponseStream());
source = sr.ReadToEnd();
getResponse.Close();
return source;
}
I've tried several ways of doing this and I have gotten it to return my profile, but it returns it as if I was not logged in and you can't actually view my profile (because it is set to private)
So I need my login info to be included somehow when requesting my profile.
This had me stumped for a while when I was making something similar. You need to make sure you're setting the proper cookies for your request after you've logged in. Try adding the new cookies to the CookieCollection object before applying them to your last request.
// ^ previous code ^
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
StreamReader sr = new StreamReader(getResponse.GetResponseStream());
string source = sr.ReadToEnd();
Uri facebookPage = new Uri("http://www.facebook.com"); // .GetCookies() only accepts a Uri
cookies.Add(request.CookieContainer.GetCookies(facebookPage));
getResponse.Close(); // Always make sure you close your responses
// V requesting your profile code V
You have to start a CookieCollection at the beginning and use it for all requests.
A CookieCollection updates itself with new cookies whenever you do a request so using cookies.Add() was also messing it up.
But I got it working fine now.
Related
I am trying to write some code to connect to an HTTPS site that uses Siteminder authentication.
I keep getting a 401. Any ideas?
I have read a few different things on here but none have really seemed all that helpful. I am also using Fiddler/Firefox Tamper to snoop what's going on.
Here is what I've got so far in regards to code:
try
{
Uri uri = new Uri("https://websiteaddresshere");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri) as HttpWebRequest;
request.Accept = "text/html, application/xhtml+xml, */*";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko";
// request.Connection = "Keep-Alive";
// request.Method = "Get";
// request.Accept = "text";
request.AllowAutoRedirect = true;
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
Cookie emersoncookie = new Cookie("SMCHALLENGE", "YES");
emersoncookie.Domain = "mydomain";
emersoncookie.Path = "/";
// authentication
var cache = new CredentialCache();
cache.Add(uri, "False", new NetworkCredential("myusername", "mypassword"));
request.Credentials = cache;
// response.
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
XmlTextReader reader = new XmlTextReader(stream);
MessageBox.Show(stream.ToString());
}
}
}
catch (WebException exception)
{
string responseText;
using (var reader = new StreamReader(exception.Response.GetResponseStream()))
{
responseText = reader.ReadToEnd();
MessageBox.Show(responseText.ToString());
}
}
After doing some more reading on the MSDN website I decided to go a different route.
I ended up making this a service since it will need to be a service at the end of the day and did the following:
CookieContainer emersoncookie = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)
WebRequest.Create("https://websiteaddress");
request.Credentials = new NetworkCredential("username", "password");
request.CookieContainer = emersoncookie;
request.Method = "GET";
HttpWebResponse response = (HttpWebResponse)
request.GetResponse();
Stream resStream = response.GetResponseStream();
using (Stream output = File.OpenWrite(#"c:\\somefolder\\somefile.someextention"))
using (Stream input = resStream)
{
input.CopyTo(output);
}
To anyone that might be running into Siteminder authentication issues, this piece of code works pretty well.
I couldn't get Jasen's code to work. Maybe your SM setup is different from mine. But with SiteMinder it's generally a two step authentication process. The code block below works for me:
//Make initial request
RestClient client = new RestClient("http://theResourceDomain/myApp");
client.CookieContainer = new CookieContainer();
IRestResponse response = client.Get(new RestRequest("someProduct/orders"));
//Now add credentials.
client.Authenticator = new HttpBasicAuthenticator("username", "password");
//Get resource from the SiteMinder URI which will redirect to the correct API URI upon authentication.
response = client.Get(new RestRequest(response.ResponseUri));
Although this uses RestSharp, it can be easily replicated using HttpClient or even HttpWebRequest.
I'm trying to log in to website. Following settings works when i use putty:
GET /WindchillCProdA/netmarkets/jsp/netmarkets/view.jsp HTTP/1.1
Host: www.host_address.com
Cookie: action_number=0
Authorization: Basic bG9naW46cGFzc3dvcmQ=
I've used following code to make the same thing in C#:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.webpage.com/WindchillCProdA/netmarkets/jsp/netmarkets/view.jsp");
request.Method = "GET";
CookieContainer gaCookies = new CookieContainer();
gaCookies.Add(new Cookie("action_number", "0") { Domain = request.Host });
request.Host = www.host_address.com;
request.CookieContainer = gaCookies;
request.UserAgent = "HTTP/1.1";
request.Headers["Authorization"] = "Basic " + "bG9naW46cGFzc3dvcmQ=";
I'm receiving:
The remote server returned an error: (401) Unauthorized.
I suspect that i have to fill somehow view.jsp because my parameters are not used in this case.
use post method and write on request stream the values,
i Checked the web site and i was redirected to another page the code should be as bellow
//there could be some hash check the scripts via inspect Element
string postData = String.Format("email={0}&newPassword={1}", "yourusername","yourPassword");
var request = (HttpWebRequest)WebRequest.Create("https://secure.systemsecure.com/host/login/23.htm");
getRequest.Method = WebRequestMethods.Http.Post;
getRequest.CookieContainer = new CookieContainer();
getRequest.UserAgent ="Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko)Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version10;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
byte[] byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
Stream newStream = getRequest.GetRequestStream();
newStream.Write(byteArray, 0, byteArray.Length);
newStream.Close();
var getResponse = (HttpWebResponse)getRequest.GetResponse();
Hi I am using a facebook posts feeding application which will read all the new posts from my facebook page and will save to my sharepoint list. Earlier, i.e, before June it has worked properly it feeds all the posts from fabebook to my Share point list. But nowadays its throws an error while getting response from the Facebook authorize url. I don't know what went wrong. Please help me if you guys have any suggestion to resolve this issue. I am appending my code part below.
private void btnSendToList_Click(object sender, EventArgs e)
{
try
{
if (ConfigurationSettings.AppSettings["fbClientID"] != null)
strClientID = ConfigurationSettings.AppSettings["fbClientID"];
if (ConfigurationSettings.AppSettings["fbRedirectURL"] != null)
strURL = ConfigurationSettings.AppSettings["fbRedirectURL"];
if (ConfigurationSettings.AppSettings["fbCltSecret"] != null)
strCltSecret = ConfigurationSettings.AppSettings["fbCltSecret"];
if (ConfigurationSettings.AppSettings["fbUserId"] != null)
strUserId = ConfigurationSettings.AppSettings["fbUserId"];
if (ConfigurationSettings.AppSettings["SPSiteURL"] != null)
strSiteURL = ConfigurationSettings.AppSettings["SPSiteURL"];
CookieCollection cookies = new CookieCollection();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.facebook.com");
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookies);
//Get the response from the server and save the cookies from the first request..
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
cookies = response.Cookies;
string getUrl = "https://www.facebook.com/login.php?login_attempt=1";
string postData = String.Format("email={0}&pass={1}", "testuser#gmail.com", "test123$"); // Masking credentials.
getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.CookieContainer = request.CookieContainer;
getRequest.CookieContainer.Add(cookies);
getRequest.Method = WebRequestMethods.Http.Post;
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
newStream = getRequest.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
getResponse = (HttpWebResponse)getRequest.GetResponse();
getRequest = (HttpWebRequest)WebRequest.Create(string.Format("https://graph.facebook.com/oauth/authorize?client_id={0}&redirect_uri={1}", strClientID, "http://www.facebook.com/connect/login_success.html"));
getRequest.Method = WebRequestMethods.Http.Get;
getRequest.CookieContainer = request.CookieContainer;
getRequest.CookieContainer.Add(cookies);
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
//getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
getResponse = (HttpWebResponse)getRequest.GetResponse();
The above line throws WebExceptopn which is teeling like Process has timed out.
strAccessToken = getResponse.ResponseUri.Query;
strAccessToken = strAccessToken.Replace("#_=_", "");
strAccessToken = strAccessToken.Replace("?code=", "");
newStream.Close();
if (!string.IsNullOrEmpty(strAccessToken))
strCode = strAccessToken;
if (!string.IsNullOrEmpty(strCode) && !string.IsNullOrEmpty(strClientID) &&
!string.IsNullOrEmpty(strURL) && !string.IsNullOrEmpty(strCltSecret) &&
!string.IsNullOrEmpty(strUserId))
{
SaveToList();
}
}
catch (Exception ex)
{
LogError(ex);
}
}
Hi Finally i got the solution.
Here i am using getResponse = (HttpWebResponse)getRequest.GetResponse();
The problem is we can use only one HttpWebResponse at a time. In my case i am using the same object in two times without any disposing and closing. That's make me the error.
So I updated my code like this.
byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
newStream = getRequest.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
getResponse = (HttpWebResponse)getRequest.GetResponse();
getresponse.Close();
This solved my error. Thanks
Hi ever body I am new in c# webscraping.
I want to login to http://telelisting.ca/ but I cant. I have used WebClient, WebBrowser and now HttpWebRequest, but the same errors occured all the time which is server exception 500.
Can any body help me what i am doing wrong??
CookieCollection cookies = new CookieCollection();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://telelisting.ca/");
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookies);
//Get the response from the server and save the cookies from the first request..
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
cookies = response.Cookies;
string getUrl = "http://telelisting.ca/";
string postData = String.Format("ctl00_LoginPopup1_Login1_UserName={0}&ctl00_LoginPopup1_Login1_Password={1}&__LASTFOCUS={2}&__EVENTTARGET={3}&__EVENTARGUMENT={4}&__VIEWSTATE={5}&__EVENTVALIDATION={6}&ctl00_LoginPopup1_Login1_LoginButton={7}&ctl00_LoginPopup1_hiddenImagePath={8}&ctl00_cphContent_hiddenVideoDialogTitle={9}&ctl00_cphContent_hiddenShowDefaultVideo={10}", "fg#xyz.com", "abcede", "", "", "", "/wEPDwUKMTYzMzk2MDQwNQ9kFgJmD2QWAgIDDxYEHgVjbGFzc2UeCG9udW5sb2FkBQ9PbkVuZFNlc3Npb24oKTsWAgIBD2QWDAIHD2QWBAIDD2QWAgIDDw8WBB4EVGV4dAUVRm9yZ290IHlvdXIgcGFzc3dvcmQ/HgtOYXZpZ2F0ZVVybAUWfi9wYXNzd29yZHJlY292ZXIuYXNweGRkAgUPDxYCHg9WYWxpZGF0aW9uR3JvdXAFGGN0bDAwJExvZ2luUG9wdXAxJExvZ2luMWRkAgkPZBYCAgEPFgIeBGhyZWYFHmphdmFzY3JpcHQ6U2hvd1ZpZGVvQ29udGVudCgpO2QCCw9kFgICAQ9kFgYCCw8PFgIfAwURL0RlZmF1bHQuYXNweD9sPTBkZAIND2QWAgIBD2QWAmYPFgIfAAURRW1wdHlTaG9wcGluZ0NhcnQWAgIBDw8WAh8CBXgmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtkZAIZDw8WAh4HVmlzaWJsZWhkZAIND2QWAgIBD2QWAgIBDw8WAh8CBTM8c3Ryb25nPjxhIGhyZWY9IkRlZmF1bHQuYXNweCI+SG9tZTwvYT4gPiA8L3N0cm9uZz5kZAIPD2QWBAIJDw8WAh8CBdgJPG1hcnF1ZWUgU0NST0xMREVMQVk9JzE1MCcgPkF1ZyAxOCAtIE5ldyBDbGllbnQgaW4gPGI+TWFnb2cnPC9iPi4uLi4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtBdWcgMTYgLSBOZXcgQ2xpZW50IGluIDxiPkxvbmRvbic8L2I+Li4uLiZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwO0F1ZyAxNiAtIE5ldyBDbGllbnQgaW4gPGI+U3QuIE1hcnlzJzwvYj4uLi4uJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7QXVnIDE2IC0gTmV3IENsaWVudCBpbiA8Yj5CYXJyaWUnPC9iPi4uLi4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtBdWcgMTQgLSBOZXcgQ2xpZW50IGluIDxiPkJhcnJpZSc8L2I+Li4uLiZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwO0F1ZyAxNCAtIE5ldyBDbGllbnQgaW4gPGI+VmFuY291dmVyJzwvYj4uLi4uJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7QXVnIDEzIC0gTmV3IENsaWVudCBpbiA8Yj52YXVnaGFuJzwvYj4uLi4uJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7QXVnIDEzIC0gTmV3IENsaWVudCBpbiA8Yj5Mb25ndWV1aWwnPC9iPi4uLi4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtBdWcgMTMgLSBOZXcgQ2xpZW50IGluIDxiPlRvcm9udG8nPC9iPi4uLi4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtBdWcgMTIgLSBOZXcgQ2xpZW50IGluIDxiPlN0ZS1DYXRoZXJpbmUtZGUtbGEtSi4tQy4nPC9iPi4uLi4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtBdWcgMDkgLSBOZXcgQ2xpZW50IGluIDxiPk5ld21hcmtldCAnPC9iPi4uLi4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtBdWcgMDggLSBOZXcgQ2xpZW50IGluIDxiPkJyYW1wdG9uJzwvYj4uLi4uJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7QXVnIDA3IC0gTmV3IENsaWVudCBpbiA8Yj5WYW5jb3V2ZXInPC9iPi4uLi4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDtBdWcgMDcgLSBOZXcgQ2xpZW50IGluIDxiPk90dGF3YSc8L2I+Li4uLiZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwO0F1ZyAwNyAtIE5ldyBDbGllbnQgaW4gPGI+VHJvaXMtUml2acOocmVzJzwvYj4uLi4uJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7PC9tYXJxdWVlPmRkAhEPZBYCZg8PFgIfAgXyAzxoMyBjbGFzcz0idGl0bGVuYXYiPg0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBSZWZlcnJhbCBQcm9ncmFtPC9oMz4NCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxwIGlkPSJyMjUiPjEgTW9udGggRnJlZSEgPC9wPiAgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgPHAgaWQ9InRlc3RpbW9uaWFsbmF2Ij4NCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgUmVmZXIgY29sbGVhZ3VlcyBhbmQgZnJpZW5kcyB0byBUZWxlbGlzdGluZyBhbmQgcmVjZWl2ZSAxIE1vbnRoIEZyZWUgYWNjZXNzIHRvIG91ciBwaG9uZWJvb2sgc2VydmljZSBmb3IgZXZlcnkgcmVmZXJyYWwgdGhhdCByZXN1bHRzIGluIGEgcGFpZCBzdWJzY3JpcHRpb24uIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvcD4NCjxhIHJ1bmF0PSJzZXJ2ZXIiIGhyZWY9IlJlZmVycmFsTG9naW4uYXNweCIgY2xhc3M9ImJ1eW5vdyI+UmVmZXIgTm93PC9hPiAgICAgICANCmRkAhEPZBYCAgEPZBYMAgMQPCsACQEADxYEHghEYXRhS2V5cxYAHgtfIUl0ZW1Db3VudGZkZGQCBQ8PZA8QFgFmFgEWAh4OUGFyYW1ldGVyVmFsdWUFBWVuLVVTFgECBmRkAgkQPCsACQEADxYEHwcWAB8IZmRkZAILDw9kDxAWAmYCARYCFgIfCWQWAh8JBQVlbi1VUxYCAgMCBmRkAg8QPCsACQEADxYEHwcWAB8IAgFkZBYCZg9kFgICAQ8PFgIfAgW1AUkndmUgYmVlbiB1c2luZyBUZWxlbGlzdGluZyBzZXJ2aWNlcyAgZm9yIHRoZSBsYXN0IDMgeWVhcnMgbm93IGFuZCB0aGV5IHJlYWxseSBoZWxwIG1lIGNvbmNlbnRyYXRlIG9uIGJ1aWxkaW5nIG15IGJ1c2luZXNzIGFuZCBpbiBjbG9zaW5nIG1vcmUgb3Bwb3J0dW5pdGllcyBldmVyeSB5ZWFyLg0KDQpNLiBTbWl0aC5kZAIRDw9kDxAWAWYWARYCHwkFBWVuLVVTFgECBmRkGAEFHl9fQ29udHJvbHNSZXF1aXJlUG9zdEJhY2tLZXlfXxYCBSNjdGwwMCRMb2dpblBvcHVwMSRMb2dpbjEkUmVtZW1iZXJNZQUpY3RsMDAkTG9naW5Qb3B1cDEkTG9naW4xJExvZ2luSW1hZ2VCdXR0b25RUp74UUPL72rYSQL3cfYMy9DWZU4l5e0kF3vCJf5fkQ==", "/wEWCQL9vYfeCAKVyobIBgKB94O0CwKz7JHYDQLsu7CWDgL31sOaAQLmjZ/bDQLto+ilAwLC4KCXAqOJh1kiYkswzxstW/o4u1wgJbA0/EnFO2L8sDvV8fVl", "Log In", "/HSIControl", "Telelisting Instruction Video", "");
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.CookieContainer = new CookieContainer();
getRequest.CookieContainer.Add(cookies); //recover cookies First request
getRequest.Method = WebRequestMethods.Http.Post;
//Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
getRequest.ContentType = "text/html";
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
byte[] byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
Stream newStream = getRequest.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
newStream.Close();
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
string sourceCode = sr.ReadToEnd();
Debug.WriteLine(sourceCode);
}
A server error in the 500 range means something gone wrong server side. From the looks of your request, I think it might be related to the Viewstate variable you are posting. The server will try to reconstruct the page according to your submitted state. It's not something you just copy and paste. Have you tried only sending the parameters that relate to the login itself (name, password, the name of the button clicked)?
For your work see fiddler run the same request and see the cookies match and other post parameters.
And also try :
postparameter= UrlEncode(postparameter)
Before converting them to byte and see if that works.
Maybe it works that way.
Why dont you go to the exact page you want to scrape in your browser and then export the full COOKIE set for manual attachment to your script's header?
I'm trying to login on facebook and retrive a token by using this link:
https://www.facebook.com/dialog/oauth?client_id=282892925078054&redirect_uri=https://www.facebook.com/&response_type=token
My code looks like this, but i get an invalid link when i'm requesting the link above.
CookieCollection cookies = new CookieCollection();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.facebook.com/login.php?login_attempt=1");
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookies);
//Get the response from the server and save the cookies from the first request..
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
cookies = response.Cookies;
string getUrl = "https://www.facebook.com/login.php?login_attempt=1";
string postData = String.Format("email={0}&pass={1}", email, pass);
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.CookieContainer = new CookieContainer();
getRequest.CookieContainer.Add(cookies); //recover cookies First request
getRequest.Method = WebRequestMethods.Http.Post;
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
byte[] byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
Stream newStream = getRequest.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
newStream.Close();
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
cookies = getResponse.Cookies;
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
string sourceCode = sr.ReadToEnd();
}
//Get the token
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("https://www.facebook.com/dialog/oauth?client_id=282892925078054&redirect_uri=https://www.facebook.com/&response_type=token");
getRequest.CookieContainer = new CookieContainer();
getRequest.CookieContainer.Add(cookies);
webRequest.AllowAutoRedirect = false;
HttpWebResponse rresponse = (HttpWebResponse)webRequest.GetResponse();
if (rresponse.StatusCode == HttpStatusCode.Redirect)
{
Console.WriteLine("redirected to: " + rresponse.GetResponseHeader("Location"));
}
Please help me. Thanks in advance.
https://www.facebook.com/dialog/oauth?client_id=282892925078054&redirect_uri=https://www.facebook.com/&response_type=token
The URL does not allow the application configuration.: The application settings do not allow one or more of the URLs. Internet Site URL and Canvas URL or domain URLs must be sub-domains of application domains.