I've recently been looking on how to get data from a website using C#. I tried using the WebBrowser object to navigate and log in, and that worked fine, but I keep getting the same problem over and over again: when I navigate to the wanted page I get disconnected.
I've tried several things like making sure that only one HtmlDocument exists but I still get logged out.
TLDR: how do you stay logged in, from page to page, while navigating a website with WebBrowser? Or are there better alternatives?
EDIT: So far I have the following code;
currentWebBrowser = new WebBrowser();
currentWebBrowser.DocumentText = #"<head></head><body></body>";
currentWebBrowser.Url = new Uri("about:blank");
currentWebBrowser.Navigate("http://google.com");
HttpWebRequest Req = (HttpWebRequest) WebRequest.Create("http://google.com");
Req.Proxy = null;
Req.UseDefaultCredentials = true;
HttpWebResponse Res = (HttpWebResponse)Req.GetResponse();
currentWebBrowser.Document.Cookie = Res.Cookies.ToString();
At which moment should I get the cookies? And is my code correct?
You have to preserve the cookies returned from your login request and reuse those cookies on all subsequent requests - the authentication cookie tells the server that you are in fact logged in already. E.g. see here on how to do that.
Related
I'm trying to get cookie-response from a website.
For example each time I visit google.com I would like to retrieve new cookie values, is this possible? Below I'll have a code using webbrowser to retrieve cookies :
var cookies = FullWebBrowserCookie.GetCookieInternal(webBrowser3.Url,false);
I tried httpwebrequest also using codes I found on this site, but I can not get new cookie-values from the website.
All help would be appreciated
Regards,
dennis
Ok so I found a working method over here but i can not put this code in a loop.
var StringURL = "http://www.google.com";
HttpWebRequest request = null;
request = HttpWebRequest.Create(StringURL) as HttpWebRequest;
HttpWebResponse TheRespone = (HttpWebResponse)request.GetResponse();
String setCookieHeader = TheRespone.Headers[HttpResponseHeader.SetCookie];
textBox18.Text = TheRespone.Headers[HttpResponseHeader.SetCookie];
If i put this code into my application, which runs in a loop the application get stuck. When I put this code on a button, I can push button once and it prints out the cookie perfect, but when I hit the button again the app will get stuck also. So if there is anyone with a solution for this please be so kind to explain what I'm doing wrong.
Hello i making a simple httpwebrequest and then i read (StreamReader) the response and just want to get the html page of website,but i get only one laber(only one element of the page) in the browser all fine(i see all page) but when i try to set cookies to Deny\disable i also in the browser get this label(only one element of the page) and all is disappear.Sow i getting opinion if after i disabled cookies in browser i get the same page(like in code) that mean my HttpWebRequest is have settings cookies=deny/disable.
You can go to https://www.bbvanetcash.com/local_kyop/KYOPSolicitarCredenciales.html and disable cookies with F12 and you will see the difrance and also this page with one label.
Sow this my code any ideas what i need to change here?
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create("https://www.bbvanetcash.com/local_kyop/KYOPSolicitarCredenciales.html");
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
Stream streamResponseLogin = myHttpWebResponse.GetResponseStream();
StreamReader streamReadLogin = new StreamReader(streamResponseLogin);
LoginInfo = streamReadLogin.ReadToEnd();
Your code is receiving complete page content, but it cannot receive the dynamic contents. This is happening because the page you are trying to access relies on Cookies for maintaining session as well as JavaScript (it is using jQuery) for loading dynamic contents and providing rich user experience.
To successfully receive the whole page, your code must
support retrieving, storing and sending cookie objects across various HttpRequest and HttpResponse.
be able to execute JavaScript code to load the dynamic contents/markup of the page
To test 'if your code is receiving proper values or not' visit the site Web Sniffer and put your URL there.
As you can try on web-sniffer site, for www.google.com, the response you are getting is a redirect instruction.... that means, even to access the Google's home page, your code must understand HTTP status messages (302 there).
I have a URL that I'd like to submit. This will cause a redirect on the target page. I'd then like to get the new URL.
This will occur inside of a library. So no winforms or WPF. Is there some type of web browser object available at this level (similar to the winforms web browser object)?
I don't need to do any page scraping. Just access to the new URL. But some type of eventing will need to be available so I'll know when it is ok to grab the new URL.
You can use HttpWebRequest:
HttpWebRequest httpWebRequest = (HttpWebRequest) WebRequest.Create(<submit URL>);
httpWebRequest.AllowAutoRedirect = true;
httpWebRequest.GetResponse();
// address after all redirections:
Uri address = httpWebRequest.Address;
Hello Everyone i have the follow code
Uri site = new Uri("http://www.receita.fazenda.gov.br/pessoajuridica/cnpj/cnpjreva/Cnpjreva_Solicitacao2.asp");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(site);
CookieContainer cookies = new CookieContainer();
request.CookieContainer = cookies;
//Console.WriteLine(cookies.GetCookieHeader(site));
//Get the response and print out the cookies again
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Console.WriteLine(cookies.GetCookieHeader(site));
}
Console.Write("end");
Console.ReadKey();
Well this code returns the Cookie OK but if i change the URI to http://www8.receita.fazenda.gov.br/SimplesNacional/Aplicacoes/ATBHE/ConsultaOptantes.app/ConsultarOpcao.aspx it's not show correctly it's return empty
Anyone can help me to solve this problem ?
The server at the new URL does not attempt to set any cookies, so of course no cookies are in the container. Modify your ASP code to set a cookie and it will work.
Add the following to your cookie: , domain=.receita.fazenda.gov.br. This says that your cookie can be used with any sub-domain of receita.fazenda.gov.br. Have a look at the RFC on cookies or use this Wikipedia article.
UPDATE: Re-reading your OP, there may be a few things that could be going awry.
One thing is that the site is not setting a cookie when making a request for www8.receita.fazenda.gov.br (as mentioned by others). Or, the site did not set the domain field of the cookie sent with the response for the request to www.receita.fazenda.gov.br. I think it's likely the former—the cookie is not being set by the web server when the request is being made.
Another thing is perhaps you forgot to request the cookie header for the correct site?? Above, you have a line of code that reads Console.WriteLine(cookies.GetCookieHeader(site));, where site is hardcoded to a System.Uri that is different from www8.receita.fazenda.gov.br. If that's the case, then you are requesting cookies for a different site than the one for which the request was made.
I am trying to login to a HTTPS website and then navigate to download a report using c# (its an xml report) ?
I have managed to login OK via cookies/headers etc - but whenever I navigate to the link once logged in, my connection takes me to the "logged out" page ?
Anyone know what would cause this ?
Make sure the CookieContainer you use for your login is the same one you use when downloading the actual report.
var cookies = new CookieContainer();
var wr1 = (HttpWebRequest) HttpWebRequest.Create(url1);
wr1.CookieContainer = cookies;
// do login here with wr1
var wr2 = (HttpWebRequest) HttpWebRequest.Create(url2);
wr2.CookieContainer = cookies;
// get the report with wr2
It can be any number of reasons. Did you pass in the cookie to the download request? Did you pass a referrer URL?
The best way to check is to record a working HTTP request with Wireshark or any number of Firefox extensions or Fiddler.
Then try to recreate the request in C#