I have a URL that I'd like to submit. This will cause a redirect on the target page. I'd then like to get the new URL.
This will occur inside of a library. So no winforms or WPF. Is there some type of web browser object available at this level (similar to the winforms web browser object)?
I don't need to do any page scraping. Just access to the new URL. But some type of eventing will need to be available so I'll know when it is ok to grab the new URL.
You can use HttpWebRequest:
HttpWebRequest httpWebRequest = (HttpWebRequest) WebRequest.Create(<submit URL>);
httpWebRequest.AllowAutoRedirect = true;
httpWebRequest.GetResponse();
// address after all redirections:
Uri address = httpWebRequest.Address;
Related
I am trying to make a small app that can log in automatically on a website, get certain texts on the website and return to user.
To show what I have, I did below to make it log in,
System.Windows.Forms.HtmlDocument doc = logger.Document as System.Windows.Forms.HtmlDocument;
try
{
doc.GetElementById("loginUsername").SetAttribute("value", "myusername");
doc.GetElementById("loginPassword").SetAttribute("value", "mypassword");
doc.GetElementById("loginSubmit").InvokeMember("click");
And below to load html of the page
WebClient myClient = new WebClient();
Stream response = myClient.OpenRead(webbrowser.Url);
StreamReader reader = new StreamReader(response);
string src = reader.ReadToEnd(); // finally reading html and saving in variable
Now, it successfully loaded html but html of the page where it's not logged in. Is there a way to refer to current html somehow? Or another way to achieve my goals. Thank you for reading!
Use the Webclient class so you can use sessions and cookies.
check this Q&A: Using WebClient or WebRequest to login to a website and access data
Why don't you make REST API calls and send the data like username and password from your code itself?
Is there any Web API for the URL ? If yes , you can simply call the service and pass on the required parameters. The API shall return in JSON/XML which you can parse and extract information
Hello i making a simple httpwebrequest and then i read (StreamReader) the response and just want to get the html page of website,but i get only one laber(only one element of the page) in the browser all fine(i see all page) but when i try to set cookies to Deny\disable i also in the browser get this label(only one element of the page) and all is disappear.Sow i getting opinion if after i disabled cookies in browser i get the same page(like in code) that mean my HttpWebRequest is have settings cookies=deny/disable.
You can go to https://www.bbvanetcash.com/local_kyop/KYOPSolicitarCredenciales.html and disable cookies with F12 and you will see the difrance and also this page with one label.
Sow this my code any ideas what i need to change here?
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create("https://www.bbvanetcash.com/local_kyop/KYOPSolicitarCredenciales.html");
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
Stream streamResponseLogin = myHttpWebResponse.GetResponseStream();
StreamReader streamReadLogin = new StreamReader(streamResponseLogin);
LoginInfo = streamReadLogin.ReadToEnd();
Your code is receiving complete page content, but it cannot receive the dynamic contents. This is happening because the page you are trying to access relies on Cookies for maintaining session as well as JavaScript (it is using jQuery) for loading dynamic contents and providing rich user experience.
To successfully receive the whole page, your code must
support retrieving, storing and sending cookie objects across various HttpRequest and HttpResponse.
be able to execute JavaScript code to load the dynamic contents/markup of the page
To test 'if your code is receiving proper values or not' visit the site Web Sniffer and put your URL there.
As you can try on web-sniffer site, for www.google.com, the response you are getting is a redirect instruction.... that means, even to access the Google's home page, your code must understand HTTP status messages (302 there).
I'm trying to make an application in VS in C #.
Wanted navigate in webBrowser I have on my form through proxy.
But I can not.
With this code I can access a page by proxy.
But if I click within the page. I am already browsing by webBrowser directly without going through proxy.
proxyURI = new System.Uri("http://" + myProxy);
proxy = new System.Net.WebProxy(proxyURI);
request = (HttpWebRequest)WebRequest.Create("http://stackoverflow.com/");
request.Proxy = proxy;
response = (HttpWebResponse)request.GetResponse();
receiveStream = response.GetResponseStream();
webBrowser1.DocumentStream = receiveStream;
To make things easier is not possible to place the proxy in the WebBrowser?
Like:
webBrowser1.Document.GetElementById("login").byProxy("MyProxy").InvokeMember("click");
My apology but I'm noob in C #
Don't mix WebRequest and WebBrowser in this case, they have completely different Net sessions.
Use WebBrowser alone, and set the proxy with UrlMkSetSessionOption/INTERNET_OPTION_PROXY. Do this before you create WebBrowser (Otherwise, I'm not sure if existing WebBrowser instance picks up new proxy settings instantly).
I've recently been looking on how to get data from a website using C#. I tried using the WebBrowser object to navigate and log in, and that worked fine, but I keep getting the same problem over and over again: when I navigate to the wanted page I get disconnected.
I've tried several things like making sure that only one HtmlDocument exists but I still get logged out.
TLDR: how do you stay logged in, from page to page, while navigating a website with WebBrowser? Or are there better alternatives?
EDIT: So far I have the following code;
currentWebBrowser = new WebBrowser();
currentWebBrowser.DocumentText = #"<head></head><body></body>";
currentWebBrowser.Url = new Uri("about:blank");
currentWebBrowser.Navigate("http://google.com");
HttpWebRequest Req = (HttpWebRequest) WebRequest.Create("http://google.com");
Req.Proxy = null;
Req.UseDefaultCredentials = true;
HttpWebResponse Res = (HttpWebResponse)Req.GetResponse();
currentWebBrowser.Document.Cookie = Res.Cookies.ToString();
At which moment should I get the cookies? And is my code correct?
You have to preserve the cookies returned from your login request and reuse those cookies on all subsequent requests - the authentication cookie tells the server that you are in fact logged in already. E.g. see here on how to do that.
I am trying to login to a HTTPS website and then navigate to download a report using c# (its an xml report) ?
I have managed to login OK via cookies/headers etc - but whenever I navigate to the link once logged in, my connection takes me to the "logged out" page ?
Anyone know what would cause this ?
Make sure the CookieContainer you use for your login is the same one you use when downloading the actual report.
var cookies = new CookieContainer();
var wr1 = (HttpWebRequest) HttpWebRequest.Create(url1);
wr1.CookieContainer = cookies;
// do login here with wr1
var wr2 = (HttpWebRequest) HttpWebRequest.Create(url2);
wr2.CookieContainer = cookies;
// get the report with wr2
It can be any number of reasons. Did you pass in the cookie to the download request? Did you pass a referrer URL?
The best way to check is to record a working HTTP request with Wireshark or any number of Firefox extensions or Fiddler.
Then try to recreate the request in C#