Today i tried to write a program which checks some checkboxes for me on a webpage and then clicks on a button.
For this purpose i tried to use the webbrowser, but how can I set the state of a checkbox there? Searching the internet for hours but no luck only managed to navigate to the webpage with the checkboxes.
One approach would be to write a Bookmarklet, where you create a bookmark that runs JavaScript code, and the other would be to avoid the web browser altogether and instead just send a request directly to the web server that looks like it would if you had checked the checkboxes and clicked the button. Using a tool like wget or curl can make the latter option pretty easy.
Here's a sample URL that you could use to go for the Bookmarklet approach:
javascript:document.getElementById('theCheckBox').setAttribute('checked', 'checked');document.getElementById('theForm').submit();
The easiest way to do the second approach would be to use a tool like Firebug or Fiddler to monitor what a request looks like when you manually submit a page with your checkboxes checked and then construct similar requests through curl.
Using a WebBrowser control is not really a good approach here. The purpose of this control is to display a webpage, not to automate user interaction with it.
The most easy and most reliable solution is to use HttpWebRequest to directly talk to the server, sending a (most likely) POST request.
Related
I'm displaying a website in a C# WebBrowser. But I would like to display only the search part not the whole website so it won't bo so big on the screen. This is the website http://www.buscacep.correios.com.br/ and I would like to display only the Busca CEP - Endereço Box. Any ideas of how I can do this? I tryed to use htmlagilitypack but it has very little documentation and I couldn't understand it.
The WebBrowser control isn't really designed for what you're asking. You probably could go through all the page elements and remove anything that isn't part of the search box, but that's a lot of work for very little value.
However, there's a bright side. As mentioned in a comment, you should be able to POST directly to the search page. Use a program like Fiddler to find out what form values are being passed to the server with the request. Then you can re-create that request from your own application (using a WebClient or HttpClient). The result will be HTML, which you can display in your WebBrowser by setting the returned HTML to the WebBrowser's DocumentText property.
I'm programming an autocomplete scraper in C# .Net 2.0. I tried a lot of tricks to make an Input Box on a specific website to show its autocomplete suggestions. Because I'm filling it with HtmlElement.SetAttribute("Blah"), it doesn't act as if I was filling it by hand, so it doesn't show the autocomplete suggestions
SendKeys is a very bad solution since it is imprevisible and crappy.
I tried HtmlElement.InvokeMember("WithEveryOptionPossible") and it does nothing.
I tried the SendMessage PInvoke and I saw no impact on my WebBrowser.
Is there an existing solution to send a key or a virtual key to WebBrowser so I could trigger the autosuggestion of the input box?
[edit]
I've been able to see what is being fetching via Wireshark. Didn't think about it before. I only needed to convert and attach my WebBrowser cookie to a WebRequest cookie and use the direct autocomplete URL. Now I can fastly get autocomplete suggestions without Timer and SetAttribute. (Until I get banned.) In fact, a WebBrowser is not needed anymore, since only a session-cookie is important, and no SetAttribute are needed. I think the former question has maybe no solution.
is it possible to take a screenshot of what a textbox holds when the user presses the sumbit button for example?
EDIT: this is an aspx webpage
In short, no it is not possible to do this in a consistent, cross browser fashion (that I am aware of). If your textbox was implemented inside of a flash movie, it would be possible to take a 'screenshot' of what the flash movie was displaying when a button was pressed (discussion on this subject available here). But otherwise, you are going to have to do this processing on the server.
You could simulate this process by having the server render a copy of the page itself (feeding it the data the user entered) and then doing what you wanted with it from there. There are free and paid for solutions to assist you in taking a screenshot of a website (browse options available here).
On the client side I think you're stuck with the limitations of javascript, which might not be possible. Here is another question that is very similar to yours:
Take a screenshot of a webpage with JavaScript?
In the general sense, no, you can't. However if you have a constrained environment (e.g. kiosk, intranet), you can create a browser plug-in which can essentially do anything, including snapping a screenshot and sending it to the server.
If you have lots of control over the environment, you can create your own web browser which can take screenshots. In fact, I've done this with C#. I just wrote an app that hosts a browser control and sends screenshots to the server on certain key presses or at a user-defined interval.
I need to create c# code to go to site, fill in form submit record.
I have tried using seleniumhq but this creates tests..I need more of a script i can run once to help me register some users for my site
any ideas??
You can create c# code that posts data to the form page for the site that you are trying to register the people for. Otherwise depending on the browser you could use something like iMacros to automate the form filling. The other answers are also correct. Unless it is a site you control none of these methods will work if the site uses captcha or other methods to prevent automatic form filling.
Edit: Something like http://geekswithblogs.net/rakker/archive/2006/04/21/76044.aspx would be a good start.
If you are trying to automate mouse clicks and keyboard strokes, you can try AutoIt.
i would use an HttpWebRequest Object to submit the form, a quick google found this:
http://netomatix.com/HttpPostData.aspx
http://watin.sourceforge.net
I'm working on a web scraping application and was testing it with ebay. The thing is that the application should follow the link "Next" (the one at the bottom of the page that should go to the next page of results) but it kinda stays on the same page (yea, i'm actually not sure about that). If you try to open ebay and search for any term that will give a result with multiple pages, and then either copy the link of "Next" and paste it on a new window or right click it and select open in a new tab/window, it will stay on the same page. I tested it on Chrome and IE8. So my question is what are these browsers doing when they actually follow the link (when I just click on it) so that I can do the same with my scraping application? (Oh, and by the way I'm working on C#)
In the case of eBay it is just a normal link (at least on http://www.ebay.com, look for page 2 of TV's) so the problem is probably with your code (are you storing cookies for instance?). From your description it sounds that it's an AJAX request, which would go "under the hood" and gets XML from the server which is rendered by JavaScript on the client side.
Traditionally, AJAX requests are hard to follow. In the case of ebay, however, I'd suggest use the interface that ebay has to query for information. If you are building a generalized web crawler, then stay away from the AJAX requests. Google doesn't bother either, most of the time.
I did a element.InvokeMember("click"); (where element is an HtmlElement) and it worked. Not sure why though. I'll take a look at that HTTP GET thing anyway.