we are tasked with basically emulating a browser to fetch webpages, looking to automate tests on different web pages. This will be used for (ideally) console-ish applications that run in the background and generate reports.
We tried going with .NET and the WatiN library, but it was built on a Marshalled IE, and so it lacked many features that we hacked in with calls to unmanaged native code, but at the end of the day IE is not thread safe nor process safe, and many of the needed features could only be implemented by changing registry values and it was just terribly unflexible.
Proxy support
JavaScript support- we have to be able to parse the actual DOM after any javascript has executed (and hopefully an event is raised to handle any ajax calls)
Ability to save entire contents of page including images FROM THE loaded page's CACHE to a separate location
ability to clear cookies/cache, get the cookies/cache, etc.
Ability to set headers and alter post data for any browser call
Process and/or thread safe would be ideal
And for the love of drogs, an API that isn't completely cryptic
Languages acceptable C++, C#, Python, anything that can be a simple little background application that is somewhat bearable and doesn't have a completely "untraditional" syntax like Ruby.
From my own research, and believe me I am terrible at google searches, I have heard good things about WebKit... would the Qt module QtWebKit handle all these features?
You might try one of these:
http://code.google.com/p/spynner/
http://code.google.com/p/pywebkitgtk/
I know you mentioned you don't like Ruby syntax (neither do I), but I just have to chime in and say that Watir is probably the best thing out there for what you are trying to do.
EDIT: There appears to be a Java counter-part called Watij
I've only been digging into this recently myself, so I couldn't say that this does everything you've listed, but check out GeckoFx.
From the site: GeckoFX is an open-source component which makes it easy to embed Mozilla Gecko (Firefox) into any .NET Windows Forms application. Written in clean, fully commented C#, GeckoFX is the perfect replacement for the default Internet Explorer-based WebBrowser control.
As for my own impressions: it has blown away the default .NET WebBrowser in both performance and stability.
Related
I have to create a webbroswer in wpf that supports all the sites and scripts and play video , I have created but some site got crashed and not supporting .
Please provide solution.
What you ran into are probably error dialogs caused by JavaScript script errors. WPF WebBrowser control currently doesn't support disabling these dialogs and ignoring the errors. The WebBrowser control is based on Internet Explorer core and will have a hard time dealing with broken JavaScript, webpages will also sometimes report the web browser as old and inform the visitor about it, like FlipKart does.
One solution is to use WindowsFormsHost and host a Windows Forms equivalent - WebBrowser control - in it. The WF WebBrowser offers a property called SupressScriptErrors, which is not implemented by the WPF WebBrowser. That will hide the error dialogs for you, but you will still occasionaly receive a warning from the website saying they don't support the web browser used. That is out of your control unless you want to take a route of hacking the control to send a different User-Agent string and that will only fool broken UA detection, feature detection will still fail for things like HTML5 video, WebRTC and the like. More on this here.
You can also choose to embed Gecko (Firefox rendering engine) or Chromium. see here for more information on embedding alternative rendering cores as an alternative to IE COM wrapper offered by the framework.
Edit: this is also a possible solution, as it would seem. You will have to test it for yourself, though. I expect it to have less overhead than the WindowsFromHost element and although this uses reflection to develop against implementation, not interface, my opinion is it's safe to say the underlaying API in unlikely to change.
I'm browsing the net now for days, but still couldn't find what I'm looking for. Basically I'd like to develop my application interface, the GUI using HTML and CSS, with of course keeping the events, and the chance to modify parts of the layout at runtime (like a content of a div). The application is a database manager displaying visually the datas, and updating them at runtime.
IE aka using System.Windows.Forms.WebBrowser, this is what I try to avoid, it depends on the client system, stops at version 7, hard to link with the application
HTMLayout - http://www.terrainformatica.com/htmlayout/ : seems interesting, but the .net c# part, the nabu-library is dead
Sciter - http://www.terrainformatica.com/sciter/ : from the same group as HTMLayout, but couldn't really understand the integration, and I'm not sure if this is what I want - also couldn't find .net c# infos
Awesomium - http://awesomium.com/ : which really seems nice at first, but it needs .net 4 , and we have to stay at .net 2 - also it seems too much to handle, to use, and .net wiki is a little weak: http://wiki.awesomium.net/
and finally Gecko http://code.google.com/p/geckofx/ and Webkit http://code.google.com/p/open-webkit-sharp/ - but currently I'm not sure that they can be used to render the whole application, they just seem to be a WebBrowser with another engine.
Did I miss something?
Sciter integration principles are outlined here and here.
Sciter and HTMLayout both use the same integration principles and API architecture.
As of .NET wrapper, check these projects:
https://github.com/midiway/SciterSharp
https://code.google.com/p/expemerent/
Not sure if this fits your requirements but I currently use Webkit.NET http://webkitdotnet.sourceforge.net/
Essentially I use its Webkit Browser Control for the interface, store the HTML/CSS markup as variables and then I use C# to perform server-side logic, then output the results to the Browser Control (C# in this case acting in place of PHP since I am not including a portable web server with my application).
I did want to update it to achieve some newer functionality/standards so this article was most helpful.
http://peterdn.com/post/(First!)-Using-WebKit-nightly-builds-with-WebKit-NET.aspx
While talking with a friend over yahoo messenger, I told him would be really cool to make a bot to answer with generic messages when someone starts a conversation. Upon thinking about what I told him, I realized it would be quite interesting to do something like that. The problem is that I don't know much about win32.
So my question is this: how do you 'link' a process to both another one and the windows environment? The goal would be to have an application running in the background which makes some sort of a query to see what windows are opened and when a new yahoo messenger conversation window appears it should send a list of keystroke events to that window.
I could use either C# or VC++ for the programming part and I can use any help: either specific answers or tips that could help me - e.g.: what to google for. So far my google research only came up with some apps/dlls/code that do that for you and some scripting stuff and I'm not exactly searching for that. I want to do all the work myself so I can learn from it.
It seems like you basically want to control other applications.
There are roughly 2 ways to do this on windows
1 - Use the low level windows API to blindly fire keyboard and mouse events at your target application.
The basic way this works is using the Win32 SendInput method, but there's a ton of other work you have to do to find window handles, etc, etc
2 - Use a higher level UI automation API to interact with the application in a more structured manner.
The best (well, newest anyway) way to do this is using the Microsoft UI Automation API which shipped in windows vista and 7 (it's available on XP as well). Here's the MSDN starter page for it.
We use the microsoft UI automation API at my job for automated UI testing of our apps, and it's not too bad. Beware though, that no matter how you chose to solve this problem, it is fraught with peril, and whether or not it works at all depends on the target application.
Good luck
Not quite the same domain as what you're looking for, BUT this series of blog posts will tell you what you need to know (and some other cool stuff).
http://www.codingthewheel.com/archives/how-i-built-a-working-poker-bot
If you really want to learn everything from scratch, then you should use C++ and native WIN32 API functions.
If you want to play a bit with C#, then you should look the pinvoke.net site and Managed Windows API project.
What you'll surely need is the Spy++ tool.
http://pinvoke.net/ seems to be the website you are looking for. The site explains how to use Windows API functions in higher level languages. Search on pinvoke for any of the functions I've listed below and it gives you the code necessary to be able to use these functions in your application.
You'll likely want to use the FindWindow function to find the window in which you're interested.
You'll need the process ID, so use GetWindowThreadProcessId to grab it.
Next, you'll need to use OpenProcess allow for reading of the process's memory.
Afterwards, you'll want to use ReadProcessMemory to read into the process's memory to see what happening with it.
Lastly, you'll want to use the PostMessage function to send key presses to the window handle.
Welcome to the wonderful world of Windows API programming.
Check out Autohotkey. This is the fastest way to do what you want.
I have a webapplication that heavily uses JQuery. I would like to start using silverlight in its place if its possible. Instead of using jquery ajax features and other jquery features I would like to use silverlight. is this possible to do with silverlight?
Different tools. jQuery is best for manipulating the DOM. Silverlight is (ostensibly) best for rich internet apps, not manipulating the DOM. If your web application tries to create a cool UI using JavaScript, Silverlight might be a good fit to replace tons of JavaScript code.
If you application uses JQuery now replacing it with Silverlight will most likely not only entail redoing your JQuery features but also any HTML, CSS, forms processing and really everything about your application depending on what it does and how it is current designed.
I agree with Jonathon, they are really are two different tools.
Thanks.
Nick
nickgs.com
Technically, Yes.
But you must understand that you will have to rewrite all your application from the bottom. Microsoft propose the RIA Services as the best practice design to use with Silverlight but with this will make you change all your design and even the Data layer.
So the answer is Yes. But you would better stick with jQuery.
#Luke101, you can design an whole website using Silverlight no probs. The question though is what is your target audience?
Silverlight doesn't come shipped with the browsers as yet so a lot of computers don't have it yet and so need to download it.
Given the amount of paranoia out there I think people are a little hesitant to download something when a broswer asks you to.
The environment you'll be working in i don't think will be HTML any longer and you'll be working in xaml.
Also consider your target browser. There are still a lot of people using <gasp> IE6 and i'm not sure if SL will run on it.
What is the motivation behind moving to SL? If it's because it looks cool, don't forget a lot of that comes down to design and if you're not a design person you're likely to make a bigger hash of it than if you were working in HTML.
You may also want to look at how you integrate with a designer. I know MS Expression is designed to make the job easier but it's still an early product. Also, does your designer know about Expression or Blend and know how to use it?
What I'm getting at is there is a large learning curve here which equates to someones money and you may not see anything for some time.
Yes.
Silverlight runs in the browser, jQuery (JavaScript) also runs in the browser. So they can technically replace each other, assuming all the functionality you want in one exists in the other (including support in various browsers).
Yes. Silverlight with RIA services can replace a jQuery AJAX site. Depending on the site's features, you may even be able to break up the features of the site into silverlight "widgets" for individual functionality. The danger is that Silverlight can quickly balloon into an application that is difficult to update (though it will guarantee cross browser compatibility). Silveright is entering version 4 and it's still not finished yet, so it's changing rapidly.
I need JavaScript working almost perfectly in my application that should be able to access web content and let user take a control of it. However, webbrowser component pops new windows in separate instance / application of Internet Explorer and dotBrowser doesn't work with JavaScript properly.
Please, give me some advice - I'd even take some working example and enhance it with function I need it for.
Give a look to GeckoFX, is on open-source component for embedding the Mozilla Gecko (Firefox) in .NET applications.
I could be mistaken, but I believe new windows can be controlled (or at least suppressed) using the Web Browser control in .NET. Have a look at the NewWindow event.
I have had experience embedding both WebKit (Apple Core, used in Chrome) and XULRunner (Gecko) cores in .NET winforms applications. Let me give you some advice:
GeckoFX is your best bet. It is licensed under MPL and it works pretty much out of the box.
Follow the instructions carefully and pull the exact XUL version that is indicated. I would say it works better than the WebBrowser control. There is some talk of a significant delay when loading XUL for the first time but I've found it to be negligible. Branding and the like is fully customizable via external files.
WebKit is embeddable with SwiftDotNet. If all you need is solid javascript performance then this may be an option. If you need to be able to download files, then you have some work cut out for you.
Your downside with both routes is a nasty ~7-8mb overhead, even with compression (although I was able to get 6.5mb with lz compression in NSIS). If you are not creating distributable consumer-facing software then this may not be a problem for you.