I am familiar with nunit for unit testing of the business layer however I am looking now to automate the test of the win forms gui layer.
I have seen watin and the watin recorder for automating tests on web application by accessing the controls and automating them. However I am struggling to find a watin equivalent for windows forms (written in c# or vb.net) preferably that is open source.
Does one exist or are all products based on recording mouse and keyboard presses?
Update: I have looked at this blog post on white and it seems the sort of thing I am looking for. The blog post raises some issues but as white is only in version 0.6 these may be resolved. Be interested if others have used white or any others for comparison.
Check out http://www.codeplex.com/white and http://nunitforms.sourceforge.net/. We've used the White project with success.
Same Answer to a previous question
Edit
The White project has moved, and is now located on GitHub as part of TestStack.
AutomatedQA's TestComplete is a good testing application to automate GUI testing. It supports more than just Windows Forms also, so you can reuse it for other applications. It is not open source and this is the best that I have found. I haven't seen an open source equivalent to WatiN. It does have a free trial, for you decide if you like it or not. The main reason I went with it, is that it really is cost effective, compared to other testing applications.
As a new alternative, I can give you FlaUI (https://github.com/Roemer/FlaUI). It is basically a complete re-write of white with more features and a clean code-base.
WinAppDriver is a Selenium-like UI test automation service for testing Windows applications including Windows Forms applications. It can be used with Appium, a test automation framework.
The list
Please check the updated list at the: List of Automated Testing (TDD/BDD/ATDD/SBE) Tools and Frameworks for .NET: User Interface Testing
Framework
Comment
Coded UI
Discontinued
FlaUI
NUnitForms
Discontinued
Squish GUI Tester
TestComplete
TestStack.White
WinAppDriver
What Microsoft recommends
WinAppDriver
Microsoft recommends to use WinAppDriver:
Windows Application Driver (WinAppDriver) is a service to support
Selenium-like UI Test Automation on Windows Applications. This service
supports testing Universal Windows Platform (UWP), Windows Forms
(WinForms), Windows Presentation Foundation (WPF), and Classic Windows
(Win32) apps on Windows 10 PCs.
Coded UI (Visual Studio ≤ 2019)
Previously, Coded UI, a Visual Studio built-in feature and part of the UI Automation, was recommended for application UI testing (it's deprecated now):
Coded UI Test for automated UI-driven functional testing is
deprecated. Visual Studio 2019 is the last version where Coded UI Test
will be available. We recommend using Selenium for testing web apps
and Appium with WinAppDriver for testing desktop and UWP apps.
Consider Xamarin.UITest for testing iOS and Android apps using the
NUnit test framework.
Automated tests that drive your application through its user interface
(UI) are known as coded UI tests (CUITs). These tests include
functional testing of the UI controls. They let you verify that the
whole application, including its user interface, is functioning
correctly. Coded UI Tests are particularly useful where there is
validation or other logic in the user interface, for example in a web
page. They are also frequently used to automate an existing manual
test.
Read more at: https://learn.microsoft.com/en-us/visualstudio/test/use-ui-automation-to-test-your-code
As far as I know, White is an abstraction layer over the top of Microsoft's UI Automation framework. I have written a similar layer that we use internally on our projects and it works great. So White would definattley be worth a look
Microsoft have released the source to UI Automation, so if necessary you should be able to debug right down the whole stack if necessary.
The really cool thing is that with licence cost, you can scale up and run as many machines as you like for execution.
We run inside VSTS and link our results to requirements, but you can use c# express and nUnit and get first class tools and languages for little to no cost.
Here are some links from MSDN Magazine on automatic testing code:
Using UIAutomation Bugslayer March 2007
Using PowerShell Test Run December 2007
Tester, a utility for recording mouse clicks and keystrokes, then playing them back & program checking behaviour. Excellent for unmanaged code. Uses windows handles so may not work well for managed code. Bugslayer March 2002.
You could check out the Microsoft UI Automation framework. This has been included in .NET since version 3.0. This is actually what the White framework uses anyway.
You may also use Winium(https://github.com/2gis/Winium) that works on multiple Windows platform besides Windows 10 and is similar to Selenium with extra features that support controlling the application remotely.
Related
Blueprism gives the possibility to spy elements (like buttons and textboxes) in both web-browsers and windows applications. How can I spy (windows-based only) applications using Python, R, Java, C++, C# or other, anything but not Blueprism, preferrably opensource.
For web-browsers, I know how to do this, without being an expert. Using Python or R, for example, I can use Selenium or RSelenium, to spy elements of a website using different ways such as CSS selector, xpath, ID, Class Name, Tag, Text etc.
But for Applications, I have no clue. BluePrism has mainly two different App spying modes which are WIN32 and Active Accessibility. How can I do this type of spying and interacting with an application outside of Blueprism, preferrably using an opensource language?
(only interested in windows-based apps for now)
The aim is of course to create robots able to navigate the apps as a human would do.
I guess you are using selenium for web-browsers
There is also some projects for windows based applications working with a Windows Driver.
Take a look to the project on github, it may be what you are looking for.
https://github.com/2gis/Winium
https://github.com/microsoft/WinAppDriver
Autoit: https://www.autoitscript.com/site/
It also comes with an identify mode for application elements and is has a big community
There is a free version of Blue Prism now :) Also Blue Prism uses win32, active accessibility and UI Automation which is a newer for of the older active accessibility.
To do this yourself without looking into Blue Prism you would need to know how to use UIA with C#/VB.new or C++. There are libraries however given that Blue Prism now has a free version I would recommend using that. Anything specific can be developed withing a code stage within Blue Prism.
I need to interact with a windows application by clicking a button in my application. More precisely: how can I write in a textbox or scroll a dropdown menu?
There are multiple ways to do this depending on what kind of application you are automating. If you are automating a WPF application I would suggesting using Microsoft UI Automation (UIA). There are some nice wrappers written around UIA like TestStack.White and FlaUI. FlaUI is the more modern of the two and supports UIAv3 using a COM wrapper. TestStack.White is built on top of a UIAv2 using the managed wrapper in the .NET Framework which is no longer supported.
If you are automating anything else besides a WPF application you can do straight PInvokes to SendMessage. I would suggest staying away from that method and using the Microsoft UIA framework since sending windows messages can get quite verbose. If you really want to go with the Win32 route I suggest using something like AutoIT to automate your application.
Lastly, you will want to download the Windows SDK and run Inspect.exe from that. Inspect.exe is the application that will allow you to see the properties you are attempting to query and the patterns that are available. Applications like AutoIT have essentially built their own Inspect.exe or reference and application call UISpy which is also in the Windows SDK but mostly superseded by Inspect.exe.
System.Windows.Automation is EXTREMELY slow.
I execute:
element.FindAll(TreeScope.Children, Condition.TrueCondition);
Obtaining only 30 child elements may take 1000ms on a very fast computer.
I have even seen it hanging forever while getting the child elements of a Tree in a QT application.
Is this a known problem?
I cannot find any usefull answer after googling a lot.
System.Windows.Automation is EXTREMELY slow.
System.Windows.Automation is full of bugs. It may not return all children of an AutomationElement, which is a very severe bug.
Apart from that the implementation is not thread safe.
System.Windows.Automation is deprecated. Do not use it!
In the MSDN you find the following note:
UI Automation was first available in Windows XP as part of the
Microsoft .NET Framework. Although an unmanaged C++ API was also
published at that time, the usefulness of client functions was limited
because of interoperability issues. For Windows 7, the API has been
rewritten in the Component Object Model (COM).
Although the library functions introduced in the earlier version of
UI Automation are still documented, they should not be used in new
applications.
The solution to slow performance is to use the new IUIAutomationElement COM interface instead of the old System.Windows.Automation C# interface. After that the code will be running lightning fast!
Apart from that the new interface offers much more patterns and Microsoft is extending it continously. In the Windows 10 SDK (UIAutomationClient.h and UIAutomationCore.h) several patterns and properties have been added which are not available in the .NET Automation framework.
The following patterns are available in the COM version of UIAutomation which do not exist in System.Windows.Automation:
IUIAutomationLegacyIAccessiblePattern
IUIAutomationObjectModelPattern
IUIAutomationAnnotationPattern
IUIAutomationTextPattern2
IUIAutomationStylesPattern
IUIAutomationSpreadsheetPattern
IUIAutomationSpreadsheetItemPattern
IUIAutomationTransformPattern2
IUIAutomationTextChildPattern
IUIAutomationDragPattern
IUIAutomationDropTargetPattern
IUIAutomationTextEditPattern
IUIAutomationCustomNavigationPattern
Additionally the following Control types have been added:
AppBar
SemanticZoom
Additionally the following Element's have been added:
IUIAutomationElement2
IUIAutomationElement3
IUIAutomationElement4
And what concerns the bugs: The new COM UIAutomation Framework is very well designed and I could not find bugs on the client side of the framework which is a great progress compared to System.Windows.Automation. But several missing features and even bugs on the server side of the framework. On the server side each GUI framework must implement an UIAutomation provider (see MSDN: Interfaces for Providers). So these problems differ depending on what type of application you are automating because each GUI framework has it's own problems:
In the Native Windows GUI features are missing: Lots of controls do not implement the patterns that they should implement. For example a SplitButton in a native Toolbar should implement the Invoke pattern to click the button and the ExpandCollapse pattern to open the drop-down menu. But the ExpandCollapse pattern is missing which makes it difficult to use SplitButtons. If you obtain a Toolbar SplitButton by IUIAutomation->ElementFromPoint() and then ask for it's parent you will get a crippled element. And the Pager control cannot be automated at all.
Also in WPF applications there are controls that are implemented buggy by Microsoft: For example if you have a Calendar control you see two buttons at the top to switch to the next/previous month. If you execute the Invoke pattern on these buttons you will get an UIA_E_NOTSUPPORTED error. But this is not a bug on the client side of the framework, because for other buttons the Invoke pattern works correctly. This is a bug in the WPF Automation server. And if you test IUIAutomationTextRange with a WPF RichTextBox, you will find that several commands are not implemented: Select() and ScrollIntoView() do simply nothing.
For .NET Forms applications Microsoft did not make much effort to support them. The .NET Calendar control cannot be automated at all. The entire control is not even recognized as Calendar. It has the ControlType "Pane" with no child elements in it. The same applies to the DateTimePicker. And for complex controls like DataGrid and PropertyGrid the only implemented pattern is LegacyIAccessible which is a poor support. These controls should implement at least the Table and the Grid and the ScrollItem pattern.
Also Internet Explorer cannot be automated because elements outside the visible area cannot be scrolled automatically into view due to missing coordinates. (The Bounds are returned as an empty rectangle) And the ScrollItem pattern is not implemented. (Yes, I know that Internet Explorer has been replaced with Edge in Windows 10, but the UIAutomation framework exists since Windows 7 and Microsoft did not implement a usefull automation support in Internet Explorer in all these years)
I saw even complete crashes of the automated application. For example Visual Studio and TotalCommander will crash if you execute certain automation commands on a certain control. Here - once again - the bug lies in the server side implementation of the framework.
Summary: We have a great framework with limited usefullness. The Microsoft team that developed the new UIAutomation framework did a great job, but the other areas in Microsoft (the native GUI, WPF, .NET and Internet Explorer team) do not support this framework. This is very sad because only a small effort would have to be made to offer a better functionality. But it seems that the users who use UIAutomation in the first place (handicapped people) are not a profitable market.
I'm looking for a library that can be used in native .NET code, just like any .NET assembly. The purpose of the library must be to automate Windows (push a button, select a window, send keys, record & playback, that sort of thing).
So: the library is supposed to be used natively in .NET, but the automation itself must be able to target any native or .NET Windows application that can receive user input.
Suggestions so far:
benPearce suggested AutoIt. It has a DLL, which is native Win32 but not native .NET and cannot be used without use of .NET Interop.
Chris Dunaway suggested Global Mouse Keyboard Lib. This came closest, but is not an automation lib. It just helps setting up keyboard and mouse hooks.
pm100 suggested Microsoft's WPF UI Automation. This one is pretty good, albeit that it's not available if you develop in .NET 2.0 and it requires the WPF to be installed on the system. It can, however, automate everything from Win32 apps to HTML in a browser.
JasonTrue suggested WebAI from ArtOfTest. This is a testing framework mainly geared towards browsers and web applications. It is unfortunately not well suitable for use for Windows automation.
If nothing else appears available, I'll probably choose Microsoft's UI Automation and upgrade any projects that require it that are still in .NET 2.0 to .NET 3.5, if possible. But I hope for a more widely applicable automation framework (.NET prior to 2.0 does not need to be supported).
Have you looked at the White framework?
I have used AutoIt in the past with success.
microsoft's own built in one is fine
http://msdn.microsoft.com/en-us/library/ms747327.aspx
not restricted to wpf as some seem to think.
If you haven't seen it yet, and a commercial library is acceptable, you might check out Ranorex:
http://www.ranorex.com/
I used Ranorex 1.5 quite a bit to write small C# UI automation utilities. It was pretty effective! Development seemed faster compared to using the MS UI Automation API directly, since Ranorex has a lot of useful convenience methods already available.
I haven't used Ranorex 2 very much yet, though.
In Ranorex 1.5, there was also support for traditional Win32 development in C++, but I didn't use it. As far as I know, that's still available in Ranorex 2.
I can't speak to the quality of the record/playback support in Ranorex since I never used that feature.
One final plus: Their support team was really responsive and helpful anytime I emailed them.
This library is pretty interesting and is fairly simple. Perhaps it will help you.
Check out Tools for automated GUI testing on windows
I would still suggest FlaUI for autoamating .Net Desktop,Mobile apps. Its based on Microsoft UIA libraries and have support for external controls like the DevExpress Grid too
Moreover, it is built on top of TestStack.White so indeed a very good library and has a github page also
I have used WebAii from ArtOfTest with a fair degree of success in automating integration testing for a Silverlight app. It also supports WinForms and Web applications.
Microsoft UI Automation, the successor to Active Accessibility, can do almost all of the Windows UI automation you would need.
How about CSharpScript, here's an article about it on Codeproject, and here's the link to the main website. Furthermore, it is familiar C#, scripted which can be used to automate anything.
Hope this helps,
Best regards,
Tom.
I am considering porting a WPF application to Silverlight. However, the WPF application uses Watin to spawn IE processes and automate certain tasks.
My question is if there is a way to automate tasks in Silverlight by spawning popups and performing tasks in them similar to how you would with Watin.
I haven't tried this tool yet, but it is the closest I've found to Watin for Silverlight. It is WebAii from Telerik and it is a free download. I ran across it the other day and plan to try it soon.
WebAii Testing Framework plugs
directly into the Silverlight
application it automates and has
access to every single element/object
in the entire Silverlight application.
Beside the ability to perform basic
automation actions like clicking,
moving and setting text, the tool
gives you access to complex properties
on UI elements such as brushes,
borders and even transform matrices.
What’s best is that you can set most
of these properties, which is crucial
for test verifications and
synchronizations.
As far as I know WatiN is a web application test framework, but your question seems to indicate that you are using WatiN to automate tasks in your application. In that context I will try to answer your question.
A Silverlight application is executed in a sandboxed environment and is unable to start new processes on the local computer. However, a Silverlight application can interact with the browser object model that is used to host the Silverlight application. This allows for some interaction with the local environment. If you can fit your use of WatiN into this model you are able to do it from Silverlight but in general you should consider a Silverlight application to live in a sandboxed environment.
Have you had a look at the Silverlight Automation Peer?
http://msdn.microsoft.com/en-us/library/cc645045(VS.95).aspx