I've got a desktop application written in C# created using VS2008 Pro and unit tested with Nunit framework and Testdriven.net plugin for VS2008. I need to conduct system testing on the application.
I've previously done web based system tests using Bad Boy and Selenium plugin for Firefox, but I'm new to Visual Studio and C#.
I would appreciate if someone could share their advice regarding this.
System testing will likely need to be done via the UI. This gives you two options:
1) You can manually conduct the test cases by clicking on elements.
2) You can automate the test cases by programming against the UI. There are plenty of commercial tools to do this or you can use a programming framework like the Microsoft UI Automation Framework. These tend to use the accessibility APIs built into Windows to access your UI.
Whether you go the manual or automated route depends on how many times you will be running the tests. If you are just going to run them once or twice, don't spend the time automating. You will never earn it back. If you are going to run them often, automating can be very handy.
A word of caution: Automating the UI isn't hard, but it is very brittle. If the application is changing a lot, the tests will require a lot of maintenance.
As Thomas Owens commented on your question, first you must decide what kind of system testing you want to do. But assuming you want start with Functional System Tests. Prepare use cases you want to automate. Than you must find proper tool.
Just for start:
AtoIT – is not test atomization tool but it lets automate some tasks. So you could record/script use cases. Not really recommended, but can be done.
HP QuickTestPro – easily can be done with this tool via recording/scripting but it is expensive, so maybe not worth it for personal use.
IBM Robot – as HP QTP.
Powershell – you could write scripts in powershell and execute them. If you would use dedicated ide-like tools for powershell you could record test also. I did some web automation via powershell and it worked. With a bit of work probably you could script around your desktop app.
And the best would be to try different tools, and use one that suits you best. Try this link and this link.
System tests usually have use cases, end to end scenarios and other scripted functions that real people execute. These are the tests that don't lend themselves well to automation as they are asking your unit-tested cogs to work with each other. You might have great unit tests for your "nuts" and your "wrenches" but only a comprehensive system test will let you know if you have the right sized wrench for the nut at hand, how to select/return it from/to the drawer, etc.
In short - manual tests.
If you're willing to put money down, you could look at something like TestComplete.
Although I haven't really used it yet (our company just bought it), it seems quite nice. You can record clicks and keypresses and stuff, define success criteria, and it will replay the test for you later. It appears to be quite smart about UI changes - it remembers which button you clicked, not just the (x,y) of each click.
It's scriptable, or drag-and-drop programmable.
I'm not affiliated in any way, and this is not an endorsement, because I haven't really formed an opinion of it yet.
Perhaps NUnitForms could be useful for you?
Related
Sorry to ask such a simple question. I am a newbie.
I am setting up my website. I try to create a UnitTest Project for it, but I simply cannot not create a reference for it. Even though I have created a namespace for my class method, Visual Studio doesn't allow me to use the method and to see the namespace at all.
Is UnitTesting website completely different from UnitTesting normal programs?
Any reference online available?
Thank you
You might want to consider using the new Fakes support in the Visual Studio 2012. In short, you can detour the call to the web site API and let the call routed to your supplied delegate. This allows you to isolate testing your code in a modualized fashion.
You can find more detail at http://msdn.microsoft.com/en-us/library/hh549175.aspx
Regards,
Patrick Tseng
Visual Studio ALM team.
use this article.if u are user mvc use this question.and this stie
You've probably heard the old adage, "Better, faster, cheaper, pick any two." If you want something good and fast, it isn't going to be cheap, and if you want something fast and cheap, it isn't going to be very good. Cheaper, faster, and better means that we need to write more code more quickly, right? If only it were that simple. Learning to type faster might satisfy two of the requirements, but it isn't going to make the software you develop any better. So how do we make software better? What does "better" mean?
"Better" means producing flexible, maintainable software with a low number of defects; better software is all about long-term maintainability. To achieve this, a key design decision is to ensure that components are loosely coupled. Loosely coupled software has many benefits. One that stands out is that it improves our ability to test solutions. If we write software that can be broken down easily into small parts, it becomes easier to test. It sounds simple when you word it that way, but the amount of software in use today that is difficult to test or maintain shows that it is not as straightforward as we might like to think. Software needs to be coupled to do anything useful, but developers need tools and techniques to decrease coupling so that solutions are easier to test.
I assume that your website code is in one project, and your unit test code is in another project in the same solution? In order for code in one project to access the code of another project, you have to add a reference to the other project:
Right-click on the unit test project and click 'Add Reference...'
Open the 'Projects' tab, select the website project and click 'OK'.
You should not unit test your website. You should unit test your business layer code. Testing your website would probably be UI testing, you can use tools for this like Selenium. I would recommend reading up on unit testing, and unit testing frameworks (MSTest, NUnit etc). You should layer your application so you can test your classes, objects, data access etc, all in absolute isolation.
Example:
A Manage Users page may show an administrator a list of all the users, filtered by a filter criteria.
The page should only handle the UI side of the problem, as in, it should only be concerned about displaying the data. It should send off a request to another object to get a list of users. Maybe something like a UserRepository which has a method called GetUsers which takes in a filter criteria.
This is where unit testing comes in. You would test (by mocking, perhaps) that when given a very specific list of users in the database, and a certain filter criteria, that the GetUsers method returns the list of users you expect.
I been trying MS Test manager 2010 and man it sucks bad. It's got huge potential but right now it sucks and is too buggy and crashes on all my development machines every few mins.
So is there any free software what can sort of do it the same thing?
Like what I liked about it is this.
It had a nice check list on the side you would go through each step manually and once you verified that step passed you put passed.
It was better than word since other people could log in use the same tests.
Once you did it manually it had a play back button that you just hit and it would rerun the test with your clicks. At the end you just manually verified if it all passed. This made it faster on trying multiple browsers ( note this is different than programs like selenium)
You could write shared steps that you could use in multiple tests and only have to change in one test( even though this also had many problems it is a good concept).
So what is out there. I am doing asp.net mvc, C# and using VS 2010
I am looking something like this ( just free though)
http://www.testuff.com/static/images/screenshots/big/runner.png
You might be interested in Selenium.
Take a look at WatiN
Or try WebDriver: http://google-opensource.blogspot.com/2009/05/introducing-webdriver.html
I have an application that I'm building that has had concurrency problems in the past.
This was before implementing any LINQ error handling.
Now, I have some LINQ error handling added to my code, and I was wondering if you could give me tips about how to stress test the hell out of my application. It is super important that everything works when I deploy this thing, so your input would help.
I have two boxes set up at my desk right now to simulate two users doing whatever.
Edit:
Environment: Windows XP
App Type: WinForm
User Count: 15
I strongly suggest using CHESS, a free tool from Microsoft for finding and reproducing concurrency bugs. I have found it invaluable, indispensible, and generally life-saving.
A lot of this depends on the way your application is set up. Do you have unit tests defined? Is it a multi-layer (data/UI/business/whatever) app where you don't need direct human interaction to test the functions that you care about?
There are third part testing applications that let you set up and run scripts for testing your applications (often by recording steps that you do and then running them). WinRunner is but one example. These test systems are usually configurable to some degree.
You can also create a test project (this will be a lot easier if you have a multi-tiered application), and effectively "roll your own" test application. That may give you more control over your tests - you can either choose to do tests with predictable output in a predictable order, or generate random data and test things in random orders at random intervals, or some combination thereof.
I am building a prototype for a web-based application and was considering building the front-end in HTML, which can then be reused later for the actual application. I had done a Flash-based prototype earlier, which embedded the .swf into a C# executable. Flash made for rapid turnaround time while the Windows application provided unlimited access to fancy API's for DB access and sound.
I want to consider something similar for this one too. Does this approach make sense? I am particularly concerned about the way the HTML would communicate with the container app. From what I understand out of preliminary research, it would be only through JavaScript, which might quickly get unwieldy. This is especially so because unlike the Flash-based prototype which implemented a lot of its functionality in the .swf, the HTML UI will depend entirely upon the shell to maintain state. Also, I don't need anything more than access to a database. So a desktop application might be overkill.
Another alternative that comes to mind is to build the prototype using PHP and deploy it with a portable server stack such as Server2Go or XAMPP. But I've never done something like this before. Anybody here shed some light on drawbacks of this approach?
The key requirement is rapid iterations of the UI, reusable front-end code and simplified deployment without any installations or configuration.
Some of the best programming advice I've seen came from Code Complete, and was along the lines of, "evolutionary prototypes are fine things, and throwaway prototypes are fine things, but you run into trouble when you try to make one from the other." That is, know which type of prototype you're developing, and respect it. If you're developing a throwaway prototype, don't permit yourself to use any of it, however tempting it may be, in the production system. And if you're developing an evolutionary prototype - one intended to become the production system - don't compromise quality in any way.
It sounds like you're trying to get both, the rapid development of a throwaway and the reusability of an evolutionary prototype - and you can't. Make up your mind, and stand by it. You can't have your cake and eat it, too.
I think you off to the wrong start, here. Why would you want your prototype to be fully functional? A prototype is intended to be throw-away and to help flesh out requirements and UI. If you need full functionality, why not just skip to the final product? If prototyping is really something you want to do, I suggest looking into a specialized prototyping tool.
Are you prototyping the user interface for a customer? If you are, consider something less unwieldy like paper prototypes or presentation software (like PowerPoint) until you get the UI nailed down. If you can establish the UI and are clear about the customer's requirements, you can then develop the application in whatever the actual platform is going to be with a clear model in mind.
In my current project, I prototyped the UI in PowerPoint first. In a subsequent iteration, I used static web pages and some jQuery plugins to simulate actual user interaction. That proved to be very effective in demonstrating the interface, and I didn't have to build the application first.
I would join in on folks suggesting paper prototyping as the "idea", but not necessarily the implementation. The biggest point here is that tools such as HTML or Flash let you get "bogged down" in the details - what does this color look like? What's the text on this thing? Lots of time can pass by that way. Instead, what you should be focusing on is user flows.
One tool that keeps the spirit of paper prototyping without all the "paper" drawbacks is Balsamiq: http://www.balsamiq.com/demos/mockups/Mockups.html. It was covered by Jeff and Joel in one of the Stack Overflow podcasts; I've been using it for my own projects for a while. It's freeware, and it does its job magnificently.
If you know C# then another option you can look at is Silverlight. You can then leverage your knowledge of C# and/or JavaScript and interact with a rich object model.
Would that do what you are looking for? The installation would be minimal on the part of the client - download and install the Silverlight plugin
If prototyping is something you truly wish to accomplish here, paper and pencil will be your best friends. You can draw out as many iterations as necessary. While none of this is ultimately useful later on once you begin coding, it is as quick and rapid is it goes.
As mentioned previously, there are many prototyping tools which have a bit of a learning curve, but an alternative to consider would be using a framework such as CakePHP or Ruby on Rails which make for fast application logic and leave customizing the front end being the primary hard work left. And plus, you're left with a mostly functional application when you're done with your prototyping which can be tweaked as needed.
In either scenario, you're paying with your time either upfront (in the case with learning a new framework), of over time in payments (with the case of prototyping on paper or coding by hand).
I am re-building our poorly designed web application from scratch, and wanted to get into TDD and basically "do things right" as kind of a learning project. What tools, processes, and resources are recommended to do it "right" from the start? I will be working alone as the architect and developer, with the backup of a business analyst and business owners for usability testing and use cases.
EDIT:
Right now we use SourceSafe for source control, is there any reason technologically that I'd want to try to get us to switch to subversion?
EDIT #2:
Looks like the consensus is:
Cruise Control.NET
Subversion(if I want to stop using SourceSafe)
ASP.NET MVC
NUnit for unit testing
Resharper
I highly recommend that you take a look at MVC for ASP.NET if you want to make unit testing a high priority in your development process. It sounds like it is perfect for what you are trying to do.
I would also recommend CruiseControl.NET for continuous integration (this is important if your team is going to grow).
Subversion is my favorite source control system when I am working on small teams. Use Tortoise SVN for Windows Explorer integration.
An answer to your source control question...
Redesigning an app from the ground up will probably be a time-consuming project, I wouldn't waste time changing source control unless you already know exactly which one you will use and have experience setting it up.
Visual SourceSafe gets the job done, especially in a 1 person effort, and its already in place so run with it.
We are using a setup with Visual Studio 2008, Resharper 4.1, Subversion for sourcecontrol, Cruise control for automated builds and the build in unit testing for all our automated tests. and Linq2Sql for or mapping. You could swap out anything but VS (obviously) and resharper (it's so cool) but you could easily use another sourcecontrol, or mapper or unit testing tool.
Here are some tools that can make it easier and safer to work (Googling the names will bring up the relevant pages):
Subversion - Source control
NUnit - Testing framework
CruiseControl.Net - Automated builds
Visual Source safe has a strict locking policy so that only one person can work on a file at a time....CVS or subversion allows multiple users to work on the same file at the same time.
All of the suggestions here are good, but there is no magic bullet. You'll have to look at how big your app is, how many users, how is it deployed, etc. to make your architectural, process, tool set, and other decisions. TDD, for instance, is a good methodology, but not the only good methodology for "doing things right". Another one, CruiseControl is awesome, but in a single developer project, it is probably overkill.
Be consistent in whatever you do is my best suggestion - if you go with TDD, GO WITH TDD if you know what I mean.
We re-wrote our website like you're doing and we are using C# with MVC. Its great. We use Microsoft's SourceSafe to control our code and it works awesome. Since you are the only developer it will depend on what you like.
Microsoft's sourcesafe allows us to create a branch that we can work off can keep under source control, and we can switch between both easily. (I really haven't used subversion to much so I can't comment on it.)
We use NUnit to test/ mock out our code. It super easy to mock them out. We created a class that will save and read the objects.
The save function:
Stream stream = File.Open(simplePath, FileMode.OpenOrCreate);
BinaryFormatter bwriter = new BinaryFormatter();
bwriter.Serialize(stream, actual);
The read function:
Stream stream = File.Open(simplePath, FileMode.Open, FileAccess.Read, FileShare.Read);
BinaryFormatter bwriter = new BinaryFormatter();
object returnObject = bwriter.Deserialize(stream);
We've used NUnit to mock out xml and SQL.
Good luck
If you're about to set up a fresh instance of subversion and continuous integration, starting green from a VSS background, these two free packages will likely save you days (or weeks) of time:
Visual SVN Server
Sets up everything needed for a subversion server, including Windows AD auth and an admin GUI. Free, you may consider supporting their excellent VisualSVN VS addin for source control integration in Visual Studio. Alternatively, can look at AnkhSVN
TeamCity
A Continuous Integration package (alternative to CruiseControl.NET) from JetBrains (makers of ReSharper, a fantastic tool, as mentioned) which is free for the professional version (up to 20 users and 3 build servers).
These two packages are some of the easiest installs around, challenging VSS itself :-)
Using SVN may take a little adjustment; but with the excellent doco for whichever client you pick (AnkSVN, VisualSVN, TortoiseSVN, or some combination), you'll be fine.
Besides, you know where to find people eager to answer any questions you might have in exchange for Rep ;-)
Check out TypeMock or Rhino Mocks. Mocking can save you so much time and pain when you're unit testing a web application.
If you are just starting out then I would change as little as possible (especially since you are the only developer), so stick with Sourcesafe. There is absolutely nothing wrong with it in your situation.
Later down the line, you might look towards the MS Team System, or perhaps go for other 3rd Party tools.
Personally I'm not a fan of Subversion, but I recognise that it's a popular tool across the industry.
As for TDD-specific software, I can't offer any advice. Do you have preferred tools for UML or whatever formal methods you are using?
One thing to mention: Be a 100% sure that you understand what the program's doing and what's it supposed to do, before doing any changes. A 'bad' softwer often turns out to be 'not that bad' after understanding the whole situation.
Sourcesafe can be ok, especially for one person, but when there'll be more and more ppl on the team, the lock model can get annoying, but for the time of being: stick with it