web frontend for C# unit tests projects - c#

The project I'm working on has a bunch of service-tier unit tests using Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute. I want to look into some web frontend automated generation tool for these tests. I don't care if I need to use some other framework like NUnit. I need some decent way to have an easy web frontend for looking at test results, that also allows adding new tests in an easy manner.
After a bit of investigation I realised that we already have TeamCity for the builds. Do I need anything else to setup test browsing from teamcity?

We use Cruise Control, NAnt, SubVersion and NUnit together to provide continuous integration. Every commit triggers a build and runs all the unit tests. The cruise control dashboard show build results, unit test results and code coverage for each build. Is that the kind of thing you are looking to do or do you want some kind of web based ad hoc test runner?

Continuous Integration systems normally let you do this and usually have a web front end.
I know that you could set this up using CruiseControl.Net (which is free), the other system that has been recommended to me in the past is TeamCity so I'm sure that could do this too (and its also free as long as you don't configure too many projects).

Related

how to execute test cases in better way for windows application using winappdriver and C#

I am working on an existing test automation framework that uses the following for UI tests on a windows application:-
C# for creating UI cases
winAppDriver to interact with UI objects
nUnit for validation
specflow for BDD
I don't have much idea about microsoft projects. Here is a simple structure of the project:-
Application
Source
Modules
features
Tests
Each test folder has a "app.config" file in it that supplies the config for the test to work like DB username, pwd, services urls etc.
We execute our test case from "Test Explorer" pane in VS Enterprise which is not the best way if the test cases need to be run remotely in jenkins. As I said I don't have much frame of reference when it comes to Microsoft apps. So, here are some questions to which I have been looking for answers; there seems to be no definite consensus online. Just curious to know how others are maintaining their projects. Here goes:-
is there a jenkins friendly way of running these test cases? using a cmd line or a runner file perhaps?
if I find a way to accomplish #1 how do I inject app.config properties at runtime?
how does one execute these cases on remote machine? Mine is a desktop windows app. What would a high-level strategy look like? I assume I will have to get a remote machine and install the app on that machine?
any pointers, resources to read about would be helpful. Just looking for a nudge in the right direction.
Since you are using nUnit, it has a default approach of running tests from console see here
It looks like you need to pass parameters from the test runner into the runtime. You can use this approach
Execute on remote machine strategy depends on your current infrastructure. E. g. if you are using Gitlab, you should set up the GitLab CI runner into your machine and set up the GitLab pipeline.
looking for a nudge in the right direction
See CI\CD test run best practices

Why would I need to use NUnit with Selenium WebDriver?

I'm just getting started with Selenium, using VS2012 and C#. I'm not sure if using a separate testing framework such as NUnit or HtmlUnit is necessary. I tried out the google search example that is available online in C# without using NUnit and it worked fine.
So my question at this point is, why would i need to use NUnit with Selenium?
You don't NEED to use NUnit (or any other unit testing framework) with Selenium if you don't want to. However, there may be instances where you might want to use NUnit (or others) to leverage the benefits of other things. For example:
If you have existing unit tests it keeps everything in one place. (If that's the way you want to organise things)
If you already use NUnit (or your preferred unit testing framework) for unit tests then you can re-use the same test runner (e.g. NUnit console, NUnit GUI, ReSharper, etc.) you use for NUnit meaning you can run all your tests (NUnit and Selenium) with one button press/keyboard shortcut.
If you use continuous integration it can run your selenium tests through the existing NUnit (or which ever you prefer) test runner which means you don't have to configure your continuous integration server for the selenium tests separately.
The above assumes that you have unit tests already. If you don't already have unit tests, or you are only interested in the Selenium tests (For example, we have a development team and a tester, we write unit tests, the tester writes Selenium tests and they are run independently of each other) then there is no need to add that extra layer.
Unit tests frameworks and selenium test for different things. Unit tests typically look at a single unit of code at a time (although in practice, I find it often spills out into adjacent units especially if they are small and deterministic). Selenium looks at a web page (or series of pages) as a single test. Selenium's tests need a system to have many of its components integrated together to run the test. It is therefore testing at a higher level as it is checking many things at once. (e.g. that the system can cope with requests, that the responses arrive back, that the responses contain the expected data, that pressing buttons on the page do the correct things, go to the correct pages, etc.)
Ultimately, just because you can do something doesn't mean you should. Running Selenium tests through a unit testing framework is a convenience if you have to handle both. It may work for you, it may not.

Is it possible to run specflow tests programmatically?

In my project we had BDD tests which I have written using specflow, nUnit and Watin. I run these tests from visual studio using resharper. Now I want to expose these features and scenarios to non technical people and I want them to run these tests.
Something like I want to list all the tests in a browser and user should be able to run those tests by clicking on them. Can this be achieved ? Is there any addin ?
Currently we use Team Foundation Server as our build server.
TeamCity, a Continuous integration server by JetBrains provides this as a webbased functionality. It even provides statistics and test output results.
It supports nUnit out of the box.
SpecFlow and Watin are supported with some configuration.
The BIGGEST problem you are going to have is that the plain text feature file, automatically gets converted to a xxx.feature.cs file by the SpecFlow Visual Studio plugin. So your process is this,
Modify xxxx.feature file
Find some way to get the SpecFlow plugin to generate xxx.feature.cs
Compile
Run tests by using NUnit/Xunit (as configured)
Gather and present test success report
To me this is a process has a name, I'd called it development.
BDD however is a different process, it's all about collaboration and communication with the business in order to devise a specification. In the beginning there were no tools, but the process still worked.
A number of my co-workers have been using BDD techniques on a variety of real-world projects and have found the techniques very successful. The JBehave story runner – the part that verifies acceptance criteria – is under active development.
Dan North - Introducing BDD 2006
Don't get caught up on the tools alone or you'll miss the vital part of the process. You'll get so much benefit by working with your BA to define the new specification together collaboratively.
P.S. Another way to consider this is that the specification and code should always be in step. Just by defining a new example, we don't magically move the code forwards to meet that example. Instead the most common practice is to develop the code to meet the new example, and then check in the new specification and code as a single change set.
You can use the Pickles project to produce stakeholder-friendly documentation (including HTML) from the Gherkin specifications in your source control.
https://github.com/picklesdoc/pickles
There's no facility for running the tests from the HTML. It's open-source so perhaps you can extend it this way... however, I personally don't see the value in having non-technical users actually execute the specifications. I would have your continuous integration server run the SpecFlow tests and generate a step definition report periodically. The non-technical users can then browse to these reports to see current project status.
To give access to your feature files to non technical people you can use http://www.speclog.net/
Spec log will allow non tech to edit and create new features and will automatically synchronise them with TFS.
Unfortunately it's not free and you can't run the specs from that tool.

Email all the test results once nunit finishes the tests

I wrote unit tests using NUnit. Once all the tests are done, I want to email the Test results to all my team. Is there a way to do it?
Usually this is done using an automated build tool like Cruise Control. It checks your code out of version control, builds the app, runs all the tests, packages the app, and sends it to the first deployment server. Team members can view the complete results of the build and test cycle using a browser to check the build server dashboard.
I'd prefer that to getting e-mailed test results. E-mail would soon become an annoyance.
Since you are using TFS, you can make use of NUnit for Team Build which would make your NUnit test results visible in the build log and incorporated into the data warehouse for reporting.
There is also NUnitForVS which can publish results within TFS.
Going one of the above routes will allow you to leverage the current CI environment and is a much better approach to emailing the results as they surface.

Whats a great way to perfom integration testing?

We have written our own integration test harness where we can write a number of "operations" or tests, such as "GenerateOrders". We have a number of parameters we can use to configure the tests (such as the number of orders). We then write a second operation to confirm the test has passed/Failed (i.e. there are(nt) orders).
The tool is used for
Integration Testing
Data generation
End-to-end testing (By mixing and matching a number of tests)
It seems to work well, however requires development experience to maintain and write new tests. Our test team would like to get involved, who have little C# development experience.
We are just about to start a new Greenfield project and I am doing some research into the optimum way to write and maintain integration tests.
The questions are as follows:
How do you perform integration testing?
What tool do you use for it (FitNess?, Custom?, NUnit)?
I am looking forward to peoples suggestions/comments.
Thanks in advance,
David
Integration testing may be done at a user interface level (via automated functional tests - AFT) or service/api interface level.
There are several tools in both cases:
I have worked on projects that successfully used Sahi or Selenium for AFT of web apps, white for AFT for .NET WPF or winforms apps, swtBot for AFT of Eclipse Rich client apps and frankenstein for AFT of Java swing apps.
Fitnesse is useful for service/api level tests or for tests that run just below the UI. When done right, it has the advantage of having business-readable tests i.e. non-developers can read and understand the tests. Tools like NUnit are less useful for this purpose. SOAPUI is particularly suited for testing SOAP web services.
Factors to consider:
Duration: Can you tolerate 8 hour test runs?
Brittleness: AFTs can be quite brittle against an evolving application (e.g. ids and positions of widgets may change). Adequate skill and effort is needed to not hard code the changing parts.
Fidelity: How close to real world do you want it to be? e.g. You may have to mock out interactions with a payment gateway unless the provider provides you a test environment that you can pummel with your tests.
Some nuances are captured here.
Full disclosure: The author is associated with the organization behind most (not all) of the above free and open source tools.
You could try the Concordion framework for writing user acceptance tests in HTML files. It takes a BDD-style approach. There is a .Net port as well
It's not out of Beta yet, but StoryTeller looks promising:

Categories