I wrote unit tests using NUnit. Once all the tests are done, I want to email the Test results to all my team. Is there a way to do it?
Usually this is done using an automated build tool like Cruise Control. It checks your code out of version control, builds the app, runs all the tests, packages the app, and sends it to the first deployment server. Team members can view the complete results of the build and test cycle using a browser to check the build server dashboard.
I'd prefer that to getting e-mailed test results. E-mail would soon become an annoyance.
Since you are using TFS, you can make use of NUnit for Team Build which would make your NUnit test results visible in the build log and incorporated into the data warehouse for reporting.
There is also NUnitForVS which can publish results within TFS.
Going one of the above routes will allow you to leverage the current CI environment and is a much better approach to emailing the results as they surface.
Related
I am working on an existing test automation framework that uses the following for UI tests on a windows application:-
C# for creating UI cases
winAppDriver to interact with UI objects
nUnit for validation
specflow for BDD
I don't have much idea about microsoft projects. Here is a simple structure of the project:-
Application
Source
Modules
features
Tests
Each test folder has a "app.config" file in it that supplies the config for the test to work like DB username, pwd, services urls etc.
We execute our test case from "Test Explorer" pane in VS Enterprise which is not the best way if the test cases need to be run remotely in jenkins. As I said I don't have much frame of reference when it comes to Microsoft apps. So, here are some questions to which I have been looking for answers; there seems to be no definite consensus online. Just curious to know how others are maintaining their projects. Here goes:-
is there a jenkins friendly way of running these test cases? using a cmd line or a runner file perhaps?
if I find a way to accomplish #1 how do I inject app.config properties at runtime?
how does one execute these cases on remote machine? Mine is a desktop windows app. What would a high-level strategy look like? I assume I will have to get a remote machine and install the app on that machine?
any pointers, resources to read about would be helpful. Just looking for a nudge in the right direction.
Since you are using nUnit, it has a default approach of running tests from console see here
It looks like you need to pass parameters from the test runner into the runtime. You can use this approach
Execute on remote machine strategy depends on your current infrastructure. E. g. if you are using Gitlab, you should set up the GitLab CI runner into your machine and set up the GitLab pipeline.
looking for a nudge in the right direction
See CI\CD test run best practices
I'm currently using Selenium Webdriver with C#. I've successfully executed my test in remote webdriver as well using selenium GRID.
I just configured 5 instances of FF, Chrome and IE in my Grid settings and when I ran my test project on chrome browser, I noticed that just only one instance of chrome is picked. Is this the expected behavior? I was initially in an assumption that the number of tests in a single project will be distributed across multiple browser instances based on the maxinstance and maxsessions. But not sure why it is using just one browser instance for the whole project. Please do let me know if I need to do anything to use more than one browser instance to share/run the test.
Unfortunately the standard NUnit runner doesn't support parallelization out of the box.
There are a few alternative unit testing frameworks that you might want to look into that do support parallelized runs like MbUnit or PnUnit.
One workaround is to split up your test. Some common ways are by DLL, namespace, test name, or category. You could then run your NUnit test in parallel using a MSBuild script.
The final command would look something like this c:\proj> msbuild /m:8 RunTests.xml
Check out the answers to this question for more details: How can I run NUnit tests in parallel?
In my project we had BDD tests which I have written using specflow, nUnit and Watin. I run these tests from visual studio using resharper. Now I want to expose these features and scenarios to non technical people and I want them to run these tests.
Something like I want to list all the tests in a browser and user should be able to run those tests by clicking on them. Can this be achieved ? Is there any addin ?
Currently we use Team Foundation Server as our build server.
TeamCity, a Continuous integration server by JetBrains provides this as a webbased functionality. It even provides statistics and test output results.
It supports nUnit out of the box.
SpecFlow and Watin are supported with some configuration.
The BIGGEST problem you are going to have is that the plain text feature file, automatically gets converted to a xxx.feature.cs file by the SpecFlow Visual Studio plugin. So your process is this,
Modify xxxx.feature file
Find some way to get the SpecFlow plugin to generate xxx.feature.cs
Compile
Run tests by using NUnit/Xunit (as configured)
Gather and present test success report
To me this is a process has a name, I'd called it development.
BDD however is a different process, it's all about collaboration and communication with the business in order to devise a specification. In the beginning there were no tools, but the process still worked.
A number of my co-workers have been using BDD techniques on a variety of real-world projects and have found the techniques very successful. The JBehave story runner – the part that verifies acceptance criteria – is under active development.
Dan North - Introducing BDD 2006
Don't get caught up on the tools alone or you'll miss the vital part of the process. You'll get so much benefit by working with your BA to define the new specification together collaboratively.
P.S. Another way to consider this is that the specification and code should always be in step. Just by defining a new example, we don't magically move the code forwards to meet that example. Instead the most common practice is to develop the code to meet the new example, and then check in the new specification and code as a single change set.
You can use the Pickles project to produce stakeholder-friendly documentation (including HTML) from the Gherkin specifications in your source control.
https://github.com/picklesdoc/pickles
There's no facility for running the tests from the HTML. It's open-source so perhaps you can extend it this way... however, I personally don't see the value in having non-technical users actually execute the specifications. I would have your continuous integration server run the SpecFlow tests and generate a step definition report periodically. The non-technical users can then browse to these reports to see current project status.
To give access to your feature files to non technical people you can use http://www.speclog.net/
Spec log will allow non tech to edit and create new features and will automatically synchronise them with TFS.
Unfortunately it's not free and you can't run the specs from that tool.
I have a Lab Management environment (2010), with deployed agents and controller, TFS, etc. I have Coded UI tests running successfully on my virtual environments, reporting results to TFS.
What I want to achieve is, to be able to communicate with test agents from my code, essentially acting on behalf of the controller, to run tests. Alternatively, it could be ok to instruct controller itself to schedule a run, but the emphasis is that I want to do it from code.
Do you have any idea how to do that?
Thanks.
You can run your tests from the command line using 3 different executables as described here
http://msdn.microsoft.com/en-us/library/ms182486.aspx
The project I'm working on has a bunch of service-tier unit tests using Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute. I want to look into some web frontend automated generation tool for these tests. I don't care if I need to use some other framework like NUnit. I need some decent way to have an easy web frontend for looking at test results, that also allows adding new tests in an easy manner.
After a bit of investigation I realised that we already have TeamCity for the builds. Do I need anything else to setup test browsing from teamcity?
We use Cruise Control, NAnt, SubVersion and NUnit together to provide continuous integration. Every commit triggers a build and runs all the unit tests. The cruise control dashboard show build results, unit test results and code coverage for each build. Is that the kind of thing you are looking to do or do you want some kind of web based ad hoc test runner?
Continuous Integration systems normally let you do this and usually have a web front end.
I know that you could set this up using CruiseControl.Net (which is free), the other system that has been recommended to me in the past is TeamCity so I'm sure that could do this too (and its also free as long as you don't configure too many projects).