I have a suite of SpecFlow tests that are using the MSTest framework. I run them via Microsoft Test manager. What I want to know is if I can get it to attach a file to the run.
My code generates a HTML file that i'd like attached so that users can go into the results for test in MTM and simply open it up.
I saw a previous question had:
TestContext.AddResultFile(testPassedFile);
But when I tried to add TestContext to my "[AfterScenario]" method it doesnt have a method called AddResultFile.
Does anyone know how I might be able to achieve adding this HTML file to the results.
AFAIK there is no way of accessing the TestContext from within the StepBindings:
Access TestContext in SpecFlow Step Binding class
The only way I see is to write your own generator for the tests so that the generated test-code writes the TestContext for example to SpecFlow's ScenarioContext, so that you can access it from the step bindings.
If you want to take all that hassle, you might take a look at https://jessehouwing.net/specflow-custom-unit-test-generator/.
In SpecFlow 3, you can get the TestContext via content injection, and store a reference to it, which can then be accessed in the step definitions:
using Microsoft.VisualStudio.TestTools.UnitTesting;
using TechTalk.SpecFlow;
[Binding]
public class Context
{
private static TestContext test_context;
[BeforeScenario]
private static void SetContext(ScenarioContext context)
{
test_context = context.ScenarioContainer.Resolve<TestContext>();
}
public static void Attach(string file_name)
{
test_context.AddResultFile(file_name);
}
}
Related
I am using Microsoft's WinAppDriver in conjunction with Visual Studio 2015 unit tests to automate testing of Windows applications. These tests are being run from the command line using VSTest.exe because I can pass a .runsettings file as a parameter to specify certain test settings that I may have to change in the future. I want to be able to reference the .runsettings file directly from my test methods without needing to create a bunch of global variables at the beginning of my code in the Setup method. Although I am using multiple classes, I am more or less doing it like so:
protected static string basicFile;
[ClassInitialize]
public static void Setup(TestContext context)
{
var basicFile = context.Properties["basic"].ToString();
}
[TestMethod]
public void BasicTest(){
OpenFile(basicFile);
}
Where context.Properties[] references the key in my .runsettings file.
The reason I cannot simply do
[TestMethod]
public void BasicTest(TestContext context){
var basicFile = context.Properties["basic"].ToString();
OpenFile(basicFile);
}
is because test methods cannot accept any parameters. So, is there any way I can directly reference the .runsettings file within the test method without using context.Properties?
I've just switched to C# project from Java and providing .runsettings in our test automation project also gave me a hard time. I also faced the problem of passing TestContext parameter through several classes.
So, my solution may be not so accurate but it worked:
[TestClass]
public class Operations : TestBase
{
public void CreateRelationship()
{
// Add test logic here
}
}
Test Base class:
[TestClass]
public abstract class TestBase
{
[AssemblyInitialize]
public static void ContextInitialize(TestContext context)
{
DriverUtils.Initialize(context);
}
}
And DriverUtils in turn:
public static class DriverUtils
{
private static IWebDriver driver;
private static TestContext testContext;
private static string testEnvironment = string.Empty;
public static void Initialize(TestContext context)
{
testContext = context;
testEnvironment = Convert.ToString(testContext.Properties["TestEnvironmentUrl"]);
}
}
The .runsettings file looks exactly like in the examples, but I left "TestEnvironmentUrl" parameter blank. Then I added .runsettings file to TFS artifacts and later in TFS release config added path to the file in 'Run Functional UI Tests' section. Then I was able to override 'TestEnvironmentUrl' with the real Server URL.
Still haven't found the implementation of [AssemblyInitialize], but I suppose it makes possible to pass the test context from child test classes to parent TestBase.
I recently asked a question but but did not get an answer that I could act on. I think this was due to the long code sample included. I have decided to post another question with a much smaller code sample. I want to unit test the below method to make sure that it does work and to make sure that it deletes all .xml files in a specified directory.
private static void DeleteXmlFiles(string XmlFileDirectory)
{
foreach (var file in Directory.GetFiles(XmlFileDirectory, "*.Xml"))
{
File.Delete(file);
}
}
Does anybody have any unit testing code snippet that I can look at which would help me in this case?
The below is all i have in the Test method which basically is not much:
[Test]
public void can_delete_all_files_from_specified_directory()
{
string inputDir = #"C:\TestFiles\";
var sut = new FilesUtility();
var deleteSuccess = sut.
}
In order to unit-test your method, you should test it in isolation. I.e. there should not be any real classes like Directory or File your SUT interacts with. So, you have three options:
Create abstraction which your class will depend on, and mock that abstraction for tests (Moq will do that)
Mock static methods for test. Moq can't help here, but that is possible with TypeMock, Moles, JustMock etc
Do acceptance or integration testing instead of unit testing (Specflow is good for writing acceptance tests)
Last approach is pretty simple - create new folder before each test runs, and delete it after test run
private string path = #"C:\TestFiles\";
[SetUp]
public void Setup()
{
Directory.CreateDirectory(path);
}
[TearDown]
public void Deardown()
{
Directory.Delete(path);
}
[Test]
public void ShouldRemoveAllFilesFromSpecifiedDirectory()
{
// seed directory with sample files
FileUtility.DeleteXmlFiles(path);
Assert.False(Directory.EnumerateFiles(path).Any());
}
There's no way of testing methods like these with Moq or any other free mocking framework. That's because they cannot mock methods other than virtuals or interface implementations (which is roughly the same under the hood anyway).
To fake (not mock) system methods like File.Delete(...) or static methods of any kind, you'll need something like Typemock (commercial) or MS Moles (not very user-friendly).
As a workaround, you could create a test directory along with some files in your test, call DeleteXmlFiles(...) on it, and then check if the directory is empty. But that would be slow and also is not really a unit test but more like an integration test.
one approach to such a test might be:
Create a directory(folder) (in the unit test assembly folder)
Stick some xml files into it (you might copy thwem from another folder in the unit test assembly folder)
call you method that should delete them.
Check to see if they are gone.
a. If they are, report success,
b. If not report failure
Delete the folder created in step 1
I am checking TestContext.CurrentTestOutcome in my TestCleanup method in order to perform an action if the test did not pass (in this case, the tests are using Selenium to exercise a website and I am saving a screenshot if the test does not pass).
private static TestContext _testContext;
private static IWebDriver _driver;
[ClassInitialize]
public static void SetupTests(TestContext testContext)
{
_testContext = testContext;
_driver = new FirefoxDriver();
}
[TestCleanup]
public void TeardownTest()
{
if (_testContext.CurrentTestOutcome != UnitTestOutcome.Passed)
{
var fileName = Path.Combine(
Environment.CurrentDirectory,
string.Format("{0}.{1}.gif", _testContext.FullyQualifiedTestClassName, _testContext.TestName));
((ITakesScreenshot)driver).GetScreenshot().SaveAsFile(fileName, ImageFormat.Gif);
Console.WriteLine("Test outcome was {0}, saved image of page to '{1}'", _testContext.CurrentTestOutcome, fileName);
}
}
This works well when run on a local development PC using ReSharper, but on our build server (which uses TeamCity) the UnitTestOutcome is always Unknown, although TeamCity reports them as passed.
The documentation on MSDN is not very helpful. What can cause this value to be set to Unknown?
According to http://confluence.jetbrains.com/display/TCD8/MSTest+Support TeamCity does not support on-the-fly reporting of individual test results, it parses the tests results file to provide the results to the build step.
That would explain how TeamCity is able to report the tests as passed even though UnitTestOutcome may be unknown at the time an individual test has completed.
The link above mentions "specifics of MSTest tool" as the reason for non-on-the-fly test result reporting so I can only theorize that the same specifics may mean that TestContext is unavailable when running from your build server.
Also, the MSDN documentation for TestContext.CurrentTestOutcome does mention that Full Trust for the immediate caller is required. TeamCity could be executing the tests in a manner that is only partially trusted and therefore causing the test outcome to be Unknown.
A quick way to check if MSTest is your problem would be to switch to NUnit using:
#if NUNIT
using NUnit.Framework;
using TestClass = NUnit.Framework.TestFixtureAttribute;
using TestMethod = NUnit.Framework.TestAttribute;
using TestInitialize = NUnit.Framework.SetUpAttribute;
using TestCleanup = NUnit.Framework.TearDownAttribute;
using IgnoreAttribute = NUnit.Framework.IgnoreAttribute;
#else
using Microsoft.VisualStudio.TestTools.UnitTesting;
using IgnoreAttribute = Microsoft.VisualStudio.TestTools.UnitTesting.IgnoreAttribute;
#endif
source http://www.anotherchris.net/tools/using-team-city-for-staging-and-test-builds-with-asp-net-and-selenium/
You would have to do something similar in your TeardownTest method to use the NUnit TestContext.CurrentContext.Result.Status though.
The fix for this issue is to use a public property for TestContext, rather than using the parameter passed to the [ClassInitialize] method.
i.e.
public TestContext TestContext { get; set; }
The test runner will automatically set the property.
(This is related to another question I posted on SO)
I've recently moved from a Java-JUnit4-Webdriver environment to a C#-NUnit-Webdriver one.
In Eclipse, I could create test-suites quickly by copying test classes into a test-suite class, and after minimal formatting, run them as JUnit tests. The JUnit test-suites used the pattern below:
package com.example.testsuites;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
import com.example.TestBase;
import com.example.Test1;
import com.example.Test2;
import com.example.Test3;
import com.example.Test12;
import com.example.Test303;
#RunWith(Suite.class)
#SuiteClasses({Test1.class,
Test2.class,
Test3.class,
Test12.class,
Test303.class,
})
public class ExampleTestSuite extends TestBase {
}
Is there an equivalent way to conveniently generate NUnit test-suite classes and then execute them in Visual Studio?
I've been reading the NUnit documentation but can find no obvious equivalent. I know that with the NUnit GUI you can select tests to run via check-boxes but this does not create a permanent test-suite. And that suites can be created by adding attributes/categories etc to individual tests but this doesn't appear as flexible as the method above.
You can use NUnit Suite functionality:
namespace NUnit.Tests
{
using System;
using NUnit.Framework;
private class AllTests
{
[Suite]
public static IEnumerable Suite
{
get
{
ArrayList suite = new ArrayList();
suite.Add(typeof(OneTestCase));
suite.Add(typeof(AssemblyTests));
suite.Add(typeof(NoNamespaceTestFixture));
return suite;
}
}
}
}
However, Suites are currently not displayed in the NUnit GUI.
You can also consider using [Category] attribute instead of
Suite functionality.
See also related thread: NUnit not running Suite tests.
I am new to writing plugin for rhino 3d.
I have gone through the documentation and sample code here:
http://wiki.mcneel.com/developer/dotnetplugins
but unable to figure out how to open a .3dm file from plugin.
Can someone help me?
Thanks!!
It depends a little on what you are trying to do and which version of Rhino you are running.
If you are running Rhino 4 and using the Rhino_DotNet SDK, then you need to have your command class derive from MRhinoScriptCommand and call RhinoApp().RunScript(#"-_Open C:\path_to_model.3dm")
If you are running Rhino 5 and using the RhinoCommon SDK (recommended), then you should call RunScript in a fashion that Brian suggested above. You also need to mark your command class with the the Rhino.Commands.Style attribute of ScriptRunner
ex.
using Rhino.Commands;
[CommandStyle(ScriptRunner)]
class MyCommand : Rhino.Commands.Command
{
public override string EnglishName { get { return "MyCommand"; } }
protected override Result RunCommand(RhinoDoc doc, RunMode mode)
{
RhinoApp.RunScript(#"-_Open C:\model.3dm");
}
}
This will open the 3dm file and make it the active document.
On the other hand if you just want to read the 3dm file into memory and inspect the contents of it, I would recommend using the Rhino.FileIO.File3dm class in RhinoCommon. There is a static Read function on that class that you can use.
You can script the Open command from inside a plug-in using:
Rhino.RhinoApp.RunScript() to script the open command. For example:
Rhino.RhinoApp.RunScript(#"-_Open C:\model.3dm");