I'm using FitNesse to do some testing with Fixtures written in C#. One of my fixtures kicks off some testing suites to run in Ab Initio in a Unix environment. I would like to be able to kill the whole test suite if one of the tests within it fail. I imagine I would need some kind of return value from the test suite (on the unix box) and then that would get passed back up to my fixture which would kill FitNesse (from within my C# fixture). This is what my KickOff() method looks like right now:
public string KickOff()
{
var data = new Dictionary<string, string>();
foreach (var row in System.IO.File.ReadAllLines(unixConfigFile))
data.Add(row.Split('=')[0], String.Join("=", row.Split('=').Skip(1).ToArray()));
string server = data["servername"];
string userId = data["username"];
string password = data["password"];
StringEncryptor3DES encryptor = new StringEncryptor3DES("fitnesse");
password = encryptor.Decrypt(password);
UnixScriptRunner runner = new UnixScriptRunner(server, userId, password, unixPath, scriptName,EtlType.AbInitio);
return runner.Run() & EtlStatus.IsEtlSuccessful(runner.LogFile,EtlType.AbInitio) ? "True":"False";
}
I think I need something that catches the value of EtlStatus.IsEtlSuccessful(), if it is false it will terminte FitNesse. The two questions I have are this:
Is this reasoning correct?
What is the code needed to terminite/kill FitNesse (or end the test suite if there is a more graceful way)?
Edit: I did a little more research and it looks like it is the 'runner' that I have to kill. Still not sure how to do this though...
If you're using Slim, you can throw an exception with "StopTest" in the class name to stop the test.
http://www.fitnesse.org/FitNesse.UserGuide.WritingAcceptanceTests.SliM.ExceptionHandling
If you're using Fit, you can throw an AbandonStoryTestExecption to stop the test.
http://fitsharp.github.io/Fit/AbandonStoryTest.html
There's a new Fit feature coming in the next release (any day now!) to throw AbandonTestSuiteException to stop the entire test suite. There's no comparable feature with Slim AFAIK.
Related
I am writing a UI test. This is to check for the error 404 page which I have enabled in Web.Config via;
<customErrors mode="On" redirect="~/Errors/"/>
This all works fine, however I only have custom errors set to "On" whilst in the "UAT" development environment. If I am in "Dev" or "IST" then I still want to see the default ASP.Net errors.
So now back to the UI test using Selenium
public string GetAlertBoxDetails()
{
IWebElement alertBox = _driver.FindElement(By.CssSelector(".alert.alert-danger"));
return alertBox.Text;
}
As you can see I am detecting the Bootstrap ".alert.alert-danger" box and returning the text inside. I then check to see if this text contains "Sorry, that page doesn't exist.". I am using Specflow for the text story.
[Then(#"The user should be told that no such page exists")]
public void ThenTheUserShouldBeToldThatNoSuchPageExists()
{
string alertboxDetail = GetAlertBoxDetails();
Assert.IsTrue(alertboxDetail.Contains("Sorry, that page doesn't exist."), "Couldn't find the message \"Sorry, that page doesn't exist.\"");
}
This all works fine, however I would only like this test to run in the UAT environment. This is because the element ".alert.alert-danger" will only be found if customErrors is set to "Off". For this I have included this step in the test.
[Given(#"I am in the UAT environment")]
public void GivenIAmInTheUATEnvironment()
{
var env = EnvironmentType;
if (env != EnvironmentType.Uat)
{
Assert.Inconclusive($"Cannot run this test on environment: {env}. " +
$"This test is only for the UAT environment.");
}
else
{
Assert.IsTrue(true);
}
}
Again this works fine. My only problem is that I do not want to use "Assert.Inconclusive" I would rather "Assert.Pass" and say the test passed if it is carried out in a non-UAT environment.
I see XUnit has an Assert.Pass function, but can this be done in MsTest? To force a test to pass WITHOUT continuing to the next Assert. In specflow I am running the "given" step I would like to stop it from continuing to the "Then" step.
WRT NUnit, you can try Assert.Pass. I can't try it out myself right now as I'm traveling. My uncertainty is that I'm not sure if it will prevent the test from being run if you do it in the SetUp, which is what Given maps to.
My view is that accepting the behavior you looking for, all the code belongs in the test itself and not the Given. What Given would normally do would be to actually create the situation you expect, i.e. change the environment. That's obviously not possible here so I would simply put the environment check in the test itself. I wouldn't even use Assert.Pass unless you want a special message, I'd just skip the test code if the environment is wrong. As a side benefit, this approach works for all three test frameworks.
Although you didn't ask, I have to say that the instructions you have been given to show the test as passing even if it is not run seem pretty crazy to me!
With Team Foundation Server, given a WorkItem of type "Test Suite," how can I write a query to select all Test Cases associated to that Test Suite?
Unfortunately, there are no work item links created between Test Plans, Suites and Cases. So although they are Work Items, they don't have links. This means that a default query isn't possible.
A work around is tagging all test cases in a suite with the name of the suite. You can then use a query that filters on the work item tags.
You can go even further and automate the creation of tags by using some Web Hooks and Azure Functions (or some other hosted API) magic. This allows you to create a Web Hook that listens for the creation (or updates) to Test Cases. By using some of the code mentioned in the other posts you can retrieve the Test Suite of the Test Case and then use the REST API to add it as a Tag to the Test Case.
You may need to use this Interface ITestSuiteBase.
AllTestCases
Gets the read-only collection of test cases for this suite and all hierarchical children.
TestCases
Gets a read-only collection of test cases.
More info from MSDN
Here is a example code:
public static List<TestCase> GetAllTestCaseFromSuite(ITestPlan testPlan, int suiteId, bool includeExecutionStatus = true)
{
List<TestCase> testCases = new List<TestCase>();
testPlan.Refresh();
ITestSuiteBase currentSuite = testPlan.Project.TestSuites.Find(suiteId);
currentSuite.Refresh();
foreach (var currentTestCase in currentSuite.TestCases)
{
TestCase testCaseToAdd = new TestCase(currentTestCase.TestCase, currentSuite, testPlan, includeExecutionStatus);
if (!testCases.Contains(testCaseToAdd))
{
testCases.Add(testCaseToAdd);
}
}
log.InfoFormat("Load all test cases in the suite with Title= \"{0}\" id = \"{1}\"", currentSuite.Title, currentSuite.Id);
return testCases;
}
More details you can refer this blog: Manage TFS Test Cases C# Code
If you are using TFS 2015 or higher,
you can check this link :
usage
TestCaseExplorer Tool
list-bugs-and-the-test-cases-that-test-them
if Not using TFS 2015 or higher :
For now, there is no way to create ordinary TFS query via Web interface and not API call or custom coding to get list of Test Cases belongs to a specific Test Suite. support-querying-for-all-test-cases-in-a-specifed
Or try old tool : test-plans-test-suites-test-cases-mapping
"Show tests from child suites" is the option that you want. To see the screenshot click here. No need for the query.
As the name of the option says it lists all the child tests from the suite. You might need Test Manager TFS plugin for this.
So I use RemoteTesttRunner to run a couple of tests with selenium and they work perfectly the first time I run them. But if I try to run them a second time the web browser don't show up and he fail the test directly.
I run the tests like this:
var location = AppDomain.CurrentDomain.BaseDirectory;
var runner = new RemoteTestRunner();
runner.Load(new TestPackage(Path.Combine(location,"/my_test.dll")));
var result = runner.Run(new NullListener(), TestFilter.Empty, false, LoggingThreshold.All);
return result;
And the test I run look like this:
[Test]
public void AutomationTest()
{
ChromeDriver driver = new ChromeDriver()
driver.Navigate().GoToUrl("www.google.se")
}
So is there something I need to clean after every test or why does the web browser not show up after my first test run? As it run just fine the first time I can't find any wrong with the code.
Should add that I don't get any exception or anything like that, it just don't show up.
I am new to JINT, and trying to just do some basic tests to kind of learn the ropes. My first attempt was to just store some javascript in my database, load it, and execute it in a unit test. So that looks essentially like this....
[Fact]
public void can_use_jint_engine() {
using (var database = DocumentStore()) {
using (var session = database.OpenSession()) {
var source = session.Load<Statistic>("statistics/1");
// join the list of strings into a single script
var script = String.Join("\n", source.Scripting);
// this will create the script
// console.log("this is a test from jint.");
//
var engine = new Jint.Engine();
// attempt to execute the script
engine.Execute(script);
}
}
}
And it doesn't work, I get this error, which makes absolutely no sense to me, and I cannot find any documentation on.
Jint.Runtime.JavaScriptExceptionconsole is not defined at
Jint.Engine.Execute(Program program) at
Jint.Engine.Execute(String source) at
SampleProject.Installers.Instanced.__testing_installer.can_use_jint_engine()
in _testing_installer.cs: line 318
Can anyone assist in shedding some light on this? I'm pretty confused at this point.
With JavaScript there are three entities - we care about. The host (browser, your application etc), the engine (JINT in this case) and the script ("console.log(...)") in this case.
JavaScript defines a bunch of functions and object as part of the language, but console is not one of them. By convention, browsers define a console object that can be used in the manner you describe. However, since your app is not a browser (and JINT does not do this by itself), there's no console object defined in your namespace (globals).
What you need to do is add a console object that will be accessible in JINT. You can find how to do this in the docs, but here's a simple example of how to add a log function to the engine so it can be used from the JS code (example taken from github).
var engine = new Engine()
.SetValue("log", new Action<object>(Console.WriteLine))
;
engine.Execute(#"
function hello() {
log('Hello World');
};
hello();
");
I'm working on implementing a reasonably simple XML serializer/deserializer (log file parser) application in C# .NET with VS 2008. I have about 50 unit tests right now for various parts of the code (mostly for the various serialization operations), and some of them seem to be failing mostly at random when they deal with file I/O.
The way the tests are structured is that in the test setup method, I create a new empty file at a certain predetermined location, and close the stream I get back. Then I run some basic tests on the file (varying by what exactly is under test). In the cleanup method, I delete the file again.
A large portion (usually 30 or more, though the number varies run to run) of my unit tests will fail at the initialize method, claiming they can't access the file I'm trying to create. I can't pin down the exact reason, since a test that will work one run fails the next; they all succeed when run individually.
What's the problem here? Why can't I access this file across multiple unit tests?
Relevant methods for a unit test that will fail some of the time:
[TestInitialize()]
public void LogFileTestInitialize()
{
this.testFolder =
System.Environment.GetFolderPath(
System.Environment.SpecialFolder.LocalApplicationData
);
this.testPath = this.testFolder + "\\empty.lfp";
System.IO.File.Create(this.testPath).Close();
}
[TestMethod()]
public void LogFileConstructorTest()
{
string filePath = this.testPath;
LogFile target = new LogFile(filePath);
Assert.AreNotEqual(null, target);
Assert.AreEqual(this.testPath, target.filePath);
Assert.AreEqual("empty.lfp", target.fileName);
Assert.AreEqual(this.testFolder + "\\empty.lfp.lfpdat", target.metaPath);
}
[TestCleanup()]
public void LogFileTestCleanup()
{
System.IO.File.Delete(this.testPath);
}
And the LogFile() constructor:
public LogFile(String filePath)
{
this.entries = new List<Entry>();
this.filePath = filePath;
this.metaPath = filePath + ".lfpdat";
this.fileName = filePath.Substring(filePath.LastIndexOf("\\") + 1);
}
The precise error message:
Initialization method
LogFileParserTester.LogFileTest.LogFileTestInitialize
threw exception.
System.IO.IOException:
System.IO.IOException: The process
cannot access the file
'C:\Users\<user>\AppData\Local\empty.lfp'
because it is being used by another
process..
You should be mocking the file system access, and not actually reading/writing files in your unit tests.
Sounds like some of the tests are being run at the same time. Do the individual tests write to the file, or just read it? If they're read only, I'm sure we can make a minor change to enable them to run concurrently. More details?