I'd like to programmatically retrieve information related to test results. After a test run, I want to go through the test results and, for each test failed, to get the name of the failed test, the reason why it failed (validation rule message or extraction rule message) and the value of one key contained in the test context.
I couldn't find anything that I can use on how to dig into trx and webtestResult files. Does anyone have any hints for me?
Thanks,
Paul
After searching a bit more I came upon this:
https://blogs.msdn.microsoft.com/slumley/2009/11/11/vsts-2010-feature-api-for-processing-web-test-results/
The RequestResult property returns a WebTestRequestResult object, which exposes 2 properties, ValidationRuleResults and ExtractionRuleResults, through which the needed information can be retrieved.
Related
As mentioned in the blog below, could the attribute, _bpf_accountid_value be used when not found in the default solution? Trying something similar with leads, I am trapping the error "the given key not present in dictionary".
I am on the lead form and for testing purposes I traverse the BPF and without manually finishing it,
I am trying to make my custom workflow "Finish" it, on demand.
I hesitate to show my code because it is commented with various attempts and will only loose the focus of this specific question.
Please help.
https://community.dynamics.com/365/b/crmmemories/posts/finish-a-business-process-flow-in-c
I just verified in my instance as the BPF will have its own entity, I can find those attributes. You can check too using CRM REST Builder or the native Advanced find - download fetchxml.
There is a difference between OOB like Lead entity BPF and custom entity BPF like below: (_bpf_xxx_projectinspectionsid_value vs _leadid_value)
https://crmdev.crm.dynamics.com/api/data/v9.1/xxx_inspectionflows(00000000-0000-0000-0000-000000000000)?$select=_bpf_xxx_projectinspectionsid_value,bpf_name,businessprocessflowinstanceid
https://crmdev.crm.dynamics.com/api/data/v9.1/leadtoopportunitysalesprocesses(00000000-0000-0000-0000-000000000000)?$select=businessprocessflowinstanceid,_leadid_value
My requirement is to test individually the dynamic list of URL's using specflow for success code 200
Current Approach: URL list is hard coded in the scenario outline like below wherein I am able to test each URL successfully. Here my test runs as many number of URL's present in the scenario outline and I can easily identify the failed tests if any.
Scenario Outline: URL test
Given list of URL's
When I launch each URL
Then I should expect 200 HTTP status code
Examples:
| URL |
| url 1 |
| url 2 |
| url..n|
New Approach: I am thinking to get the list of URL from the web service dynamically instead of hard coding in the scenario outline.
Scenario Outline: URL test
Given a service to get list of URL's
When I call the API get
Then I should expect 200 HTTP status code
With the above approach am able to get the list of URL's and by iterating am able to launch them individually. The problem here is since there is no scenario outline, In a single test, complete list of URL's will be executed.
I need a way wherein I can create a dynamic data set and each URL testing will happen in its own test method, rather executing list of URL's in a single test method.
Tools Used:
C# , NUnit , Specflow , Resharper, Visual Studio
Am using ReSharper to execute my tests and I looked about dynamic Table / Instance concepts .
Since I cannot (yet) add comments I will supply my comment as an answer.
You probably shouldn't use a scenario in this situation. Scenarios are meant to describe the behavior of the system from a business perspective (in other words business value).
Here it seems that you want to make sure a list of uri's is responding fine. This sounds more as an internal integration test than an actual business verification.
I suggest putting this in a single integration test. There you can simply create a collection of failing uri's. At the end of the test you can simply assert whether the collection contains 0 elements.
You can't dynamically generate table data during tests execution because .cs files are already compiled.
Write parameterized unit tests instead.
Or of course, you can write a tool which will parse your urls, rewrite .feature file, rebuild SpecFlow solution and run tests. JFF :)
I am trying to run a SpecFlow scenario from code instead of through Test Explorer or the command line. Has someone managed to do this?
From a scenario, I can extract the method name and test method with recursion, but I cannot run this scenario method. It seems to need a proper initialize and teardown, but I could't manage to do this.
My first thought was to use the TechTalk.SpecFlow.TestRunner class, but it doesn't seem to have a scenario selection method.
EDIT on why I want to do this:
We want to run specific scenarios from TFS. It is very cumbersome to connect TestMethods to WorkItems in TFS, because:
You can only assign one testmethod to one workitem
For each workitem you have to search the method name, with in itself is a hassle, because the list is very long with lots of specflow scenarios.
When your specflow scenario gets a different name (which happens a lot), TFS cannot find the correct method anymore
Specflow Scenario Outlines get practically unusable, while they are a very powerful feature.
I want to create a mechanism where each automated workitem gets the same method assigned. This method extracts the workitem id and search and executes the scenario(s) with this workitem tagged.
I had a similar problem since my tests have some dependencies between Scenarios (shame on me, but it saves tons of copy-paste lines per Feature file). In most cases I would stick to isolated Scenarios of course.
I used reflection
Find all Types with a DescriptionAttribute (aka Features)
Find their MethodInfos with a TestAttribute and DescriptionAttribute (aka Scenarios)
Store them to a Dictionary
Call them by "Title of the Feature/Title of the Scenario" with Activator.CreateInstance and Invoke
You have to set the (private) field "testRunner" according to your needs of course.
Many classes in the .NET framework (especially in the socket/network classes, which is what I'm looking at) use System.Net.GlobalLog (an internal class) to log trace messages somewhere. You can view example uses of things like GlobalLog.Assert and GlobalLog.Print in the SslState class:
SslStream source code
This is different from the System.Net.Logging (also internal) class, uses of which can also be found throughout the socket/network classes.
For System.Net.Logging, I know I can use a <system.diagnostics> configuration block in App.Config and that will result in System.Net.Logging messages getting logged if configured properly. However, this does not appear to influence System.Net.GlobalLog.
After searching around for about an hour, I cannot seem to find any information about locating the output of System.Net.GlobalLog. Does anyone know how to locate/view/control the output of this?
As you stated, GlobalLog is an internal class to the System.Net assembly. Without being able to modify the System.Net assembly you won't get access to that class.
That said, you might want to review the following: http://www.123aspx.com/rotor/RotorSrc.aspx?rot=42941
It looks like you have to have compiler flags TRAVE and DEBUG set in order to get it to work.. but I'm not seeing where it actually does anything with the logged information. The comments suggest that it is supposed to look for an environment variable setting and dump the log to a text file somewhere on the system; however the code at that page seems either incomplete or it simply wasn't finished.
My guess is that you need to find some other way of getting access to the logging info you want.
I don't have enough rep to put this as a comment on the existing answer but I wanted to figure out what was happening in the TcpClient and Socket classes but also found they couldn't be stepped into. The closest I got was by monitoring the Windows API calls that were being made. I used a freeware tool called API Monitor found here: http://www.rohitab.com/apimonitor.
I have a functionality that upload a file, validate it, save it to database and so on.
All of these actions are on some different classes and methods. But, if something goes wrong in any of these methods, I want to inform these errors to my front-end.
Is there a good way to manage this, let's call, "Error Class"?
Thanks!
One approach that you could take would be to create the "Error Class" like a service in your application and post the different classes and methods that perform various actions could post errors to it.
Now your front-end can either subscribe with the "Error Class" to know of errors immediately or you could return an error code the details of which can be found using the "Error Class"
Well lets say, your sequence of processing a file is
Upload
Validate
Save
Return with success result/error.
Now, in all these sequence if its same all the time, then you could just go for exception throwing mechanism and collect it at the front end. May be you could throw with an Error ID.