i'm unit testing in vs2008 and every time i run now it says
Executing the current test run will produce a new test run result, which will exceed
the limit of 25 that is currently specified. If you continue, older test run results and
their associated deployments will be deleted from the hard drive...
what does this mean, and how do i clear the older test run results? why is this important message?
This message basically means that the Unit Test project has saved/recorded 25 (your threshold) results of previously run unit tests.
By proceeding, it'll remove one from those 25 to include the results of your next run.
You can modify the 'alert' here in the Tools->Options dialog:
You can modify this number here:
Related
I am running test cases using WinAppDriver developed using Visual Studio c#. I see few test cases are flaky where first time they fail and second time they pass. I capture video of test execution and if test is pass I delete the video, if fail, I keep the video and attach to test result.
Now I want to keep the video if a test case pass on "rerun".
Is there any way to programmatically know if current test execution is a rerun?
Note: I run the test cases with DevOps pipeline and rerun is set to 3 attempts
I am trying to run my Unity Unit test with the following command:
"C:\Program Files\Unity\Hub\Editor\2019.4.3f1\Editor\Unity.exe" -runTests -quit -batchmode -projectPath "X:\MyProject" -logFile ./log.txt -testResults ./results.xml
However no reports are generated or message is given to the CMD console.
Here is what I have in the log.txt:
https://pastebin.com/aaZBkmUT
Why can't I run the tests? What am I doing wrong?
P.S.
What does that mean ERROR Failed to connect to local IPC ? May this be the cause of my issue somehow?
Try it eihter without -quit
They don't have it in the example. The thing is that afaik Tests run async so you might just be shutting down Unity before the results are available
Alternatively I think you could add the -runSynchronously
If included, the test run will run tests synchronously, guaranteeing that all tests runs in one editor update call. Note that this is only supported for EditMode tests, and that tests which take multiple frames (i.e. [UnityTest] tests, or tests with [UnitySetUp] or [UnityTearDown] scaffolding) will be filtered out.
to make sure your editor stays alive until the tests are done synchronously
How do you execute millions of unit test quickly, meaning 20 to 30 minutes?
Here is the scenario:
You are releasing certain hardware and you have, let's say, 2000 unit tests.
You are releasing new hardware and you have additional 1000 tests for that.
Each new hardware will include tests, but also you have to run and execute every previous one, and the number gets bigger as does execution time.
During development, this is solved by categorizing, using the TestCategory attribute and running only what you need to.
The CI, however, must run every single test. As the number increases, executing time is slower and sometimes times out. The .testrunconfig is already set for parallelTestCount execution, but over time this does not solve the issue permanently.
How would you solve this?
It seems like with each update on Visual Studio 2017, execution time changes. We currently have over 6000 tests, of which 15 to 20% are unit tests, and the rest are integration tests.
The bottleneck seemed to be the CI server itself, running on a single machine. 70% to 80% of the tests are asynchronous, and analysis showed that there are no blocking I/O operations. Besides IO, we do not use databases, caching so there is that.
Now, we are in the process of migrating to Jenkins and using its Parallel Test Executor Plugin to parallelize the tests across multiple nodes instead of a single machine. Initial testing showed that timing for executing 6000+ tests varies from 10 to 15 minutes versus the old CI which took 2 hours or stopped sometimes.
I've been working on some automation code that includes about 65 small tests. All 65 tests can be ran successfully individually. However, once we run the tests all together - they seem to fail. Once one tests fails, they all seem to fail after that. Is there a way to prevent this from happening? Is there something happening in the transition of one failing test to the next which will automatically make the next one fail?
Thanks
I have created a number of long running integration tests. They take around 5-10 minutes to complete. I don't want them to run Everytime the developers run tests from VS as it will take a long time. They are purely ad hoc tests
Is there a way to mark a test as ignored but still allow developers to run it locally if they explicitly select and run the test without removing ignore tag each time? Also i want these tests to run during overnight build via TFS and MS test runner
Suggest you look into using Test Categories.
By categorising your long running tests separately, you can choose what to include on test runs, e.g.
[TestMethod]
[TestCategory("Integration")]
public void MyLongRunningTest()
{
// Test Code
}
And to run:
mstest /testcontainer:TestProject.dll /category:"Integration"