We Keep Coding

Unit test Timeout in C#

I have a Function doSomething() which should work in a specific Time (15 seconds). If Time is out then i should throw a message that says "Timeout" but i will still waiting for the result.
so i solve that with a task:
Task<A> mytask = Task.Factory.StartNew(() => ClassA.doSomething()) ;
Mytask.wait(15);
If(mytask.isCompleted)
{
result= mytask.Result ;
}
else
{
Debug(‘ Time is out ’) ;
Mytask.wait() ;
result= mytask.Result ;
}
my Question is : how i can unit test the senario of getting Timeout
my Try :
ClassA objectA= new ClassA() ;
Clocktime t.start() ;
Expect.Call(objectA.doSomething()) ;
t.end
Assert.isTrue(T > 15) ;
Mocks.verifyAll() ;

Disclaimer: Even though the scenario in the question does not follow best practices, it is still useful to demonstrate how to use TaskCompletionSource for testing task-related code.
Below is an example of how you can test the second branch - the "timeout" branch in your code using TaskCompletionSource
Do:
Prefer async code over sync - not to block the threads (of course, it depends)
Don't block on async threads
Use parameters and named constants instead of "magic values"
Keep your unit tests fast and focused on a single scenario
Avoid testing built-in functionality, like Wait timeout
Implement cooperative cancellation https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/task-cancellation
Use proper instrumentation - logging long running task is just an example, in PROD use ETW - https://learn.microsoft.com/en-us/windows/win32/etw/event-tracing-portal
I have modified your code a bit, to align to these recommendations.
I hope it's clear, if not - please let me know how I can help :-)
// This is the class that does the "long running work"
public class ClassA<T>
{
public async Task<T> doSomething(CancellationToken cancellationToken)
{
const int magicNumberOfIterations = 100;
var magicLengthOfTimeInSeconds = TimeSpan.FromSeconds(1);
for (int i = 0; i < magicNumberOfIterations; i++)
{
if (cancellationToken.IsCancellationRequested)
cancellationToken.ThrowIfCancellationRequested();
await Task.Delay(magicLengthOfTimeInSeconds); // time - consuming work
}
return default;
}
}
// This is the class that "detects" long running work based on timeout
public class SlowTaskLogger<T> {
// Don't use this in prod - this actually blocks a thread to write a metric
// Use ETW instead - https://learn.microsoft.com/en-us/windows/win32/etw/event-tracing-portal
public Task<T> RunAndLogSlowRunningTask(TimeSpan logAfterTimeout, Task<T> task)
{
if (task.Wait(logAfterTimeout)) // DO NOT use in PROD
return task; // don't block on async threads
LogWarningForLongRunningTask(task);
return task;
}
private void LogWarningForLongRunningTask(Task<T> task)
{
longRunningTasks.Add(task); // TODO: logging
}
private List<Task<T>> longRunningTasks = new(); // NOTE: for testing, replace with partial mock
public IReadOnlyCollection<Task<T>> LongRunningTasks => longRunningTasks;
}
The test could look something like this
public class InterviewTaskTests
{
private SlowTaskLogger<string> slowTaskLogger = new();
[Fact]
public async Task LogSlowRunningTaskTest()
{
var timeout = TimeSpan.FromMilliseconds(15);
var completionSource = new TaskCompletionSource<string>();
var expectedResult = "foo";
var taskWithLogging = slowTaskLogger.RunAndLogSlowRunningTask(timeout, completionSource.Task);
await Task.Delay(timeout);
// check that timeout method was called
// you can use partial mocks instead of private state for production code
Assert.Contains(taskWithLogging, slowTaskLogger.LongRunningTasks);
// complete the task and ensure proper result is returned
completionSource.SetResult(expectedResult);
var actualResult = await taskWithLogging;
Assert.Equal(expectedResult, actualResult);
}
}

Forcing certain code to always run on the same thread

We have an old 3rd party system (let's call it Junksoft® 95) that we interface with via PowerShell (it exposes a COM object) and I'm in the process of wrapping it in a REST API (ASP.NET Framework 4.8 and WebAPI 2). I use the System.Management.Automation nuget package to create a PowerShell in which I instantiate Junksoft's COM API as a dynamic object that I then use:
//I'm omitting some exception handling and maintenance code for brevity
powerShell = System.Management.Automation.PowerShell.Create();
powerShell.AddScript("Add-Type -Path C:\Path\To\Junksoft\Scripting.dll");
powerShell.AddScript("New-Object Com.Junksoft.Scripting.ScriptingObject");
dynamic junksoftAPI = powerShell.Invoke()[0];
//Now we issue commands to junksoftAPI like this:
junksoftAPI.Login(user,pass);
int age = junksoftAPI.GetAgeByCustomerId(custId);
List<string> names = junksoftAPI.GetNames();
This works fine when I run all of this on the same thread (e.g. in a console application). However, for some reason this usually doesn't work when I put junksoftAPI into a System.Web.Caching.Cache and use it from different controllers in my web app. I say ususally because this actually works when ASP.NET happens to give the incoming call to the thread that junksoftAPI was created on. If it doesn't, Junksoft 95 gives me an error.
Is there any way for me to make sure that all interactions with junksoftAPI happen on the same thread?
Note that I don't want to turn the whole web application into a single-threaded application! The logic in the controllers and elswhere should happen like normal on different threads. It should only be the Junksoft interactions that happen on the Junksoft-specific thread, something like this:
[HttpGet]
public IHttpActionResult GetAge(...)
{
//finding customer ID in database...
...
int custAge = await Task.Run(() => {
//this should happen on the Junksoft-specific thread and not the next available thread
var cache = new System.Web.Caching.Cache();
var junksoftAPI = cache.Get(...); //This has previously been added to cache on the Junksoft-specific thread
return junksoftAPI.GetAgeByCustomerId(custId);
});
//prepare a response using custAge...
}

You can create your own singleton worker thread to achieve this. Here is the code which you can plug it into your web application.
public class JunkSoftRunner
{
private static JunkSoftRunner _instance;
//singleton pattern to restrict all the actions to be executed on a single thread only.
public static JunkSoftRunner Instance => _instance ?? (_instance = new JunkSoftRunner());
private readonly SemaphoreSlim _semaphore;
private readonly AutoResetEvent _newTaskRunSignal;
private TaskCompletionSource<object> _taskCompletionSource;
private Func<object> _func;
private JunkSoftRunner()
{
_semaphore = new SemaphoreSlim(1, 1);
_newTaskRunSignal = new AutoResetEvent(false);
var contextThread = new Thread(ThreadLooper)
{
Priority = ThreadPriority.Highest
};
contextThread.Start();
}
private void ThreadLooper()
{
while (true)
{
//wait till the next task signal is received.
_newTaskRunSignal.WaitOne();
//next task execution signal is received.
try
{
//try execute the task and get the result
var result = _func.Invoke();
//task executed successfully, set the result
_taskCompletionSource.SetResult(result);
}
catch (Exception ex)
{
//task execution threw an exception, set the exception and continue with the looper
_taskCompletionSource.SetException(ex);
}
}
}
public async Task<TResult> Run<TResult>(Func<TResult> func, CancellationToken cancellationToken = default(CancellationToken))
{
//allows only one thread to run at a time.
await _semaphore.WaitAsync(cancellationToken);
//thread has acquired the semaphore and entered
try
{
//create new task completion source to wait for func to get executed on the context thread
_taskCompletionSource = new TaskCompletionSource<object>();
//set the function to be executed by the context thread
_func = () => func();
//signal the waiting context thread that it is time to execute the task
_newTaskRunSignal.Set();
//wait and return the result till the task execution is finished on the context/looper thread.
return (TResult)await _taskCompletionSource.Task;
}
finally
{
//release the semaphore to allow other threads to acquire it.
_semaphore.Release();
}
}
}
Console Main Method for testing:
public class Program
{
//testing the junk soft runner
public static void Main()
{
//get the singleton instance
var softRunner = JunkSoftRunner.Instance;
//simulate web request on different threads
for (var i = 0; i < 10; i++)
{
var taskIndex = i;
//launch a web request on a new thread.
Task.Run(async () =>
{
Console.WriteLine($"Task{taskIndex} (ThreadID:'{Thread.CurrentThread.ManagedThreadId})' Launched");
return await softRunner.Run(() =>
{
Console.WriteLine($"->Task{taskIndex} Completed On '{Thread.CurrentThread.ManagedThreadId}' thread.");
return taskIndex;
});
});
}
}
}
Output:
Notice that, though the function was launched from the different threads, some portion of code got always executed always on the same context thread with ID: '5'.
But beware that, though all the web requests are executed on independent threads, they will eventually wait for some tasks to get executed on the singleton worker thread. This will eventually create a bottle neck in your web application. This is anyway your design limitation.

Here is how you could issue commands to the Junksoft API from a dedicated STA thread, using a BlockingCollection class:
public class JunksoftSTA : IDisposable
{
private readonly BlockingCollection<Action<Lazy<dynamic>>> _pump;
private readonly Thread _thread;
public JunksoftSTA()
{
_pump = new BlockingCollection<Action<Lazy<dynamic>>>();
_thread = new Thread(() =>
{
var lazyApi = new Lazy<dynamic>(() =>
{
var powerShell = System.Management.Automation.PowerShell.Create();
powerShell.AddScript("Add-Type -Path C:\Path\To\Junksoft.dll");
powerShell.AddScript("New-Object Com.Junksoft.ScriptingObject");
dynamic junksoftAPI = powerShell.Invoke()[0];
return junksoftAPI;
});
foreach (var action in _pump.GetConsumingEnumerable())
{
action(lazyApi);
}
});
_thread.SetApartmentState(ApartmentState.STA);
_thread.IsBackground = true;
_thread.Start();
}
public Task<T> CallAsync<T>(Func<dynamic, T> function)
{
var tcs = new TaskCompletionSource<T>(
TaskCreationOptions.RunContinuationsAsynchronously);
_pump.Add(lazyApi =>
{
try
{
var result = function(lazyApi.Value);
tcs.SetResult(result);
}
catch (Exception ex)
{
tcs.SetException(ex);
}
});
return tcs.Task;
}
public Task CallAsync(Action<dynamic> action)
{
return CallAsync<object>(api => { action(api); return null; });
}
public void Dispose() => _pump.CompleteAdding();
public void Join() => _thread.Join();
}
The purpose of using the Lazy class is for surfacing a possible exception during the construction of the dynamic object, by propagating it to the callers.
...exceptions are cached. That is, if the factory method throws an exception the first time a thread tries to access the Value property of the Lazy<T> object, the same exception is thrown on every subsequent attempt.
Usage example:
// A static field stored somewhere
public static readonly JunksoftSTA JunksoftStatic = new JunksoftSTA();
await JunksoftStatic.CallAsync(api => { api.Login("x", "y"); });
int age = await JunksoftStatic.CallAsync(api => api.GetAgeByCustomerId(custId));
In case you find that a single STA thread is not enough to serve all the requests in a timely manner, you could add more STA threads, all of them running the same code (private readonly Thread[] _threads; etc). The BlockingCollection class is thread-safe and can be consumed concurrently by any number of threads.

If you did not say that was a 3rd party tool, I would have asumed it is a GUI class. For practical reasons, it is a very bad idea to have multiple threads write to them. .NET enforces a strict "only the creating thread shall write" rule, from 2.0 onward.
WebServers in general and ASP.Net in particular use a pretty big thread pool. We are talking 10's to 100's of Threads per Core. That means it is really hard to nail any request down to a specific Thread. You might as well not try.
Again, looking at the GUI classes might be your best bet. You could basically make a single thread with the sole purpose of immitating a GUI's Event Queue. The Main/UI Thread of your average Windows Forms application, is responsible for creating every GUI class instance. It is kept alive by polling/processing the event queue. It ends onlyx when it receies a cancel command, via teh Event Queue. Dispatching just puts orders into that Queue, so we can avoid Cross-Threading issues.

xUnit not always catching async exception

I created a xUnit test that calls a method twice to ensure the 2nd call throws an exception saying the class instance is busy. Pretty straightforward.
[Fact]
public async Task RunTwice() {
var P = Create();
Task T1 = Task.Run(() => P.Run("test", null));
Task T2 = Task.Run(() => P.Run("test", null));
await Assert.ThrowsAsync<InvalidOperationException>(() => Task.WhenAll(new Task[] { T1, T2 }));
}
Problem is, this test will randomly either succeed or fail!
The Run method starts like this
public IProcess WorkProcess;
private readonly object lockToken = new object();
public virtual CompletionStatus Run(string fileName, string arguments) {
IProcess P;
lock (lockToken) {
if (WorkProcess != null)
throw new InvalidOperationException();
P = factory.Create();
WorkProcess = P;
}
...
Is this a buy in xUnit or am I doing something wrong? I know async support was added in xUnit 1.9. I'm using v2.4.1.
When I look at debug information, when the test fails, the exception is still thrown.

Looks like you're doing something wrong. Task.Run just schedules the work to the thread pool. Perhaps they'll try to call Run at the same time, perhaps one will happen after the other. There's a race condition inherent in what you're trying to do.
To solve this, you'll need some way for the unit test to control the code within Run and "pause" it.

Verify that task is being awaited

I have the following code which i'd like to test:
private Task _keepAliveTask; // get's assigned by object initializer
public async Task EndSession()
{
_cancellationTokenSource.Cancel(); // cancels the _keepAliveTask
await _logOutCommand.LogOutIfPossible();
await _keepAliveTask;
}
It is important that the EndSession Task only ends once the `_keepAliveTask' ended. However, I'm struggling to find a way to test it reliably.
Question: How do i unit test the EndSession method and verify that the Task returned by EndSession awaits the _keepAliveTask.
For demonstration purposes, the unit test could look like that:
public async Task EndSession_MustWaitForKeepAliveTaskToEnd()
{
var keepAliveTask = new Mock<Task>();
// for simplicity sake i slightly differ from the other examples
// by passing the task as method parameter
await EndSession(keepAliveTask);
keepAliveTask.VerifyAwaited(); // this is what i want to achieve
}
Further criterias:
- reliable test (always passes when implementation is correct, always fails when implementation is wrong)
- cannot take longer than a few milliseconds (it's a unit test, after all).
I have already taken several alternatives into considerations which i'm documenting below:
non-async method
If there wouldn't be the call to _logOutCommand.LogOutIfPossible() it would be quite simple: i'd just remove the async and return _keepAliveTask instead of awaiting it:
public Task EndSession()
{
_cancellationTokenSource.Cancel();
return _keepAliveTask;
}
The unit test would look (simplified):
public void EndSession_MustWaitForKeepAliveTaskToEnd()
{
var keepAliveTask = new Mock<Task>();
// for simplicity sake i slightly differ from the other examples
// by passing the task as method parameter
Task returnedTask = EndSession(keepAliveTask);
returnedTask.Should().be(keepAliveTask);
}
However, there's two arguments against this:
i have multiple task which need awaiting (i'm considering Task.WhenAll further down)
doing so only moves the responsibility to await the task to the caller of EndSession. Still will have to test it there.
non-async method, sync over async
Of course, I could do something similar:
public Task EndSession()
{
_cancellationTokenSource.Cancel(); // cancels the _keepAliveTask
_logOutCommand.LogOutIfPossible().Wait();
return _keepAliveTask;
}
But that is a no-go (sync over async). Plus it still has the problems of the previous approach.
non-async method using Task.WhenAll(...)
Is a (valid) performance improvement but introduces more complexity:
- difficult to get right without hiding a second exception (when both fail)
- allows parallel execution
Since performance isn't key here i'd like to avoid the extra complexity. Also, previously mentioned issue that it just moves the (verification) problem to the caller of the EndSession method applies here, too.
observing effects instead of verifying calls
Now of course instead of "unit" testing method calls etc. I could always observe effects. Which is: As long as _keepAliveTask hasn't ended the EndSession Task mustn't end either. But since I can't wait indefinite one has to settle for a timeout. The tests should be fast so a timeout like 5 seconds is a no go. So what I've done is:
[Test]
public void EndSession_MustWaitForKeepAliveTaskToEnd()
{
var keepAlive = new TaskCompletionSource<bool>();
_cancelableLoopingTaskFactory
.Setup(x => x.Start(It.IsAny<ICancelableLoopStep>(), It.IsAny<CancellationToken>()))
.Returns(keepAlive.Task);
_testee.StartSendingKeepAlive();
_testee.EndSession()
.Wait(TimeSpan.FromMilliseconds(20))
.Should().BeFalse();
}
But I really really dislike this approach:
hard to understand
unreliable
or - when it's quite reliable - it takes a long time (which unit tests shouldn't).

If all you want is to verify that EndSession is awaiting _keepAliveTask (and you really have full control over _keepAliveTask) then you can create your own awaitable type instead of Task the signals when it's awaited and check that:
public class MyAwaitable
{
public bool IsAwaited;
public MyAwaiter GetAwaiter()
{
return new MyAwaiter(this);
}
}
public class MyAwaiter
{
private readonly MyAwaitable _awaitable;
public MyAwaiter(MyAwaitable awaitable)
{
_awaitable = awaitable;
}
public bool IsCompleted
{
get { return false; }
}
public void GetResult() {}
public void OnCompleted(Action continuation)
{
_awaitable.IsAwaited = true;
}
}
Since all you need to await something is that has a GetAwaiter method that returns something with IsCompleted, OnCompleted and GetResult you can use the dummy awaitable to make sure _keepAliveTask is being awaited:
_keepAliveTask = new MyAwaitable();
EndSession();
_keepAliveTask.IsAwaited.Should().BeTrue();
If you use some mocking framework you can instead make Task's GetAwaiter return our MyAwaiter.

Use TaskCompletionSource and set its result at a known time.
Verify that before setting the result, the await on EndSession hasn't completed.
Verify that after setting the result, the await on EndSession has completed.
A simplified version could look like the following (using nunit):
[Test]
public async Task VerifyTask()
{
var tcs = new TaskCompletionSource<bool>();
var keepAliveTask = tcs.Task;
// verify pre-condition
Assert.IsFalse(keepAliveTask.IsCompleted);
var waitTask = Task.Run(async () => await keepAliveTask);
tcs.SetResult(true);
await waitTask;
// verify keepAliveTask has finished, and as such has been awaited
Assert.IsTrue(keepAliveTask.IsCompleted);
Assert.IsTrue(waitTask.IsCompleted); // not needed, but to make a point
}
You can also add a short delay at the waitTask to ensure any synchronous execution would be faster, something like:
var waitTask = Task.Run(async () =>
{
await Task.Delay(1);
await keepAliveTask;
});
And if you don't trust your unit test framework to deal correctly with async, you can set a completed flag as part of the waitTask, and check for that in the end. Something like:
bool completed = false;
var waitTask = Task.Run(async () =>
{
await Task.Delay(1);
await keepAliveTask;
completed = true;
});
// { .... }
// at the end of the method
Assert.IsTrue(completed);

How to speed up task<t> with httpclient

I have a processes where I need to make ~100 http api calls to a server and process the results. I've put together this commandexecutor which builds a list of commands and then runs them async. To make about 100 calls and parse the result is taking over 1 minute. 1 request using a browser give me a response in ~100ms. You would think that ~100 calls would be around 10 seconds. I believe that I am doing something wrong and that this should go much faster.
public static class CommandExecutor
{
private static readonly ThreadLocal<List<Command>> CommandsToExecute =
new ThreadLocal<List<Command>>(() => new List<Command>());
private static readonly ThreadLocal<List<Task<List<Candidate>>>> Tasks =
new ThreadLocal<List<Task<List<Candidate>>>>(() => new List<Task<List<Candidate>>>());
public static void ExecuteLater(Command command)
{
CommandsToExecute.Value.Add(command);
}
public static void StartExecuting()
{
foreach (var command in CommandsToExecute.Value)
{
Tasks.Value.Add(Task.Factory.StartNew<List<Candidate>>(command.GetResult));
}
Task.WaitAll(Tasks.Value.ToArray());
}
public static List<Candidate> Result()
{
return Tasks.Value.Where(x => x.Result != null)
.SelectMany(x => x.Result)
.ToList();
}
}
The Command that I am passing into this list creates a new httpclient, calls the getasync on that client with a url, converts the string response to an object then hydrates a field.
protected void Initialize()
{
_httpClient = new HttpClient();
_httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("text/plain"));
}
protected override void Execute()
{
Initialize();
var task = _httpClient.GetAsync(string.Format(Url, Input));
Result = ConvertResponseToObjectAsync(task).Result;
Result.ForEach(x => x.prop = value);
}
private static Task<Model> ConvertResponseToObjectAsync(Task<HttpResponseMessage> task)
{
return task.Result.Content.ReadAsAsync<Model>(
new MediaTypeFormatter[]
{
new Formatter()
});
}
Can you pick up on my bottleneck or have any suggestions on how to speed this up.
EDIT
making these changes made it down to 4 seconds.
protected override void Execute()
{
Initialize();
_httpClient.GetAsync(string.Format(Url, Input))
.ContinueWith(httpResponse => ConvertResponseToObjectAsync(httpResponse)
.ContinueWith(ProcessResult));
}
protected void ProcessResult(Task<Model> model)
{
Result = model.Result;
Result.ForEach(x => x.prop = value);
}

Stop creating new HttpClient instances. Everytime you dispose a HttpClient instance it closes the TCP/IP connection. Create one HttpClient instance and re-use it for every request. HttpClient can make multiple requests on multiple different threads at the same time.

Avoid the use of task.Result in ConvertResponseToObjectAsync and then again in Execute. Instead chain these on to the original GetAsync task with ContinueWith.
As it stands today, Result will block execution of the current thread until the other task finishes. However, your threadpool will quickly get backed up by tasks waiting on other tasks that have nowhere to run. Eventually (after waiting for a second), the threadpool will add an additional thread to run and so this will eventually finish, but it's hardly efficient.
As a general principle, you should avoid ever accessing Task.Result except in a task continuation.
As a bonus, you probably don't want to be using ThreadLocalStorage. ThreadLocalStorage stores an instance of the item stored in it on each thread where it is accessed. In this case, it looks like you want a thread-safe but shared form of storage. I would recommend ConcurrentQueue for this sort of thing.