We Keep Coding

c# - How to execute code on the main thread until a Task finishes

I'd like to understand the TPL a little bit better. How would I for example write something to the screen until a task finishes. I can't call 'await' because I want the main thread to be notified instead of proactively calling 'await' and I don't want to stop execution until the task is finished.
Some example code:
var task = Task.Run(()=>
{
Task.Delay(10000);
});
while(true)
{
Console.WriteLine("Running...");
//I want to exit the loop the second 'task' finishes
//and print 'finished'
}

There are a a couple of ways you cam achieve this:
First you can use Lambda expression in Order to invoke your action, but look at the code:
using System;
using System.Threading;
using System.Threading.Tasks;
public class Example
{
public static void Main()
{
Console.WriteLine("Application thread ID: {0}",
Thread.CurrentThread.ManagedThreadId);
var t = Task.Run(() => { Console.WriteLine("Task thread ID: {0}",
Thread.CurrentThread.ManagedThreadId);
} );
t.Wait();
}
}
// The example displays the following output:
// Application thread ID: 1
//
notice the t.Wait():
The call to the Wait method ensures that the task completes and
displays its output before the application ends. Otherwise, it is
possible that the Main method will complete before the task finishes.
So we understand that it's imperative to call the Wait() method in order make sure that the task completes and displays its output.
You can use the second way also:
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
public class Example
{
public static void Main()
{
var list = new ConcurrentBag<string>();
string[] dirNames = { ".", ".." };
List<Task> tasks = new List<Task>();
foreach (var dirName in dirNames) {
Task t = Task.Run( () => { foreach(var path in Directory.GetFiles(dirName))
list.Add(path); } );
tasks.Add(t);
}
Task.WaitAll(tasks.ToArray());
foreach (Task t in tasks)
Console.WriteLine("Task {0} Status: {1}", t.Id, t.Status);
Console.WriteLine("Number of files read: {0}", list.Count);
}
}
For more reference see Task.Run Method.
One highlight regarding your quesion:
taken from Asynchronous programming with async and await (C#):
An async method typically contains one or more occurrences of an await
operator, but the absence of await expressions doesn’t cause a
compiler error. If an async method doesn’t use an await operator to
mark a suspension point, the method executes as a synchronous method
does, despite the async modifier. The compiler issues a warning for
such methods.
That implies that either way you will have to wait for your tasks to finish and the main thread will have to wait this way or another.

The ContinueWith function is a method available on the task that allows executing code after the task has finished execution. In simple words it allows continuation.
Things to note here is that ContinueWith also returns one Task. That means you can attach ContinueWith one task returned by this method.
Task<string> t = Task.Run(() => LongRunningOperation("Continuewith", 500));
t.ContinueWith((t1) =>
{
Console.WriteLine("Running...");
});

You can achieve your goal using creating separate function to use in Task.Run and passing parameter by reference. Function should be something like below.
private void PerformTask(ref bool isComplete)
{
System.Threading.Thread.Sleep(5000);
isComplete = true;
}
Call above function from Task.Run. Your current function should be like below.
bool isComplete = false;
System.Threading.Tasks.Task.Run(() => PerformTask(ref isComplete));
while (!isComplete)
{
Console.WriteLine("Running...");
System.Threading.Thread.Sleep(1000);
}

Try to do something like this:
var task = Task.Run(() =>
{
Task.Delay(10000).Wait();
});
bool terminate = false;
while (!task.GetAwaiter().IsCompleted && !terminate)
{
// do something
if (task.GetAwaiter().IsCompleted) break;
// do something heavy
if (task.GetAwaiter().IsCompleted) break;
// do another heavy operation
for (int i = 0; i < 10000; i++)
{
// it took too long!
if (i == 1000)
{
terminate = true;
break;
}
}
}

Observable (and cancelable) loop

I'm creating an emulator. The core of the emulation runs in an infinite loop like this:
while (true)
{
UpdateMachineState();
}
I would like to introduce Reactive Extensions to execute this loop into another thread and to make it cancelable, but I'm completely lost.
Since my emulator is a GUI application (Universal Windows), I don't wan't to block the UI thread.
It should look like:
...
while (true)
{
if (machine.IsHalted)
{
observer.OnCompleted;
}
observer.OnNext(machine.GetState());
cancellationToken.ThrowIfCancellationRequested();
}
...
The created sequence would eventually complete when the emulator enters the "halted" state. Otherwise, it will keep pushing States (an object that represents its internal state) forever.
I've tried with Observable.Create, but the overload that provides a CancellationToken requires a Task<Action>.

Here's how you do it in Rx:
void Main()
{
var scheduler = new EventLoopScheduler();
var loop = scheduler.Schedule(a =>
{
UpdateMachineState();
a();
});
Thread.Sleep(1);
loop.Dispose();
}
public void UpdateMachineState()
{
Console.Write(".");
}
The overload on .Schedule that I used takes a Action<Action> as the parameter. You simply call the inner action if you want the the action to be rescheduled - so the above code effectively creates the infinite loop.
You then call .Dispose() on the return from the .Schedule call to cancel the loop.
Another alternative is to use the .Generate operator:
var scheduler = new EventLoopScheduler();
var query =
Observable
.Generate(0, x => true, x => x, x => machine.GetState(), scheduler);
var subscription = query.Subscribe(x => Console.Write("."));
Thread.Sleep(1);
subscription.Dispose();

How to wait for all Timer Tasks to be done?

I've got multiple System.Threading.Timer which starts in parallel. In the end, I have got a Task.Wait to wait till all tasks are done. But it doesn't wait for all, how can I make it wait for all?
private List<Task> todayTasks = new List<Task>();
foreach (var item in todayReport)
{
todayTasks.Add(SetupTimer(item.Exec_Time, item.Report_Id));
}
Task.WaitAll(todayTasks.ToArray());
--SetupTimer--
private Task SetupTimer(DateTime alertTime, int id)
{
DateTime current = DateTime.Now;
TimeSpan timeToGo = alertTime.TimeOfDay - current.TimeOfDay;
if (timeToGo < TimeSpan.Zero) {
//TODO: ERROR time already passed
}
ExecCustomReportService executeCustom = new ExecCustomReportService();
return Task.Run(
() => new Timer(
x => executeCustom.AdhockReport(id), null, timeToGo, Timeout.InfiniteTimeSpan
)
);
}

You're really better off using a tool that is suited to the job. I'd suggest Microsoft's Reactive Framework (Rx). Then you can do this:
var query =
from item in todayReport.ToObservable()
from report in Observable.Start(() => executeCustom.AdhockReport(item.Report_Id))
select report;
IDisposable subscription =
query
.Subscribe(
report =>
{
/* Do something with each report */
},
() =>
{
/* Do something when finished */
});
You just need to NuGet "System.Reactive".

As #YacoubMassad stated in the comment, your task is merely creating the timer and returns.
What you could do is get rid of the timer and use Task.Delay:
return Task.Delay(timeToGo).ContinueWith(t=> executeCustom.AdhockReport(id));

Fire and forget with TPL

I have following code:
public void Func()
{
...
var task = httpClient.PostAsync(...);
var onlyOnRanToCompletionTask = task
.ContinueWith(
t => OnPostAsyncSuccess(t, notification, provider, account.Name),
TaskContinuationOptions.OnlyOnRanToCompletion);
var onlyOnFaultedTask = task
.ContinueWith(
t => OnPostAsyncAggregateException(t, notification.EntityId),
TaskContinuationOptions.OnlyOnFaulted);
return true;
}
Func is not async and I would like to have something like fire and forget but with continuation function. I don't want to have that function async. The scenario is that this function is called in some kind of loop for group of objects to handle them. For me the problem seems that e.g. onlyOnRanToCompletionTask can be deleted when we finish Func execution.
Thank you in advance!

But your code should be performed as expected. Simple example:
void Main()
{
HandlingMyFuncAsync();
Console.WriteLine("Doing some work, while 'fire and forget job is performed");
Console.ReadLine();
}
public void HandlingMyFuncAsync()
{
var task = MyFuncAsync();
task.ContinueWith(t => Console.WriteLine(t), TaskContinuationOptions.OnlyOnRanToCompletion);
}
public async Task<string> MyFuncAsync()
{
await Task.Delay(5000);
return "A";
}
produces
Doing some work, while 'fire and forget job is performed
[end after 5 sec]
A

If I were you I'd look at using Microsoft's Reactive Framework (Rx) for this. Rx is designed for this kind of thing.
Here's some basic code:
void Main()
{
int[] source = new [] { 1, 2, 3 };
IObservable<int> query =
from s in source.ToObservable()
from u in Observable.Start(() => Func(s))
select s;
IDisposable subscription =
query
.Subscribe(
x => Console.WriteLine($"!{x}!"),
() => Console.WriteLine("Done."));
}
public void Func(int x)
{
Console.WriteLine($"+{x} ");
Thread.Sleep(TimeSpan.FromSeconds(1.0));
Console.WriteLine($" {x}-");
}
When I run that I get this kind of output:
+1
+2
+3
1-
!1!
2-
!2!
3-
!3!
Done.
It triggers off all of the calls and waits for the results to come in. It also let's you know when it is done.
Here's a more practical example showing how to use async and Rx to return a result:
void Main()
{
int[] source = new [] { 1, 2, 3 };
var query =
from s in source.ToObservable()
from u in Observable.FromAsync(() => Func(s))
select new { s, u };
IDisposable subscription =
query
.Subscribe(
x => Console.WriteLine($"!{x.s},{x.u}!"),
() => Console.WriteLine("Done."));
}
public async Task<int> Func(int x)
{
Console.WriteLine($"+{x} ");
await Task.Delay(TimeSpan.FromSeconds(1.0));
Console.WriteLine($" {x}-");
return 10 * x;
}
That gives:
+1
+2
+3
1-
!1,10!
3-
2-
!2,20!
!3,30!
Done.
Without seeing your full code I can't easily give you a complete example that you can use, but Rx will also handle opening an closing on your DB contexts using the Observable.Using operator.
You can get Rx by Nugetting "System.Reactive".

Unwrapping IObservable<Task<T>> into IObservable<T> with order preservation

Is there a way to unwrap the IObservable<Task<T>> into IObservable<T> keeping the same order of events, like this?
Tasks: ----a-------b--c----------d------e---f---->
Values: -------A-----------B--C------D-----E---F-->
Let's say I have a desktop application that consumes a stream of messages, some of which require heavy post-processing:
IObservable<Message> streamOfMessages = ...;
IObservable<Task<Result>> streamOfTasks = streamOfMessages
.Select(async msg => await PostprocessAsync(msg));
IObservable<Result> streamOfResults = ???; // unwrap streamOfTasks
I imagine two ways of dealing with that.
First, I can subscribe to streamOfTasks using the asynchronous event handler:
streamOfTasks.Subscribe(async task =>
{
var result = await task;
Display(result);
});
Second, I can convert streamOfTasks using Observable.Create, like this:
var streamOfResults =
from task in streamOfTasks
from value in Observable.Create<T>(async (obs, cancel) =>
{
var v = await task;
obs.OnNext(v);
// TODO: don't know when to call obs.OnComplete()
})
select value;
streamOfResults.Subscribe(result => Display(result));
Either way, the order of messages is not preserved: some later messages that
don't need any post-processing come out faster than earlier messages that
require post-processing. Both my solutions handle the incoming messages
in parallel, but I'd like them to be processed sequentially, one by one.
I can write a simple task queue to process just one task at a time,
but perhaps it's an overkill. Seems to me that I'm missing something obvious.
UPD. I wrote a sample console program to demonstrate my approaches. All solutions by far don't preserve the original order of events. Here is the output of the program:
Timer: 0
Timer: 1
Async handler: 1
Observable.Create: 1
Observable.FromAsync: 1
Timer: 2
Async handler: 2
Observable.Create: 2
Observable.FromAsync: 2
Observable.Create: 0
Async handler: 0
Observable.FromAsync: 0
Here is the complete source code:
// "C:\Program Files (x86)\MSBuild\14.0\Bin\csc.exe" test.cs /r:System.Reactive.Core.dll /r:System.Reactive.Linq.dll /r:System.Reactive.Interfaces.dll
using System;
using System.Reactive;
using System.Reactive.Concurrency;
using System.Reactive.Linq;
using System.Threading.Tasks;
class Program
{
static void Main()
{
Console.WriteLine("Press ENTER to exit.");
// the source stream
var timerEvents = Observable.Timer(TimeSpan.Zero, TimeSpan.FromSeconds(1));
timerEvents.Subscribe(x => Console.WriteLine($"Timer: {x}"));
// solution #1: using async event handler
timerEvents.Subscribe(async x =>
{
var result = await PostprocessAsync(x);
Console.WriteLine($"Async handler: {x}");
});
// solution #2: using Observable.Create
var processedEventsV2 =
from task in timerEvents.Select(async x => await PostprocessAsync(x))
from value in Observable.Create<long>(async (obs, cancel) =>
{
var v = await task;
obs.OnNext(v);
})
select value;
processedEventsV2.Subscribe(x => Console.WriteLine($"Observable.Create: {x}"));
// solution #3: using FromAsync, as answered by #Enigmativity
var processedEventsV3 =
from msg in timerEvents
from result in Observable.FromAsync(() => PostprocessAsync(msg))
select result;
processedEventsV3.Subscribe(x => Console.WriteLine($"Observable.FromAsync: {x}"));
Console.ReadLine();
}
static async Task<long> PostprocessAsync(long x)
{
// some messages require long post-processing
if (x % 3 == 0)
{
await Task.Delay(TimeSpan.FromSeconds(2.5));
}
// and some don't
return x;
}
}

Combining #Enigmativity's simple approach with #VMAtm's idea of attaching the counter and some code snippets from this SO question, I came up with this solution:
// usage
var processedStream = timerEvents.SelectAsync(async t => await PostprocessAsync(t));
processedStream.Subscribe(x => Console.WriteLine($"Processed: {x}"));
// my sample console program prints the events ordered properly:
Timer: 0
Timer: 1
Timer: 2
Processed: 0
Processed: 1
Processed: 2
Timer: 3
Timer: 4
Timer: 5
Processed: 3
Processed: 4
Processed: 5
....
Here is my SelectAsync extension method to transform IObservable<Task<TSource>> into IObservable<TResult> keeping the original order of events:
public static IObservable<TResult> SelectAsync<TSource, TResult>(
this IObservable<TSource> src,
Func<TSource, Task<TResult>> selectorAsync)
{
// using local variable for counter is easier than src.Scan(...)
var counter = 0;
var streamOfTasks =
from source in src
from result in Observable.FromAsync(async () => new
{
Index = Interlocked.Increment(ref counter) - 1,
Result = await selectorAsync(source)
})
select result;
// buffer the results coming out of order
return Observable.Create<TResult>(observer =>
{
var index = 0;
var buffer = new Dictionary<int, TResult>();
return streamOfTasks.Subscribe(item =>
{
buffer.Add(item.Index, item.Result);
TResult result;
while (buffer.TryGetValue(index, out result))
{
buffer.Remove(index);
observer.OnNext(result);
index++;
}
});
});
}
I'm not particularly satisfied with my solution as it looks too complex to me, but at least it doesn't require any external dependencies. I'm using here a simple Dictionary to buffer and reorder task results because the subscriber need not to be thread-safe (the subscriptions are neved called concurrently).
Any comments or suggestions are welcome. I'm still hoping to find the native RX way of doing this without custom buffering extension method.

The RX library contains three operators that can unwrap an observable sequence of tasks, the Concat, Merge and Switch. All three accept a single source argument of type IObservable<Task<T>>, and return an IObservable<T>. Here are their descriptions from the documentation:
Concat
Concatenates all task results, as long as the previous task terminated successfully.
Merge
Merges results from all source tasks into a single observable sequence.
Switch
Transforms an observable sequence of tasks into an observable sequence producing values only from the most recent observable sequence. Each time a new task is received, the previous task's result is ignored.
In other words the Concat returns the results in their original order, the Merge returns the results in order of completion, and the Switch filters out any results from tasks that didn't complete before the next task was emitted. So your problem can be solved by just using the built-in Concat operator. No custom operator is needed.
var streamOfResults = streamOfTasks
.Select(async task =>
{
var result1 = await task;
var result2 = await PostprocessAsync(result1);
return result2;
})
.Concat();
The tasks are already started before they are emitted by the streamOfTasks. In other words they are emerging in a "hot" state. So the fact that the Concat operator awaits them the one after the other has no consequence regarding the concurrency of the operations. It only affects the order of their results. This would be a consideration if instead of hot tasks you had cold observables, like these created by the Observable.FromAsync and Observable.Create methods, in which case the Concat would execute the operations sequentially.

Is the following simple approach an answer for you?
IObservable<Result> streamOfResults =
from msg in streamOfMessages
from result in Observable.FromAsync(() => PostprocessAsync(msg))
select result;

To maintain the order of events you can funnel your stream into a TransformBlock from TPL Dataflow. The TransformBlock would execute your post-processing logic and will maintain the order of its output by default.
using System;
using System.Collections.Generic;
using System.Reactive.Linq;
using System.Threading.Tasks;
using System.Threading.Tasks.Dataflow;
using NUnit.Framework;
namespace HandlingStreamInOrder {
[TestFixture]
public class ItemHandlerTests {
[Test]
public async Task Items_Are_Output_In_The_Same_Order_As_They_Are_Input() {
var itemHandler = new ItemHandler();
var timerEvents = Observable.Timer(TimeSpan.Zero, TimeSpan.FromMilliseconds(250));
timerEvents.Subscribe(async x => {
var data = (int)x;
Console.WriteLine($"Value Produced: {x}");
var dataAccepted = await itemHandler.SendAsync((int)data);
if (dataAccepted) {
InputItems.Add(data);
}
});
await Task.Delay(5000);
itemHandler.Complete();
await itemHandler.Completion;
CollectionAssert.AreEqual(InputItems, itemHandler.OutputValues);
}
private IList<int> InputItems {
get;
} = new List<int>();
}
public class ItemHandler {
public ItemHandler() {
var options = new ExecutionDataflowBlockOptions() {
BoundedCapacity = DataflowBlockOptions.Unbounded,
MaxDegreeOfParallelism = Environment.ProcessorCount,
EnsureOrdered = true
};
PostProcessBlock = new TransformBlock<int, int>((Func<int, Task<int>>)PostProcess, options);
var output = PostProcessBlock.AsObservable().Subscribe(x => {
Console.WriteLine($"Value Output: {x}");
OutputValues.Add(x);
});
}
public async Task<bool> SendAsync(int data) {
return await PostProcessBlock.SendAsync(data);
}
public void Complete() {
PostProcessBlock.Complete();
}
public Task Completion {
get { return PostProcessBlock.Completion; }
}
public IList<int> OutputValues {
get;
} = new List<int>();
private IPropagatorBlock<int, int> PostProcessBlock {
get;
}
private async Task<int> PostProcess(int data) {
if (data % 3 == 0) {
await Task.Delay(TimeSpan.FromSeconds(2));
}
return data;
}
}
}

Rx and TPL can be easily combined here, and TPL do save the order of events, by default, so your code could be something like this:
using System.Threading.Tasks;
using System.Threading.Tasks.Dataflow;
static async Task<long> PostprocessAsync(long x) { ... }
IObservable<Message> streamOfMessages = ...;
var streamOfTasks = new TransformBlock<long, long>(async msg =>
await PostprocessAsync(msg)
// set the concurrency level for messages to handle
, new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = Environment.ProcessorCount });
// easily convert block into observable
IObservable<long> streamOfResults = streamOfTasks.AsObservable();
Edit: Rx extensions meant to be a reactive pipeline of events for UI. As this type of applications are in general single-threaded, so messages are being handled with saving the order. But in general events in C# aren't thread safe, so you have to provide some additional logic to same the order.
If you don't like the idea to introduce another dependency, you need to store the operation number with Interlocked class, something like this:
// counter for operations get started
int operationNumber = 0;
// counter for operations get done
int doneNumber = 0;
...
var currentOperationNumber = Interlocked.Increment(ref operationNumber);
...
while (Interlocked.CompareExchange(ref doneNumber, currentOperationNumber + 1, currentOperationNumber) != currentOperationNumber)
{
// spin once here
}
// handle event
Interlocked.Increment(ref doneNumber);