We Keep Coding

Async Change Streams in MongoDB

I want to listen to a change stream for a collection, without blocking the thread.
In the code below the ForEachAsync will block forever, processing changes as they occur.
using (var cursor = await collection.WatchAsync())
{
await cursor.ForEachAsync(change =>
{
// process change event
});
}
I have tried removing "await" and storing the task in a variable instead, but no changes will be processed before i actually either await or .Wait() the task later.
I would expect that even though I don't await the task, the events would still be processed.
What can I do, so my entire program doesn't block to listen for changes?
I am considering wrapping the functionality in a new thread, but is that really necessary?

I don't have enough reputation to comment (yet) so my feedback in the form of an answer:
You should be working with events (listeners, handlers) instead of await/async in this situation.
It's difficult to elaborate on this without having more information on what you're trying to accomplish.

So, I figured out that the following works...
static void Main(string[] args)
{
Task watchTask = WatchCollection();
// Do other stuff here, while watching...
}
private static async Task WatchCollection()
{
using (var cursor = collection.Watch())
{
await cursor.ForEachAsync(change =>
{
// process change event
});
}
}
While I was attempting to do it like this, which doesn't work:
static void Main(string[] args)
{
using (var cursor = collection.Watch())
{
Task watchTask cursor.ForEachAsync(change =>
{
// process change event
});
}
// Do other stuff here, while watching
}
So, the actual ForEachAsync task is awaited, while the outer Task which wraps the entire functionality is not awaited... I am either too tired or not familiar enough with Tasks to give a good explanation of why one works and the other doesn't.

Interrupt method execution on demand

I do have a method for which the execution time might take a while (approx. 2 minutes) running inside a task:
public Task BaseWorkerTask { get; set; }
public void Initialize()
{
BaseWorkerTask = Task.Run(() =>
{
BaseWorker();
});
}
public void BaseWorker()
{
Server.Speak(Packet.Greeting);
while (WaitServerResponse())
{
DoSomethingElse();
}
var response = Server.GetResponse();
if (response.Equals("abc"))
{
DoAbcTask();
}
else if (response.Equals("def"))
{
[...]
}
[...]
}
I would like to be able to interrupt it's execution for a given period of time in order to do some other server calls. What is important to note here, is that I want it to "pause" script execution at current line in the BaseWorker method, just where it's exactly at.
Something like:
if (!BaseWorkerTask.IsCompleted)
{
// Pause to do something else
// BaseWorkerTask.StopAtCurrentPlace();
DoSomethingElse();
// Once something else is done, resume BaseWorkerTask
// BaseWorkerTask.Resume();
}
Can such approach be achieved?
I have been trying to play with blocking a thread, however I was only able to block it once the method has finished using endless loop with System.Threading.Thread.Sleep(...).

Thread.Suspend is probably the closest you'll get, but it's marked as obsolete with the following (relevant) warning:
Do not use the Suspend and Resume methods to synchronize the activities of threads. You have no way of knowing what code a thread is executing when you suspend it. If you suspend a thread while it holds locks during a security permission evaluation, other threads in the AppDomain might be blocked. If you suspend a thread while it is executing a class constructor, other threads in the AppDomain that attempt to use that class are blocked. Deadlocks can occur very easily.
In short, anything that can do this would be a bad idea for exactly the same reasons as above.
Attempting to use it on thread-pool threads virtually assures you of malfunction.
If you want to do this, you'll have to think of a way that the task can co-operate in its own pausing. Likely, this will involve peppering the code with synchronization stuff.
What's taking so long anyway?

I would use some kind of a bool flag, similar to a cancellation token. Set it to true when you want to suspend execution of the task, and inside of the BaseWorker check the value of the variable at various places. If it's true, use Thread.Sleep to halt execution until it's set to false.
Example:
bool suspendExecution = false;
public void Initialize()
{
BaseWorkerTask = Task.Run(() =>
{
BaseWorker(ref suspendExecution);
});
}
public void BaseWorker(ref bool suspend)
{
while (WaitServerResponse())
{
if (suspend)
{
while (suspend)
{
Thread.Sleep(1000);
}
}
DoSomethingElse();
}
}

Alternative in a situation of recurring Task demand

I have observer module which takes care of subscriptions of some reactive stream I have created from Kafka. Sadly I need to Poll in order to receive messages from kafka, so I need to dedicate one background thread for that. My first solution was this one:
public void Poll()
{
if (Interlocked.Exchange(ref _state, POLLING) == NOTPOLLING)
{
Task.Run(() =>
{
while (CurrentSubscriptions.Count != 0)
{
_consumer.Poll(TimeSpan.FromSeconds(1));
}
_state = NOTPOLLING;
});
}
}
Now my reviewer suggested that I should Task because it have statuses and can be checked if they are running or not. This led to this code:
public void Poll()
{
// checks for statuses: WaitingForActivation, WaitingToRun, Running
if (_runningStatuses.Contains(_pollingTask.Status)) return;
_pollingTask.Start(); // this obviously throws exception once Task already completes and then I want to start it again
}
Task remained pretty much the same but check changed, now since my logic is that I want to start polling when I have subscriptions and stop when I don't I need to sort of re-use the Task, but since I can't I am wondering do I need to go back to my first implementation or is there any other neat way of doing this that right now I am missing?

I am wondering do I need to go back to my first implementation or is there any other neat way of doing this that right now I am missing?
Your first implementation looks fine. You might use a ManualResetEventSlim instead of enum and Interlocked.Exchange, but that's essentially the same... as long as you have just two states.

I think I made a compromise and removed Interlocked API for MethodImpl(MethodImpl.Options.Synchronized) it lets me have simple method body without possibly confusing Interlocked API code for eventual newcomer/inexperienced guy.
[MethodImpl(MethodImplOptions.Synchronized)]
public void Poll()
{
if (!_polling)
{
_polling = true;
new Task(() =>
{
while (_currentSubscriptions.Count != 0)
{
_consumer.Poll(TimeSpan.FromSeconds(1));
}
_polling = false;
}, TaskCreationOptions.LongRunning).Start();
}
}

Does Parallel.ForEach Block?

Does the .net function Parallel.ForEach block the calling thread? My guess as to the behavior is one of these:
Yes, it blocks until the slowest item executing returns.
No, it doesn't block and returns control immediately. The items to run in parallel are done on background threads.
Or perhaps something else is happening, anyone know for sure?
This question came up when implementing this in a logging class:
public class MultipleLoggingService : LoggingServiceBase
{
private readonly List<LoggingServiceBase> loggingServices;
public MultipleLoggingService(List<LoggingServiceBase> loggingServices)
{
this.loggingServices = loggingServices;
LogLevelChanged += OnLogLevelChanged;
}
private void OnLogLevelChanged(object sender, LogLevelChangedArgs args)
{
loggingServices.ForEach(l => l.LogLevel = LogLevel);
}
public override LogMessageResponse LogMessage(LogMessageRequest request)
{
if (request.LogMessage)
Parallel.ForEach(loggingServices, l => l.LogMessage(request));
return new LogMessageResponse{MessageLogged = request.LogMessage};
}
}
Notice the LogMessage method calls some other logging services. I need that part to return immediately, so it doesn't block the calling thread.
Update: Based on comments from others (we have confirmed the behavior is #1). So I have taken advice to use the Task library and rewritten the loop like this:
if (request.LogMessage)
foreach (var loggingService in loggingServices)
Task.Factory.StartNew(() => loggingService.LogMessage(request));

Number 1 is correct; Parallel.ForEach does not return until the loop has completed. If you don't want that behavior, you can simply execute your loop as a Task and run it on another thread.

Re your update, StartNew in a normal foreach() :
This may not be the most optimal for large collections, and you don't get a point to handle errors.
Your loggingServices probably doesn't hold thousands of items but the errorhandling remains a point .
Consider:
Task.Factory.StartNew(() =>
{
try
{
Parallel.ForEach(loggingServices, l => l.LogMessage(request));
}
catch(SomeException ex)
{
// at least try to log it ...
}
});

Simplest way to do a fire and forget method in C#?

I saw in WCF they have the [OperationContract(IsOneWay = true)] attribute. But WCF seems kind of slow and heavy just to do create a nonblocking function. Ideally there would be something like static void nonblocking MethodFoo(){}, but I don't think that exists.
What is the quickest way to create a nonblocking method call in C#?
E.g.
class Foo
{
static void Main()
{
FireAway(); //No callback, just go away
Console.WriteLine("Happens immediately");
}
static void FireAway()
{
System.Threading.Thread.Sleep(5000);
Console.WriteLine("5 seconds later");
}
}
NB: Everyone reading this should think about if they actually want the method to finish. (See #2 top answer) If the method has to finish, then in some places, like an ASP.NET application, you will need to do something to block and keep the thread alive. Otherwise, this could lead to "fire-forget-but-never-actually-execute", in which case,of course, it would be simpler to write no code at all. (A good description of how this works in ASP.NET)

ThreadPool.QueueUserWorkItem(o => FireAway());
(five years later...)
Task.Run(() => FireAway());
as pointed out by luisperezphd.

For C# 4.0 and newer, it strikes me that the best answer is now given here by Ade Miller: Simplest way to do a fire and forget method in c# 4.0
Task.Factory.StartNew(() => FireAway());
Or even...
Task.Factory.StartNew(FireAway);
Or...
new Task(FireAway).Start();
Where FireAway is
public static void FireAway()
{
// Blah...
}
So by virtue of class and method name terseness this beats the
threadpool version by between six and nineteen characters depending on
the one you choose :)
ThreadPool.QueueUserWorkItem(o => FireAway());

For .NET 4.5:
Task.Run(() => FireAway());

To add to Will's answer, if this is a console application, just throw in an AutoResetEvent and a WaitHandle to prevent it exiting before the worker thread completes:
Using System;
Using System.Threading;
class Foo
{
static AutoResetEvent autoEvent = new AutoResetEvent(false);
static void Main()
{
ThreadPoolQueueUserWorkItem(new WaitCallback(FireAway), autoEvent);
autoEvent.WaitOne(); // Will wait for thread to complete
}
static void FireAway(object stateInfo)
{
System.Threading.Thread.Sleep(5000);
Console.WriteLine("5 seconds later");
((AutoResetEvent)stateInfo).Set();
}
}

An easy way is to create and start a thread with parameterless lambda:
(new Thread(() => {
FireAway();
MessageBox.Show("FireAway Finished!");
}) {
Name = "Long Running Work Thread (FireAway Call)",
Priority = ThreadPriority.BelowNormal
}).Start();
By using this method over ThreadPool.QueueUserWorkItem you can name your new thread to make it easier for debugging. Also, don't forget to use extensive error handling in your routine because any unhandled exceptions outside of a debugger will abruptly crash your application:

The recommended way of doing this when you are using Asp.Net and .Net 4.5.2 is by using QueueBackgroundWorkItem. Here is a helper class:
public static class BackgroundTaskRunner
{
public static void FireAndForgetTask(Action action)
{
HostingEnvironment.QueueBackgroundWorkItem(cancellationToken => // .Net 4.5.2 required
{
try
{
action();
}
catch (Exception e)
{
// TODO: handle exception
}
});
}
/// <summary>
/// Using async
/// </summary>
public static void FireAndForgetTask(Func<Task> action)
{
HostingEnvironment.QueueBackgroundWorkItem(async cancellationToken => // .Net 4.5.2 required
{
try
{
await action();
}
catch (Exception e)
{
// TODO: handle exception
}
});
}
}
Usage example:
BackgroundTaskRunner.FireAndForgetTask(() =>
{
FireAway();
});
or using async:
BackgroundTaskRunner.FireAndForgetTask(async () =>
{
await FireAway();
});
This works great on Azure Web Sites.
Reference: Using QueueBackgroundWorkItem to Schedule Background Jobs from an ASP.NET Application in .NET 4.5.2

Calling beginInvoke and not catching EndInvoke is not a good approach. Answer is simple:
The reason that you should call EndInvoke is because the results of the invocation (even if there is no return value) must be cached by .NET until EndInvoke is called. For example if the invoked code throws an exception then the exception is cached in the invocation data. Until you call EndInvoke it remains in memory. After you call EndInvoke the memory can be released. For this particular case it is possible the memory will remain until the process shuts down because the data is maintained internally by the invocation code. I guess the GC might eventually collect it but I don't know how the GC would know that you have abandoned the data vs. just taking a really long time to retrieve it. I doubt it does. Hence a memory leak can occur.
More can be found on http://haacked.com/archive/2009/01/09/asynchronous-fire-and-forget-with-lambdas.aspx

Almost 10 years later:
Task.Run(FireAway);
I would add exception handling and logging inside FireAway

The simplest .NET 2.0 and later approach is using the Asynchnonous Programming Model (ie. BeginInvoke on a delegate):
static void Main(string[] args)
{
new MethodInvoker(FireAway).BeginInvoke(null, null);
Console.WriteLine("Main: " + Thread.CurrentThread.ManagedThreadId);
Thread.Sleep(5000);
}
private static void FireAway()
{
Thread.Sleep(2000);
Console.WriteLine("FireAway: " + Thread.CurrentThread.ManagedThreadId );
}

The simplest way to do fire-and-forget is to use the discard pattern:
_ = MyFireAndForgetTask(myParameters);
This notifies your method that the result of your Task will not be needed and execution of the thread is not stalled.
Please note that the Task must call Task.Run within it to be asynchronous using this pattern. Using our previous method as an example:
Task MyFireAndForgetTask(myParameters)
{
return Task.Run(/* put Task, Func<T>, or Action here*/);
}
If this step is ignored, the Task will run synchronously and will not behave as expected.
Furthermore the assignment pattern can be used. This is useful for when the method runs until the last line but hangs until the Task is finished. We will utilize Task.Wait() for this. Using our previous method as an example:
void MyCustomEventHandler(object sender, EventArgs args)
{
/* perform some code here */
var myTask = MyFireAndForgetTask(myParameters);
/* perform some more code here; thread is not blocked */
/// Hang the method until the Task is completed.
/// using "await myTask;" is equivalent.
myTask.Wait();
}
This will perform a fire-and-forget-till-completion, which is mandatory on some platforms (i.e. ASP.NET).

If you want to test in Console keep in mind that Console.ReadKey() or something like that is needed before Console loses its thread by Press any key to continue ...
public static void Main()
{
Task.Factory.StartNew(async () =>
{
await LongTaskAsync();
}, TaskCreationOptions.LongRunning).ConfigureAwait(false);
Console.WriteLine("Starts immediately");
Console.ReadKey();
}
static async Task LongTaskAsync()
{
await Task.Delay(5000);
Console.WriteLine("After 5 seconds delay");
}