We Keep Coding

Handle exceptions with TPL Dataflow blocks

I have a simple tpl data flow which basically does some tasks.
I noticed when there is an exception in any of the datablocks, it wasn't getting caught in the initial parent block caller.
I have added some manual code to check for exception but doesn't seem the right approach.
if (readBlock.Completion.Exception != null
|| saveBlockJoinedProcess.Completion.Exception != null
|| processBlock1.Completion.Exception != null
|| processBlock2.Completion.Exception != null)
{
throw readBlock.Completion.Exception;
}
I had a look online to see what's a suggested approach but didn't see anything obvious.
So I created some sample code below and was hoping to get some guidance on a better solution:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Threading.Tasks.Dataflow;
namespace TPLDataflow
{
class Program
{
static void Main(string[] args)
{
try
{
//ProcessB();
ProcessA();
}
catch (Exception e)
{
Console.WriteLine("Exception in Process!");
throw new Exception($"exception:{e}");
}
Console.WriteLine("Processing complete!");
Console.ReadLine();
}
private static void ProcessB()
{
Task.WhenAll(Task.Run(() => DoSomething(1, "ProcessB"))).Wait();
}
private static void ProcessA()
{
var random = new Random();
var readBlock = new TransformBlock<int, int>(x =>
{
try { return DoSomething(x, "readBlock"); }
catch (Exception e) { throw e; }
}); //1
var braodcastBlock = new BroadcastBlock<int>(i => i); // ⬅ Here
var processBlock1 = new TransformBlock<int, int>(x =>
DoSomethingAsync(5, "processBlock1")); //2
var processBlock2 = new TransformBlock<int, int>(x =>
DoSomethingAsync(2, "processBlock2")); //3
//var saveBlock =
// new ActionBlock<int>(
// x => Save(x)); //4
var saveBlockJoinedProcess =
new ActionBlock<Tuple<int, int>>(
x => SaveJoined(x.Item1, x.Item2)); //4
var saveBlockJoin = new JoinBlock<int, int>();
readBlock.LinkTo(braodcastBlock, new DataflowLinkOptions
{ PropagateCompletion = true });
braodcastBlock.LinkTo(processBlock1,
new DataflowLinkOptions { PropagateCompletion = true }); //5
braodcastBlock.LinkTo(processBlock2,
new DataflowLinkOptions { PropagateCompletion = true }); //6
processBlock1.LinkTo(
saveBlockJoin.Target1); //7
processBlock2.LinkTo(
saveBlockJoin.Target2); //8
saveBlockJoin.LinkTo(saveBlockJoinedProcess,
new DataflowLinkOptions { PropagateCompletion = true });
readBlock.Post(1); //10
//readBlock.Post(2); //10
Task.WhenAll(processBlock1.Completion,processBlock2.Completion)
.ContinueWith(_ => saveBlockJoin.Complete());
readBlock.Complete(); //12
saveBlockJoinedProcess.Completion.Wait(); //13
if (readBlock.Completion.Exception != null
|| saveBlockJoinedProcess.Completion.Exception != null
|| processBlock1.Completion.Exception != null
|| processBlock2.Completion.Exception != null)
{
throw readBlock.Completion.Exception;
}
}
private static int DoSomething(int i, string method)
{
Console.WriteLine($"Do Something, callng method : { method}");
throw new Exception("Fake Exception!");
return i;
}
private static async Task<int> DoSomethingAsync(int i, string method)
{
Console.WriteLine($"Do SomethingAsync");
throw new Exception("Fake Exception!");
await Task.Delay(new TimeSpan(0, 0, i));
Console.WriteLine($"Do Something : {i}, callng method : { method}");
return i;
}
private static void Save(int x)
{
Console.WriteLine("Save!");
}
private static void SaveJoined(int x, int y)
{
Thread.Sleep(new TimeSpan(0, 0, 10));
Console.WriteLine("Save Joined!");
}
}
}

I had a look online to see what's a suggested approach but didn't see anything obvious.
If you have a pipeline (more or less), then the common approach is to use PropagateCompletion to shut down the pipe. If you have more complex topologies, then you would need to complete blocks by hand.
In your case, you have an attempted propagation here:
Task.WhenAll(
processBlock1.Completion,
processBlock2.Completion)
.ContinueWith(_ => saveBlockJoin.Complete());
But this code will not propagate exceptions. When both processBlock1.Completion and processBlock2.Completion complete, saveBlockJoin is completed successfully.
A better solution would be to use await instead of ContinueWith:
async Task PropagateToSaveBlockJoin()
{
try
{
await Task.WhenAll(processBlock1.Completion, processBlock2.Completion);
saveBlockJoin.Complete();
}
catch (Exception ex)
{
((IDataflowBlock)saveBlockJoin).Fault(ex);
}
}
_ = PropagateToSaveBlockJoin();
Using await encourages you to handle exceptions, which you can do by passing them to Fault to propagate the exception.

Propagating errors backward in the pipeline is not supported in the TPL Dataflow out of the box, which is especially annoying when the blocks have a bounded capacity. In this case an error in a block downstream may cause the blocks in front of it to block indefinitely. The only solution I know is to use the cancellation feature, and cancel all blocks in case anyone fails. Here is how it can be done. First create a CancellationTokenSource:
var cts = new CancellationTokenSource();
Then create the blocks one by one, embedding the same CancellationToken in the options of all of them:
var options = new ExecutionDataflowBlockOptions()
{ BoundedCapacity = 10, CancellationToken = cts.Token };
var block1 = new TransformBlock<double, double>(Math.Sqrt, options);
var block2 = new ActionBlock<double>(Console.WriteLine, options);
Then link the blocks together, including the PropagateCompletion setting:
block1.LinkTo(block2, new DataflowLinkOptions { PropagateCompletion = true });
Finally use an extension method to trigger the cancellation of the CancellationTokenSource in case of an exception:
block1.OnFaultedCancel(cts);
block2.OnFaultedCancel(cts);
The OnFaultedCancel extension method is shown below:
public static class DataflowExtensions
{
public static void OnFaultedCancel(this IDataflowBlock dataflowBlock,
CancellationTokenSource cts)
{
dataflowBlock.Completion.ContinueWith(_ => cts.Cancel(), default,
TaskContinuationOptions.OnlyOnFaulted |
TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Default);
}
}

at the first look, if have only some minor points (not looking at your architecture). it seems to me that you have mixed some newer and some older constructs. and there are some code parts which are unnecessary.
for example:
private static void ProcessB()
{
Task.WhenAll(Task.Run(() => DoSomething(1, "ProcessB"))).Wait();
}
using the Wait()-method, if any exceptions happen, they will be wrapped in a System.AggregateException. in my opinion, this is better:
private static async Task ProcessBAsync()
{
await Task.Run(() => DoSomething(1, "ProcessB"));
}
using async-await, if an exception occurs, the await statement rethrows the first exception which is wrapped in the System.AggregateException. This allows you to try-catch for concrete exception types and handle only cases you really can handle.
another thing is this part of your code:
private static void ProcessA()
{
var random = new Random();
var readBlock = new TransformBlock<int, int>(
x =>
{
try { return DoSomething(x, "readBlock"); }
catch (Exception e)
{
throw e;
}
},
new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 1 }); //1
Why catch an exception only to rethrow it? in this case, the try-catch is redundant.
And this here:
private static void SaveJoined(int x, int y)
{
Thread.Sleep(new TimeSpan(0, 0, 10));
Console.WriteLine("Save Joined!");
}
It is much better to use await Task.Delay(....). Using Task.Delay(...), your application will not freeze.

Shared context between tests

I have following code:
public class Batcher<TPayload> : IBatcher<TPayload>
{
private static readonly BufferBlock<BatchElement<TPayload>> BufferBlock = new BufferBlock<BatchElement<TPayload>>(new DataflowBlockOptions
{
EnsureOrdered = true
});
private readonly TransformBlock<BatchElement<TPayload>, BatchElement<TPayload>> BufferInterceptor;
private readonly TransformBlock<BatchElement<TPayload>, BatchElement<TPayload>> TimeoutInterceptor;
public EventsBatcher(int size, int interval, IMagicService magicService, ILogger<Batcher<TPayload, TStrategy>> logger)
{
BufferInterceptor =
new TransformBlock<BatchElement<TPayload>, BatchElement<TPayload>>(x =>
{
logger.LogInformation($"Get a message with value: {x}");
return x;
});
TimeoutInterceptor =
new TransformBlock<BatchElement<TPayload>, BatchElement<TPayload>>(x =>
{
logger.LogInformation($"Move out from transformation block with a value: {x}");
return x;
});
var batchBlock = new BatchBlock<BatchElement<TPayload>>(size, new GroupingDataflowBlockOptions()
{
EnsureOrdered = true
});
var timer = new Timer(async _ =>
{
try
{
batchBlock.TriggerBatch();
var data = await batchBlock.ReceiveAsync();
if (!data.Any() && data.SomeLogic())
return;
await magicService.PushMessageAsync(batchElement.Payload);
}
catch (Exception e)
{
logger.LogError($"Error occurs while trying to invoke action on batch", e);
}
}, null, 0, 500);
var timeoutBlock = new TransformBlock<BatchElement<TPayload>, BatchElement<TPayload>>(v =>
{
timer.Change(interval, Timeout.Infinite);
return v;
});
TimeoutInterceptor.LinkTo(batchBlock);
timeoutBlock.LinkTo(TimeoutInterceptor);
BufferInterceptor.LinkTo(timeoutBlock);
BufferBlock.LinkTo(BufferInterceptor);
}
public async Task<Result<Unit>> SendAsync(BatchElement<TPayload> msg, CancellationToken token = new CancellationToken())
{
try
{
var result = await BufferBlock.SendAsync(msg, token);
return result
? ResultFactory.CreateSuccess()
: ResultFactory.CreateFailure<Unit>("Message was refused by queue");
}
catch (Exception e)
{
return ResultFactory.CreateFailure<Unit>(e.Message);
}
}
}
Which responsibility is to evaluate somehow data every x milliseconds. I try to write unit tests to that to be sure that everything works fine. Those tests are here:
public class BatcherTests
{
public EventsBatcher<int> Initialize(Dictionary<DateTime, int> output)
{
var busMock = new Mock<IMagicService>();
busMock.Setup(x => x.PushMessageAsync(It.IsAny<int>()))
.Callback<Data>((data) =>
{
output.Add(DateTime.Now, data);
}).Returns(Task.CompletedTask);
var loggerMock = new Mock<ILogger<Batcher<int>>>();
return new Batcher<int>(
2,
5000,
busMock.Object,
loggerMock.Object
);
}
[Fact]
public async Task Batcher_ShouldRemoveDuplicatedMessages()
{
var output = new Dictionary<DateTime, int>();
var batcher = Initialize(output);
var first = await batcher.SendAsync(new MockEvent { Payload = 1 });
var second = await batcher.SendAsync(new MockEvent { Payload = 1 });
(first.IsSuccess && second.IsSuccess).ShouldBeTrue();
while (output.Count != 2)
{
}
output.Count.ShouldBe(2);
output.First().Value.ShouldBe(1);
output.Last().Value.ShouldBe(1);
output.Clear();
}
[Fact]
public async Task Batcher_WhenSizeIsSetTo2AndWeSend3Items_ReturnTwoBatchedItemsWithDateIntervalPlusMinus5000msAndAllSendRequestsEndsWithSuccess()
{
var output = new Dictionary<DateTime, int>();
var batcher = Initialize(output);
var first = await batcher.SendAsync(new MockEvent { Payload = 1 });
var second = await batcher.SendAsync(new MockEvent { Payload = 1 });
var third = await batcher.SendAsync(new MockEvent { Payload = 1 });
(first.IsSuccess && second.IsSuccess && third.IsSuccess).ShouldBeTrue();
while (output.Count != 2) //never ends because there are already two elements in output dictionary
{
}
output.Count.ShouldBe(2);
output.First().Value.ShouldBe(2);
output.Last().Value.ShouldBe(1);
var interval = (output.Last().Key - output.First().Key).TotalSeconds;
(interval >= 4.5d && interval <= 5.5d).ShouldBeTrue();
output.Clear();
}
}
But the strange thing is that when I run them separately they end up with a success status. But when I run them all together one of them seems to stuck. This is because a dictionary which is passed to a logic method has 2 elements inside while starting a test. I don't see here a possibility of shared context since stub class is created at the beginning of test cases, the same with a dictionary. Is there something that I missing? I also try to split those test cases to separe classes but the same behavior occurs.

There is shared stated, but it is not in the test (directly).
Your BufferBlock is declared as static in the class Batcher<TPayload>. There is your shared state.
private static readonly BufferBlock<BatchElement<TPayload>> BufferBlock = new BufferBlock<BatchElement<TPayload>>(new DataflowBlockOptions
{
EnsureOrdered = true
});
When multiple tests are executed that shared block is linked to the other blocks multiple times.

cancel async task if running

I have the following method called on several occasions (e.g onkeyup of textbox) which asynchronously filters items in listbox.
private async void filterCats(string category,bool deselect)
{
List<Category> tempList = new List<Category>();
//Wait for categories
var tokenSource = new CancellationTokenSource();
var token = tokenSource.Token;
//HERE,CANCEL TASK IF ALREADY RUNNING
tempList= await _filterCats(category,token);
//Show results
CAT_lb_Cats.DataSource = tempList;
CAT_lb_Cats.DisplayMember = "strCategory";
CAT_lb_Cats.ValueMember = "idCategory";
}
and the following task
private async Task<List<Category>> _filterCats(string category,CancellationToken token)
{
List<Category> result = await Task.Run(() =>
{
return getCatsByStr(category);
},token);
return result;
}
and I would like to test whether the task is already runing and if so cancel it and start it with the new value. I know how to cancel task, but how can I check whether it is already running?

This is the code that I use to do this :
if (_tokenSource != null)
{
_tokenSource.Cancel();
}
_tokenSource = new CancellationTokenSource();
try
{
await loadPrestatieAsync(_bedrijfid, _projectid, _medewerkerid, _prestatieid, _startDate, _endDate, _tokenSource.Token);
}
catch (OperationCanceledException ex)
{
}
and for the procedure call it is like this (simplified of course) :
private async Task loadPrestatieAsync(int bedrijfId, int projectid, int medewerkerid, int prestatieid,
DateTime? startDate, DateTime? endDate, CancellationToken token)
{
await Task.Delay(100, token).ConfigureAwait(true);
try{
//do stuff
token.ThrowIfCancellationRequested();
}
catch (OperationCanceledException ex)
{
throw;
}
catch (Exception Ex)
{
throw;
}
}
I am doing a delay of 100 ms because the same action is triggered rather quickly and repeatedly, a small postpone of 100 ms makes it look like the GUI is more responsive actually.

It appears you are looking for a way to get an "autocomplete list" from text entered in a text box, where an ongoing async search is canceled when the text has changed since the search was started.
As was mentioned in the comments, Rx (Reactive Extensions), provides very nice patterns for this, allowing you to easily connect your UI elements to cancellable asynchronous tasks, building in retry logic, etc.
The less than 90 line program below, shows a "full UI" sample (unfortunately excluding any cats ;-). It includes some reporting on the search status.
I have created this using a number of static methods in the RxAutoComplete class, to show how to this is achieved in small documented steps, and how they can be combined, to achieve a more complex task.
namespace TryOuts
{
using System;
using System.Linq;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.Reactive.Linq;
using System.Threading;
// Simulated async search service, that can fail.
public class FakeWordSearchService
{
private static Random _rnd = new Random();
private static string[] _allWords = new[] {
"gideon", "gabby", "joan", "jessica", "bob", "bill", "sam", "johann"
};
public async Task<string[]> Search(string searchTerm, CancellationToken cancelToken)
{
await Task.Delay(_rnd.Next(600), cancelToken); // simulate async call.
if ((_rnd.Next() % 5) == 0) // every 5 times, we will cause a search failure
throw new Exception(string.Format("Search for '{0}' failed on purpose", searchTerm));
return _allWords.Where(w => w.StartsWith(searchTerm)).ToArray();
}
}
public static class RxAutoComplete
{
// Returns an observable that pushes the 'txt' TextBox text when it has changed.
static IObservable<string> TextChanged(TextBox txt)
{
return from evt in Observable.FromEventPattern<EventHandler, EventArgs>(
h => txt.TextChanged += h,
h => txt.TextChanged -= h)
select ((TextBox)evt.Sender).Text.Trim();
}
// Throttles the source.
static IObservable<string> ThrottleInput(IObservable<string> source, int minTextLength, TimeSpan throttle)
{
return source
.Where(t => t.Length >= minTextLength) // Wait until we have at least 'minTextLength' characters
.Throttle(throttle) // We don't start when the user is still typing
.DistinctUntilChanged(); // We only fire, if after throttling the text is different from before.
}
// Provides search results and performs asynchronous,
// cancellable search with automatic retries on errors
static IObservable<string[]> PerformSearch(IObservable<string> source, FakeWordSearchService searchService)
{
return from term in source // term from throttled input
from result in Observable.FromAsync(async token => await searchService.Search(term, token))
.Retry(3) // Perform up to 3 tries on failure
.TakeUntil(source) // Cancel pending request if new search request was made.
select result;
}
// Putting it all together.
public static void RunUI()
{
// Our simple search GUI.
var inputTextBox = new TextBox() { Width = 300 };
var searchResultLB = new ListBox { Top = inputTextBox.Height + 10, Width = inputTextBox.Width };
var searchStatus = new Label { Top = searchResultLB.Height + 30, Width = inputTextBox.Width };
var mainForm = new Form { Controls = { inputTextBox, searchResultLB, searchStatus }, Width = inputTextBox.Width + 20 };
// Our UI update handlers.
var syncContext = SynchronizationContext.Current;
Action<Action> onUITread = (x) => syncContext.Post(_ => x(), null);
Action<string> onSearchStarted = t => onUITread(() => searchStatus.Text = (string.Format("searching for '{0}'.", t)));
Action<string[]> onSearchResult = w => {
searchResultLB.Items.Clear();
searchResultLB.Items.AddRange(w);
searchStatus.Text += string.Format(" {0} maches found.", w.Length > 0 ? w.Length.ToString() : "No");
};
// Connecting input to search
var input = ThrottleInput(TextChanged(inputTextBox), 1, TimeSpan.FromSeconds(0.5)).Do(onSearchStarted);
var result = PerformSearch(input, new FakeWordSearchService());
// Running it
using (result.ObserveOn(syncContext).Subscribe(onSearchResult, ex => Console.WriteLine(ex)))
Application.Run(mainForm);
}
}
}

C# infinitive task loop using Task<> class + cancellation

I`m trying to make a small class for the multithreading usage in my WinForm projects.
Tried Threads(problems with UI), Backgroundworker(smth went wrong with UI too, just leave it now:)), now trying to do it with Task class. But now, can`t understand, how to make an infinitive loop and a cancelling method (in class) for all running tasks.
Examples i found is to be used in 1 method.
So, here is a structure & code of currently working part (Worker.css and methonds used in WinForm code).
Worker.css
class Worker
{
public static int threadCount { get; set; }
public void doWork(ParameterizedThreadStart method)
{
Task[] tasks = Enumerable.Range(0, 4).Select(i => Task.Factory.StartNew(() => method(i))).ToArray();
}
}
usage on
Form1.cs
private void Start_btn_Click(object sender, EventArgs e)
{
Worker.threadCount = 1; //actually it doesn`t using now, number of tasks is declared in class temporaly
Worker worker = new Worker();
worker.doWork(Job);
string logString_1 = string.Format("Starting {0} threads...", Worker.threadCount);
log(logString_1);
}
public static int j = 0;
private void Job(object sender)
{
Worker worker = new Worker();
Random r = new Random();
log("Thread "+Thread.CurrentThread.ManagedThreadId +" is working...");
for (int i = 0; i < 5; i++)
{
j++;
log("J==" + j);
if (j == 50)
{
//worker.Stop();
log("STOP");
}
}
Thread.Sleep(r.Next(500, 1000));
}
So, it run an example 4 threads, they executed, i got J==20 in my log, it`s ok.
My question is, how to implement infinitive loop for the tasks, created by Worker.doWork() method.
And also to make a .Stop() method for the Worker class (which should just stop all tasks when called). As i understand it`s related questions, so i put it in 1.
I tryed some solutions, but all of them based on the CancellationToken usage, but i have to create this element only inside of the Worker.doWork() method, so i can`t use the same token to create a Worker.Stop() method.
Someone can help? threads amount range i have to use in this software is about 5-200 threads.
using J computation is just an example of the the easy condition used to stop a software work(stop of tasks/threads).
In real, stop conditions is mostly like Queue<> is finished, or List<> elements is empty(finished).

Finally, get it works.
class Worker
{
public static int threadCount { get; set; }
Task[] tasks;
//ex data
public static string exception;
static CancellationTokenSource wtoken = new CancellationTokenSource();
CancellationToken cancellationToken = wtoken.Token;
public void doWork(ParameterizedThreadStart method)
{
try
{
tasks = Enumerable.Range(0, 4).Select(i => Task.Factory.StartNew(() =>
{
while (!cancellationToken.IsCancellationRequested)
{
method(i);
}
}, cancellationToken)).ToArray();
}
catch (Exception ex) { exception = ex.Message; }
}
public void HardStop()
{
try
{
using (wtoken)
{
wtoken.Cancel();
}
wtoken = null;
tasks = null;
}
catch (Exception ex) { exception = ex.Message; }
}
}
But if i`m using this method to quit cancellationToken.ThrowIfCancellationRequested();
Get a error:
when Job() method reach J == 50, and worker.HardStop() function called, program window crashes and i get and exception "OparetionCanceledException was unhandled by user code"
on this string
cancellationToken.ThrowIfCancellationRequested();
so, whats wrong? i`m already put it in try{} catch(){}
as i understood, just some boolean properties should be changed in Task (Task.IsCancelled == false, Task.IsFaulted == true) on wtoken.Cancel();

I'd avoid all of the mucking around with tasks and use Microsoft's Reactive Framework (NuGet "Rx-Main") for this.
Here's how:
var r = new Random();
var query =
Observable
.Range(0, 4, Scheduler.Default)
.Select(i =>
Observable
.Generate(0, x => true, x => x, x => x,
x => TimeSpan.FromMilliseconds(r.Next(500, 1000)),
Scheduler.Default)
.Select(x => i))
.Merge();
var subscription =
query
.Subscribe(i => method(i));
And when you want to cancel the calls to method just do this:
subscription.Dispose();
I've tested this and it works like a treat.
If I wrap this up in your worker class then it looks like this:
class Worker
{
private Random _r = new Random();
private IDisposable _subscription = null;
public void doWork()
{
_subscription =
Observable
.Range(0, 4, Scheduler.Default)
.Select(n =>
Observable
.Generate(
0, x => true, x => x, x => x,
x => TimeSpan.FromMilliseconds(_r.Next(500, 1000)),
Scheduler.Default)
.Select(x => n))
.Merge()
.Subscribe(i => method(i));
}
public void HardStop()
{
_subscription.Dispose();
}
}

Pause and Resume Subscription on cold IObservable

Using Rx, I desire pause and resume functionality in the following code:
How to implement Pause() and Resume() ?
static IDisposable _subscription;
static void Main(string[] args)
{
Subscribe();
Thread.Sleep(500);
// Second value should not be shown after two seconds:
Pause();
Thread.Sleep(5000);
// Continue and show second value and beyond now:
Resume();
}
static void Subscribe()
{
var list = new List<int> { 1, 2, 3, 4, 5 };
var obs = list.ToObservable();
_subscription = obs.SubscribeOn(Scheduler.NewThread).Subscribe(p =>
{
Console.WriteLine(p.ToString());
Thread.Sleep(2000);
},
err => Console.WriteLine("Error"),
() => Console.WriteLine("Sequence Completed")
);
}
static void Pause()
{
// Pseudocode:
//_subscription.Pause();
}
static void Resume()
{
// Pseudocode:
//_subscription.Resume();
}
Rx Solution?
I believe I could make it work with some kind of Boolean field gating combined with thread locking (Monitor.Wait and Monitor.Pulse)
But is there an Rx operator or some other reactive shorthand to achieve the same aim?

Here's a reasonably simple Rx way to do what you want. I've created an extension method called Pausable that takes a source observable and a second observable of boolean that pauses or resumes the observable.
public static IObservable<T> Pausable<T>(
this IObservable<T> source,
IObservable<bool> pauser)
{
return Observable.Create<T>(o =>
{
var paused = new SerialDisposable();
var subscription = Observable.Publish(source, ps =>
{
var values = new ReplaySubject<T>();
Func<bool, IObservable<T>> switcher = b =>
{
if (b)
{
values.Dispose();
values = new ReplaySubject<T>();
paused.Disposable = ps.Subscribe(values);
return Observable.Empty<T>();
}
else
{
return values.Concat(ps);
}
};
return pauser.StartWith(false).DistinctUntilChanged()
.Select(p => switcher(p))
.Switch();
}).Subscribe(o);
return new CompositeDisposable(subscription, paused);
});
}
It can be used like this:
var xs = Observable.Generate(
0,
x => x < 100,
x => x + 1,
x => x,
x => TimeSpan.FromSeconds(0.1));
var bs = new Subject<bool>();
var pxs = xs.Pausable(bs);
pxs.Subscribe(x => { /* Do stuff */ });
Thread.Sleep(500);
bs.OnNext(true);
Thread.Sleep(5000);
bs.OnNext(false);
Thread.Sleep(500);
bs.OnNext(true);
Thread.Sleep(5000);
bs.OnNext(false);
It should be fairly easy for you to put this in your code with the Pause & Resume methods.

Here it is as an application of IConnectableObservable that I corrected slightly for the newer api (original here):
public static class ObservableHelper {
public static IConnectableObservable<TSource> WhileResumable<TSource>(Func<bool> condition, IObservable<TSource> source) {
var buffer = new Queue<TSource>();
var subscriptionsCount = 0;
var isRunning = System.Reactive.Disposables.Disposable.Create(() => {
lock (buffer)
{
subscriptionsCount--;
}
});
var raw = Observable.Create<TSource>(subscriber => {
lock (buffer)
{
subscriptionsCount++;
if (subscriptionsCount == 1)
{
while (buffer.Count > 0) {
subscriber.OnNext(buffer.Dequeue());
}
Observable.While(() => subscriptionsCount > 0 && condition(), source)
.Subscribe(
v => { if (subscriptionsCount == 0) buffer.Enqueue(v); else subscriber.OnNext(v); },
e => subscriber.OnError(e),
() => { if (subscriptionsCount > 0) subscriber.OnCompleted(); }
);
}
}
return isRunning;
});
return raw.Publish();
}
}

Here is my answer. I believe there may be a race condition around pause resume, however this can be mitigated by serializing all activity onto a scheduler. (favor Serializing over synchronizing).
using System;
using System.Reactive.Concurrency;
using System.Reactive.Disposables;
using System.Reactive.Linq;
using System.Reactive.Subjects;
using Microsoft.Reactive.Testing;
using NUnit.Framework;
namespace StackOverflow.Tests.Q7620182_PauseResume
{
[TestFixture]
public class PauseAndResumeTests
{
[Test]
public void Should_pause_and_resume()
{
//Arrange
var scheduler = new TestScheduler();
var isRunningTrigger = new BehaviorSubject<bool>(true);
Action pause = () => isRunningTrigger.OnNext(false);
Action resume = () => isRunningTrigger.OnNext(true);
var source = scheduler.CreateHotObservable(
ReactiveTest.OnNext(0.1.Seconds(), 1),
ReactiveTest.OnNext(2.0.Seconds(), 2),
ReactiveTest.OnNext(4.0.Seconds(), 3),
ReactiveTest.OnNext(6.0.Seconds(), 4),
ReactiveTest.OnNext(8.0.Seconds(), 5));
scheduler.Schedule(TimeSpan.FromSeconds(0.5), () => { pause(); });
scheduler.Schedule(TimeSpan.FromSeconds(5.0), () => { resume(); });
//Act
var sut = Observable.Create<IObservable<int>>(o =>
{
var current = source.Replay();
var connection = new SerialDisposable();
connection.Disposable = current.Connect();
return isRunningTrigger
.DistinctUntilChanged()
.Select(isRunning =>
{
if (isRunning)
{
//Return the current replayed values.
return current;
}
else
{
//Disconnect and replace current.
current = source.Replay();
connection.Disposable = current.Connect();
//yield silence until the next time we resume.
return Observable.Never<int>();
}
})
.Subscribe(o);
}).Switch();
var observer = scheduler.CreateObserver<int>();
using (sut.Subscribe(observer))
{
scheduler.Start();
}
//Assert
var expected = new[]
{
ReactiveTest.OnNext(0.1.Seconds(), 1),
ReactiveTest.OnNext(5.0.Seconds(), 2),
ReactiveTest.OnNext(5.0.Seconds(), 3),
ReactiveTest.OnNext(6.0.Seconds(), 4),
ReactiveTest.OnNext(8.0.Seconds(), 5)
};
CollectionAssert.AreEqual(expected, observer.Messages);
}
}
}

It just works:
class SimpleWaitPulse
{
static readonly object _locker = new object();
static bool _go;
static void Main()
{ // The new thread will block
new Thread (Work).Start(); // because _go==false.
Console.ReadLine(); // Wait for user to hit Enter
lock (_locker) // Let's now wake up the thread by
{ // setting _go=true and pulsing.
_go = true;
Monitor.Pulse (_locker);
}
}
static void Work()
{
lock (_locker)
while (!_go)
Monitor.Wait (_locker); // Lock is released while we’re waiting
Console.WriteLine ("Woken!!!");
}
}
Please, see How to Use Wait and Pulse for more details