We Keep Coding

Using BufferBlock as Observable without consuming the elements

We use BufferBlocks to build a small simulation tool where we want to find areas that takes a long time to complete. Producers and Consumers of the blocks will essentially sleep for x amount of time and then post a message to another block.
We decided to use an Observer pattern. Howver, I see some behavior I did not expect. Whenever the OnNext method of the observers is called the BufferBlock is empty (Count == 0). This is problematic as I want only 1 observer to be able to fetch the value from the queue.
Is there a way to change this behavior? If not, how should I handle consumption from the BufferBlocks?
Currently I want to be able to do something similar to post the messages and have all Observers try to fetch it:
public void OnNext(Object value)
{
var res =this.AsConsumer().ConsumeQueue.ReceiveAsync().Result;
Thread.Sleep(this.TimeToConsume );
ProduceQueue.Post(someOtherValue);
}
I have written some tests to show the behavior of the BufferBlock.
[Test]
public void
WhenObservingMocks_CallsOnNextForAllMocks()
{
var firstObserver = new Mock<IObserver<int>>();
var secondObserver = new Mock<IObserver<int>>();
var block = new BufferBlock<int>();
block.AsObservable().Subscribe(firstObserver.Object);
block.AsObservable().Subscribe(secondObserver.Object);
block.Post(2);
Thread.Sleep(TimeSpan.FromMilliseconds(50));
firstObserver.Verify(e => e.OnNext(It.IsAny<int>()), Times.Once);
secondObserver.Verify(e => e.OnNext(It.IsAny<int>()), Times.Once);
}
[Test]
public void
WhenHavingObservers_DoesConsumesTheElementFromQueue()
{
var firstObserver = new Mock<IObserver<int>>();
var secondObserver = new Mock<IObserver<int>>();
var block = new BufferBlock<int>();
block.AsObservable().Subscribe(firstObserver.Object);
block.AsObservable().Subscribe(secondObserver.Object);
block.Post(2);
Assert.Zero(block.Count);
}
[Test]
public void
WhenPostingOnce_CanOnlyReceiveOnce()
{
var block = new BufferBlock<int>();
block.Post(2);
Assert.True(block.TryReceive(out int _));
Assert.False(block.TryReceive(out int _));
}

Exception handling in RX.Net when using ToEventPattern and Timeout

I am writing some code using RX in C# that must interface with an older system by emitting events.
In summary, I have an observable and need to emit one event when the observable completes and another event if a timeout exception is detected. The main problem is how best to handle the exception.
I'm relatively new to RX, so although I have found a solution, I can't be sure that there isn't a better or more appropriate way that uses the RX extensions better.
This is not the real code but indicates the pattern of my thinking:
public delegate void SuccessHandler(object sender, SuccessEventArgs e);
public event SuccessHandler OnSuccess;
public delegate void TimeoutHandler(object sender, TimeoutEventArgs e);
public event TimeoutHandler OnTimeout;
var id;
var o = Observable.Return() // <- this would be a fetch from an asynchronous source
.Where(r=>r.status=="OK")
.Timeout(new Timespan(0,0,30)
.Do(r=> {
id=r.Id // <-- Ugh! I know this shouldn't be done!
}
.Subscribe(r => {
var statusResponse= new StatusResponse()
{
Id = r.Id
Name = r.Name
Message = "The operation completed successfully",
Status = Status.Success
};
if (OnSuccess == null) return;
OnSuccess (this, new SuccessEventArgs(statusResponse);
},
e =>
{
_logger.LogError(e, "A matching response was not returned in a timely fashion");
if (OnTimeout == null) return;
OnTimeout(this, new TimeoutEventArgs(id));
});
If I didn't need to detect and act upon the timeout it would be fine; I have already worked out how to substitute the Subscribe for ToEventPattern:
...
.Select(r =>
{
var statusResponse= new StatusResponse()
{
Id = r.Id
Name = r.Name
Message = "The operation completed successfully",
Status = Status.Success
};
return new EventPattern<SuccessEventArgs>(this, new SuccessEventArgs(statusResponse));
})
.ToEventPattern();
However, I'd like to be able to detect the timeout (and possibly other exceptions). my experiments with Catch have been unsuccessful because I can't seem to get the types to line up correctly, probably because I don't really understand what is going on.
I'd very much appreciate opinions on this. Is this an acceptable solution? How can I improve it? Can anyone point me to some good online references that will explain how this kind of flow-control and exception handling can be done (all the examples I've seen so far seem to stop short of the real-world case where you want to emit an event and combine that with exception handling).
Thanks in advance

You can branch from observables quite easily, e.g.
var a = Observable.Range(0, 10);
var b = a.Select(x => x * x);
var c = a.Select(x => x * 10);
A word of warning - if the observable is cold, this will cause the producer function to run for each subscription. Look up the difference between hot and cold observables if this isn't clear.
I've created a solution that creates two branches from the source observable and turns each into an event:
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Hello World!");
var service = new Service();
var apiCall = service.CallApi();
apiCall.OnSuccess.OnNext += (_, __) => Console.WriteLine("Success!");
apiCall.OnTimeout.OnNext += (_, __) => Console.WriteLine("Timeout!");
Console.ReadLine();
}
}
class SuccessEventArgs{}
class TimeoutEventArgs{}
class ApiCall
{
public IEventPatternSource<SuccessEventArgs> OnSuccess {get;}
public IEventPatternSource<TimeoutEventArgs> OnTimeout {get;}
public ApiCall(IEventPatternSource<SuccessEventArgs> onSuccess, IEventPatternSource<TimeoutEventArgs> onTimeout)
{
OnSuccess = onSuccess;
OnTimeout = onTimeout;
}
}
class Service
{
public ApiCall CallApi()
{
var apiCall = Observable
.Timer(TimeSpan.FromSeconds(3))
.Do(_ => Console.WriteLine("Api Called"))
.Select(_ => new EventPattern<SuccessEventArgs>(null, new SuccessEventArgs()))
// .Timeout(TimeSpan.FromSeconds(2)) // uncomment to time out
.Timeout(TimeSpan.FromSeconds(4))
// the following two lines turn the "cold" observable "hot"
// comment them out and see how often "Api Called" is logged
.Publish()
.RefCount();
var success = apiCall
// ignore the TimeoutException and return an empty observable
.Catch<EventPattern<SuccessEventArgs>, TimeoutException>(_ => Observable.Empty<EventPattern<SuccessEventArgs>>())
.ToEventPattern();
var timeout = apiCall
.Materialize() // turn the exception into a call to OnNext rather than OnError
.Where(x => x.Exception is TimeoutException)
.Select(_ => new EventPattern<TimeoutEventArgs>(null, new TimeoutEventArgs()))
.ToEventPattern();
return new ApiCall(success, timeout);
}
}

Observable (and cancelable) loop

I'm creating an emulator. The core of the emulation runs in an infinite loop like this:
while (true)
{
UpdateMachineState();
}
I would like to introduce Reactive Extensions to execute this loop into another thread and to make it cancelable, but I'm completely lost.
Since my emulator is a GUI application (Universal Windows), I don't wan't to block the UI thread.
It should look like:
...
while (true)
{
if (machine.IsHalted)
{
observer.OnCompleted;
}
observer.OnNext(machine.GetState());
cancellationToken.ThrowIfCancellationRequested();
}
...
The created sequence would eventually complete when the emulator enters the "halted" state. Otherwise, it will keep pushing States (an object that represents its internal state) forever.
I've tried with Observable.Create, but the overload that provides a CancellationToken requires a Task<Action>.

Here's how you do it in Rx:
void Main()
{
var scheduler = new EventLoopScheduler();
var loop = scheduler.Schedule(a =>
{
UpdateMachineState();
a();
});
Thread.Sleep(1);
loop.Dispose();
}
public void UpdateMachineState()
{
Console.Write(".");
}
The overload on .Schedule that I used takes a Action<Action> as the parameter. You simply call the inner action if you want the the action to be rescheduled - so the above code effectively creates the infinite loop.
You then call .Dispose() on the return from the .Schedule call to cancel the loop.
Another alternative is to use the .Generate operator:
var scheduler = new EventLoopScheduler();
var query =
Observable
.Generate(0, x => true, x => x, x => machine.GetState(), scheduler);
var subscription = query.Subscribe(x => Console.Write("."));
Thread.Sleep(1);
subscription.Dispose();

Custom Rx operator for throttling only when there's a been a recent value

I'm trying to create an Rx operator that seems pretty useful, but I've suprisingly not found any questions on Stackoverflow that match precisely. I'd like to create a variation on Throttle that lets values through immediately if there's been a period of inactivity. My imagined use case is something like this:
I have a dropdown that kicks off a web request when the value is changed. If the user holds down the arrow key and cycles rapidly through the values, I don't want to kick off a request for each value. But if I throttle the stream then the user has to wait out the throttle duration every time they just select a value from the dropdown in the normal manner.
So whereas a normal Throttle looks like this:
I want to create ThrottleSubsequent that look like this:
Note that marbles 1, 2, and 6 are passed through without delay because they each follow a period of inactivity.
My attempt at this looks like the following:
public static IObservable<TSource> ThrottleSubsequent<TSource>(this IObservable<TSource> source, TimeSpan dueTime, IScheduler scheduler)
{
// Create a timer that resets with each new source value
var cooldownTimer = source
.Select(x => Observable.Interval(dueTime, scheduler)) // Each source value becomes a new timer
.Switch(); // Switch to the most recent timer
var cooldownWindow = source.Window(() => cooldownTimer);
// Pass along the first value of each cooldown window immediately
var firstAfterCooldown = cooldownWindow.SelectMany(o => o.Take(1));
// Throttle the rest of the values
var throttledRest = cooldownWindow
.SelectMany(o => o.Skip(1))
.Throttle(dueTime, scheduler);
return Observable.Merge(firstAfterCooldown, throttledRest);
}
This seems to work, but I'm having a difficult time reasoning about this, and I get the feeling there are some edge cases here where things might get screwy with duplicate values or something. I'd like to get some feedback from more experienced Rx-ers as to whether or not this code is correct, and/or whether there is a more idiomatic way of doing this.

Well, here's a test suite (using nuget Microsoft.Reactive.Testing):
var ts = new TestScheduler();
var source = ts.CreateHotObservable<char>(
new Recorded<Notification<char>>(200.MsTicks(), Notification.CreateOnNext('A')),
new Recorded<Notification<char>>(300.MsTicks(), Notification.CreateOnNext('B')),
new Recorded<Notification<char>>(500.MsTicks(), Notification.CreateOnNext('C')),
new Recorded<Notification<char>>(510.MsTicks(), Notification.CreateOnNext('D')),
new Recorded<Notification<char>>(550.MsTicks(), Notification.CreateOnNext('E')),
new Recorded<Notification<char>>(610.MsTicks(), Notification.CreateOnNext('F')),
new Recorded<Notification<char>>(760.MsTicks(), Notification.CreateOnNext('G'))
);
var target = source.ThrottleSubsequent(TimeSpan.FromMilliseconds(150), ts);
var expectedResults = ts.CreateHotObservable<char>(
new Recorded<Notification<char>>(200.MsTicks(), Notification.CreateOnNext('A')),
new Recorded<Notification<char>>(450.MsTicks(), Notification.CreateOnNext('B')),
new Recorded<Notification<char>>(500.MsTicks(), Notification.CreateOnNext('C')),
new Recorded<Notification<char>>(910.MsTicks(), Notification.CreateOnNext('G'))
);
var observer = ts.CreateObserver<char>();
target.Subscribe(observer);
ts.Start();
ReactiveAssert.AreElementsEqual(expectedResults.Messages, observer.Messages);
and using
public static class TestingHelpers
{
public static long MsTicks(this int i)
{
return TimeSpan.FromMilliseconds(i).Ticks;
}
}
Seems to pass. If you wanted to reduce it, you could turn it into this:
public static IObservable<TSource> ThrottleSubsequent2<TSource>(this IObservable<TSource> source, TimeSpan dueTime, IScheduler scheduler)
{
return source.Publish(_source => _source
.Window(() => _source
.Select(x => Observable.Interval(dueTime, scheduler))
.Switch()
))
.Publish(cooldownWindow =>
Observable.Merge(
cooldownWindow
.SelectMany(o => o.Take(1)),
cooldownWindow
.SelectMany(o => o.Skip(1))
.Throttle(dueTime, scheduler)
)
);
}
EDIT:
Publish forces sharing of a subscription. If you have a bad (or expensive) source observable with subscription side-effects, Publish makes sure you only subscribe once. Here's an example where Publish helps:
void Main()
{
var source = UglyRange(10);
var target = source
.SelectMany(i => Observable.Return(i).Delay(TimeSpan.FromMilliseconds(10 * i)))
.ThrottleSubsequent2(TimeSpan.FromMilliseconds(70), Scheduler.Default) //Works with ThrottleSubsequent2, fails with ThrottleSubsequent
.Subscribe(i => Console.WriteLine(i));
}
static int counter = 0;
public IObservable<int> UglyRange(int limit)
{
var uglySource = Observable.Create<int>(o =>
{
if (counter++ == 0)
{
Console.WriteLine("Ugly observable should only be created once.");
Enumerable.Range(1, limit).ToList().ForEach(i => o.OnNext(i));
}
else
{
Console.WriteLine($"Ugly observable should only be created once. This is the {counter}th time created.");
o.OnError(new Exception($"observable invoked {counter} times."));
}
return Disposable.Empty;
});
return uglySource;
}

Call service on each different thread and wait for all service reponse

As I am working on one Windows service project, I am finding trouble to integrate below functionality.
I need to call 5 different web services on 5 different thread in one call (in single method).
I must have to wait for all 5 service's response and then I have to perform next task.
Question: How would I wait for all service's response which is called by each different thread? and then I perform my next task moving further.
Thanks in advance.

If you are using .NET 4.0, you can use the Task Parallel Library.
Start each web service call with a new Task and then call Task.WaitAll passing arraying of tasks created.
Task[] taskList = new Task[5];
taskList[0] = System.Threading.Tasks.Task.Factory.StartNew(()=> ServiceCall1());
taskList[1] = System.Threading.Tasks.Task.Factory.StartNew(()=> ServiceCall2());
taskList[2] = System.Threading.Tasks.Task.Factory.StartNew(()=> ServiceCall3());
taskList[3] = System.Threading.Tasks.Task.Factory.StartNew(()=> ServiceCall4());
taskList[4] = System.Threading.Tasks.Task.Factory.StartNew(()=> ServiceCall5());
System.Thread.Tasks.Task.WaitAll(taskList);

You can use the Thread.Join() method in order to wait on a thread.
Create a new thread per service call:
var service1 = new Thread(() => StartService1());
var service2 = new Thread(() => StartService2());
var service3 = new Thread(() => StartService3());
var service4 = new Thread(() => StartService4());
var service5 = new Thread(() => StartService5());
Wait for all threads to terminate:
if (service1.Join() && service2.Join() && service3.Join() &&
service4.Join() && service5.Join())
// Continue...

You can do a couple of things. First option, you can spawn 5 threads from your service, have each thread contact a web-service, and wait for all the threads to finish in your main thread (you can call Thread.Join on all of them - just call Join on the first thread, then on the second etc...), and when they're all done continue your processing.
It's the simplest thing. Although if you care about performance and scale, you might want to use asynchronous communications, and call the 5 services on the main thread and wait for 5 callbacks. I'd worry about that only if performance becomes an issue.

You can use System.Threading.Tasks.Parallel.Invoke
void ServiceN()
{
/////
}
Parallel.Invoke(Service1, Service2, Service3, Service4, Service5);

You could use the Fask.Factory Class. This way, you could add various checks or handle exceptions / timeouts of child-processes.
Task parent = new Task(() =>
{
var cts = new CancellationTokenSource();
var tf = new TaskFactory<Int32>(cts.Token,
TaskCreationOptions.AttachedToParent,
TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Default);
var childTasks = new[] {
tf.StartNew(() => StartService1()),
tf.StartNew(() => StartService2()),
tf.StartNew(() => StartService3()),
tf.StartNew( () => StartService4()),
tf.StartNew( () => StartService5())
};
});
parent.ContinueWith(p =>
{
//(..do something..)
}, TaskContinuationOptions.None);
parent.Start();

Use the System.Threading.ThreadPool.SetMaxThreads ?
static void Main(string[] args)
{
System.Threading.ThreadPool.SetMaxThreads(5 ,5);
for (int i = 0; i < 100; i++)
{
System.Threading.ThreadPool.QueueUserWorkItem(p => { Console.WriteLine(p.ToString()); System.Threading.Thread.Sleep(10000); }, i);
}
Console.Read();
}
For Update :
And you also could revert it after creating threads like
int a, b;
ThreadPool.GetMaxThreads(out a , out b);
ThreadPool.SetMaxThreads(5, 5);
// your code
ThreadPool.SetMaxThreads(a, b);