I would like to convert a blocking IEnumerable (possibly infinite) into messages sent to an Actor. What is the best way to do this? I am currently attempting to create a Task within an actor on PreStart and inside the task have it send messages to but it doesn't seem to be working.
I've read some pages about preferring to use PipeTo to wrap a Task but that seems to only be used to retrieve a single result rather than have a separate process continually sending values.
using System;
using System.Collections;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Akka.Actor;
namespace Junk
{
public class Program
{
public static void Main()
{
var system = ActorSystem.Create("my-system");
var actor = system.ActorOf(Props.Create(() => new Manager()));
Console.ReadLine();
}
}
public class Manager : ReceiveActor
{
private Task _task;
public Manager() {
Receive<uint>(msg => { Console.WriteLine($"received {msg}"); });
}
protected override void PreStart() {
Console.WriteLine($"{Self} - PreStart --------------");
startTask();
}
private void startTask() {
_task = Task.Factory.StartNew(() => {
BlockingEnumerable source = new BlockingEnumerable();
Console.WriteLine($"task starting loop -------------");
foreach (uint v in source) {
Self.Tell(v);
}
});
}
}
public class BlockingEnumerable : IEnumerable<uint>
{
public IEnumerator<uint> GetEnumerator() { return new BlockingEnumerator(); }
IEnumerator IEnumerable.GetEnumerator() { return GetEnumerator(); }
}
public class BlockingEnumerator : IEnumerator<uint>
{
private uint value = 0;
public uint Current => value;
object IEnumerator.Current => Current;
public void Dispose() { }
public bool MoveNext() { Thread.Sleep(2000); value += 1; return true; }
public void Reset() { value = 0; }
}
}
Related
I am using C# and I have an enumerator and I am reading the data inside the enumerator sequentially.
This is a third party library object and does not support Parallel.Foreach
while(enumerator.Next())
{
var item = enumerator.Read();
ProcessItem(item);
}
ProcessItem(Item item)
{
// Is lock required here
if(item.prop == "somevalue")
this._list.Add(item);
}
I want to achieve multithreading here while reading the content.
while(enumerator.Next())
{
// This code should run in a multi-threaded way
var item = enumerator.Read();
// ProcessItem method puts these items on a class level list property
// Is there any Lock required?
ProcessItem(item);
}
I am new to multithreading. Please share any code samples which satisfies the above requirement.
Yes, some locking required. you can achieve it using lock or using a concurrent collection type.
using lock:
ProcessItem(Item item)
{
if(item.prop == "somevalue")
{
lock(_list)
{
_list.Add(item);
}
}
}
Edit: based on detail you provided, you can wrap the enumerator from external lib using your own enumerator like below so you can use Parallel.ForEach on it:
We assume the enumerator you got is something like MockEnumerator, we wrap it in a normal IEnumerator, and IEnumerable so we are able to use Parallel.ForEach to read in parallel.
class Program
{
class Item
{
public int SomeProperty { get; }
public Item(int prop)
{
SomeProperty = prop;
}
}
class MockEnumerator
{
private Item[] _items = new Item[] { new Item(1), new Item(2) };
private int _position = 0;
public bool Next()
{
return _position++ < _items.Length;
}
public Item Read()
{
return _items[_position];
}
}
class EnumeratorWrapper : IEnumerator<Item>, IEnumerable<Item>
{
private readonly MockEnumerator _enumerator;
public EnumeratorWrapper(MockEnumerator enumerator)
{
this._enumerator = enumerator;
}
public Item Current => _enumerator.Read();
object IEnumerator.Current => Current;
public void Dispose()
{
}
public IEnumerator<Item> GetEnumerator()
{
throw new NotImplementedException();
}
public bool MoveNext()
{
return _enumerator.Next();
}
public void Reset()
{
}
IEnumerator IEnumerable.GetEnumerator()
{
return this;
}
}
private static List<Item> _list = new List<Item>();
static void Main(string[] args)
{
var enumerator = new EnumeratorWrapper(new MockEnumerator());
Parallel.ForEach(enumerator, item =>
{
if (item.SomeProperty == 1)//someval
{
lock (_list)
{
_list.Add(item);
}
}
});
}
}
This is a good example for task-based parallelization. Each processing of an item corresponds to a task. Hence, you can change the loop to the following:
var tasks = new List<Task<int>>();
while(enumerator.MoveNext())
{
var item = enumerator.Current;
Task<int> task = new Task<int>(() => ProcessItem(item));
task.Start();
tasks.Add(task);
}
foreach(Task<int> task in tasks)
{
int i = task.Result;
classList.Add(i);
}
Note that the synchronization on the classList is implicitly given by first spawning all tasks in the while loop and then merging the results in the foreach loop. The synchronization is specifically given by the access to Result which waits until the corresponding task is finished.
I have a Director that supplies Builders to Threads.
Once an event is fired I need to dispose the Builder as it holds native resources BUT only when all Threads are done.
I keep the builders in a Cache so it's possible to think of it as to Dispose when number of references is zero if the director release the Builder.
I know about the circular references problem and It's not relevant in this scenario because only the Director supplies the references.
I know that I should implement the Finalize,SuppressFinalize pattern to avoid memory leak if thread died before disposing the Builder.
Consider the following example where its hard to tell who should actually dispose the builder:
using System;
using System.Collections.Generic;
using System.Threading;
namespace ConsoleApplication4
{
class Program
{
static void Main()
{
var builderId = 1;
var dictionary = new Dictionary<int, MyBuilder> {{builderId, new MyBuilder()}};
var director = new Director(dictionary);
for (var i =0 ; i < 5; i++)
{
new Worker(director.GetBuilder(builderId));
}
director.Dispose(builderId);
Thread.Sleep(10000);
}
}
public class Director
{
private readonly Dictionary<int, MyBuilder> _cache;
public Director(Dictionary<int, MyBuilder> cache)
{
this._cache = cache;
}
public MyBuilder GetBuilder(int i)
{
MyBuilder builder;
_cache.TryGetValue(i, out builder);
return builder;
}
public void Dispose(int i)
{
MyBuilder builder;
_cache.TryGetValue(i, out builder);
builder?.Dispose();
}
}
public class MyBuilder :IDisposable
{
public void Dispose()
{
Console.WriteLine("Should I dispose?");
}
public void Build()
{
Console.WriteLine("Build");
}
}
public class Worker
{
public void Work(MyBuilder builder)
{
builder.Build();
builder.Dispose();
}
public Worker(MyBuilder builder)
{
var t = new Thread(() => Work(builder));
t.Start();
}
}
}
Can the various extension methods be changed to allow the FireAndForget call in the unit test below to be more elegant than I have it?
using System;
using System.Threading.Tasks;
namespace MyNamespace
{
public static class AsyncAdditions
{
private static async Task FireAndForgetInternal<T>(Func<Task<T>> task, Action<T> completion) {
T value = await task();
completion(value);
}
public static void FireAndForget<T>(Func<Task<T>> task, Action<T> completion) {
AsyncAdditions.FireAndForgetInternal(task, completion).DoNotAwait();
}
/// <summary>Empty method gets rid of stupid compiler warning</summary>
public static void DoNotAwait(this Task task) {}
}
}
the test:
using System;
using System.Threading.Tasks;
using NUnit.Framework;
using MyNamespace;
namespace MyTests
{
[TestFixture()]
public class AsyncTests
{
public async Task<int> GetAsyncInteger()
{
return await Task.Factory.StartNew(() => 6);
}
public int MyNumber { get; set;}
public void SlowSynchronousJob() {
for (int i = 0; i < 1E7; i++) {
}
}
[Test()]
public void TestSetter() {
AsyncAdditions.FireAndForget(this.GetAsyncInteger, n => this.MyNumber = n);
this.SlowSynchronousJob();
Assert.AreEqual(6, this.MyNumber);
}
}
}
I have a situation where I have multiple producers and multiple consumers. The producers enters a job into a queue. I chose the BlockingCollection and it works great since I need the consumers to wait for a job to be found. However, if I use the GetConsumingEnumerable() feature the order of the items in the collection change... this is not what I need.
It even says in MSDN http://msdn.microsoft.com/en-us/library/dd287186.aspx
that it does not preserve the order of the items.
Does anyone know an alternative for this situation?
I see that the Take method is available but does it also provide a 'wait' condition for the consumer threads?
It says http://msdn.microsoft.com/en-us/library/dd287085.aspx
'A call to Take may block until an item is available to be removed.' Is it better to use TryTake? I really need the thread to wait and keep checking for a job.
Take blocks the thread till something comes available.
TryTake as the name implies tries to do so but returns a bool if it fails or succeeds.
Allowing for more flex using it:
while(goingOn){
if( q.TryTake(out var){
Process(var)
}
else{
DoSomething_Usefull_OrNotUseFull_OrEvenSleep();
}
}
instead of
while(goingOn){
if( var x = q.Take(){
//w'll wait till this ever will happen and then we:
Process(var)
}
}
My votes are for TryTake :-)
EXAMPLE:
public class ProducerConsumer<T> {
public struct Message {
public T Data;
}
private readonly ThreadRunner _producer;
private readonly ThreadRunner _consumer;
public ProducerConsumer(Func<T> produce, Action<T> consume) {
var q = new BlockingCollection<Message>();
_producer = new Producer(produce,q);
_consumer = new Consumer(consume,q);
}
public void Start() {
_producer.Run();
_consumer.Run();
}
public void Stop() {
_producer.Stop();
_consumer.Stop();
}
private class Producer : ThreadRunner {
public Producer(Func<T> produce, BlockingCollection<Message> q) : base(q) {
_produce = produce;
}
private readonly Func<T> _produce;
public override void Worker() {
try {
while (KeepRunning) {
var item = _produce();
MessageQ.TryAdd(new Message{Data = item});
}
}
catch (ThreadInterruptedException) {
WasInterrupted = true;
}
}
}
public abstract class ThreadRunner {
protected readonly BlockingCollection<Message> MessageQ;
protected ThreadRunner(BlockingCollection<Message> q) {
MessageQ = q;
}
protected Thread Runner;
protected bool KeepRunning = true;
public bool WasInterrupted;
public abstract void Worker();
public void Run() {
Runner = new Thread(Worker);
Runner.Start();
}
public void Stop() {
KeepRunning = false;
Runner.Interrupt();
Runner.Join();
}
}
class Consumer : ThreadRunner {
private readonly Action<T> _consume;
public Consumer(Action<T> consume,BlockingCollection<Message> q) : base(q) {
_consume = consume;
}
public override void Worker() {
try {
while (KeepRunning) {
Message message;
if (MessageQ.TryTake(out message, TimeSpan.FromMilliseconds(100))) {
_consume(message.Data);
}
else {
//There's nothing in the Q so I have some spare time...
//Excellent moment to update my statisics or update some history to logfiles
//for now we sleep:
Thread.Sleep(TimeSpan.FromMilliseconds(100));
}
}
}
catch (ThreadInterruptedException) {
WasInterrupted = true;
}
}
}
}
}
USAGE:
[Fact]
public void ConsumerShouldConsume() {
var produced = 0;
var consumed = 0;
Func<int> produce = () => {
Thread.Sleep(TimeSpan.FromMilliseconds(100));
produced++;
return new Random(2).Next(1000);
};
Action<int> consume = c => { consumed++; };
var t = new ProducerConsumer<int>(produce, consume);
t.Start();
Thread.Sleep(TimeSpan.FromSeconds(5));
t.Stop();
Assert.InRange(produced,40,60);
Assert.InRange(consumed, 40, 60);
}
my Code is :
using System;
using System.Threading;
using System.Collections;
using System.Collections.Generic;
namespace IEnumerable
{
public class MyEnumerable<T> : IEnumerable<T>
{
public MyEnumerable(T[] items)
{
this.items = items;
}
public IEnumerator<T> GetEnumerator()
{
return new NestedEnumerator(this);
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
// The enumerator definition.
class NestedEnumerator : IEnumerator<T>
{
public NestedEnumerator(MyEnumerable<T> coll)
{
Monitor.Enter(coll.items.SyncRoot);
this.index = -1;
this.coll = coll;
}
public T Current
{
get { return current; }
}
object IEnumerator.Current
{
get { return Current; }
}
public bool MoveNext()
{
if (++index >= coll.items.Length)
{
return false;
}
else
{
current = coll.items[index];
return true;
}
}
public void Reset()
{
current = default(T);
index = 0;
}
public void Dispose()
{
try
{
current = default(T);
index = coll.items.Length;
}
finally
{
Monitor.Exit(coll.items.SyncRoot);
}
}
private MyEnumerable<T> coll;
private T current;
private int index;
}
private T[] items;
}
public class EntryPoint
{
static void Main()
{
MyEnumerable<int> integers = new MyEnumerable<int>(new int[] { 1, 2, 3, 4 });
foreach (int n in integers)
{
Console.WriteLine(n);
}
}
}
}
I am implementing this piece of code But i get an error. Can anybody help me what to do to error free this code? please help.
My Errors are :
1->'IEnumerable' is a 'namespace' but is used like a 'type'
2->'IEnumerable.MyEnumerable' does not implement interface member 'System.Collections.IEnumerable.GetEnumerator()'. 'IEnumerable.MyEnumerable.GetEnumerator()' cannot implement 'System.Collections.IEnumerable.GetEnumerator()' because it does not have the matching return type of 'System.Collections.IEnumerator'.
The error is likely that you're using IEnumerable as your namespace when it's already a type in the system.
This means that your references to IEnumerable are refering to the current namespace and not the IEnumerable you're meaning to use (System.Collections.IEnumerable).
Try changing your namespace to something else.
Listen to what the compiler is telling you and change your namespace to something sensible (like BlingBlingWoo)
...
using System.Collections.Generic;
namespace BlingBlingWoo
{
public class MyEnumerable<T> : IEnumerable<T>
...
Whats happening here is:
Error : 'IEnumerable' is a 'namespace' but is used like a 'type'
You are trying to use the type IEnumerable, but you've created a namespace called IEnumerable and so the compiler things its a namespace.