I'm writing a wrapper around a 3rd party library, and it has a method to scan the data it manages. The method takes a callback method that it calls for each item in the data that it finds.
e.g. The method is essentially: void Scan(Action<object> callback);
I want to wrap it and expose a method like IEnumerable<object> Scan();
Is this possible without resorting to a separate thread to do the actual scan and a buffer?
You can do this quite simply with Reactive:
class Program
{
static void Main(string[] args)
{
foreach (var x in CallBackToEnumerable<int>(Scan))
Console.WriteLine(x);
}
static IEnumerable<T> CallBackToEnumerable<T>(Action<Action<T>> functionReceivingCallback)
{
return Observable.Create<T>(o =>
{
// Schedule this onto another thread, otherwise it will block:
Scheduler.Later.Schedule(() =>
{
functionReceivingCallback(o.OnNext);
o.OnCompleted();
});
return () => { };
}).ToEnumerable();
}
public static void Scan(Action<int> act)
{
for (int i = 0; i < 100; i++)
{
// Delay to prove this is working asynchronously.
Thread.Sleep(100);
act(i);
}
}
}
Remember that this doesn't take care of things like cancellation, since the callback method doesn't really allow it. A proper solution would require work on the part of the external library.
You should investigate the Rx project — this allows an event source to be consumed as an IEnumerable.
I'm not sure if it allows vanilla callbacks to be presented as such (it's aimed at .NET events) but it would be worth a look as it should be possible to present a regular callback as an IObservable.
Here is a blocking enumerator (the Scan method needs to run in a separate thread)
public class MyEnumerator : IEnumerator<object>
{
private readonly Queue<object> _queue = new Queue<object>();
private ManualResetEvent _event = new ManualResetEvent(false);
public void Callback(object value)
{
lock (_queue)
{
_queue.Enqueue(value);
_event.Set();
}
}
public void Dispose()
{
}
public bool MoveNext()
{
_event.WaitOne();
lock (_queue)
{
Current = _queue.Dequeue();
if (_queue.Count == 0)
_event.Reset();
}
return true;
}
public void Reset()
{
_queue.Clear();
}
public object Current { get; private set; }
object IEnumerator.Current
{
get { return Current; }
}
}
static void Main(string[] args)
{
var enumerator = new MyEnumerator();
Scan(enumerator.Callback);
while (enumerator.MoveNext())
{
Console.WriteLine(enumerator.Current);
}
}
You could wrap it in a simple IEnumerable<Object>, but I would not recommend it. IEnumerable lists implies that you can run multiple enumerators on the same list, which you can't in this case.
How about this one:
IEnumerable<Object> Scan()
{
List<Object> objList = new List<Object>();
Action<Object> action = (obj) => { objList.Add(obj); };
Scan(action);
return objList;
}
Take a look at the yield keyword -- which will allow you to have a method that looks like an IEnumerable but which actually does processing for each return value.
Related
I am using C# and I have an enumerator and I am reading the data inside the enumerator sequentially.
This is a third party library object and does not support Parallel.Foreach
while(enumerator.Next())
{
var item = enumerator.Read();
ProcessItem(item);
}
ProcessItem(Item item)
{
// Is lock required here
if(item.prop == "somevalue")
this._list.Add(item);
}
I want to achieve multithreading here while reading the content.
while(enumerator.Next())
{
// This code should run in a multi-threaded way
var item = enumerator.Read();
// ProcessItem method puts these items on a class level list property
// Is there any Lock required?
ProcessItem(item);
}
I am new to multithreading. Please share any code samples which satisfies the above requirement.
Yes, some locking required. you can achieve it using lock or using a concurrent collection type.
using lock:
ProcessItem(Item item)
{
if(item.prop == "somevalue")
{
lock(_list)
{
_list.Add(item);
}
}
}
Edit: based on detail you provided, you can wrap the enumerator from external lib using your own enumerator like below so you can use Parallel.ForEach on it:
We assume the enumerator you got is something like MockEnumerator, we wrap it in a normal IEnumerator, and IEnumerable so we are able to use Parallel.ForEach to read in parallel.
class Program
{
class Item
{
public int SomeProperty { get; }
public Item(int prop)
{
SomeProperty = prop;
}
}
class MockEnumerator
{
private Item[] _items = new Item[] { new Item(1), new Item(2) };
private int _position = 0;
public bool Next()
{
return _position++ < _items.Length;
}
public Item Read()
{
return _items[_position];
}
}
class EnumeratorWrapper : IEnumerator<Item>, IEnumerable<Item>
{
private readonly MockEnumerator _enumerator;
public EnumeratorWrapper(MockEnumerator enumerator)
{
this._enumerator = enumerator;
}
public Item Current => _enumerator.Read();
object IEnumerator.Current => Current;
public void Dispose()
{
}
public IEnumerator<Item> GetEnumerator()
{
throw new NotImplementedException();
}
public bool MoveNext()
{
return _enumerator.Next();
}
public void Reset()
{
}
IEnumerator IEnumerable.GetEnumerator()
{
return this;
}
}
private static List<Item> _list = new List<Item>();
static void Main(string[] args)
{
var enumerator = new EnumeratorWrapper(new MockEnumerator());
Parallel.ForEach(enumerator, item =>
{
if (item.SomeProperty == 1)//someval
{
lock (_list)
{
_list.Add(item);
}
}
});
}
}
This is a good example for task-based parallelization. Each processing of an item corresponds to a task. Hence, you can change the loop to the following:
var tasks = new List<Task<int>>();
while(enumerator.MoveNext())
{
var item = enumerator.Current;
Task<int> task = new Task<int>(() => ProcessItem(item));
task.Start();
tasks.Add(task);
}
foreach(Task<int> task in tasks)
{
int i = task.Result;
classList.Add(i);
}
Note that the synchronization on the classList is implicitly given by first spawning all tasks in the while loop and then merging the results in the foreach loop. The synchronization is specifically given by the access to Result which waits until the corresponding task is finished.
I have a method that is accessed from multiple threads at the same time and I want to make sure that only 1 thread can be inside of a body of any method.
Can this code be refactored to something more generic? (Apart from Locking inside the State property?
public class StateManager : IStateManager
{
private readonly object _lock = new object();
public Guid? GetInfo1()
{
lock (_lock)
{
return State.Info1;
}
}
public void SetInfo1(Guid guid)
{
lock (_lock)
{
State.Info1 = guid;
}
}
public Guid? GetInfo2()
{
lock (_lock)
{
return State.Info2;
}
}
public void SetInfo2(Guid guid)
{
lock (_lock)
{
State.Info2 = guid;
}
}
}
Maybe something like:
private void LockAndExecute(Action action)
{
lock (_lock)
{
action();
}
}
Then your methods might look like this:
public void DoSomething()
{
LockAndExecute(() => Console.WriteLine("DoSomething") );
}
public int GetSomething()
{
int i = 0;
LockAndExecute(() => i = 1);
return i;
}
I'm not sure that's really saving you very much however and return values are a bit of a pain.
Although you could work around that by adding another method like this:
private T LockAndExecute<T>(Func<T> function)
{
lock (_lock)
{
return function();
}
}
So now my GetSomething method is a lot cleaner:
public int GetSomething()
{
return LockAndExecute(() => 1 );
}
Again, not sure you are gaining much in terms of less typing, but at least you know every call is locking on the same object.
While your gains may be pretty minimal in the case where all you need to do is lock, I could imagine a case where you had a bunch of methods something like this:
public void DoSomething()
{
// check some preconditions
// maybe do some logging
try
{
// do actual work here
}
catch (SomeException e)
{
// do some error handling
}
}
In that case, extracting all the precondition checking and error handling into one place could be pretty useful:
private void CheckExecuteAndHandleErrors(Action action)
{
// preconditions
// logging
try
{
action();
}
catch (SomeException e)
{
// handle errors
}
}
Using Action or Function Delegate.
Creating a method like
public T ExecuteMethodThreadSafe<T>(Func<T> MethodToExecute)
{
lock (_lock)
{
MethodToExecute.Invoke();
}
}
and using it like
public T GetInfo2(Guid guid)
{
return ExecuteMethodThreadSafe(() => State.Info2);
}
I would like to add what I ended up putting together, using some of the ideas presented by Matt and Abhinav in order to generalize this and make it as seamless as possible to implement.
private static readonly object Lock = new object();
public static void ExecuteMethodThreadSafe<T>(this T #object, Action<T> method) {
lock (Lock) {
method(#object);
}
}
public static TResult ExecuteMethodThreadSafe<T, TResult>(this T #object, Func<T, TResult> method) {
lock (Lock) {
return method(#object);
}
}
Which can then be extended in ways like this (if you want):
private static readonly Random Random = new Random();
public static T GetRandom<T>(Func<Random, T> method) => Random.ExecuteMethodThreadSafe(method);
And then when implemented could look something like this:
var bounds = new Collection<int>();
bounds.ExecuteMethodThreadSafe(list => list.Add(15)); // using the base method
int x = GetRandom(random => random.Next(-10, bounds[0])); // using the extended method
int y = GetRandom(random => random.Next(bounds[0])); // works with any method overload
I want to reduce multiple events into a single delayed action. After some trigger occurs I expect some more similar triggers to come, but I prefer not to repeat the resulting delayed action. The action waits, to give a chance of completion to the burst.
The question: How can I do it in an elegant reusable way?
Till now I used a property to flag the event and trigger a delayed action like below:
public void SomeMethod()
{
SomeFlag = true; //this will intentionally return to the caller before completing the resulting buffered actions.
}
private bool someFlag;
public bool SomeFlag
{
get { return someFlag; }
set
{
if (someFlag != value)
{
someFlag = value;
if (value)
SomeDelayedMethod(5000);
}
}
}
public async void SomeDelayedMethod(int delay)
{
//some bufferred work.
await Task.Delay(delay);
SomeFlag = false;
}
below is a shorter way, but still not generic or reusable... I want something concise that packages the actions and the flag, and keeps the functionality (returning to the caller before execution is complete (like today)). I also need to be able to pass an object reference to this action)
public void SerializeAccountsToConfig()
{
if (!alreadyFlagged)
{
alreadyFlagged = true;
SerializeDelayed(5000, Serialize);
}
}
public async void SerializeDelayed(int delay, Action whatToDo)
{
await Task.Delay(delay);
whatToDo();
}
private bool alreadyFlagged;
private void Serialize()
{
//some buferred work.
//string json = JsonConvert.SerializeObject(Accounts, Formatting.Indented);
//Settings1.Default.Accounts = json;
//Settings1.Default.Save();
alreadyFlagged = false;
}
Here's a thread-safe and reusable solution.
You can create an instance of DelayedSingleAction, and in the constructor you pass the action that you want to have performed. I believe this is thread safe, though there is a tiny risk that it will restart the timer just before commencing the action, but I think that risk would exist no matter what the solution is.
public class DelayedSingleAction
{
private readonly Action _action;
private readonly long _millisecondsDelay;
private long _syncValue = 1;
public DelayedSingleAction(Action action, long millisecondsDelay)
{
_action = action;
_millisecondsDelay = millisecondsDelay;
}
private Task _waitingTask = null;
private void DoActionAndClearTask(Task _)
{
Interlocked.Exchange(ref _syncValue, 1);
_action();
}
public void PerformAction()
{
if (Interlocked.Exchange(ref _syncValue, 0) == 1)
{
_waitingTask = Task.Delay(TimeSpan.FromMilliseconds(_millisecondsDelay))
.ContinueWith(DoActionAndClearTask);
}
}
public Task Complete()
{
return _waitingTask ?? Task.FromResult(0);
}
}
See this dotnetfiddle for an example which invokes one action continuously from multiple threads.
https://dotnetfiddle.net/el14wZ
Since you're interested in RX here simple console app sample:
static void Main(string[] args)
{
// event source
var burstEvents = Observable.Interval(TimeSpan.FromMilliseconds(50));
var subscription = burstEvents
.Buffer(TimeSpan.FromSeconds(3)) // collect events 3 seconds
//.Buffer(50) // or collect 50 events
.Subscribe(events =>
{
//Console.WriteLine(events.First()); // take only first event
// or process event collection
foreach (var e in events)
Console.Write(e + " ");
Console.WriteLine();
});
Console.ReadLine();
return;
}
Based on the solution proposed by Andrew, here is a more generic solution.
Declaration and instance creation of the delayed action:
public DelayedSingleAction<Account> SendMailD;
Create the instance inside a function or in the constructor (this can be a collection of such actions each working on a different object):
SendMailD = new DelayedSingleAction<Account>(SendMail, AccountRef, 5000);
repeatedly call this action
SendMailD.PerformAction();
Send mail is the action you will "burst control". Its signature matches :
public int SendMail(Account A)
{}
Here is the updated class
public class DelayedSingleAction<T>
{
private readonly Func<T, int> actionOnObj;
private T tInstance;
private readonly long millisecondsDelay;
private long _syncValue = 1;
public DelayedSingleAction(Func<T, int> ActionOnObj, T TInstance, long MillisecondsDelay)
{
actionOnObj = ActionOnObj;
tInstance = TInstance;
millisecondsDelay = MillisecondsDelay;
}
private Task _waitingTask = null;
private void DoActionAndClearTask(Task _)
{
Console.WriteLine(string.Format("{0:h:mm:ss.fff} DelayedSingleAction Resetting SyncObject: Thread {1} for {2}", DateTime.Now, System.Threading.Thread.CurrentThread.ManagedThreadId, tInstance));
Interlocked.Exchange(ref _syncValue, 1);
actionOnObj(tInstance);
}
public void PerformAction()
{
if (Interlocked.Exchange(ref _syncValue, 0) == 1)
{
Console.WriteLine(string.Format("{0:h:mm:ss.fff} DelayedSingleAction Starting the timer: Thread {1} for {2}", DateTime.Now, System.Threading.Thread.CurrentThread.ManagedThreadId, tInstance));
_waitingTask = Task.Delay(TimeSpan.FromMilliseconds(millisecondsDelay)).ContinueWith(DoActionAndClearTask);
}
}
public Task Complete()
{
return _waitingTask ?? Task.FromResult(0);
}
}
This is a design question, not a bug fix problem.
The situation is this. I have a lot of collections and objects contained in one class. Their contents are only changed by a single message handler thread. There is one other thread which is doing rendering. Each frame it iterates through some of these collections and draws to the screen based on the value of these objects. It does not alter the objects in any way, it is just reading their values.
Now when the rendering is being done, if any of the collections are altered, my foreach loops in the rendering method fail. How should I make this thread safe? Edit: So I have to lock the collections outside each foreach loop I run on them. This works, but it seems like a lot of repetitive code to solve this problem.
As a short, contrived example:
class State
{
public object LockObjects;
public List<object> Objects;
// Called by message handler thread
void HandleMessage()
{
lock (LockObjects)
{
Objects.Add(new object());
}
}
}
class Renderer
{
State m_state;
// Called by rendering thread
void Render()
{
lock (m_state.LockObjects)
{
foreach (var obj in m_state.Objects)
{
DrawObject(obj);
}
}
}
}
This is all well and good, but I'd rather not put locks on all my state collections if there's a better way. Is this "the right" way to do it or is there a better way?
The better way is to use begin/end methods and separated lists for your both threads and synchronization using auto events for example. It will be lock-free to your message handler thread and enables you to have a lot of render/message handler threads:
class State : IDisposable
{
private List<object> _objects;
private ReaderWriterLockSlim _locker;
private object _cacheLocker;
private List<object> _objectsCache;
private Thread _synchronizeThread;
private AutoResetEvent _synchronizationEvent;
private bool _abortThreadToken;
public State()
{
_objects = new List<object>();
_objectsCache = new List<object>();
_cacheLocker = new object();
_locker = new ReaderWriterLockSlim();
_synchronizationEvent = new AutoResetEvent(false);
_abortThreadToken = false;
_synchronizeThread = new Thread(Synchronize);
_synchronizeThread.Start();
}
private void Synchronize()
{
while (!_abortThreadToken)
{
_synchronizationEvent.WaitOne();
int objectsCacheCount;
lock (_cacheLocker)
{
objectsCacheCount = _objectsCache.Count;
}
if (objectsCacheCount > 0)
{
_locker.EnterWriteLock();
lock (_cacheLocker)
{
_objects.AddRange(_objectsCache);
_objectsCache.Clear();
}
_locker.ExitWriteLock();
}
}
}
public IEnumerator<object> GetEnumerator()
{
_locker.EnterReadLock();
foreach (var o in _objects)
{
yield return o;
}
_locker.ExitReadLock();
}
// Called by message handler thread
public void HandleMessage()
{
lock (_cacheLocker)
{
_objectsCache.Add(new object());
}
_synchronizationEvent.Set();
}
public void Dispose()
{
_abortThreadToken = true;
_synchronizationEvent.Set();
}
}
Or (the simpler way) you can use ReaderWriteerLockSlim (Or just locks if you sure you have only one reader) like in the following code:
class State
{
List<object> m_objects = new List<object>();
ReaderWriterLockSlim locker = new ReaderWriterLockSlim();
public IEnumerator<object> GetEnumerator()
{
locker.EnterReadLock();
foreach (var o in Objects)
{
yield return o;
}
locker.ExitReadLock();
}
private List<object> Objects
{
get { return m_objects; }
set { m_objects = value; }
}
// Called by message handler thread
public void HandleMessage()
{
locker.EnterWriteLock();
Objects.Add(new object());
locker.ExitWriteLock();
}
}
Humm... have you tried with a ReaderWriterLockSlim ? Enclose each conllection with one of this, and ensure you start a read or write operation each time you access it.
Just for the heck of it I'm trying to emulate how JRuby generators work using threads in C#.
Also, I'm fully aware that C# has built in support for yield return, I'm just toying around a bit.
I guess it's some sort of poor mans coroutines by keeping multiple callstacks alive using threads. (even though none of the callstacks should execute at the same time)
The idea is like this:
The consumer thread requests a value
The worker thread provides a value and yields back to the consumer thread
Repeat untill worker thread is done
So, what would be the correct way of doing the following?
//example
class Program
{
static void Main(string[] args)
{
ThreadedEnumerator<string> enumerator = new ThreadedEnumerator<string>();
enumerator.Init(() =>
{
for (int i = 1; i < 100; i++)
{
enumerator.Yield(i.ToString());
}
});
foreach (var item in enumerator)
{
Console.WriteLine(item);
};
Console.ReadLine();
}
}
//naive threaded enumerator
public class ThreadedEnumerator<T> : IEnumerator<T>, IEnumerable<T>
{
private Thread enumeratorThread;
private T current;
private bool hasMore = true;
private bool isStarted = false;
AutoResetEvent enumeratorEvent = new AutoResetEvent(false);
AutoResetEvent consumerEvent = new AutoResetEvent(false);
public void Yield(T item)
{
//wait for consumer to request a value
consumerEvent.WaitOne();
//assign the value
current = item;
//signal that we have yielded the requested
enumeratorEvent.Set();
}
public void Init(Action userAction)
{
Action WrappedAction = () =>
{
userAction();
consumerEvent.WaitOne();
enumeratorEvent.Set();
hasMore = false;
};
ThreadStart ts = new ThreadStart(WrappedAction);
enumeratorThread = new Thread(ts);
enumeratorThread.IsBackground = true;
isStarted = false;
}
public T Current
{
get { return current; }
}
public void Dispose()
{
enumeratorThread.Abort();
}
object System.Collections.IEnumerator.Current
{
get { return Current; }
}
public bool MoveNext()
{
if (!isStarted)
{
isStarted = true;
enumeratorThread.Start();
}
//signal that we are ready to receive a value
consumerEvent.Set();
//wait for the enumerator to yield
enumeratorEvent.WaitOne();
return hasMore;
}
public void Reset()
{
throw new NotImplementedException();
}
public IEnumerator<T> GetEnumerator()
{
return this;
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return this;
}
}
Ideas?
There are many ways to implement the producer/consumer pattern in C#.
The best way, I guess, is using TPL (Task, BlockingCollection). See an example here.