I am consuming one data streaming source which pushes data in interval >= 5 seconds. I have one observer like below to receive push notifications:
public class ConsoleDataObserver : IObserver<DataPipeEvent>
{
private DataPipeType _dataPipeType;
private LimitedConcurrencyLevelTaskScheduler _scheduler;//This is based on msdn parallel extensions examples
private TaskFactory _factory;
public ConsoleDataObserver(DataPipeType afDataPipeType)
{
_dataPipeType = afDataPipeType;
_scheduler = new LimitedConcurrencyLevelTaskScheduler(5);
_factory = new TaskFactory(_scheduler);
}
public void OnNext(DataPipeEvent dataPipeEvent)
{
_factory.StartNew(() => ProcessDataNow(dataPipeEvent));
}
private void ProcessDataNow(DataPipeEvent dataPipeEvent)
{
Thread.Sleep(8000); //just want to simulate long running tasks
}
public void OnError(Exception error)
{
Console.WriteLine("Provider has sent an error");
}
public void OnCompleted()
{
Console.WriteLine("Provider has terminated sending data");
}
}
I have following requirements:
In my OnNext, I don't want to block main thread and want to do long running processing tasks in other thread. I am using TaskScheduler to used ThreadPool. Is it good implementation? OnNext can get 500-1000 events per second. ProcessDataNow will log in case of any exception.
Related
I am trying to display messages to a list box in a WinForms application but it is not working. I am using the most recent Azure namespace, hence using asynchronous methods.
Below is Program.cs:
namespace App
{
public class Program
{
static ServiceBusClient client;
static ServiceBusProcessor processor;
public static List<string> data = new List<string>();
[STAThread]
public static async Task MessageHandler(ProcessMessageEventArgs args)
{
string body = args.Message.Body.ToString();
data.Add(body);
// complete the message. messages is deleted from the subscription.
await args.CompleteMessageAsync(args.Message);
}
public static void Main(string[] args)
{
Application.Run(new Form1());
}
public static async Task MainAsync()
{
client = new ServiceBusClient(_serviceBusConn);
// create a processor that we can use to process the messages
processor = client.CreateProcessor(_serviceBustopic, _ServiceBusSubscription, new ServiceBusProcessorOptions());
try
{
// add handler to process messages
processor.ProcessMessageAsync += MessageHandler;
// add handler to process any errors
processor.ProcessErrorAsync += ErrorHandler;
// start processing
await processor.StartProcessingAsync();
}
finally
{
await processor.DisposeAsync();
await client.DisposeAsync();
}
}
}
}
//end of Program.cs
And the Form.cs:
namespace App
{
public partial class Form1 : Form
{
public static List<string> AppNames = new List<string>();
public Form1()
{
InitializeComponent();
}
public static async Task receiveMessage()
{
await Program.MainAsync();
AppNames = Program.data;
}
public async void button1_Click(object sender, EventArgs e)
{
await receiveMessage();
for (int i = 0; i < AppNames.Count; i++)
{
listBox1.Items.Add("item" + AppNames[i].ToString());
}
}
}
}
There is a console version of this program that is functional, but I cannot seem to get it to display the messages in this Winforms Application. Some debugging showed me that the program was getting into the Main async. method upon the button being clicked, but it was not going into the Message Handler despite messages being sent through the service bus.
The pattern that you're using for the Service Bus client and processor isn't going to work in your scenario.
In MainAsync, when you call StartProcessingAsync, the method will return once the processor has started. Execution is then going to reach the finally block, where the processor and client are disposed. At that point, the processor is not running and, therefore, is not receiving messages.
Each time button1_Click runs, you're creating a new set of clients, establishing a new connection to Azure, and then immediately throwing them away.
The processor is intended to be a long-lived type that runs continuously in the background, calling back into your code as messages are available. Likewise, the client is intended to be used as a singleton for the lifetime of your application. I'd suggest reading through the Service Bus "Hello World" sample, which would help to explain some of the types and recommended patterns for use.
In a small project i am working on i have the neccessity for a component to execute a components shutdown code in the same thread that it was initialized in. However unlike in WPF/Winforms/Web the synchronizationcontext which takes care of this does not work.
My guess is that the lack of a synchronization context is the issue that causes the lack of utilization for ConfigureAwait(true).
Does someone know how to properly implement this?
I read this article but could not make any sense of it yet. Perhaps it was too late yesterday.
Minimal Repro:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace ConsoleSyncContext
{
class Program
{
static async Task Main(string[] args)
{
Console.WriteLine($"Thread: {Thread.CurrentThread.ManagedThreadId}");
await SomeBackgroundWorkAsync();
// if this is the same thread as above the question is solved.
Console.WriteLine($"Thread: {Thread.CurrentThread.ManagedThreadId}");
}
private static async Task SomeBackgroundWorkAsync()
{
await Task.Run(() => { });
}
}
}
As you already figured out, console application by default doesn't have synchronization context, so ConfigureAwait has no effect, and continuation after your await SomePageLoad() will run on random thread pool thread. Note that using async main method is essentially equivalent to this:
static async Task AsyncMain() { ... } // your `async Task Main method`
// real Main method generated by compiler
static void RealMain() {
AsyncMain().GetAwaiter().GetResult();
}
In your case you don't need any synchronization context though. What you want is initialize CefSharp on main thread and shutdown CefSharp on main thread. So instead of using async Main - you can do the same as above, but initialize and shutdown Cef outside of async method:
static void Main(string[] args) {
// starting with thread 1
Cef.Initialize(new CefSettings());
try {
AsyncMain(args).GetAwaiter().GetResult();
}
finally {
// we are on main thread here
Cef.Shutdown();
}
}
static async Task AsyncMain(string[] args) {
await SomePageLoad(); // more stuff here
}
Edit: if you insist on using synchronization context then it can be done, but will add a lot of complications for nothing. Out goal is create synchronization context which will run all actions on the same thread. This case be done with simple actions queue, here is basic implementation (don't use it in production, provided as an example only, no exception handling and so on):
class CustomSyncContext : SynchronizationContext {
private readonly BlockingCollection<WorkItem> _queue = new BlockingCollection<WorkItem>(new ConcurrentQueue<WorkItem>());
private readonly Thread _thread;
public CustomSyncContext() {
// start new thread which will handle all callbacks
_thread = new Thread(() => {
// set outselves as current sync context for this thread
SynchronizationContext.SetSynchronizationContext(this);
foreach (var item in _queue.GetConsumingEnumerable()) {
try {
// execute action
item.Action();
}
finally {
// if this action is synchronous, signal the caller
item.Signal?.Set();
}
}
});
_thread.Start();
}
public override void Post(SendOrPostCallback d, object state) {
// Post means acion is asynchronous, just queue and forget
_queue.Add(new WorkItem(() => d(state), null));
}
public override void Send(SendOrPostCallback d, object state) {
// Send means action is synchronous, wait on a single until our thread executes it
using (var signal = new ManualResetEvent(false)) {
_queue.Add(new WorkItem(() => d(state), signal));
signal.WaitOne();
}
}
public void Shutdown() {
// signal thread that no more callbacks are expected
_queue.CompleteAdding();
}
public void WaitForShutdown() {
_thread.Join();
}
private class WorkItem {
public WorkItem(Action action, ManualResetEvent signal) {
Action = action;
Signal = signal;
}
public Action Action { get; }
public ManualResetEvent Signal { get; }
}
}
And your code then becomes:
var ctx = new CustomSyncContext();
ctx.Send(async (_) => {
try {
// starting with thread 1
Cef.Initialize(new CefSettings());
// this method returns on thread 4
await SomePageLoad();
}
finally {
Cef.Shutdown();
// signal the context we are done, so that main thread can unblock
ctx.Shutdown();
Console.WriteLine("done");
}
}, null);
ctx.WaitForShutdown();
Now your code runs on custom synchronization context, and continuation after await SomePageLoad(); will be posted to that synchronization context and executed by our thread (the same thread which inited CefSharp) (no ConfigureAwait(true) is needed, as it's already true by default). Note that we achieved nothing useful - we have one more thread, and our main thread is still blocked waiting for the whole operation to complete (there is no sensible way around that).
Edit 2: here is variation which does not require separate thread, but is not much better:
class CustomSyncContext : SynchronizationContext {
private readonly BlockingCollection<WorkItem> _queue = new BlockingCollection<WorkItem>(new ConcurrentQueue<WorkItem>());
public override void Post(SendOrPostCallback d, object state) {
// Post means acion is asynchronous, just queue and forget
_queue.Add(new WorkItem(() => d(state), null));
}
public override void Send(SendOrPostCallback d, object state) {
// Send means action is synchronous, wait on a single until our thread executes it
using (var signal = new ManualResetEvent(false)) {
_queue.Add(new WorkItem(() => d(state), signal));
signal.WaitOne();
}
}
public void Shutdown() {
// signal thread that no more callbacks are expected
_queue.CompleteAdding();
}
public void Start() {
// now we run the loop on main thread
foreach (var item in _queue.GetConsumingEnumerable()) {
try {
// execute action
item.Action();
}
finally {
// if this action is synchronous, signal the caller
item.Signal?.Set();
}
}
}
private class WorkItem {
public WorkItem(Action action, ManualResetEvent signal) {
Action = action;
Signal = signal;
}
public Action Action { get; }
public ManualResetEvent Signal { get; }
}
}
static async Task Main(string[] args) {
var ctx = new CustomSyncContext();
// set sync context
SynchronizationContext.SetSynchronizationContext(ctx);
// now execute our async stuff
var task = DoStuff().ContinueWith(x => ctx.Shutdown());
// now run the loop of sync context on the main thread.
// but, how do we know when to stop? Something from outside should singal that
// in the case signal is completion of DoStuff task
// note that most of the time main thread is still blocked while waiting for items in queue
ctx.Start();
}
private static async Task DoStuff() {
try {
// starting with thread 1
Cef.Initialize(new CefSettings());
// this method returns on thread 4
await SomePageLoad();
}
finally {
Cef.Shutdown();
// signal the context we are done, so that main thread can unblock
Console.WriteLine("done");
}
}
Your problem is indeed the lack of a Synchronisation context. You can't use ConfigureAwait(true) as this implies that you need to return to the original scheduler/context which does not exist.
Custom implementation
A very simple implementation that ought to do the trick is the one found here. Basically two steps.
Implement a custom synchronization context
public class CustomSynchronizationContext : SynchronizationContext
{
public override void Post(SendOrPostCallback action, object state)
{
SendOrPostCallback actionWrap = (object state2) =>
{
SynchronizationContext.SetSynchronizationContext(new CustomSynchronizationContext());
action.Invoke(state2);
};
var callback = new WaitCallback(actionWrap.Invoke);
ThreadPool.QueueUserWorkItem(callback, state);
}
public override SynchronizationContext CreateCopy()
{
return new CustomSynchronizationContext();
}
public override void Send(SendOrPostCallback d, object state)
{
base.Send(d, state);
}
public override void OperationStarted()
{
base.OperationStarted();
}
public override void OperationCompleted()
{
base.OperationCompleted();
}
}
Initaliaze it and use it
static void Main()
{
var context = new CustomSynchronizationContext();
SynchronizationContext.SetSynchronizationContext(context);
AsyncEx library
You can also use the AsyncEx library
private static void Main(string[] args)
{
AsyncContext.Run(() => AsyncMethod(args));
}
static async void AsyncMethod(string[] args)
{
await something().ConfigureAwait(true);
}
I have an event based scenario in iis application(c#).I want to start threads and in the background when event is come then threads do their job.My problem is in some scenario only one thread is working.When I check threads status all threads looks startted.As I understand if one thread finish work and sleep the other thread is sleep and waiting too.How can I solve this multithreading issue?I want all my threads are working in the background.When event is come then each related thread do it is job and sleep again with independent from each other.
In the application begins call ProcessMyMessages() at once.And All applicaiton lifecyle threads are working in background
.................
public void ProcessMyMessages() {
Dictionary<string,string> messages=new Dictionary<string,string>();
messages.Add("message1","value1");
messages.Add("message2","value2");
foreach(var item in messages)
{
StartMessageExecuter(message,value);
}
}
public void StartMessageExecuter(string message,string value)
{
Thread t=new Thread(()=>ExecuteMessage(message,value));
t.IsBackground=true;
t.Start();
}
internal void ExecuteMessage(string message,string value)
{
ThreadMessage m=new ThreadMessage();
m.ProcessMessage(message,value);
}
.....
public class ThreadMessage
{
private AutoResetEvent receiveCompleteEvent=null;
public void ProcessMessage(string message,string value)
{
using(receiveCompleteEvent=new AutoResetEvent(false))
{
DoJob(message,value);
receiveCompleteEvent.WaitOne();
}
}
public DoJob(string message,string value)
{
//Here is event base if message is comming then executed
}
}
I have a service running some different tasks in a loop until the service is stopped.
However one of these tasks i calling a web service and this call can take several minutes to complete. I want to be able to stop the service instantly, 'cancelling' the web service call without calling Thread.Abort because that causes some strange behavior even if the only thing the thread is doing is calling this web service method.
How can i cancel or break from a synchronous method call (if it's even possible)?
Or should I try a different approach?
I have tried to use the AutoResetEvent and then calling Thread.Abort which is working fine in the below code sample, but when implementing this solution in the actual service I get some unexpected behavior probably because of what's going on in the external libraries I'm using.
AutoResetEvent and Thread.Abort:
class Program
{
static void Main(string[] args)
{
MainProgram p = new MainProgram();
p.Start();
var key = Console.ReadKey();
if (key.Key == ConsoleKey.Q)
p.Stop();
}
}
class MainProgram
{
private Thread workerThread;
private Thread webServiceCallerThread;
private volatile bool doWork;
public void Start()
{
workerThread = new Thread(() => DoWork());
doWork = true;
workerThread.Start();
}
public void Stop()
{
doWork = false;
webServiceCallerThread.Abort();
}
private void DoWork()
{
try
{
while (doWork)
{
AutoResetEvent are = new AutoResetEvent(false);
WebServiceCaller caller = new WebServiceCaller(are);
webServiceCallerThread = new Thread(() => caller.TimeConsumingMethod());
webServiceCallerThread.Start();
// Wait for the WebServiceCaller.TimeConsumingMethod to finish
WaitHandle.WaitAll(new[] { are });
// If doWork has been signalled to stop
if (!doWork)
break;
// All good - continue
Console.WriteLine(caller.Result);
}
}
catch (Exception e)
{
Console.Write(e);
}
}
}
class WebServiceCaller
{
private AutoResetEvent ev;
private int result;
public int Result
{
get { return result; }
}
public WebServiceCaller(AutoResetEvent ev)
{
this.ev = ev;
}
public void TimeConsumingMethod()
{
try
{
// Simulates a method running for 1 minute
Thread.Sleep(60000);
result = 1;
ev.Set();
}
catch (ThreadAbortException e)
{
ev.Set();
result = -1;
Console.WriteLine(e);
}
}
}
Can someone suggest a solution to this issue?
Try this
public void Start()
{
workerThread = new Thread(() => DoWork());
doWork = true;
workerThread.IsBackground = true;
workerThread.Start();
}
A thread is either a background thread or a foreground thread.
Background threads are identical to foreground threads, except that
background threads do not prevent a process from terminating. Once all
foreground threads belonging to a process have terminated, the common
language runtime ends the process. Any remaining background threads
are stopped and do not complete.
For more details see http://msdn.microsoft.com/en-us/library/system.threading.thread.isbackground.aspx
The solution is really this simple: Don't make calls that block for several minutes unless you want to block for several minutes. If there is no way to do a particular thing without blocking, potentially for several minutes, complain loudly to whoever wrote the code that imposes that painful requirement (or fix it yourself, if possible).
Once you've made the call, it's too late. You're committed. If the function you are calling doesn't provide a safe way to abort it, then there's no safe way.
As all you want to do is make one an asynchonrous web service call at a time and on each response make another call you can dispense with the worker thread and simply make an aynchronous call, register a callback and make another async call from the callback:
class Program
{
private static WebServiceCaller.TCMDelegate _wscDelegate;
private static readonly WebServiceCaller _wsCaller = new WebServiceCaller();
static void Main(string[] args)
{
_wscDelegate = _wsCaller.TimeConsumingMethod;
MakeWSCallAsync();
Console.WriteLine("Enter Q to quit");
while (Console.ReadLine().ToUpper().Trim()!="Q"){}
}
public static void MakeWSCallAsync()
{
_wscDelegate.BeginInvoke(OnWSCallComplete, null);
}
public static void OnWSCallComplete(IAsyncResult ar)
{
Console.WriteLine("Result {0}", _wscDelegate.EndInvoke(ar));
MakeWSCallAsync();
}
}
class WebServiceCaller
{
public delegate int TCMDelegate();
public int TimeConsumingMethod()
{
try
{
// Simulates a method running for 1 minute
Thread.Sleep(1000);
return 1;
}
catch (ThreadAbortException e)
{
return -1;
}
}
}
No blocking (well, the console thread is blocking on ReadLine()) and no windows kernal mode sync objects (AutoResetEvent) which are expensive.
I'm having a small background thread which runs for the applications lifetime - however when the application is shutdown, the thread should exit gracefully.
The problem is that the thread runs some code at an interval of 15 minutes - which means it sleeps ALOT.
Now in order to get it out of sleep, I toss an interrupt at it - my question is however, if there's a better approach to this, since interrupts generate ThreadInterruptedException.
Here's the gist of my code (somewhat pseudo):
public class BackgroundUpdater : IDisposable
{
private Thread myThread;
private const int intervalTime = 900000; // 15 minutes
public void Dispose()
{
myThread.Interrupt();
}
public void Start()
{
myThread = new Thread(ThreadedWork);
myThread.IsBackground = true; // To ensure against app waiting for thread to exit
myThread.Priority = ThreadPriority.BelowNormal;
myThread.Start();
}
private void ThreadedWork()
{
try
{
while (true)
{
Thread.Sleep(900000); // 15 minutes
DoWork();
}
}
catch (ThreadInterruptedException)
{
}
}
}
There's absolutely a better way - either use Monitor.Wait/Pulse instead of Sleep/Interrupt, or use an Auto/ManualResetEvent. (You'd probably want a ManualResetEvent in this case.)
Personally I'm a Wait/Pulse fan, probably due to it being like Java's wait()/notify() mechanism. However, there are definitely times where reset events are more useful.
Your code would look something like this:
private readonly object padlock = new object();
private volatile bool stopping = false;
public void Stop() // Could make this Dispose if you want
{
stopping = true;
lock (padlock)
{
Monitor.Pulse(padlock);
}
}
private void ThreadedWork()
{
while (!stopping)
{
DoWork();
lock (padlock)
{
Monitor.Wait(padlock, TimeSpan.FromMinutes(15));
}
}
}
For more details, see my threading tutorial, in particular the pages on deadlocks, waiting and pulsing, the page on wait handles. Joe Albahari also has a tutorial which covers the same topics and compares them.
I haven't looked in detail yet, but I wouldn't be surprised if Parallel Extensions also had some functionality to make this easier.
You could use an Event to Check if the Process should end like this:
var eventX = new AutoResetEvent(false);
while (true)
{
if(eventX.WaitOne(900000, false))
{
break;
}
DoWork();
}
There is CancellationTokenSource class in .NET 4 and later which simplifies this task a bit.
private readonly CancellationTokenSource cancellationTokenSource =
new CancellationTokenSource();
private void Run()
{
while (!cancellationTokenSource.IsCancellationRequested)
{
DoWork();
cancellationTokenSource.Token.WaitHandle.WaitOne(
TimeSpan.FromMinutes(15));
}
}
public void Stop()
{
cancellationTokenSource.Cancel();
}
Don't forget that CancellationTokenSource is disposable, so make sure you dispose it properly.
One method might be to add a cancel event or delegate that the thread will subscribe to. When the cancel event is invoke, the thread can stop itself.
I absolutely like Jon Skeets answer. However, this might be a bit easier to understand and should also work:
public class BackgroundTask : IDisposable
{
private readonly CancellationTokenSource cancellationTokenSource;
private bool stop;
public BackgroundTask()
{
this.cancellationTokenSource = new CancellationTokenSource();
this.stop = false;
}
public void Stop()
{
this.stop = true;
this.cancellationTokenSource.Cancel();
}
public void Dispose()
{
this.cancellationTokenSource.Dispose();
}
private void ThreadedWork(object state)
{
using (var syncHandle = new ManualResetEventSlim())
{
while (!this.stop)
{
syncHandle.Wait(TimeSpan.FromMinutes(15), this.cancellationTokenSource.Token);
if (!this.cancellationTokenSource.IsCancellationRequested)
{
// DoWork();
}
}
}
}
}
Or, including waiting for the background task to actually have stopped (in this case, Dispose must be invoked by other thread than the one the background thread is running on, and of course this is not perfect code, it requires the worker thread to actually have started):
using System;
using System.Threading;
public class BackgroundTask : IDisposable
{
private readonly ManualResetEventSlim threadedWorkEndSyncHandle;
private readonly CancellationTokenSource cancellationTokenSource;
private bool stop;
public BackgroundTask()
{
this.threadedWorkEndSyncHandle = new ManualResetEventSlim();
this.cancellationTokenSource = new CancellationTokenSource();
this.stop = false;
}
public void Dispose()
{
this.stop = true;
this.cancellationTokenSource.Cancel();
this.threadedWorkEndSyncHandle.Wait();
this.cancellationTokenSource.Dispose();
this.threadedWorkEndSyncHandle.Dispose();
}
private void ThreadedWork(object state)
{
try
{
using (var syncHandle = new ManualResetEventSlim())
{
while (!this.stop)
{
syncHandle.Wait(TimeSpan.FromMinutes(15), this.cancellationTokenSource.Token);
if (!this.cancellationTokenSource.IsCancellationRequested)
{
// DoWork();
}
}
}
}
finally
{
this.threadedWorkEndSyncHandle.Set();
}
}
}
If you see any flaws and disadvantages over Jon Skeets solution i'd like to hear them as i always enjoy learning ;-)
I guess this is slower and uses more memory and should thus not be used in a large scale and short timeframe. Any other?