I have 4 classes that perform different actions, each of the 2 tasks should run syncronously.
My current implementation works fine, but sometimes there are strange lags and some of the processors do not run. This could be the threads issue I suppose.
I would like to refactor the code to use Hangfire library or any other way to keep the program working correctly. I'm not sure how to property do that and would appreciate any help.
public void Run()
{
var processors1 = new CommonProcessor[] { new AProcessor(), new BProcessor() };
//AProcessor should be first!
var processors2 = new CommonProcessor[] { new CProcessor(), new DProcessor() };
//CProcessor should be first!
Task task1 = Task.Run(() => RunSyncProcess(processors1);
Task task2 = Task.Run(() => RunSyncProcess(processors2);
Task.WaitAll(task1, task2);
}
private void RunSyncProcess(CommonProcessor[] processors)
{
while (true)
{
foreach (var processor in processors)
{
// do some job
}
Thread.Sleep(frequency);
}
}
You are using Tasks the wrong way. Tasks are supposed to be non blocking or short term lived items.
Basically what happens here is that you launch Tasks which never end and never release their thread. This will result in blocking some threads of the ThreadPool.
There are multiple way to change things:
1) Non blocking tasks:
public void Run()
{
var processors1 = new CommonProcessor[] { new AProcessor(), new BProcessor() };
//AProcessor should be first!
var processors2 = new CommonProcessor[] { new CProcessor(), new DProcessor() };
//CProcessor should be first!
Task task1 = RunSyncProcess(processors1);
Task task2 = RunSyncProcess(processors2);
Task.WhenAll(task1, task2);
}
private async Task RunSyncProcess(CommonProcessor[] processors)
{
while (true)
{
foreach (var processor in processors)
{
// do some job
}
await Task.Delay(TimeSpan.FromMilliseconds(frequency));//will free threadpool while waiting
}
}
2) Using blocking threads but without impacting threadpool:
public void Run()
{
var processors1 = new CommonProcessor[] { new AProcessor(), new BProcessor() };
//AProcessor should be first!
var processors2 = new CommonProcessor[] { new CProcessor(), new DProcessor() };
//CProcessor should be first!
Thread t1 = new Thread(() => RunSyncProcess(processors1));
t1.Start();
Thread t2 = new Thread(() => RunSyncProcess(processors1));
t2.Start();
t1.Join();
t2.Join();
}
private void RunSyncProcess(CommonProcessor[] processors)
{
while (true)
{
foreach (var processor in processors)
{
// do some job
}
Thread.Sleep(frequency);
}
}
Hangfire is primarily used for fire and forget tasks where you can queue, schedule and requeue jobs, if thats what you want to achieve you can install via nuget and then use the following syntax
BackgroundJob.Enqueue(() => RunSyncProcess(processors1));
so in order to refactor your code you would need to decide whether you want to schedule a job or whether you want to wait for a successful completion of a previous task prior to waiting for a new task, really depends on what you want to achieve.
public void Run()
{
var processors1 = new CommonProcessor[] { new AProcessor(), new BProcessor() };
//AProcessor should be first!
var processors2 = new CommonProcessor[] { new CProcessor(), new DProcessor() };
//CProcessor should be first!
BackgroundJob.Enqueue(() => RunSyncProcess(processors1));
BackgroundJob.Enqueue(() => RunSyncProcess(processors2));
}
public void RunSyncProcess(CommonProcessor[] processors)
{
while (true)
{
foreach (var processor in processors)
{
// do some job
}
}
}
You won't have to await all as these will kick of behind the scene and your UI will be responsive. Remember that the methods will need to be public when you want to use hangfire.
Related
I have to consume from a Kafka topic, get the message and do some json clean and filter job, then I need to produce the new message to another Kafka topic, my code is like this:
public static YamlMappingNode configs;
public static void Main(string[] args)
{
using (var reader = new StreamReader(Path.Combine(Directory.GetCurrentDirectory(), ".gitlab-ci.yml")))
{
var yaml = new YamlStream();
yaml.Load(reader);
//find variables
configs = (YamlMappingNode)yaml.Documents[0].RootNode;
configs = (YamlMappingNode)configs.Children.Where(k => k.Key.ToString() == "variables")?.FirstOrDefault().Value;
}
CancellationTokenSource cts = new CancellationTokenSource();
Console.CancelKeyPress += (_, e) => {
e.Cancel = true; // prevent the process from terminating.
cts.Cancel();
};
Run_ManualAssign(configs, cts.Token);
}
public static async void Run_ManualAssign(YamlMappingNode configs, CancellationToken cancellationToken)
{
var brokerList = configs.Where(k => k.Key.ToString() == "kfk_broker")?.FirstOrDefault().Value.ToString();
var topics = configs.Where(k => k.Key.ToString() == "input_kfk_topic")?.FirstOrDefault().Value.ToString();
var config = new ConsumerConfig
{
// the group.id property must be specified when creating a consumer, even
// if you do not intend to use any consumer group functionality.
GroupId = new Guid().ToString(),
BootstrapServers = brokerList,
// partition offsets can be committed to a group even by consumers not
// subscribed to the group. in this example, auto commit is disabled
// to prevent this from occurring.
EnableAutoCommit = true
};
using (var consumer =
new ConsumerBuilder<Ignore, string>(config)
.SetErrorHandler((_, e) => Console.WriteLine($"Error: {e.Reason}"))
.Build())
{
//consumer.Assign(topics.Select(topic => new TopicPartitionOffset(topic, 0, Offset.Beginning)).ToList());
consumer.Assign(new TopicPartitionOffset(topics, 0, Offset.End));
//var producer = new ProducerBuilder<Null, string>(config).Build();
try
{
while (true)
{
try
{
var consumeResult = consumer.Consume(cancellationToken);
/// Note: End of partition notification has not been enabled, so
/// it is guaranteed that the ConsumeResult instance corresponds
/// to a Message, and not a PartitionEOF event.
//filter message
var result = ReadMessage(configs, consumeResult.Message.Value);
//send to kafka topic
await Run_ProducerAsync(configs, result);
}
catch (ConsumeException e)
{
Console.WriteLine($"Consume error: {e.Error.Reason}");
}
}
}
catch (OperationCanceledException)
{
Console.WriteLine("Closing consumer.");
consumer.Close();
}
}
}
#endregion
#region Run_Producer
public static async Task Run_ProducerAsync(YamlMappingNode configs, string message)
{
var brokerList = configs.Where(k => k.Key.ToString() == "kfk_broker")?.FirstOrDefault().Value.ToString();
var topicName = configs.Where(k => k.Key.ToString() == "target_kafka_topic")?.FirstOrDefault().Value.ToString();
var config = new ProducerConfig {
BootstrapServers = brokerList,
};
using (var producer = new ProducerBuilder<Null, string>(config).Build())
{
try
{
/// Note: Awaiting the asynchronous produce request below prevents flow of execution
/// from proceeding until the acknowledgement from the broker is received (at the
/// expense of low throughput).
var deliveryReport = await producer.ProduceAsync(topicName, new Message<Null, string> { Value = message });
producer.Flush(TimeSpan.FromSeconds(10));
Console.WriteLine($"delivered to: {deliveryReport.TopicPartitionOffset}");
}
catch (ProduceException<string, string> e)
{
Console.WriteLine($"failed to deliver message: {e.Message} [{e.Error.Code}]");
}
}
}
#endregion
Am I doing something wrong here? The program existed immediately when executing var deliveryReport = await producer.ProduceAsync(topicName, new Message<Null, string> { Value = message });, no error message, no error code.
In the meanwhile I used Python and config the same for Producer, it works well.
Run_ManualAssign(configs, cts.Token);
For this line in the Main function, you are calling async without await in a sync function. Thus the program exit immediately after this invoke started (not finished as it is async)
You could have 2 options
Use async Main function and add await in front of this invoke.
https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-7.1/async-main
If you really want to call async function in sync function
Run_ManualAssign(configs, ts.Token).ConfigureAwait(false).GetAwaiter().GetResult();
I solved this problem but I don't know why actually. I opened an issue here.
First of all, sorry because I am so new at C# and I decided to make this question because I have been choked in this for hours.
I have an GUI that works with Google Cloud Speech services and make a Speech-to-Text operation. I share with you the whole method that runs when a button is clicked:
private async Task<object> StreamingMicRecognizeAsync(int seconds)
{
if (NAudio.Wave.WaveIn.DeviceCount < 1)
{
Console.WriteLine("No microphone!");
return -1;
}
GoogleCredential googleCredential;
using (Stream m = new FileStream(#"..\..\credentials.json", FileMode.Open))
googleCredential = GoogleCredential.FromStream(m);
var channel = new Grpc.Core.Channel(SpeechClient.DefaultEndpoint.Host,
googleCredential.ToChannelCredentials());
var speech = SpeechClient.Create(channel);
var streamingCall = speech.StreamingRecognize();
// Write the initial request with the config.
await streamingCall.WriteAsync(
new StreamingRecognizeRequest()
{
StreamingConfig = new StreamingRecognitionConfig()
{
Config = new RecognitionConfig()
{
Encoding =
RecognitionConfig.Types.AudioEncoding.Linear16,
SampleRateHertz = 48000,
LanguageCode = "es-ES",
},
InterimResults = true,
}
});
// Read from the microphone and stream to API.
object writeLock = new object();
bool writeMore = true;
var waveIn = new NAudio.Wave.WaveInEvent();
waveIn.DeviceNumber = 0;
waveIn.WaveFormat = new NAudio.Wave.WaveFormat(48000, 1);
waveIn.DataAvailable +=
(object sender, NAudio.Wave.WaveInEventArgs args) =>
{
lock (writeLock)
{
if (!writeMore) return;
streamingCall.WriteAsync(
new StreamingRecognizeRequest()
{
AudioContent = Google.Protobuf.ByteString
.CopyFrom(args.Buffer, 0, args.BytesRecorded)
}).Wait();
}
};
// Print responses as they arrive.
Task printResponses = Task.Run(async () =>
{
while (await streamingCall.ResponseStream.MoveNext(default(CancellationToken)))
{
foreach (var result in streamingCall.ResponseStream
.Current.Results)
{
foreach (var alternative in result.Alternatives)
{
Console.WriteLine(alternative.Transcript);
//Textbox1.Text = alternative.Transcript;
}
}
}
});
waveIn.StartRecording();
Console.WriteLine("Speak now.");
Result_Tone.Text = "Speak now:\n\n";
await Task.Delay(TimeSpan.FromSeconds(seconds));
// Stop recording and shut down.
waveIn.StopRecording();
lock (writeLock) writeMore = false;
await streamingCall.WriteCompleteAsync();
await printResponses;
return 0;
}
My problem is that I want to update the content of the Textbox1control but it doesn´t work. It writes perfectly the output into the console with the line Console.WriteLine(alternative.Transcript); but not into my textbox.
If someone could help I would appreciate so much his help.
The problem is that you're using Task.Run, which means your code will be running on a thread-pool thread.
Instead of calling Task.Run(), just move that code into a separate async method:
async Task DisplayResponses(IAsyncEnumerator<StreamingRecognizeResponse> responses)
{
while (await responses.MoveNext(default(CancellationToken)))
{
foreach (var result in responses.Current.Results)
{
foreach (var alternative in result.Alternatives)
{
Textbox1.Text = alternative.Transcript;
}
}
}
}
Then call that method directly (without Task.Run) from code that's already on the UI thread (e.g. an event handler).
The async machinery will make sure that after the await expression, you're back on the UI thread (the same synchronization context). So the assignment to the Text property will occur on the UI thread, and all should be well.
For example:
// This would be registered as the event handler for a button
void HandleButtonClick(object sender, EventArgs e)
{
var stream = client.StreamingRecognize();
// Send the initial config request
await stream.WriteAsync(...);
// Presumably you want to send audio data...
StartSendingAudioData(stream);
await DisplayResponses(stream.ResponseStream);
}
Tasks run on seperate threads, so you must Invoke an action that will be performed on the control's thread
Textbox1.Invoke(new Action(() =>
{
Textbox1.Text= "";
}));
Edit: For WPF, I believe the equivalent is
Textbox1.Dispatcher.Invoke(new Action(() =>
{
Textbox1.Text= "";
}));
have you tried using Dispatcher.InvokeASync()?
await Dispatcher.InvokeAsync(() => {while (await streamingCall.ResponseStream.MoveNext(default(CancellationToken)))
{
foreach (var result in streamingCall.ResponseStream
.Current.Results)
{
foreach (var alternative in result.Alternatives)
{
Textbox1.Text = alternative.Transcript;
}
}
}});
I have some kind of work manager in c# which will receive tasks to do and it will execute them. Task will be arriving from different threads, but they must be executed only one at the same time in the order they were received.
I don't want a while loop, which will be running all the time, checking if they are new tasks in the queue. Is there a built-in queue or an easy way to implement a queue which will wait for tasks and execute them synchronously without busy-waiting?
As per the comments you should look into ConcurrentQueue but also BlockingCollection and use GetConsumingEnumerable() instead of your unwanted WHILE loop
BlockingCollection<YourClass> _collection =
new BlockingCollection<YourClass>(new ConcurrentQueue<YourClass>());
_collection.Add() can be called from multiple threads
On a separate thread you can use
foreach (var message in _collection.GetConsumingEnumerable())
{}
You can use a SemaphoreSlim (https://msdn.microsoft.com/en-us/library/system.threading.semaphoreslim(v=vs.110).aspx) and a ConcurrentQueue
Example:
private delegate void TaskBody();
private class TaskManager
{
private ConcurrentQueue<TaskBody>
TaskBodyQueue = new ConcurrentQueue<TaskBody>();
private readonly SemaphoreSlim
TaskBodySemaphoreSlim = new SemaphoreSlim(1, 1);
public async void Enqueue(TaskBody body)
{
TaskBodyQueue.Enqueue(body);
await TaskBodySemaphoreSlim.WaitAsync();
Console.WriteLine($"Cycle ...");
if (TaskBodyQueue.TryDequeue(out body) == false) {
throw new InvalidProgramException($"TaskBodyQueue is empty!");
}
body();
Console.WriteLine($"Cycle ... done ({TaskBodyQueue.Count} left)");
TaskBodySemaphoreSlim.Release();
}
}
public static void Main(string[] args)
{
var random = new Random();
var tm = new TaskManager();
Parallel.ForEach(Enumerable.Range(0, 30), async number => {
await Task.Delay(100 * number);
tm.Enqueue(delegate {
Console.WriteLine($"Print {number}");
});
});
Task
.Delay(4000)
.Wait();
WaitFor(action: "exit");
}
public static void WaitFor(ConsoleKey consoleKey = ConsoleKey.Escape, string action = "continue")
{
Console.Write($"Press {consoleKey} to {action} ...");
var consoleKeyInfo = default(ConsoleKeyInfo);
do {
consoleKeyInfo = Console.ReadKey(true);
}
while (Equals(consoleKeyInfo.Key, consoleKey) == false);
Console.WriteLine();
}
I am starting multiple tasks with a parallel foreach loop.
Now I want on click of a stop button all task to be stopped. How can I do this?
Here is my code:
tasks2 = new List<Task>();
Parallel.ForEach<RssToProcess>(RssFeeds, rssFeed =>
tasks2.Add(Task.Factory.StartNew(() =>
{
string urlss = rssFeed.RssUrl;
nourl += urlss + System.Environment.NewLine;
RssReader rs = new RssReader();
rs.FeedsourceLoaded += new EventHandler(rs_FeedsourceLoaded);
rs.ItemAdded += new EventHandler(rs_ItemAdded);
rs.AllItemAdded += new EventHandler(rs_AllItemAdded);
rs.RssReaderrssitemsCountgeta += new EventHandler(rs_RssReaderrssitemsCountgeta);
rs.RdfMode = true;
RssFeed f = rs.Retrieve(rssFeed.RssUrl);
})));
You should probably be passing a CancellationToken:
http://msdn.microsoft.com/en-us/library/dd537607.aspx
Create a CancellationToken:
var tokenSource = new CancellationTokenSource();
var token = tokenSource.Token;
And pass it to your tasks:
Task.Factory.StartNew(() =>
{
... Do Work ...
}, token);
When you want to cancel, call
tokenSource.Cancel();
However, note that task cancellation is cooperative.
Within your tasks, you will need to use
token.ThrowIfCancellationRequested(); // To Abort immediately.
... OR ...
if (token.IsCancellationRequested)
{
// Exit your task manually.
}
I want to schedule a task to start in x ms and be able to cancel it before it starts (or just at the beginning of the task).
The first attempt would be something like
var _cancelationTokenSource = new CancellationTokenSource();
var token = _cancelationTokenSource.Token;
Task.Factory.StartNew(() =>
{
token.ThrowIfCancellationRequested();
Thread.Sleep(100);
token.ThrowIfCancellationRequested();
}).ContinueWith(t =>
{
token.ThrowIfCancellationRequested();
DoWork();
token.ThrowIfCancellationRequested();
}, token);
But I feel like there should be a better way, as this would use up a thread while in the sleep, during which it could be canceled.
What are my other options?
Like Damien_The_Unbeliever mentioned, the Async CTP includes Task.Delay. Fortunately, we have Reflector:
public static class TaskEx
{
static readonly Task _sPreCompletedTask = GetCompletedTask();
static readonly Task _sPreCanceledTask = GetPreCanceledTask();
public static Task Delay(int dueTimeMs, CancellationToken cancellationToken)
{
if (dueTimeMs < -1)
throw new ArgumentOutOfRangeException("dueTimeMs", "Invalid due time");
if (cancellationToken.IsCancellationRequested)
return _sPreCanceledTask;
if (dueTimeMs == 0)
return _sPreCompletedTask;
var tcs = new TaskCompletionSource<object>();
var ctr = new CancellationTokenRegistration();
var timer = new Timer(delegate(object self)
{
ctr.Dispose();
((Timer)self).Dispose();
tcs.TrySetResult(null);
});
if (cancellationToken.CanBeCanceled)
ctr = cancellationToken.Register(delegate
{
timer.Dispose();
tcs.TrySetCanceled();
});
timer.Change(dueTimeMs, -1);
return tcs.Task;
}
private static Task GetPreCanceledTask()
{
var source = new TaskCompletionSource<object>();
source.TrySetCanceled();
return source.Task;
}
private static Task GetCompletedTask()
{
var source = new TaskCompletionSource<object>();
source.TrySetResult(null);
return source.Task;
}
}
Since .NET 4.5 has now been released, there's a very simple built-in way to delay a task: just use Task.Delay(). behind the scenes, it uses the implementation that ohadsc decompiled.
The correct answer in the future will probably be Task.Delay. However, that's currently only available through the Async CTP (and in the CTP, it's on TaskEx rather than Task).
Unfortunately, because it's only in CTP, there aren't many good links to documentation for it either.
Look at the TaskFactoryExtensions_Delayed in "Parallel Programming with .NET 4 Samples".
I haven't tested this, but here is a first-pass at wrapper methods to create an initial 'Delay' Task or to continue after a Delay. If you find issues, feel free to correct.
public static Task StartDelayTask(int delay, CancellationToken token)
{
var source = new TaskCompletionSource<Object>();
Timer timer = null;
timer = new Timer(s =>
{
source.TrySetResult(null);
timer.Dispose();
}, null, delay, -1);
token.Register(() => source.TrySetCanceled());
return source.Task;
}
public static Task ContinueAfterDelay
(this Task task,
int delay, Action<Task> continuation,
CancellationToken token)
{
var source = new TaskCompletionSource<Object>();
Timer timer = null;
var startTimer = new Action<Task>(t =>
{
timer = new Timer(s =>
{
source.TrySetResult(null);
timer.Dispose();
},null,delay,-1);
});
task.ContinueWith
(startTimer,
token,
TaskContinuationOptions.OnlyOnRanToCompletion,
TaskScheduler.Current);
token.Register(() => source.TrySetCanceled());
return source.Task.ContinueWith(continuation, token);
}
You can use Token.WaitHandle.WaitOne(int32 milliseconds) overload method to specify number of milliseconds to wait for your task. But key difference between Thread.Sleep(xxx) and Token.WaitHandle.WaitOne(xxx) that later blocks thread until the time specified elapsed or the token has been canceled.
Here is an example
void Main()
{
var tokenSource = new CancellationTokenSource();
var token = tokenSource.Token;
var task = Task.Factory.StartNew(() =>
{
// wait for 5 seconds or user hit Enter key cancel the task
token.WaitHandle.WaitOne(5000);
token.ThrowIfCancellationRequested();
Console.WriteLine("Task started its work");
});
Console.WriteLine("Press 'Enter' key to cancel your task");
Console.Read();
tokenSource.Cancel();
}