We Keep Coding

How to do a while loop on an async retrieved list?

So I have this Task:
public async Task ProcessNewMessages()
{
while (true)
{
await _exceptionHandler.WithExceptionHandling(async () =>
{
List<Message> newUncompletedNewMessages = await _newUncompletedMessagesJob.Execute();
// Do stuff
}
Thread.Sleep(30000);
}
}
The Execute returns a list of 10 messages and I want to add a while loop in the while true that runs the whole code again as long as the Execute returns me a full list of 10 items, that runs straight after the items are done rather than waiting 30 seconds every time.
The 30 seconds will be in place to keep checking if there are new messages and is ireelevant to the Task of handling them.
So to rephrase my question: How do I rerun code based on the list returned by Execute() having 10 messages?

public async Task ProcessNewMessages()
{
while (true)
{
IEnumerable<MessageToSend> uncompletedNewMessages = new List<MessageToSend>();
do
{
await _exceptionHandler.WithExceptionHandling(async () =>
{
uncompletedNewMessages = await _newUncompletedMessagesJob.Execute();
// DoStuff
});
} while (uncompletedNewMessages.Count() == 10)
await Task.Delay(30000);
}
}
I have an answer.
I read people not liking the Thread.Sleep, but this is a webjob running continuously to check for new messages. Now this doesn't need to run every second, because there might not be new message every second, but they can keep on coming in throughout the day. Let me know your suggestions how to do that differently.
Edit: Thanks for the explanation Jeroen

Using Task.Run in c# code to run the code for a specific duration [duplicate]

This question already has answers here:
Timeout pattern on task-based asynchronous method in C#
(2 answers)
Safely stop long running task
(2 answers)
Closed 2 years ago.
I have a piece of code which does some processing of data, written in c#.
string status;
log.Info("Starting the Process");
StartProcessing(); // takes 10-12 mins usually
var task = Task.Run(() =>
{
do
{
status = GetStatus();
log.Info("Status inside loop : " + status);
} while (status != "Complete")
});
if (task.Wait(TimeSpan.FromMinutes(Convert.ToDouble(15))))
{
log.Info("Processing finished");
}
else
{
log.Info("Process did not complete in 15 mins");
log.Info("Stopping the Process");
StopProcessing(); // this takes 2-3 mins usually.
log.Info("Process stopped");
}
StartProcessing() method actually start some background processing of the data. It doesn't return any value or wait for the method to finish. So we added a do-while loop to check the status of the processing. If the status is complete , then come of the loop and proceed further.
Now the requirement has changed to put a timeout for the processing. If the processing is taking more than 5 mins, then we have to stop the process. So I have wrapped my code in Task.Run as shown above and written a if else condition to check the time.
This doesn't seem to work as expected, because when I run my code, this is log information I'm getting.
Starting the Process
Status inside loop : Processing
Status inside loop : Processing
Status inside loop : Processing
Status inside loop : Processing --> this line is repeated multiple times with in 15 mins.
Process did not complete in 15 mins.
Stopping the Process
Status inside loop : Processing
Status inside loop : Processing
Status inside loop : Processing --> why is it going back to do while after coming out ?
Process stopped
The execution is going back to do while even after coming out. Is there anything wrong am I doing here?
Any suggestions are very helpful.

You need to add code that terminates your background task after the given time. For this, you best introduce a CancellationToken to your processing task. Do something like:
CancellationTokenSource source = new CancellationTokenSource(TimeSpan.FromMinutes(5));
StartProcessing(source.Token);
and then, within your processing task, regularly call token.IsCancellationRequested and abort if this is true.

If you have access to source code of StartProcess, then like #PMF said, passing CancellationToken to the method is the efficient way.
Otherwise, you can create your own cancelable task using TaskCompletionSource and CancellationTokenSource:
Task<bool> ProcessAsync(TimeSpan timeout)
{
var tcs = new TaskCompletionSource<bool>();
var cts = new CancellationTokenSource();
// If task canceled, set to return false
cts.Token.Register(() => tcs.TrySetResult(false));
// Cancel after timeout
cts.CancelAfter(timeout);
// Start process.
StartProcess();
// Start waiting until complete or canceled.
Task.Run(() =>
{
while (GetStatus() != "Complete" && !cts.IsCancellationRequested)
{
}
if (cts.IsCancellationRequested)
{
// Task has been canceled due to timeout.
tcs.TrySetResult(false);
}
else
{
// Task done. Status is "Completed".
tcs.TrySetResult(true);
}
});
return tcs.Task;
}
Then await ProcessAsync:
// Or ProcessAsync(TimeSpan.FromMinutes(5)).Result for sync method
var isCompleted = await ProcessAsync(TimeSpan.FromMinutes(5));
if (!isCompleted)
{
StopProcess();
}

ParallelProcess blocks indefinitely for Task.WaitAll randomly , processes fine most of the time

I have a process that runs periodically every 5 mins. It picks up some records from the db and calls code to Register, in parallel 25 tasks are called, waits for execution of all those parallel tasks to complete and then continues processing the remaining 25 tasks at a time. It runs fine most of the times but after a few weeks or sometimes even a few days , it will just hang. It most likely is hung at wait all where one of the tasks has not completed.
We have not been able to reproduce this in a local environment or even in our test environment in a load scenario. It is a WCF application so the Register method is a service call and we are not sure if a particular call of register task is not returning , then why it isn't - there is no database error or deadlock detected, all queries that execute in that task have a timeout of 30 secs. When it halts and we kill the service , and restart then the same record executes without any issues.
====Snippet of code that executes the 25 tasks at a time -
public void RegisterApplications()
{
throttleCount = 25;
IList<OnlineApplication> onlineApplications = VDService.GetOnlineApplicationsPendingRegistration());
countTotal = onlineApplications.Count;
foreach (OnlineApplication onlineApplication in onlineApplications)
{
object parameters = new StartRegistrationParameters()
{
Application = onlineApplication,
Context = Context.CurrentOperationContext
};
Task task = Task.Factory.StartNew(StartRegisterApplication, parameters);
registrationTasks.Add(task);
onlineApplicationTasks.Add(task.Id, (long)onlineApplication.OnlineApplicationId);
counterTasksForThrottling++;
if (counterTasksForThrottling >= throttleCount)
{
countProcessed += CountTasksCompleted(registrationTasks, messageLog, onlineApplicationTasks);
counterTasksForThrottling = 0;
// initialize again.
registrationTasks.RemoveAll(t => t.Id > 0);
}
}
countProcessed += CountTasksCompleted(registrationTasks, messageLog, onlineApplicationTasks);
}
==========Count Task Completed Method ===============
private int CountTasksCompleted(List<Task> registrationTasks, StringBuilder messageLog, Dictionary<int, long> onlineApplicationTasks)
{
int countTasksCompleted = 0;
try
{
if (messageLog == null)
{
messageLog = new StringBuilder();
}
if (registrationTasks.Count > 0)
{
Task.WaitAll(registrationTasks.ToArray(),);
}
}
// catching this exception as it is thrown after all the tasks have completed and when any one of the tasks within throw an error.
// the task status in the task show us which have completed and which have faulted after this.
catch (AggregateException ex)
{
Logger.Log(ex);
}
finally
{
if (registrationTasks != null && registrationTasks.Count > 0)
{
countTasksCompleted = registrationTasks.Where(task => task.Status == TaskStatus.RanToCompletion).Count();
var failedtasks = registrationTasks.Where(task => task.Status != TaskStatus.RanToCompletion).ToList();
foreach (Task failedTask in failedtasks)
{
messageLog.AppendFormat("Id {0},", onlineApplicationTasks[failedTask.Id]);
}
}
}
return countTasksCompleted;
}

That sounds very much like a Race Condition and those are a pain to debug. If that really is the case, there is little we can help you with other then point out any potential causes of race conditions we can spot - but there is likely more in the rest of it.
Your description does make it sound like this might be a deadlock case. While you say you detected non with the DB access, if your code or the code you are using uses any form of lock statement, these can still happen.
As a start, add some more loging to the mix. For every disticnt step, add a "I am at step X" to the log. This will help you nail down the exact cause.

Proper way to implement multiple functions in windows service

I have a windows service which performs multiple task which i have separated into functions, some will take lets say 5 minutes to complete, while some will take less.
private System.Timers.Timer tim = null;
protected override void OnStart(string[] args)
{
tim = new System.Timers.Timer();
this.tim.Interval = 30000;
this.tim.Elapsed += new System.Timers.ElapsedEventHandler(this.OnTimedEvent_Tick);
tim.Enabled = true;
}
private void OnTimedEvent_Tick(Object source, System.Timers.ElapsedEventArgs e)
{
Task task0 = Task.Factory.StartNew(() => Function1()); // doing some database operations
Task task1 = Task.Factory.StartNew(() => Function2()); // doing some other database operation
Task task10 ......Up to Function10()
Task.WaitAll(task0,task1, task2, task3, task4,task5, task6,task7,task8,task9,task10);
}
Is there a draw back to the above method? if my windows service is to run lets say every 30 seconds. IF there is how do i approach it?

This would work fine in your case since there are only limited number of tasks. For cases where number of tasks that can be created are unknown, do consider using Parallel.ForEach instead. The thing that you need to handle here is exception. Put your code in a try.... catch statement.
try
{
....your code here.....
}
catch (AggregateException e)
{
var x = e.Flatten();
// Log or do what-ever
}

The correct answer would depend on what those tasks are actually doing. If all tasks must be completed prior to restarting any of them, set tim.AutoReset = false. Then after Task.WaitAll() call tim.Start(). This will ensure your wait time is between complete executions of all tasks. Otherwise, if your timer time is smaller than task execution time, you won't see any wait time.

If some of your functions will periodically take longer than timer interval (30 seconds) it will cause threads count to increase without any control. So you will end by using all possible threads which will result in processing delays. If timer interval is shorter than processing time, consider applying pause-resume timer system

How to fix resource leak because of missing EndInvoke call?

I would like to use this solution to call Console.ReadLine() with a timeout:
delegate string ReadLineDelegate();
string ReadLine(int timeoutms)
{
string resultstr = null;
ReadLineDelegate d = Console.ReadLine;
IAsyncResult result = d.BeginInvoke(null, null);
result.AsyncWaitHandle.WaitOne(timeoutms);//timeout e.g. 15000 for 15 secs
if (result.IsCompleted)
{
resultstr = d.EndInvoke(result);
Console.WriteLine("Read: " + resultstr);
}
else
{
Console.WriteLine("Timed out!");
// Bug? resource leak? No d.EndInvoke(), which blocks until Console.ReadLine() returns
}
result.AsyncWaitHandle.Close();
return resultstr;
}
but commenters warned:
every ReadLine you call sits there waiting for input.
If you call it 100 times, it creates 100 threads
which don't all go away until you hit Enter 100 times!
...especially because I want to call this repeatedly in a forever-loop.
I understand that every BeginInvoke() needs a EndInvoke() but I don't want a blocking EndInvoke call in the else branch. Somehow we need to abort the running Console.ReadLine() call rather than let it run to completion, because it may never complete.
So all this (complex) code helped me to get Console.ReadLine to return at a timeout, but does not end the Console.ReadLine to quit or otherwise go away.
How can we make this to work correctly, without running into resource leaks?
NB: I added the AsyncWaitHandle.Close() as advised by MS Calling Sync calls asynchronously

After reading a lot of comments on several similar questions, as mentioned, I come to believe there is no real solution here. The Microsoft way with Begin/EndInvoke is
rather complex, and:
not adequate
A more straightforward method is to run the synchronous call in another thread, use a timing method to keep track of the timeout, and use Thread.Abort() to get rid of the timed-out synchronous call.
Caveat:
The synchronous call may or may not support to be aborted. For example, Console.ReadLine() will be aborted OK, but if you restart the thread, no data will be read from the Console anymore.
The accepted solution on the original question on top of my posting above uses a second thread, and a timing method. However, it does not kill the sychronous call but keeps it running because it is needed for subsequent async calls, which is a fine hack.
The code for using a second thread is actually straighforward:
public class MySyncProc
{
/// <summary>
/// Value returned from the process after completion
/// </summary>
public string Result = null;
...other shared data...
public MySyncProc() { }
public void Run()
{
Result = LengthyProcess(...);
return;
}
}
public string Run(int TimeoutMs)
{
MySyncProc SyncP = new MySyncProc() { arg1 = ..., etc };
//
Thread T = new Thread(SyncP.Run);
T.Start();
//
DateTime StopTime = DateTime.Now.AddMilliseconds(TimeoutMs);
while (DateTime.Now < StopTime && SyncP.Result == null)
{
Thread.Sleep(200);
}
if (T.IsAlive)
{
T.Abort();
Console.WriteLine("Aborted thread for: {0}", Name);
}
return SyncP.Result;
}
If you don't like the polling, use the slightly more complex AutoResetEvent as in the mentioned accepted solution.