We Keep Coding

List.Add not added perfectly with Parallel and async

I have a Blazor WASM application that needs to call an API every second without blocking the UI. This codes demonstrates how I tried to do that:
List<int> testList = new();
testList.Add(1);
testList.Add(2);
testList.Add(3);
testList.Add(4);
List<int> emptyTestlist = new();
CancellationTokenSource cts;
Test();
void Test()
{
Parallel.Invoke(async () =>
{
do
{
Console.WriteLine("Start");
await Task.Delay(1000);
await Test2();
Console.WriteLine("END");
} while (true);
});
}
Console.ReadLine();
async ValueTask Test2()
{
emptyTestlist.Clear();
cts = new();
await Parallel.ForEachAsync(testList, cts.Token, async (test, token) =>
{
await Test4(test);
});
foreach (var test in emptyTestlist)
{
await Test3(test);
}
}
async Task Test4(int i)
{
await Task.Delay(300);
//Console.WriteLine("if I Add this console.WriteLine It's added perfectly");
emptyTestlist.Add(i);
Console.WriteLine($"from TEST4: {i}");
}
async Task Test3(int i)
{
Console.WriteLine($"TEST3 {i}.");
await Task.Delay(1000);
Console.WriteLine($"TEST3 {i}, after 1sec");
}
If I comment the line Console.WriteLine("if I Add this console.WriteLine It's added perfectly");, it's not adding perfectly. (emptyTestlist.Count is not always 4). But if I add Console.WriteLine before emptyTestlist.Add(i) it works correctly (emptyTestlist.Count is always 4).
I don't know how to solve it. What's the problem?

The easiest way to poll an API is to use a timer:
#code {
private List<Customer> custs=new List<Customer>();
private System.Threading.Timer timer;
protected override async Task OnInitializedAsync()
{
await base.OnInitializedAsync();
custs = await Http.GetFromJsonAsync<List<Customer>>(url);
timer = new System.Threading.Timer(async _ =>
{
custs = await Http.GetFromJsonAsync<List<Customer>>("/api/customers");
InvokeAsync(StateHasChanged);
}, null, 1000, 1000);
}
In this case InvokeAsync(StateHasChanged); is needed because the state was modified from a timer thread and Blazor has no idea the data changed.
If we wanted to add the results to a list though, we'd either have to use a lock or a thread-safe collection, like a ConcurrentQueue.
#code {
private ConcurrentQueue<Customer> custs=new ConcurrentQueue<Customer>();
private System.Threading.Timer timer;
protected override async Task OnInitializedAsync()
{
await base.OnInitializedAsync();
custs = await Http.GetFromJsonAsync<List<Customer>>(url);
timer = new System.Threading.Timer(async _ =>
{
var results = await Http.GetFromJsonAsync<List<Customer>>("/api/customers");
foreach(var c in results)
{
custs.Enqueue(c);
}
InvokeAsync(StateHasChanged);
}, null, 1000, 1000);
}
Polling an API every second just in case there's any new data isn't very efficient though. It would be better to have the API notify clients of any new data using eg SignalR or Push Notifications
Borrowing from the documentation example this would be enough to receive messages from the server:
#code {
private HubConnection hubConnection;
private List<string> messages = new List<string>();
private string userInput;
private string messageInput;
protected override async Task OnInitializedAsync()
{
hubConnection = new HubConnectionBuilder()
.WithUrl(NavigationManager.ToAbsoluteUri("/chathub"))
.Build();
hubConnection.On<string, string>("ReceiveMessage", (user, message) =>
{
var encodedMsg = $"{user}: {message}";
messages.Add(encodedMsg);
StateHasChanged();
});
await hubConnection.StartAsync();
}

Async seems to be synchronous

I am not sure if I am missing something here but more for loop seems to be executing synchronously even though I await all tasks out side of it.
Here is my code below:
static void Main(string[] args) {
var t = Start();
}
public static async Task < List < Task < TaskInfo >>> Start() {
var listOfTasks = new List < Task < TaskInfo >> ();
for (var i = 0; i <= 100; i++) {
var process = new Processor();
listOfTasks.Add(process.Process(i));
}
await Task.WhenAll(listOfTasks);
return listOfTasks;
}
I pass in the taskId to log out just to see the order the tasks execute.
Am I missing something really obvious here?
EDIT:
Changed code to this based on the answers and comments below and it still appears synchronously:
public class StartWork
{
public int TaskId { get; set; }
public Processor Processor { get;}
public StartWork()
{
Processor = new Processor();
}
}
static void Main(string[] args)
{
var t = Start();
}
public static async Task<TaskInfo[]> Start()
{
var tasks = new List<StartWork>();
for (int i = 1; i < 100; i++)
{
var work = new StartWork
{
TaskId = i
};
tasks.Add(work);
}
return await Task.WhenAll(tasks.Select(i => i.Processor.Process(i.TaskId)));
}
The function I am calling in processor class:
public Task<TaskInfo> Process(int taskId)
{
try
{
taskId = taskId + 1;
stopwatch.Start();
using (var bus = RabbitHutch.CreateBus(xxDev))
{
#event = new AutoResetEvent(false);
var replyTo = Guid.NewGuid().ToString();
var messageQueue = bus.Advanced.QueueDeclare(replyTo, autoDelete: true);
bus.Advanced.Consume(messageQueue, (payload, properties, info) =>
{
ReceivePdf(payload, properties, info);
return Task.FromResult(0);
});
taskInfo.InputFile = inputFile;
var html = File.ReadAllText(inputFile);
taskInfo.Html = html;
var message = PrepareMessage(new RenderRequest()
{
Html = Encoding.UTF8.GetBytes(html),
Options = new RenderRequestOptions()
{
PageSize = "A4",
ImageQuality = 70,
PageLoadRetryAttempts = 3
}
});
var correlation = Guid.NewGuid().ToString();
Console.WriteLine($"CorrelationId: {correlation}, TaskId {taskId}");
var props = new MessageProperties
{
CorrelationId = correlation,
ReplyTo = replyTo,
Expiration = "6000"
};
Publish(bus, props, message);
taskInfo.CorrelationId = Guid.Parse(correlation);
#event.WaitOne();
stopwatch.Stop();
taskInfo.TimeTaken = stopwatch.Elapsed;
return Task.FromResult(taskInfo);
}
}
catch (Exception e)
{
taskInfo.OutputFile = Empty;
return Task.FromResult(taskInfo);
}
}
void ReceivePdf(byte[] payload, MessageProperties properties, MessageReceivedInfo info)
{
var file = Format(outputFile, properties.CorrelationId);
taskInfo.OutputFile = file;
Console.WriteLine("Output written to " + file);
File.WriteAllBytes(file, payload);
var remaining = Interlocked.Decrement(ref outstandingRequests);
if (remaining == 0)
{
#event.Set();
}
}

This is a synchronous task
listOfTasks.Add(process.Process(i));
You are just adding items to the list.
This is also a synchronous task
process.Process(i);
The task that the function above returns is asynchronous and it will execute asynchronously in the whenAll call of your code.
Bear in mind that when all will wait for all tasks to run and if the task is trivial, since they start one after the other, will most times run sequentially by chance.
You will see some difference if the task code executed differentiated in execution time based on input.

First async doesn't mean multithread, async is used to run background task without blocking UI or to run I/O operations without bloking main thread.
Usually the operating system handles async with multithreading, but there is no guarantee.
If you want be sure to start multiple threads use Thread.Start.
Anyway in your code you force your code to run synchronously, because you call the async method start in the Main method without await.
You need to change the code to:
static async void Main(string[] args)
{
var t = await Start();
}
or without waiting to (but the program risk to terminate before the task complete):
static void Main(string[] args)
{
Task.Run(async () => {
var t = await Start();
});
}

C# async Deserializion method does not cancel

I'm running into the issue where the task being executed below runs asynchronously but is unable to be cancelled when the serializer reads the memory stream. When the user makes the cancellation request (by pressing a cancel button), a cancellation (the method cancel() is called fromt he token) is made but the task continues.
Service Class:
Async method called from LoadHelper() in Main class
public async void StartTask(Action callback, CancellationToken token)
{
await Task.Run(callback, token);
}
Main Class:
private void LoadHelper()
{
_services.GetInstance<IThreadService>().StartTask(
() => LoadHelperAsync(), _cancelService.Token);
}
Method being run async
private void LoadHelperAsync()
{
var serializer = new DataContractSerializer(typeof(UserDatabase));
string selectedDir;
ComparisonResult = null;
_services.GetInstance<IOpenFileService>().DisplayOpenFileDialog(
"Select a saved comparison",
"Comparison File(*.xml; *.cmp)|*.xml;*.cmp",
Environment.GetFolderPath(Environment.SpecialFolder.Desktop),
out selectedDir);
if (!string.IsNullOrEmpty(selectedDir))
{
_dispatchService.BeginInvoke(() => IsExecuting = true);
using (FileStream fileStream = new FileStream(selectedDir, FileMode.Open, FileAccess.Read))
using (DeflateStream compressedStream = new DeflateStream(fileStream, CompressionMode.Decompress))
using (BufferedStream regularStream = new BufferedStream(fileStream))
{
Stream memoryStream;
//Use filename to determine compression
if (selectedDir.EndsWith(".cmp", true, null))
{
memoryStream = compressedStream;
}
else
{
memoryStream = regularStream;
}
Report("Loading comparison");
Report(0);
IsExecuting = true;
ComparisonResult = (UserDatabase)serializer.ReadObject(memoryStream);
memoryStream.Close();
fileStream.Close();
IsExecuting = false;
Report("Comparison loaded");
}
_dispatchService.BeginInvoke(() => IsExecuting = false);
_dispatchService.BeginInvoke(() => ViewResults.ExecuteIfAble());
}
else
{
Report("No comparison loaded");
}
Cancellation code:
This command is binded to a "cancel" button in the view.
CancelCompare = new Command(o => _cancelService.Cancel(), o => IsExecuting);
From the CancellationService class
public class CancellationService : ICancellationService, IDisposable
{
private CancellationTokenSource _tokenSource;
public CancellationService()
{
Reset();
}
public CancellationToken Token { get; private set; }
public void Cancel()
{
_tokenSource.Cancel();
}
public void Reset()
{
_tokenSource = new CancellationTokenSource();
Token = _tokenSource.Token;
}
protected virtual void Dispose(bool disposing)
{
if (disposing)
{
_tokenSource.Cancel();
_tokenSource.Dispose();
}
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
}

Calling _tokenSource.Cancel(); won't do anything to a running task. A cancellation token interrupts task execution only if it was canceled before Task.Run
Take a look:
static void Main(string[] args)
{
using (var tokenSource = new CancellationTokenSource())
{
var aTask = StartTask(() =>
{
while (true)
{
Console.WriteLine("Nothing is going to happen.");
// Some long operation
Thread.Sleep(1000);
}
}, tokenSource.Token);
tokenSource.Cancel();
aTask.Wait();
}
Console.ReadKey();
}
static async Task StartTask(Action callback, CancellationToken cancellationToken)
{
await Task.Run(callback, cancellationToken);
}
If you want to cancel the execution, you should check the cancellation token by yourself, there is no any magic which cancels the task for you:
static void Main(string[] args)
{
using (var tokenSource = new CancellationTokenSource())
{
var aTask = StartTask(cancellationToken =>
{
while (true)
{
cancellationToken.ThrowIfCancellationRequested();
Console.WriteLine("Will stop before that if canceled.");
// Some long operation
// Also pass the token all way down
Task.Delay(1000, cancellationToken).Wait();
}
}, tokenSource.Token);
// Try 0, 500, 1500 to see the difference
Thread.Sleep(1500);
tokenSource.Cancel();
try
{
aTask.Wait();
}
catch (Exception ex)
{
// AggregateException with OperationCanceledException
Console.WriteLine("Task was canceled.");
Console.WriteLine(ex.ToString());
}
}
Console.ReadKey();
}
static async Task StartTask(Action<CancellationToken> callback, CancellationToken cancellationToken)
{
await Task.Run(() => callback(cancellationToken), cancellationToken);
}
Note that this code only illustrates how the task works, never write anything like this in production.

Task Parallel library waiting for method in loop to finish (.NET 4.0)

wonder if you can help.
I have a listview displaying a list of Customer EG (Customer1,Customer2,Customer3 etc..)
Now each customer can have 1000s of orders to process so
The problem I have is that I would like to wait for processcustomer to finish before starting with new one,and
I dont know how to code it to make my "ProcessCustomer" in the loop to finish before can proceed with the next one.
Psuedo code below:
public class form1
{
private void StartProcessingCustomer_Click(object sender, EventArgs e)
{
foreach (ListViewItem item in lvwCustomers.Items)
{
customerPresenter.ProcessCustomer(item.Tag as Customer);
}
}
}
public class CustomerPresenter
{
private void ProcessCustomer(Customer customer)
{
CustomerProcessor customerProcessor=new CustomerProcessor();
customerProcessor.OrderProcessing += OnCustomerProcessing;
customerProcessor.OrderProcessed += OnCustomerProcessed;
customerProcessor.TaskCompleted += OnCustomerCompleted;
customerProcessor.ProcessCustomer(customer);
}
}
public class CustomerProcessor
{
public CustomerProcessor()
{
}
public void ProcessCustomer(Customer customer)
{
//simplified version with psuedo code
cancelToken = new CancellationTokenSource();
var parOpts = new ParallelOptions();
parOpts.CancellationToken = cancelToken.Token;
parOpts.MaxDegreeOfParallelism = -1;
const int index = 0;
try
{
sw.Start();
processResult.StartedAt = DateTime.Now;
Task t1 = Task.Factory.StartNew(() =>
{
try
{
Parallel.ForEach(customer.Orders, parOpts, (order, loopState) =>
{
count = Interlocked.Increment(ref count);
ProcessOrder(order, count);
});
}
catch (OperationCanceledException)
{
//omitted code
}
}, cancelToken.Token).ContinueWith(result => OnTaskCompleted(
{
//omitted code
}), new CancellationTokenSource().Token, TaskContinuationOptions.None, TaskScheduler.FromCurrentSynchronizationContext());
}
catch (AggregateException ae)
{
//omitted code
}
}
private void ProcessOrder(Order order, int index)
{
//we process the order here
}
}
}

If you make your ProcessCustomer method return a Task, you can asynchronously wait for it's completion:
public Task ProcessCustomer(Customer customer)
{
return Task.Factory.StartNew(() =>
{
try
{
Parallel.ForEach(customer.Orders, parOpts, (order, loopState) =>
{
count = Interlocked.Increment(ref count);
ProcessOrder(order, count);
});
}
catch (OperationCanceledException)
{
//omitted code
}
}, cancelToken.Token).ContinueWith(result => OnTaskCompleted(
{
//omitted code
}), new CancellationTokenSource().Token, TaskContinuationOptions.None,
TaskScheduler.FromCurrentSynchronizationContext());
}
Now you can:
//I'm assuming this is an event handler.
private async void SomeEventHandler(object sender, EventArgs e)
{
foreach (ListViewItem item in lvwCustomers.Items)
{
await customerPresenter.ProcessCustomer(item.Tag as Customer);
}
}
As you're using .NET 4.0, you'll need to use Microsoft.Bcl.Async to make async-await available.

Volatile IEnlistmentNotification, TransactionScope.AsyncFlowEnabled = true and complex async/wait

This is a followup question to the following question:
Volatile IEnlistmentNotification and TransactionScope.AsyncFlowEnabled = true
The approach accepted in the question above works as long as you don't await multiple statements. Let me show an example:
public class SendResourceManager : IEnlistmentNotification
{
private readonly Action onCommit;
public SendResourceManager(Action onCommit)
{
this.onCommit = onCommit;
}
public void Prepare(PreparingEnlistment preparingEnlistment)
{
preparingEnlistment.Prepared();
}
public void Commit(Enlistment enlistment)
{
Debug.WriteLine("Committing");
this.onCommit();
Debug.WriteLine("Committed");
enlistment.Done();
}
public void Rollback(Enlistment enlistment)
{
enlistment.Done();
}
public void InDoubt(Enlistment enlistment)
{
enlistment.Done();
}
}
public class AsyncTransactionalMessageSender : ISendMessagesAsync
{
private readonly List<Message> sentMessages = new List<Message>();
public IReadOnlyCollection<Message> SentMessages
{
get { return new ReadOnlyCollection<Message>(this.sentMessages); }
}
public async Task SendAsync(Message message)
{
if (Transaction.Current != null)
{
await Transaction.Current.EnlistVolatileAsync(
new SendResourceManager(async () => await this.SendInternal(message)),
EnlistmentOptions.None);
}
else
{
await this.SendInternal(message);
}
}
private async Task SendInternal(Message message)
{
Debug.WriteLine("Sending");
await Task.Delay(1000);
this.sentMessages.Add(message);
Debug.WriteLine("Sent");
}
}
[Test]
public async Task ScopeRollbackAsync_DoesntSend()
{
var sender = new AsyncTransactionalMessageSender();
using (var tx = new System.Transactions.TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
await sender.SendAsync(new Message("First"));
await sender.SendAsync(new Message("Second"));
await sender.SendAsync(new Message("Last"));
// We do not commit the scope
}
sender.SentMessages.Should().BeEmpty();
}
[Test]
public async Task ScopeCompleteAsync_Sends()
{
var sender = new AsyncTransactionalMessageSender();
using (var tx = new System.Transactions.TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
await sender.SendAsync(new Message("First"));
await sender.SendAsync(new Message("Second"));
await sender.SendAsync(new Message("Last"));
tx.Complete();
}
sender.SentMessages.Should().HaveCount(3)
.And.Contain(m => m.Value == "First")
.And.Contain(m => m.Value == "Second")
.And.Contain(m => m.Value == "Last");
}
As soon as you introduce a Task.Delay like shown in the example above the generated asynchronous statemachine will never come back and invoke the this.sentMessages.Add(message) and Debug.WriteLine("Sent")
The problem is I currently see now way to properly enlist asynchronous code inside the enlistment notification. Any ideas how to tackle this challenge?