We Keep Coding

How do I update shared state with Parallel.ForEach

I have a WPF app that reads an Outlook .pst file, extracts each message, and saves both it and any attachments as .pdf files. After that's all done, it does some other processing on the files.
I'm currently using a plain old foreach loop for the first part. Here is a rather simplified version of the code...
// These two are used by the WPF UI to display progress
string BusyContent;
ObservableCollection<string> Msgs = new();
// See note lower down about the quick-and-dirty logging
string _logFile = #"C:\Path\To\LogFile.log";
// _allFiles is used to keep a record of all the files we generate. Used after the loop ends
List<string> _allFiles = new();
// nCurr is used to update BusyContent, which is bound to the UI to show progress
int nCurr = 0;
// The messages would really be extracted from the .pst file. Empty list used for simplicity
List<Message> messages = new();
async Task ProcessMessages() {
using StreamWriter logFile = new(_logFile, true);
foreach (Message msg in messages) {
nCurr++;
string fileName = GenerateFileName(msg);
// We log a lot more, but only one shown for simplicity
Log(logFile, $"File: {fileName}");
_allFiles.Add(fileName);
// Let the user know where we are up to
BusyContent = $"Processing message {nCurr}";
// Msgs is bound to a WPF grid, so we need to use Dispatcher to update
Application.Current.Dispatcher.Invoke(() => Msgs.Add(fileName));
// Finally we write out the .pdf files
await ProcessMessage(msg);
}
}
async Task ProcessMessage(Message msg) {
// The methods called here are omitted as they aren't relevant to my questions
await GenerateMessagePdf(msg);
foreach(Attachment a in msg.Attachments) {
string fileName = GenerateFileName(a);
// Note that we update _allFiles here as well as in the main loop
_allFiles.Add(fileName);
await GenerateAttachmentPdf(a);
}
}
static void Log(StreamWriter logFile, string msg) =>
logFile.WriteLine(DateTime.Now.ToString("yyMMdd-HHmmss.fff") + " - " + msg);
This all works fine, but can take quite some time on a large .pst file. I'm wondering if converting this to use Parallel.ForEach would speed things up. I can see the basic usage of this method, but have a few questions, mainly concerned with the class-level variables that are used within the loop...
The logFile variable is passed around. Will this cause issues? This isn't a major problem, as this logging was added as a quick-and-dirty debugging device, and really should be replaced with a proper logging framework, but I'd still like to know if what I'm dong would be an issue in the parallel version
nCurr is updated inside the loop. Is this safe, or is there a better way to do this?
_allFiles is also updated inside the main loop. I'm only adding entries, not reading or removing, but is this safe?
Similarly, _allFiles is updated inside the ProcessMessage method. I guess the answer to this question depends on the previous one.
Is there a problem updating BusyContent and calling Application.Current.Dispatcher.Invoke inside the loop?
Thanks for any help you can give.

At first, it is necessary to use thread safe collections:
ObservableConcurrentCollection<string> Msgs = new();
ConcurrentQueue<string> _allFiles = new();
ObservableConcurrentCollection can be installed through NuGet. ConcurrentQueue is located in using System.Collections.Concurrent;.
Special thanks to Theodor Zoulias for the pointing out that there is better option for ConcurentBag.
And then it is possible to use Parallel.ForEachor Task.
Parallel.ForEach uses Partitioner which allows to avoid creation more tasks than necessary. So it tries to run each method in parallel. So it is better to exclude async and await keywords of methods which participate in Parallel.ForEach.
async Task ProcessMessages()
{
using StreamWriter logFile = new(_logFile, true);
await Task.Run(() => {
Parallel.ForEach(messages, msg =>
{
var currentCount = Interlocked.Increment(ref nCurr);
string fileName = GenerateFileName(msg);
Log(logFile, $"File: {fileName}");
_allFiles.Enqueue(fileName);
BusyContent = $"Processing message {currentCount}";
ProcessMessage(msg);
});
});
}
int ProcessMessage(Message msg)
{
// The methods called here are omitted as they aren't relevant to my questions
var message = GenerateMessagePdf(msg);
foreach (Attachment a in msg.Attachments)
{
string fileName = GenerateFileName(a);
_allFiles.Enqueue(fileName);
GenerateAttachmentPdf(a);
}
return msg.Id;
}
private string GenerateAttachmentPdf(Attachment a) => string.Empty;
private string GenerateMessagePdf(Message message) => string.Empty;
string GenerateFileName(Attachment attachment) => string.Empty;
string GenerateFileName(Message message) => string.Empty;
void Log(StreamWriter logFile, string msg) =>
logFile.WriteLine(DateTime.Now.ToString("yyMMdd-HHmmss.fff") + " - " + msg);
And another way is awaiting all tasks. In this case, there is no need to exclude async and await keywords.
async Task ProcessMessages()
{
using StreamWriter logFile = new(_logFile, true);
var messageTasks = messages.Select(msg =>
{
var currentCount = Interlocked.Increment(ref nCurr);
string fileName = GenerateFileName(msg);
Log(logFile, $"File: {fileName}");
_allFiles.Enqueue(fileName);
BusyContent = $"Processing message {currentCount}";
return ProcessMessage(msg);
});
var msgs = await Task.WhenAll(messageTasks);
}

How can I properly extend the background time of an iOS app so that an HTTP request doesn't get cancelled?

I have an app (App1) that makes use of the WKWebView for a good portion of the UI. There is a scenario where an HTTP PUT request is sent from the WKWebView to a backend server to save some data. For this save operation to complete, the server will need approval thru another app (App2). The user would normally switch to App2 to approve, then switch back to App1 to see the result of the save. The problem is that when App1 gets backgrounded, it can cause the response to the save request to be cancelled, even though the save was completely successful on the backend server. There isn't any errors actually logged, but I'm fairly certain it is happening because iOS is killing the connection when the app gets suspended after it gets backgrounded. I'm basing my thoughts on this discussion.
Since the time it takes to approve the save on App2 isn't that long, I figured I could just try to extend the background time of App1, and it appears to work in the times I've tested it.
However, I want to know if this is really the best strategy, and if so, are there any recommendations on my code (For example, should I move the BeginBackgroundTask inside of the Task.Run):
I used these microsoft docs as an example.
public override async void DidEnterBackground(UIApplication application)
{
ExtendBackgroundTime(application);
}
private nint? webViewBgTaskId = null;
private CancellationTokenSource webViewBgTaskTokenSrc = null;
private void ExtendBackgroundTime(UIApplication application)
{
// cancel the previous background task that was created in this function
webViewBgTaskTokenSrc?.Cancel();
webViewBgTaskTokenSrc = null;
if (webViewBgTaskId.HasValue)
{
application.EndBackgroundTask(webViewBgTaskId.Value);
webViewBgTaskId = null;
}
var cts = new CancellationTokenSource();
nint taskId = default;
taskId = application.BeginBackgroundTask(() =>
{
cts.Cancel();
webViewBgTaskTokenSrc = null;
application.EndBackgroundTask(taskId);
webViewBgTaskId = null;
});
_ = Task.Run(async () =>
{
// For now, this is just set to 5 minutes, but in my experience,
// the background task will never be allowed to continue for that long.
// It's usually only about 30 seconds as of iOS 13.
// But this at least gives it some finite upper bound.
await Task.Delay(TimeSpan.FromMinutes(5), cts.Token);
application.EndBackgroundTask(taskId);
webViewBgTaskId = null;
}, cts.Token);
webViewBgTaskTokenSrc = cts;
webViewBgTaskId = taskId;
}

The following code snippet demonstrates registering a task to run in the background:
nint taskID = UIApplication.SharedApplication.BeginBackgroundTask( () => {});
//runs on main or background thread
FinishLongRunningTask(taskID);
UIApplication.SharedApplication.EndBackgroundTask(taskID);
The registration process pairs a task with a unique identifier, taskID, and then wraps it in matching BeginBackgroundTask and EndBackgroundTask calls. To generate the identifier, we make a call to the BeginBackgroundTask method on the UIApplication object, and then start the long-running task, usually on a new thread. When the task is complete, we call EndBackgroundTask and pass in the same identifier. This is important because iOS will terminate the application if a BeginBackgroundTask call does not have a matching EndBackgroundTask.
Note: If you want to perform Tasks During DidEnterBackground method, these tasks must be invoked on a separate thread. Therefore, sample project uses Task to invoke FinishLongRunningTask.
Task.Factory.StartNew(() => FinishLongRunningTask(taskID));

How Do I Call an Async Method from a Non-Async Method? [duplicate]

This question already has answers here:
How to call asynchronous method from synchronous method in C#?
(17 answers)
Closed 6 years ago.
I have the below method:
public string RetrieveHolidayDatesFromSource() {
var result = this.RetrieveHolidayDatesFromSourceAsync();
/** Do stuff **/
var returnedResult = this.TransformResults(result.Result); /** Where result gets used **/
return returnedResult;
}
private async Task<string> RetrieveHolidayDatesFromSourceAsync() {
using (var httpClient = new HttpClient()) {
var json = await httpClient.GetStringAsync(SourceURI);
return json;
}
}
The above does not work and seems to not return any results properly. I am not sure where I am missing a statement to force the await of a result? I want the RetrieveHolidayDatesFromSource() method to return a string.
The below works fine but it is synchronous and I believe it can be improved upon? Note that the below is synchronous in which I would like to change to Asynchronous but am unable to wrap my head around for some reason.
public string RetrieveHolidayDatesFromSource() {
var result = this.RetrieveHolidayDatesFromSourceAsync();
/** Do Stuff **/
var returnedResult = this.TransformResults(result); /** This is where Result is actually used**/
return returnedResult;
}
private string RetrieveHolidayDatesFromSourceAsync() {
using (var httpClient = new HttpClient()) {
var json = httpClient.GetStringAsync(SourceURI);
return json.Result;
}
}
Am I missing something?
Note: For some reason, when I breakpoint the above Async Method, when it gets to the line var json = await httpClient.GetStringAsync(SourceURI) it just goes out of breakpoint and I can't go back into the method.

Am I missing something?
Yes. Asynchronous code - by its nature - implies that the current thread is not used while the operation is in progress. Synchronous code - by its nature - implies that the current thread is blocked while the operation is in progress. This is why calling asynchronous code from synchronous code literally doesn't even make sense. In fact, as I describe on my blog, a naive approach (using Result/Wait) can easily result in deadlocks.
The first thing to consider is: should my API be synchronous or asynchronous? If it deals with I/O (as in this example), it should be asynchronous. So, this would be a more appropriate design:
public async Task<string> RetrieveHolidayDatesFromSourceAsync() {
var result = await this.DoRetrieveHolidayDatesFromSourceAsync();
/** Do stuff **/
var returnedResult = this.TransformResults(result); /** Where result gets used **/
return returnedResult;
}
As I describe in my async best practices article, you should go "async all the way". If you don't, you won't get any benefit out of async anyway, so why bother?
But let's say that you're interested in eventually going async, but right now you can't change everything, you just want to change part of your app. That's a pretty common situation.
In that case, the proper approach is to expose both synchronous and asynchronous APIs. Eventually, after all the other code is upgraded, the synchronous APIs can be removed. I explore a variety of options for this kind of scenario in my article on brownfield async development; my personal favorite is the "bool parameter hack", which would look like this:
public string RetrieveHolidayDatesFromSource() {
return this.DoRetrieveHolidayDatesFromSourceAsync(sync: true).GetAwaiter().GetResult();
}
public Task<string> RetrieveHolidayDatesFromSourceAsync() {
return this.DoRetrieveHolidayDatesFromSourceAsync(sync: false);
}
private async Task<string> DoRetrieveHolidayDatesFromSourceAsync(bool sync) {
var result = await this.GetHolidayDatesAsync(sync);
/** Do stuff **/
var returnedResult = this.TransformResults(result);
return returnedResult;
}
private async Task<string> GetHolidayDatesAsync(bool sync) {
using (var client = new WebClient()) {
return sync
? client.DownloadString(SourceURI)
: await client.DownloadStringTaskAsync(SourceURI);
}
}
This approach avoids code duplication and also avoids any deadlock or reentrancy problems common with other "sync-over-async" antipattern solutions.
Note that I would still treat the resulting code as an "intermediate step" on the path to a properly-asynchronous API. In particular, the inner code had to fall back on WebClient (which supports both sync and async) instead of the preferred HttpClient (which only supports async). Once all the calling code is changed to use RetrieveHolidayDatesFromSourceAsync and not RetrieveHolidayDatesFromSource, then I'd revisit this and remove all the tech debt, changing it to use HttpClient and be async-only.

public string RetrieveHolidayDatesFromSource() {
var result = this.RetrieveHolidayDatesFromSourceAsync().Result;
/** Do stuff **/
var returnedResult = this.TransformResults(result.Result); /** Where result gets used **/
return returnedResult;
}
If you add .Result to the async call, it will execute and wait for the result to arrive, forcing it to be synchronous
UPDATE:
private static string stringTest()
{
return getStringAsync().Result;
}
private static async Task<string> getStringAsync()
{
return await Task.FromResult<string>("Hello");
}
static void Main(string[] args)
{
Console.WriteLine(stringTest());
}
To address the comment: This works without any problems.

How to do C# asynchronous programming?

I'm working on an Export-Tool from a Database which I had written in Visual Basic.net before a long time.
I switched to C# and like to reprogram it in C# since I've gathered quite a little bit more experience than I had before :-)
My C# application's UI is hanging because there is a big database query to do. So I've informed myself about asynchronous programming. Tried threads and tasks as well but I'm having problems to find the right method for my concerns.
This is what I have: it's a Windows Forms application and there's a start button which creates a new thread.
The Method called Export is a non static method which I've created in a second class file named actions.cs. It's non static because the method should be reused often in the code but with different parameters.
So I instantiated the method in the Button_Clicked Event on Form1 with the corresponding parameters:
actions KuliMon = new actions()
{
ExportPath = settings.ReadActSetting("baexport", "ExportPfad", ""),
Trennzeichen = settings.ReadGlobSetting("Trennzeichen", ";"),
ConnectionString = settings.ReadGlobSetting("Mand1_odbc", ""),
SQLFile = "kuli.sql",
ExportAktion = "kuli"
};
Then I've started the thread from the Button_click event like this:
Thread ExportThread = new Thread(KuliMon.Export);
ExportThread.Start();
This works. No sticking GUI. But now there is the problem. My method Export in actions.cs is exporting the DB-Query into a csv-File but should also return the result in a string variable which then I can display in a TextBox on Form1.
By reading a little bit around I've found the Invoke-Method this helped quite a little bit. Under Thread.Start() I've added the following:
this.Invoke((MethodInvoker)delegate
{
tx_main_t1.Text = "Hello";
});
When I've clicked the button the TextBox says "hello". But instead of hello I need the Return string from my method Export running in the thread. And here's the problem how can I get the string with the queried results.
In my understanding the thread method has to call a void-method and must not return a value.
I had an idea of creating a public property string and fill the string in Export with the return value like this:
public string results { get; set; }
Instead of using return ReturnValue in Method Export I've tried
results = ReturnValue;
In Form1 I then tried to fill the TextBox with KuliMon.results but it's empty because I've made an instance of Export as I think.

Both database queries and writing to files are I/O-bound operations, so they are a natural fit for async and await.
First, you would define an asynchronous version of KuliMon.Export:
async Task<string> ExportAsync()
{
var data = await myQuery.ToListAsync();
await myStream.WriteAsync(...);
return results;
}
You can, for example, use Entity Framework 6 for asynchronous query support.
Then you can call it from your UI as such:
async void button1_Clicked(...)
{
TextBox1.Text = await myInstance.ExportAsync();
}
If you can't use asynchronous database queries for whatever reason (e.g., I think Oracle doesn't support them at this time), then you can use a background thread that calls the synchronous APIs. Note that Task.Run is a modern replacement for both Thread and BackgroundWorker:
string Export();
...
async void button1_Clicked(...)
{
TextBox1.Text = await Task.Run(() => myInstance.Export());
}

You should look in to the BackGroundWorker class
https://msdn.microsoft.com/en-us/library/cc221403%28v=vs.95%29.aspx
You can assign a function to call when the worker completes the job, and have that function update your UI.

using async await supported in .Net 4.5 and c#5 (you can get support for this in erarlier .Net versions with AsyncBridge package for 3.5 or for 4.0)
private async void button1_Click(object sender, EventArgs e)
{
button1.Enabled = false;
try
{
//Code will wait here for your method to return without blocking UI Exceptions or Result will be automatically Scheduled to UI thread
string result = await DoStuffAsync("myParameter");
}
catch
{
MessageBox.Show("Ups an error");
}
finally
{
button1.Enabled = true;
}
}
/// <summary>
/// Wraps the synchron implementation off DoStuff
/// </summary>
public Task<string> DoStuffAsync(string s)
{
return Task.Factory.StartNew(DoStuff, s); //here parameter s is forwarded to your synchronus implementation
}
/// <summary>
/// Your orginal synchron implementation with the dbQuerry
/// or long running calculations
/// </summary>
public string DoStuff(string s)
{
//do your normal code;
return result
}

HttpClient doesn't return in Windows Phone

Simple enough, here's my code:
/// <summary>
/// Fetches the JSON string from the URL.
/// </summary>
/// <param name="url">URL to download the JSON from.</param>
/// <returns>JSON formatted string</returns>
private static string GetJsonResponse(string url)
{
var result = DownloadString(url).Result;
System.Diagnostics.Debug.WriteLine("Downloaded from {0} with result = {1}", url, result);
return result;
}
private static async Task<string> DownloadString(string feedUrl)
{
var result = "";
System.Diagnostics.Debug.WriteLine("Downloading from {0}", feedUrl);
using (var client = new HttpClient())
{
result = await client.GetStringAsync(new Uri(feedUrl, UriKind.Absolute));
result.Trim();
System.Diagnostics.Debug.WriteLine("Finished download from {0}", feedUrl);
}
return result;
}
it prints the first string but there's no way to get to second one, and this means that the client.GetStringAsync doesn't return.
The URL is ok, it loads fine.
What's my problem?
EDIT 2:
(Removed as useless now)
EDIT 3:
I'm adding the methods that I'm using to let you know what's the flow of my data.
Page.xaml
<phone:LongListSelector Grid.Row="1" x:Name="UpcomingResultsList" ItemsSource="{Binding UpcomingMovies}" ItemTemplate="{StaticResource MovieListDataTemplate}" Margin="0, 10, 0, 0" />
UpcomingMovies is bound to this property
public List<MovieViewModel> UpcomingMovies
{
get
{
System.Diagnostics.Debug.WriteLine("UpcomingMovies - Begin");
var apiMovies = _serviceRT.FindUpcomingMoviesList().Result; // We get the movies in RT API's format
var movies = apiMovies.Select(result => new MovieViewModel(result)).ToList();
System.Diagnostics.Debug.WriteLine("UpcomingMovies - End");
return movies;
}
}
the _serviceRT.FindUpcomingMoviesList() method:
/// <summary>
/// Gets a list of upcoming movies.
/// </summary>
/// <returns>MovieSearchResults</returns>
public async Task<MovieSearchResults> FindUpcomingMoviesList()
{
var url = string.Format(LIST_UPCOMING, ApiKey);
var jsonResponse = await GetJsonResponse(url);
var results = Parser.ParseMovieSearchResults(jsonResponse);
return results;
}
Finally, the GetJsonResponse() is at the beginning of the question.
Now, having the full data flow, how can I bind this property and still make the download complete?

I predict that further up in your call stack, you are calling Wait or Result on a Task returned from an async method. This can easily cause deadlock, as I describe on my blog.
This happens because await will by default capture a "context" which it uses to resume the async method. In your example, this is most likely a UI context, which always has exactly one thread (the UI thread). So if you block the UI thread by calling Wait or Result, then the async method cannot re-enter the UI context to complete its execution (and you end up waiting for a Task that cannot complete).
Edit:
Since you're databinding, I recommend using the NotifyTaskCompletion type here (which will soon be part of my AsyncEx library):
public MyViewModel()
{
UpcomingMovies = NotifyTaskCompletion.Create(LoadUpcomingMoviesAsync());
}
public INotifyTaskCompletion<List<MovieViewModel>> UpcomingMovies { get; private set; }
private async Task<List<MovieViewModel>> LoadUpcomingMoviesAsync()
{
System.Diagnostics.Debug.WriteLine("UpcomingMovies - Begin");
var apiMovies = await _serviceRT.FindUpcomingMoviesList();
var movies = apiMovies.Select(result => new MovieViewModel(result)).ToList();
System.Diagnostics.Debug.WriteLine("UpcomingMovies - End");
return movies;
}
Then you can safely bind to INotifyTaskCompletion<T>.Result like this:
{Binding UpcomingMovies.Result}
Note that until you load the movies, the result will be null so your view will be empty. Also, if there is an error loading the movies, your view will always be empty. I recommend that you handle these situations by databinding to the other INotifyTaskCompletion<T> properties, e.g.:
{Binding UpcomingMovies.IsCompleted}, which will start out false and become true when the loading completes (either successfully or in error)
{Binding UpcomingMovies.IsSuccessfullyCompleted}, which becomes true only if/when the loading completes successfully
{Binding UpcomingMovies.IsFaulted}, which becomes true only if/when the loading completes with an error
{Binding UpcomingMovies.ErrorMessage}, which extracts the error message (and is null if there is no error)

HttpClient has limitations according to platform it is called from (like other network-related APIs on Windows Phone if compared with "big windows"). Maybe, this how-to will help: http://blogs.msdn.com/b/bclteam/archive/2013/02/18/portable-httpclient-for-net-framework-and-windows-phone.aspx
HttpClientHandler handler = new HttpClientHandler();
httpClient = new HttpClient(handler);
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, resourceAddress);
request.Content = streamContent;
if (handler.SupportsTransferEncodingChunked())
{
request.Headers.TransferEncodingChunked = true;
}
HttpResponseMessage response = await httpClient.SendAsync(request);