I'm getting a strange behavior in a winForms application while testing to see how it responds on different OS.
The long running operation where the unhandled AggregateException is thrown ( when tested on a XP 32bit machine) is part of a WCF (using basicHttpBinding and Streamed transfer mode) client update procedure. The structure is similar to the following code snippet. I have ommited WCF exception handling for clarity:
var tsUIthread = TaskScheduler.FromCurrentSynchronizationContext();
int filesCount = 0;
List<MessageContract> files;
private void Update()
{
var cTokenDownloadFiles = new CancellationTokenSource();
var cTokenUpdateDatabases = new CancellationTokenSource();
var task1CheckForNewFiles = Task.Factory.StartNew(() =>
{
filesCount = proxy.GetFiles();
If(filesCount == 0)
{
cTokenDownloadFiles.Cancel();
cTokenUpdateDatabases.Cancel();
}
else
files = new List<MessageContract>();
});
var task2UpdateControls = task1CheckForNewFiles.ContinueWith(result =>
{
UpdateControlsBeforeDownload();
}, CancellationToken.None, TaskContinuationOptions.None, tsUIthread);
var task3DownloadFiles = task2.ContinueWith(result =>
{
for(int i = 0; i< filesCount; i++)
{
try
{
files.Add(proxy.DownloadFile());
}
catch(IOException)
{
cTokenUpdateDatabases.Cancel();
Task.Factory.StartNew(() => ResetControls,
CancellationToken.None, TaskCreationOptions.None, tsUIthread);
retryUpdateTimer.Start(); // System.Windows.Forms.Timer
return;
}
}
}, cTokenDownloadFiles.Token);
var task4UpdateDatabases = task3DownloadFiles.ContinueWith(result =>
{
UpdateDatabases();
},cTokenUpdateDatabases.Token);
var task5UpdateControls = task4UpdateDatabases.ContinueWith(result =>
{
UpdateControlsOnUpdateFinished();
}, CancellationToken.None, TaskContinuationOptions.None, tsUIthread);
}
You'll notice that I wrap the proxy.DownloadFile() method in a try-catch block where I'm trapping an IO.IOException. As I meantioned in the beggining of my post, my WCF services use the Streamed transfer mode of the basicHttpBinding so I need to catch this type of exception when for some reason the connection to the server is lost after the operation has begun. All I do In the catch statement is, cancelling the databases update task, reset the UI controls to their primary values and start a System.Windows.Forms.Timer with an Interval of a few seconds what will execute the Update() method again if the client has internet connectivity.
While this whole procedure works as expected on my dev environment (Windows7 32bit machine) when an IOException is thrown , when I test my winforms Application on a Windows XP 32bit machine an unhandled AggregateException at System.Threading.Tasks.ExceptionHolder.Finalize() terminates the Application.
Has anyone experienced anything similar? How is it possible for the exception to be thrown only on the XP machine? I still haven't tested it on other environments.
The actual code contains more tasks and continuations, calling downstream business components and I'm kind of lost when it comes to Task exception handling with continuations spaghetti. Could you give me some examples of how I should structure the exception handling?
Related
I have a Windows service that reads data from the database and processes this data using multiple REST API calls.
Originally, this service ran on a timer where it would read unprocessed data from the database and process it using multiple threads limited using SemaphoreSlim. This worked well except that the database read had to wait for all processing to finish before reading again.
ServicePointManager.DefaultConnectionLimit = 10;
Original that works:
// Runs every 5 seconds on a timer
private void ProcessTimer_Elapsed(object sender, ElapsedEventArgs e)
{
var hasLock = false;
try
{
Monitor.TryEnter(timerLock, ref hasLock);
if (hasLock)
{
ProcessNewData();
}
else
{
log.Info("Failed to acquire lock for timer."); // This happens all of the time
}
}
finally
{
if (hasLock)
{
Monitor.Exit(timerLock);
}
}
}
public void ProcessNewData()
{
var unproceesedItems = GetDatabaseItems();
if (unproceesedItems.Count > 0)
{
var downloadTasks = new Task[unproceesedItems.Count];
var maxThreads = new SemaphoreSlim(semaphoreSlimMinMax, semaphoreSlimMinMax); // semaphoreSlimMinMax = 10 is max threads
for (var i = 0; i < unproceesedItems .Count; i++)
{
maxThreads.Wait();
var iClosure = i;
downloadTasks[i] =
Task.Run(async () =>
{
try
{
await ProcessItemsAsync(unproceesedItems[iClosure]);
}
catch (Exception ex)
{
// handle exception
}
finally
{
maxThreads.Release();
}
});
}
Task.WaitAll(downloadTasks);
}
}
To improve efficiency, I rewrite the service to run GetDatabaseItems in a separate thread from the rest so that there is a ConcurrentDictionary of unprocessed items between them that GetDatabaseItems fills and ProcessNewData empties.
The problem is that while 10 unprocessed items are send to ProcessItemsAsync, they are processed two at a time instead of all 10.
The code inside of ProcessItemsAsync calls var response = await client.SendAsync(request); where the delay occurs. All 10 threads make it to this code but come out of it two at a time. None of this code changed between the old version and the new.
Here is the code in the new version that did change:
public void Start()
{
ServicePointManager.DefaultConnectionLimit = maxSimultaneousThreads; // 10
// Start getting unprocessed data
getUnprocessedDataTimer.Interval = getUnprocessedDataInterval; // 5 seconds
getUnprocessedDataTimer.Elapsed += GetUnprocessedData; // writes data into a ConcurrentDictionary
getUnprocessedDataTimer.Start();
cancellationTokenSource = new CancellationTokenSource();
// Create a new thread to process data
Task.Factory.StartNew(() =>
{
try
{
ProcessNewData(cancellationTokenSource.Token);
}
catch (Exception ex)
{
// error handling
}
}, TaskCreationOptions.LongRunning
);
}
private void ProcessNewData(CancellationToken token)
{
// Check if task has been canceled.
while (!token.IsCancellationRequested)
{
if (unprocessedDictionary.Count > 0)
{
try
{
var throttler = new SemaphoreSlim(maxSimultaneousThreads, maxSimultaneousThreads); // maxSimultaneousThreads = 10
var tasks = unprocessedDictionary.Select(async item =>
{
await throttler.WaitAsync(token);
try
{
if (unprocessedDictionary.TryRemove(item.Key, out var item))
{
await ProcessItemsAsync(item);
}
}
catch (Exception ex)
{
// handle error
}
finally
{
throttler.Release();
}
});
Task.WhenAll(tasks);
}
catch (OperationCanceledException)
{
break;
}
}
Thread.Sleep(1000);
}
}
Environment
.NET Framework 4.7.1
Windows Server 2016
Visual Studio 2019
Attempts to fix:
I tried the following with the same bad result (two await client.SendAsync(request) completing at a time):
Set Max threads and ServicePointManager.DefaultConnectionLimit to 30
Manually create threads using Thread.Start()
Replace async/await pattern with sync HttpClient calls
Call data processing using Task.Run(async () => and Task.WaitAll(downloadTasks);
Replace the new long-running thread for ProcessNewData with a timer
What I want is to run GetUnprocessedData and ProcessNewData concurrently with an HttpClient connection limit of 10 (set in config) so that 10 requests are processed at the same time.
Note: the issue is similar to HttpClient.GetAsync executes only 2 requests at a time? but the DefaultConnectionLimit is increased and the service runs on a Windows Server. It also creates more than 2 connections when original code runs.
Update
I went back to the original project to make sure it still worked, it did. I added a new timer to perform some unrelated operations and the httpClient issue came back. I removed the timer, everything worked. I added a new thread to do parallel processing, the problem came back.
This is not a direct answer to your question, but a suggestion for simplifying your service that could make the debugging of any problem easier. My suggestion is to implement the producer-consumer pattern using an iterator for producing the unprocessed items, and a parallel loop for consuming them. Ideally the parallel loop would have async delegates, but since you are targeting the .NET Framework you don't have access to the .NET 6 method Parallel.ForEachAsync. So I will suggest the slightly wasteful approach of using a synchronous parallel loop that blocks threads. You could use either the Parallel.ForEach method, or the PLINQ like in the example below:
private IEnumerable<Item> Iterator(CancellationToken token)
{
while (true)
{
Task delayTask = Task.Delay(5000, token);
foreach (Item item in GetDatabaseItems()) yield return item;
delayTask.GetAwaiter().GetResult();
}
}
public void Start()
{
//...
ThreadPool.SetMinThreads(degreeOfParallelism, Environment.ProcessorCount);
new Thread(() =>
{
try
{
Partitioner
.Create(Iterator(token), EnumerablePartitionerOptions.NoBuffering)
.AsParallel()
.WithDegreeOfParallelism(degreeOfParallelism)
.WithCancellation(token)
.ForAll(item => ProcessItemAsync(item).GetAwaiter().GetResult());
}
catch (OperationCanceledException) { } // Ignore
}).Start();
}
Online demo.
The Iterator fetches unprocessed items from the database in batches, and yields them one by one. The database won't be hit more frequently than once every 5 seconds.
The PLINQ query is going to fetch a new item from the Iterator each time it has a worker available, according to the WithDegreeOfParallelism policy. The setting EnumerablePartitionerOptions.NoBuffering ensures that it won't try to fetch more items in advance.
The ThreadPool.SetMinThreads is used in order to boost the availability of ThreadPool threads, since the PLINQ is going to use lots of them. Without it the ThreadPool will not be able to satisfy the demand immediately, although it will gradually inject more threads and eventually will catch up. But since you already know how many threads you'll need, you can configure the ThreadPool from the start.
In case you dislike the idea of blocking threads, you can find a simple substitute of the Parallel.ForEachAsync here, based on the TPL Dataflow library. It requires installing a NuGet package.
The issue turned out to be the place where ServicePointManager.DefaultConnectionLimit is set.
In the version where HttpClient was only doing two requests at a time, ServicePointManager.DefaultConnectionLimit was being set before the threads were being created but after the HttpClient was initialized.
Once I moved it into the constructor before the HttpClient is initialized, everything started working.
Thank you very much to #Theodor Zoulias for the help.
TLDR; Set ServicePointManager.DefaultConnectionLimit before initializing the HttpClient.
I have a .NET Web-API service that is calling one to many other services. In this scenario I'm not sure if using Task.Run is the right approach. I've read other questions which have stated that using this is not the best approach, but if I want to run a process in parallel to ensure the most efficient calls I can't think of a better approach.
var membershipPersons = Task.Run(() => MembershipPersonsRepository.GetMembershipPersonsAsync(
new Dictionary<string, object>() {
{ "membershipId", membershipId},
{ "asAt", "Current"}}, 1));
Once I have made a call to the service above, I believe I need to wait for it to complete before I'm able to use the results, though perhaps my understanding of ContinueWith is incorrect as the first example seems to work, however the 2nd example doesn't.
await Task.WhenAll(membershipPersons).ContinueWith(
(
mpt =>
{
if (mpt.IsFaulted)
throw membershipPersons.Exception.InnerException;
This code does appear to correctly call multiple membershipPersons at once which is what I'm expecting.
//Get all Person Address details derived from Mem Persons
var personAddressesTasks = membershipPersons.Result.Select(ps => Task.Run(() => PersonAddressesRepository.GetPersonAddressesAsync(
new Dictionary<string, object>(){
{ "personId", ps.PersonId}, }, 1))).ToList();
IIS appears to crash at this stage on a Windows 10 machine, I'm unsure if this is due to a connection limitation. With approximately 5 membershipPersons this appears to work fine over and over, however for 10 or more this continually fails and forces me to do an IIS restart.
Task.WhenAll(personAddressesTasks).ContinueWith(
async p =>
{
I'm unsure if the next section is the correct approach. It's based of a Pluralsight course that Jon Skeet presented, though that was entirely based on a windows client using Async Await. Web API appears to just skip past this code without waiting for it to complete which causes an issue when I try and consume the services.
foreach (var task in personAddressesTasks)
{
try
{
if (task.IsCompleted)
{
var personAddressTask = await task;
var personAddress = personAddressTask?.FirstOrDefault();
if (personAddress != null)
personAddresses.Add(personAddress);
}
if (task.IsFaulted)
throw task.Exception;
}
catch (AggregateException e)
{
throw e.InnerException;
}
}
});
}
));
// Populate response entity.
EDIT 1:
Running the above was causing IIS to crash with 10 or more personAddress calls being run in parallel. Running sequentially like this seems to not cause an issue, even though it's still almost instantaneous to run.
var membershipPersons = Task.Run(() => MembershipPersonsRepository.GetMembershipPersons(
new Dictionary<string, object>() {
{ "membershipId", membershipId},
{ "asAt", "Current"}}, 1));
Task.WaitAll(membershipPersons);
foreach (var mp in membershipPersons.Result)
{
var pa = await PersonAddressesRepository.GetPersonAddresses(
new Dictionary<string, object>(){
{ "personId", mp.PersonId}, }, 1);
personAddresses.AddRange(pa);
};
Using only Await and no Task.Run certainly caused my set of calls to only run sequentially. So have stuck with Task.Run with no await.
I'd still like to identify if this is a suitable method in WebAPI and why IIS is crashing with a parallel call of 10 request.
We are trying to develop a WCF service for executing a long running task. The method implementing the long running task spawns a new task and immediately returns to the client(which in our case is an .aspx page) if the task has been successfully queued. The service is running on its own application pool with no recycling and single InstanceContextMode.
WCF Service
[OperationContract]
public bool testThreadAbortException()
{
Task.Factory.StartNew
(
() =>
{
try
{
//long operation
int i = 0;
while (i < 250)
{
int j = 0;
while (j < 2000000) j++;
i++;
}
ThreadState state = Thread.CurrentThread.ThreadState;
string dummy = "finished ";
}
catch(Exception exception)
{
ThreadState state = Thread.CurrentThread.ThreadState;
string msg = exception.Message;
Exception inner = exception.InnerException;
}
}
);
return true;
}
Client
protected void btnRun_Click(object sender, EventArgs e)
{
_default.IISHOST_ETLSchedulerServiceReference.ETLSchedulerServiceClient client = new _default.IISHOST_ETLSchedulerServiceReference.ETLSchedulerServiceClient();
bool ret = client.testThreadAbortException();
}
Now the problem is that while the testThreadAbortException method is being executed i catch the Thread was being aborted exception ( this always happends after the client has exited the event handler method ). The weird thing is that this exception is only thrown the first time (ie if i press the run button again the code executes fine). I have to restart my local IIS to replicate the error again.
Does anyone have a clue know why this happens??
Is there a better way to implement what i am trying to archive besides switching to a windows service??
As it seems its Mcafee antivirus after all that is accessing the hash.web file. Confirmed with process monitor. Go figure.....
For more info check this post. Issue similar to mine.
I am working on an application that talks to a motion controller over ethernet.
To connect to the controller I use a library provided by the supplier, to connect you create an instance of the controller than then tell it to connect, this has the chance to block for a few seconds (with no controllable timeout) if there is no controller present. This cause freeze ups in the UI.
To avoid this I thought I would be able to use Tasks to run the connection in a different thread.
ConnectionTask = Task.Factory.StartNew(() =>
{
try
{
RMCLink rmc = RMCLink.CreateEthernetLink(DeviceType.RMC70, "192.168.0.55");
RMC.Connect();
}
catch
{
this.logger.Log("Failed to connect");
}
}, TaskCreationOptions.LongRunning);
This has no effect whatsoever and the UI still locks up.
I think I am using them properly as if I replace it with the below code the UI is fine even though the separate thread takes a few seconds before the message comes out.
ConnectionTask = Task.Factory.StartNew(() =>
{
int x = 1;
while (x != 0) x++;
this.logger.Log("Failed to connect");
}, TaskCreationOptions.LongRunning);
Is there any way I can identify what is going on and prevent calls that I do not know anything about their inner workings from locking the UI thread.
Use async/await, something along the lines of:
public async void MyButton_Click(object sender, EventArgs e)
{
await CreateEthernetLink();
this.logger.Log("Connected!");
}
private async Task CreateEthernetLink()
{
var task = Task.Run(() => {
try
{
RMCLink rmc = RMCLink.CreateEthernetLink(DeviceType.RMC70, "192.168.0.55");
rmc.Connect();
}
catch
{
this.logger.Log("Failed to connect");
}});
await task;
}
The await will capture the current thread (or SynchronizationContext - in this case the UI thread which is being blocked) and restore it after the async work has been completed.
So the threading is all handled for you behind the scenes and you should notice no difference in your application other than the fact that your application no longer freezes when performing connections.
EDIT: I also noticed in your code your initializing rmc but calling connect on RMC. I don't think this is correct.
I am using the following code to invoke a method and run it asynchronously, providing feedback on the UI as it progresses. When I run the application within Visual Studio (in either debug or release config), the "lblSyncStatus" is updated, showing 10 different statuses over the course of a 60 second sync.
public async Task SyncConfigurations()
{
var progressIndicator = new Progress<string>(ReportProgress);
var repo = new SomeObject()
var cts = new CancellationTokenSource();
string result = string.Empty;
bool syncCompleted = false;
try
{
ReportProgress("Synchronizing user data...");
int request = 0;
if (Properties.Settings.Default.FirstRun)
request = await repo.Sync(progressIndicator, cts.Token);
else
request = await repo.Sync(progressIndicator, cts.Token, Properties.Settings.Default.ConfigLastUpdate);
syncCompleted = true;
result = "Synced complete.";
}
catch (OperationCanceledException ex)
{
result = "Sync failed.";
syncCompleted = false;
}
// Timestamp of sync.
if (syncCompleted)
{
Properties.Settings.Default.ConfigLastUpdate = DateTime.Now;
Properties.Settings.Default.FirstRun = false;
Properties.Settings.Default.Save();
}
ReportProgress(result);
}
void ReportProgress(string message)
{
//Update the UI to reflect the progress value that is passed back.
Application.Current.Dispatcher.Invoke(new Action(() =>
{
this.lblSyncStatus.Content = "Sync in progress: " + message;
}));
}
The issue I have however is when I publish via ClickOnce. When I publish with ClickOnce, launch the app and run it, the label is never updated. I attached the VS debugger to the app process right after launching it and I can see the async threads exiting so I know the sync is taking place. The UI is just not reflecting this.
Why does the UI update properly under debug or release within the IDE, but not once it is deployed? I invoke the SyncConfigurations method from within the Window_Loaded method like such: Task.Run(async () => await SyncConfigurations());.
If anyone could help out with this I'd appreciate it!
EDIT 1
It seems that something is causing the SyncConfigurations to fail when I instance a new SomeObject(). I am assuming an exception is being thrown and it is not bubbling up because of the async. I added the following code right after I invoke the method:
var sync = Task.Run(async () => await SyncConfigurations());
if (sync.Exception != null)
{
throw new Exception(sync.Exception.Message + "\n" + sync.Exception.StackTrace);
}
No exceptions are picked up. I also receive a message from the IDE now when I try to attach that it can't attach due to it not being a debug build (ClickOnce deployed release and I don't know how to change that).
So I guess my question is, how should I debug this outside of the IDE, where it seems to fail, and if am I handling the exceptions correctly (if one is indeed being thrown).
I was able to resolve this. I published a build with the debug configuration, attached the debugger to the process and was able to find the exception within MyObject and fix it.
When you need to debug a ClickOnce app, you need to ensure your build config is set to Debug prior to publishing. Problem solved.