We Keep Coding

C# Run async Task without blocking

How to run an async Task without blocking other tasks?
I have one function that iterates though a List but the problem is that when the function is called other functions won't work again until the first function is done. What are the ways of making the HandleAsync function non-blocking ?
public static async Task HandleAsync(Message message, TelegramBotClient bot)
{
await Search(message, bot); // This should be handled without working other possible functions. I have a function similar to this but which doesn't iterate though any list.
}
private static async Task Search(Message message, TelegramBotClient bot)
{
var textSplit = message.Text.Split(new[] {' '}, 2);
if (textSplit.Length == 1)
{
await bot.SendTextMessageAsync(message.From.Id, "Failed to fetch sales. Missing game name. ",
ParseMode.Html);
}
else
{
var search = await Program.itad.SearchGameAsync(textSplit[1], limit: 10, cts: Program.Cts);
if (search.Data != null)
{
var builder = new StringBuilder();
foreach (var deal in search.Data.List)
{
var title = deal.Title;
var plain = deal.Plain;
var shop = deal.Shop != null ? deal.Shop.Name : "N/A";
var urls = deal.Urls;
var priceNew = deal.PriceNew;
var priceOld = deal.PriceOld;
var priceCut = deal.PriceCut;
builder.AppendLine($"<b>Title:</b> {title}");
builder.AppendLine($"<b>Shop:</b> {shop}");
builder.AppendLine();
builder.AppendLine($"<b>Price:</b> <strike>{priceOld}€</strike> | {priceNew}€ (-{priceCut}%)");
var buttons = new[]
{
new[]
{
InlineKeyboardButton.WithUrl("Buy", urls.Buy.AbsoluteUri),
InlineKeyboardButton.WithUrl("History",
urls.Game.AbsoluteUri.Replace("info", "history"))
}
};
var keyboard = new InlineKeyboardMarkup(buttons);
var info = await Program.itad.GetInfoAsync(plain, cts: Program.Cts);
var image = info.Data.GameInfo.Image;
if (image == null) image = new Uri("https://i.imgur.com/J7zLBLg.png");
await TelegramBot.Bot.SendPhotoAsync(message.From.Id, new InputOnlineFile(image.AbsoluteUri),
builder.ToString(), ParseMode.Html, replyMarkup: keyboard,
cancellationToken: Program.Cts.Token);
builder.Clear();
}
}
else
{
await bot.SendTextMessageAsync(message.From.Id, "Failed to fetch sales. Game not found. ",
ParseMode.Html);
}
}

Data structure to use when gathering response from async calls

I am running this piece of code in my application.
public Task<BulkResponse<JObject>> GetRelatedObjectsAsync(IEnumerable<PrimaryObjectInfo> primaryObjectInfos)
{
var allSecondaries = new List<Tuple<int, List<JObject>>>();
var exceptionsDict = new ConcurrentDictionary<int, Exception>();
var relatedObjectsTasks = primaryObjectInfos.Select(async primaryObjectInfo =>
{
try
{
var secondaryObject = await objectManager.GetRelatedObjectsAsync(primaryObjectInfo);
allSecondaries.Add(Tuple.Create(primaryObjectInfo.Index, secondaryObject.ToList()));
}
catch (Exception ex)
{
exceptionsDict.TryAdd(primaryObjectInfo.Index, ex);
}
});
await Task.WhenAll(relatedObjectsTasks);
return ConvertToBulkResponse(allSecondaries, exceptionsDict);
}
When I run this code allSecondaries object sometimes returns valid list of results and sometimes the code ends up catching exceptions for the parallel threads I have for each primaryObjectInfo.
Async method objectManager.GetRelatedObjectsAsync() internally call 4-5 async functions and there are functions where parameters are passed by reference. (ref keyword)
Question:
Am I using the right data structure to consolidate the result from all parallel threads ?
If yes, what could be the reason I am getting different result every time?

You'd better split the execution from the results gathering:
public Task<BulkResponse<JObject>> GetRelatedObjectsAsync(
IEnumerable<PrimaryObjectInfo> primaryObjectInfos)
{
var relatedObjectsTasks = primaryObjectInfos
.Select(
async primaryObjectInfo =>
(primaryObjectInfo.Index,
await objectManager.GetRelatedObjectsAsync(primaryObjectInfo)))
.ToList();
try
{
await Task.WhenAll(relatedObjectsTasks);
}
catch
{
// observe each task's, exception
}
var allSecondaries = new List<(int index, List<JObject> related)>();
var exceptionsDict = new Dictionary<int, Exception>();
foreach (var relatedObjectsTask in relatedObjectsTasks)
{
try
{
allSecondaries.Add(relatedObjectsTask.Result);
}
catch (Exception ex)
{
exceptionsDict.Add(relatedObjectsTask.index, ex);
}
}
return ConvertToBulkResponse(allSecondaries, exceptionsDict);
}
You could look into each task's IsFaulted/Exception and IsCancelled properties instead of causing exceptions to be thrown:
foreach (var relatedObjectsTask in relatedObjectsTasks)
{
if (relatedObjectsTask.IsCancelled)
{
exceptionsDict.Add(
relatedObjectsTask.index,
new TaskCancelledException(relatedObjectsTask));
}
else if (relatedObjectsTask.IsFaulted)
{
exceptionsDict.TryAdd(relatedObjectsTask.index, ex);
}
else
{
allSecondaries.Add(relatedObjectsTask.Result);
}
}

SqlWorkflowInstanceStore WaitForEvents returns HasRunnableWorkflowEvent but LoadRunnableInstance fails

Dears
Please help me with restoring delayed (and persisted) workflows.
I'm trying to check on self-hosted workflow store is there any instance that was delayed and can be resumed. For testing purposes I've created dummy activity that is delayed and it persists on delay.
generally resume process looks like:
get WF definition
configure sql instance store
call WaitForEvents
is there event with HasRunnableWorkflowEvent.Value name and if it is
create WorkflowApplication object and execute LoadRunnableInstance method
it works fine if store is created|initialized, WaitForEvents is called, store is closed. In such case store reads all available workflows from persisted DB and throws timeout exception if there is no workflows available to resume.
The problem happens if store is created and loop is started only for WaitForEvents (the same thing happens with BeginWaitForEvents). In such case it reads all available workflows from DB (with proper IDs) but then instead of timeout exception it is going to read one more instance (I know exactly how many workflows is there ready to be resumed because using separate test database). But fails to read and throws InstanceNotReadyException. In catch I'm checking workflowApplication.Id, but it was not saved with my test before.
I've tried to run on new (empty) persistent database and result is the same :(
This code fails:
using (var storeWrapper = new StoreWrapper(wf, connStr))
for (int q = 0; q < 5; q++)
{
var id = Resume(storeWrapper); // InstanceNotReadyException here when all activities is resumed
But this one works as expected:
for (int q = 0; q < 5; q++)
using (var storeWrapper = new StoreWrapper(wf, connStr))
{
var id = Resume(storeWrapper); // timeout exception here or beginWaitForEvents continues to wait
What is a best solution in such case? Add empty catch for InstanceNotReadyException and ignore it?
Here are my tests
const int delayTime = 15;
string connStr = "Server=db;Database=AppFabricDb_Test;Integrated Security=True;";
[TestMethod]
public void PersistOneOnIdleAndResume()
{
var wf = GetDelayActivity();
using (var storeWrapper = new StoreWrapper(wf, connStr))
{
var id = CreateAndRun(storeWrapper);
Trace.WriteLine(string.Format("done {0}", id));
}
using (var storeWrapper = new StoreWrapper(wf, connStr))
for (int q = 0; q < 5; q++)
{
var id = Resume(storeWrapper);
Trace.WriteLine(string.Format("resumed {0}", id));
}
}
Activity GetDelayActivity(string addName = "")
{
var name = new Variable<string>(string.Format("incr{0}", addName));
Activity wf = new Sequence
{
DisplayName = "testDelayActivity",
Variables = { name, new Variable<string>("CustomDataContext") },
Activities =
{
new WriteLine
{
Text = string.Format("before delay {0}", delayTime)
},
new Delay
{
Duration = new InArgument<TimeSpan>(new TimeSpan(0, 0, delayTime))
},
new WriteLine
{
Text = "after delay"
}
}
};
return wf;
}
Guid CreateAndRun(StoreWrapper sw)
{
var idleEvent = new AutoResetEvent(false);
var wfApp = sw.GetApplication();
wfApp.Idle = e => idleEvent.Set();
wfApp.Aborted = e => idleEvent.Set();
wfApp.Completed = e => idleEvent.Set();
wfApp.Run();
idleEvent.WaitOne(40 * 1000);
var res = wfApp.Id;
wfApp.Unload();
return res;
}
Guid Resume(StoreWrapper sw)
{
var res = Guid.Empty;
var events = sw.GetStore().WaitForEvents(sw.Handle, new TimeSpan(0, 0, delayTime));
if (events.Any(e => e.Equals(HasRunnableWorkflowEvent.Value)))
{
var idleEvent = new AutoResetEvent(false);
var obj = sw.GetApplication();
try
{
obj.LoadRunnableInstance(); //instancenotready here if the same store has read all instances from DB and no delayed left
obj.Idle = e => idleEvent.Set();
obj.Completed = e => idleEvent.Set();
obj.Run();
idleEvent.WaitOne(40 * 1000);
res = obj.Id;
obj.Unload();
}
catch (InstanceNotReadyException)
{
Trace.TraceError("failed to resume {0} {1} {2}", obj.Id
, obj.DefinitionIdentity == null ? null : obj.DefinitionIdentity.Name
, obj.DefinitionIdentity == null ? null : obj.DefinitionIdentity.Version);
foreach (var e in events)
{
Trace.TraceWarning("event {0}", e.Name);
}
throw;
}
}
return res;
}
Here is store wrapper definition I'm using for test:
public class StoreWrapper : IDisposable
{
Activity WfDefinition { get; set; }
public static readonly XName WorkflowHostTypePropertyName = XNamespace.Get("urn:schemas-microsoft-com:System.Activities/4.0/properties").GetName("WorkflowHostType");
public StoreWrapper(Activity wfDefinition, string connectionStr)
{
_store = new SqlWorkflowInstanceStore(connectionStr);
HostTypeName = XName.Get(wfDefinition.DisplayName, "ttt.workflow");
WfDefinition = wfDefinition;
}
SqlWorkflowInstanceStore _store;
public SqlWorkflowInstanceStore GetStore()
{
if (Handle == null)
{
InitStore(_store, WfDefinition);
Handle = _store.CreateInstanceHandle();
var view = _store.Execute(Handle, new CreateWorkflowOwnerCommand
{
InstanceOwnerMetadata = { { WorkflowHostTypePropertyName, new InstanceValue(HostTypeName) } }
}, TimeSpan.FromSeconds(30));
_store.DefaultInstanceOwner = view.InstanceOwner;
//Trace.WriteLine(string.Format("{0} owns {1}", view.InstanceOwner.InstanceOwnerId, HostTypeName));
}
return _store;
}
protected virtual void InitStore(SqlWorkflowInstanceStore store, Activity wfDefinition)
{
}
public InstanceHandle Handle { get; protected set; }
XName HostTypeName { get; set; }
public void Dispose()
{
if (Handle != null)
{
var deleteOwner = new DeleteWorkflowOwnerCommand();
//Trace.WriteLine(string.Format("{0} frees {1}", Store.DefaultInstanceOwner.InstanceOwnerId, HostTypeName));
_store.Execute(Handle, deleteOwner, TimeSpan.FromSeconds(30));
Handle.Free();
Handle = null;
_store = null;
}
}
public WorkflowApplication GetApplication()
{
var wfApp = new WorkflowApplication(WfDefinition);
wfApp.InstanceStore = GetStore();
wfApp.PersistableIdle = e => PersistableIdleAction.Persist;
Dictionary<XName, object> wfScope = new Dictionary<XName, object> { { WorkflowHostTypePropertyName, HostTypeName } };
wfApp.AddInitialInstanceValues(wfScope);
return wfApp;
}
}

I'm not workflow foundation expert so my answer is based on the official examples from Microsoft. The first one is WF4 host resumes delayed workflow (CSWF4LongRunningHost) and the second is Microsoft.Samples.AbsoluteDelay. In both samples you will find a code similar to yours i.e.:
try
{
wfApp.LoadRunnableInstance();
...
}
catch (InstanceNotReadyException)
{
//Some logging
}
Taking this into account the answer is that you are right and the empty catch for InstanceNotReadyException is a good solution.

Is Object created before or after method calls?

I have three different databases that I need to check that I am connected to. This is what I originally have, which works perfectly fine.
public async Task<ServiceAvailabilityDTO> ServiceAvailabilityStatus()
{
return new ServiceAvailabilityDTO
{
IsDb1Online = await IsDb1Available(),
IsDb2Online = IsDb2Available(),
IsDb3Online = await IsDb3Available()
};
}
private async Task<bool> IsDb1Available()
{
var count = await _db1Service.GetDbCount();
if (count > 0)
return true;
return false;
}
private bool IsDb2Available()
{
if (_db2Service.GetDbCount() > 0)
return true;
return false;
}
private async Task<bool> IsDb3Available()
{
var pong = await _db3Provider.PingDb();
if(pong.Success == true && pong.Version != null)
return true;
return false;
}
Now however, I need to log exception messages in my DTO for each DB check.
public async Task<ServiceAvailabilityDTO> ServiceAvailabilityStatus()
{
return new ServiceAvailabilityDTO
{
IsDb1Online = await IsDb1Available(),
IsDb2Online = IsDb2Available(),
IsDb3Online = await IsDb3Available(this) // This is an example. I want to pass the reference of **ServiceAvailabilityDTO** to **IsDb3Available**
};
}
private async Task<bool> IsDb3Available(ServiceAvailabilityDTO availability)
{
try
{
var pong = await _db3Provider.PingDb();
if(pong.Success == true && pong.Version != null)
return true;
return false;
}
catch (Exception e)
{
var exceptionMessage = e.Message;
if (e.InnerException != null)
{
// This is what I hope to put into the object reference
exceptionMessage = String.Join("\n", exceptionMessage, e.InnerException.Message);
availability.db3Exception = exceptionMessage ;
}
return false;
}
}
My question is;
Can I keep my return method the same as in the first example, and pass the object reference to each method to store the exception and still return my bool value.
Or does the object not get created until all of the method calls have happened, and then create the object with the returned values?
I know I could just create the object normally and pass it in each
method call, but it is specifically this way of doing it that has
inspired me to ask this question, purely to be informed and learn
from.
Thanks.

No, you cannot do it like this because in the context of what you're doing this does not refer to the object you're populating, it refers to the object containing the method you're calling.
public async Task<ServiceAvailabilityDTO> ServiceAvailabilityStatus()
{
return new ServiceAvailabilityDTO
{
IsDb1Online = await IsDb1Available(),
IsDb2Online = IsDb2Available(),
IsDb3Online = await IsDb3Available(this) // here 'this' does NOT ref to ServiceAvailabilityDTO
};
}
There is no keyword which does refer to ServiceAvailabilityDTO either, so you're left with creating the object, and passing it to each method. At this point, I dont think there is much point you returning the boolean either - you may as well set the boolean property in line
public async Task<ServiceAvailabilityDTO> ServiceAvailabilityStatus()
{
var sa = new ServiceAvailabilityDTO();
await CheckDb1Available(sa);
CheckDb2Available(sa);
await CheckDb3Available(sa);
return sa;
}
(Note I've renamed the methods from Is* to Check* as the former implies a return boolean, the latter implies something going on inline.)

Threading issues with SQLite3 and C# async

I am trying to save some data to a SQLite3 database. If I do not use async, I can save the data without any problems. As soon as I try to use the following code however, I receive the following error:
{Unable to evaluate expression because the code is optimized or a native frame is on top of the call stack.}
From my UI, I invoke the following SyncDomainTablesAsync method:
private readonly IDataCoordinator _coordinator;
public Configuration(IDataCoordinator coordinator)
{
_coordinator = coordinator;
}
public async Task<int> SyncDomainTablesAsync(IProgress<string> progress, CancellationToken ct, DateTime? lastDateSynced=null, string tableName = null)
{
//Determine the different type of sync calls
// 1) Force Resync (Drop/Create Tables and Insert)
// 2) Auto Update
var domainTable = await GetDomainTablesAsync(progress,ct,lastDateSynced, tableName);
var items = domainTable.Items;
int processCount = await Task.Run<int>( async () =>
{
int p = 0;
progress.Report(String.Format("Syncing Configurations..."));
foreach (var item in items)
{
progress.Report(String.Format("Syncing {0} Information",item.Key));
var task = await SyncTableAsync(item.Value); // INVOKED BELOW
if (task) progress.Report(String.Format("Sync'd {0} {1} records", item.Value.Count,item.Key));
if (ct.IsCancellationRequested) goto Cancelled;
p += item.Value.Count;
}
Cancelled:
if (ct.IsCancellationRequested)
{
//Update Last Sync'd Records
progress.Report(String.Format("Canceling Configuration Sync..."));
ct.ThrowIfCancellationRequested();
}
else
progress.Report(String.Format("Syncing Configurations Compleleted"));
return p;
},ct);
return processCount;
}
private async Task<bool> SyncTableAsync(IEnumerable<object> items, bool includeRelationships = false)
{
try
{
//TODO: Replace with SaveObjects method
var i = await Task.Run(() => _coordinator.SaveObjects(items, includeRelationships));
if (i == 0)
return false;
}
catch(Exception ex)
{
return false;
}
return true;
}
The UI invokes the SyncDomainTablesAsync method. I then create a new Task and loop through the items that were returned from the GetDomainTablesAsync method. During each iteration I await until the SyncTableAsync method completes. Within the SyncTableAsync I am calling a SaveObject method inside of a class that implements my IDataCoordinator interface.
public override int SaveObjects(IEnumerable<object> items, Type underlyingType, bool saveRelationships = true)
{
int result = 0;
if (items == null)
throw new ArgumentNullException("Can not save collection of objects. The collection is null.");
else if (items.Count() == 0)
return 0;
// Check if table exists.
foreach (var item in items)
this.CreateTable(item.GetType(), saveRelationships);
using (SQLiteConnection connection = new SQLiteConnection(this.StorageContainerPath))
{
connection.BeginTransaction();
foreach (var item in items)
{
result += ProcessSave(item, saveRelationships, connection);
}
try
{
connection.Commit();
}
catch (SQLiteException ex)
{
connection.Rollback();
throw ex;
}
}
return result;
}
public override int CreateTable(Type type, bool createRelationalTables = false)
{
if (this.TableExists(type) == 1)
return 1;
using (SQLiteConnection cn = new SQLiteConnection(this.StorageContainerPath))
{
try
{
// Check if the Table attribute is used to specify a table name not matching that of the Type.Name property.
// If so, we generate a Sql Statement and create the table based on the attribute name.
//if (Attribute.IsDefined(type, typeof(TableAttribute)))
//{
// TableAttribute attribute = type.GetAttribute<TableAttribute>();
// Strongly typed to SQLiteCoordinator just to get a SqlQuery instance. The CreateCommand method will create a table based on 'type'
var query = new SqlQuery<SQLiteCoordinator>().CreateCommand(DataProviderTypes.Sqlite3, type);
query = query.TrimEnd(';') + ";";
cn.Execute(query);
//}
// Otherwise create the table using the Type.
//else
//{
// cn.CreateTable(type);
//}
// If we are to create relationship tables, we cascade through all relationship properties
// and create tables for them as well.
if (createRelationalTables)
{
this.CreateCascadingTables(type, cn);
}
}
catch (Exception ex)
{
return 0;
}
}
return 1;
}
The flow of the code goes
UI->SyncDomainTablesAsync->SyncTableAsync->SaveObjects->SaveTable(type)
The issue that I have is within Save Table. If I just use SaveTable synchronously I have no issues. Using it in my async method above, causes a thread abort exception. The exception is thrown within the SQLite.cs file included with SQLite.net (within the . The weird thing is that the table is created in the database, even though the exception is thrown. The error is thrown some times when the Prepare() function is called and the rest of the time when the SQLite3.Step() function is called.
public int ExecuteNonQuery ()
{
if (_conn.Trace) {
Debug.WriteLine ("Executing: " + this);
}
var r = SQLite3.Result.OK;
var stmt = Prepare (); // THROWS THE ERRROR
r = SQLite3.Step(stmt); // THROWS THE ERRROR
Finalize(stmt);
if (r == SQLite3.Result.Done) {
int rowsAffected = SQLite3.Changes (_conn.Handle);
return rowsAffected;
} else if (r == SQLite3.Result.Error) {
string msg = SQLite3.GetErrmsg (_conn.Handle);
throw SQLiteException.New (r, msg);
} else {
throw SQLiteException.New (r, r.ToString ());
}
}
I assume that because my foreach statement awaits the return of SyncTableAsync that none of the threads are closed. I am also getting a system transaction critical exception that says "attempting to access a unloaded app domain".
Am I using await/async incorrectly with Sqlite3 or is this an issue with Sqlite3 that I am not aware of.
Attached is a photo of the Parallel's stack and the exception.
EDIT
When I try to run the code above as well in unit tests, the unit tests process never dies. I have to exit Visual Studio in order to get the process to die. I am assuming something in SQLite.dll is grabbing a hold of the process when the exception is thrown and not letting go, but I am not sure.
EDIT 2
I can modify the initial method SyncDomainTablesAsync to the following and the code runs without error. The issue is my use of async and await I believe.
public async Task<int> SyncDomainTablesAsync(IProgress<string> progress, CancellationToken ct, DateTime? lastDateSynced=null, string tableName = null)
{
var domainTable = await GetDomainTablesAsync(progress,ct,lastDateSynced, tableName);
var items = domainTable.Items;
foreach (var item in items)
{
_coordinator.SaveObjects(item.Value, typeof(object), true);
}
return 1;
}