We Keep Coding

Generic TaskExecution Method with Variable Parameter functions & return types

I am trying to refactor Parallel Task Executor on 2 methods which executes on Variable Parameters & return types. can assume that both methods have same input parameters and return types but they can change.
For Eg: MyObject Task1(string,string,int), MyObject Task2(string,string,int) is valid
MyObject3 Task1(string,string,int, double), MyObject3 Task2(string,string,int,double) is valid too
currently, I am implementing it as follows. This works for one or two methods. But now I hit a case I need to do this for around 15 methods. How do I go about refactoring into its own methods and just calling it as RunTaskInParallel<ReturnTypeOfBothMethods>(Task1(params...), Task2(params...))
we can assume both methods (Task1, Task2) have same exact parameters & return types.
public async Task<UserPolicies> GetUserPoliciesAsync(string userId, string tenantId, IList<UserAssignedSettingType> userPoliciesToFetch)
{
if (FlagManager.GetSettingValue<bool>(FlagConstants.EnableMyFeature, tenantId, logSettingValue: false)
&& userPoliciesToFetch.Contains(UserAssignedSettingType.UserPermission))
{
//Read tenant Status from cache or Backend.
var tenantStatus = await GetTenantStatusFromCacheOrBackendAsync(tenantId);
if (tenantStatus.IsFlighted && tenantStatus.IsMigrated.Equals(AAEConstants.TenantMigrationCompleted, StringComparison.OrdinalIgnoreCase)) //Tenant signedup & Migrated completely to AAE Flow
{
//This is the Part that needs to be repeated for different method types.
//Here its calling OtherService Method (GetUserPoliciesAsync) and custom Logic for Man in Middle method (GetAppDocFromCacheOrBackendAsync)
var taskList = new List<Task>()
{
_policyBackendProxy.GetUserPoliciesAsync(userId, tenantId, userPoliciesToFetch)
};
//Preparing Default User teamsAppPerm document in case of BE request fail or tenantStatus.ResolutionModel == default.
TeamsAppPermissionPolicy teamsAppPermissionDocFromAAE = new TeamsAppPermissionPolicy();
if (tenantStatus.ResolutionModel.Equals(AAEConstants.TenantResolutionModelCustom, StringComparison.OrdinalIgnoreCase))
{
//only call AAE Cache or Backend for tenantStatus.ResolutionModel == Custom
taskList.Add(GetAppDocFromCacheOrBackendAsync(userId, tenantId));
}
var allTasks = Task.WhenAll(taskList);
await allTasks;
var otherPolicies = taskList[0].IsFaulted ?
new UserPolicies() : ((Task<UserPolicies>)taskList[0]).Result;
if (taskList.Count > 1) //Making sure we only read response if we hit AAE Backend. In which case tasklist length is 2.
{
teamsAppPermissionDocFromAAE = taskList[1].IsFaulted ?
new TeamsAppPermissionPolicy() : ((Task<TeamsAppPermissionPolicy>)taskList[1]).Result;
}
otherPolicies.AppsPermissionsPolicy = teamsAppPermissionDocFromAAE.ToMTTeamsAppPermissionPolicy().ToMTPolicyDocument();
//Wrap AAE TeamsApp doc response into other Policies
return otherPolicies;
}
else
{
//Tenant has not signed up or not migrated. Use existing PolicyService flow
return await _policyBackendProxy.GetUserPoliciesAsync(userId, tenantId, userPoliciesToFetch);
}
}
else
{
//AAE feature is not enabled in ECS aka Kill Switch
return await _policyBackendProxy.GetUserPoliciesAsync(userId, tenantId, userPoliciesToFetch);
}
}
Here _policyBackendProxy is the original flow and I only interject its flow if certain conditions are met which are my if Statements. else go by previous flow.
This method returns UserPolicies object but there are few other methods I need to parallel execute old method or my corresponding new method but do not want to repeat the same logic for diff methods and return types.
Another similar method but returning diff type is below.
public async Task<AcmsSettings> GetTenantDefaultPoliciesAsync(string tenantId, IList<UserAssignedSettingType> userPoliciesToFetch)
{
if (_flagManager.GetSettingValue<bool>(EcsSettingsConstants.EnableAppCatalogService, tenantId, logSettingValue: false)
&& userPoliciesToFetch.Contains(UserAssignedSettingType.AppsPermission))
{
var tenantStatus = await GetTenantStatusFromCacheOrBackendAsync(tenantId);
if (tenantStatus.IsFlighted
&& tenantStatus.IsMigrated.Equals(AAEConstants.TenantMigrationCompleted, StringComparison.OrdinalIgnoreCase)) //Tenant signedup & Migrated completely to AdminAppCatalog Flow
{
var taskList = new List<Task>();
if (tenantStatus.ResolutionModel.Equals(AAEConstants.TenantResolutionModelCustom, StringComparison.OrdinalIgnoreCase))
{
taskList.Add(GetTeamsAppDocFromCacheOrBackendAsync(null, tenantId));
}
//Preparing Default User teamsAppPerm document in case of BE request fail or tenantStatus.ResolutionModel == default.
TeamsAppPermissionPolicy teamsAppPermissionDocFromAAE = new TeamsAppPermissionPolicy();
var allTasks = Task.WhenAll(taskList);
await allTasks;
var otherPolicies = taskList[0].IsFaulted ?
new AcmsSettings() : ((Task<AcmsSettings>)taskList[0]).Result;
if (taskList.Count > 1) //Making sure we only read response if we hit AAE Backend. In which case tasklist length is 2.
{
teamsAppPermissionDocFromAAE = taskList[1].IsFaulted ?
new TeamsAppPermissionPolicy() : ((Task<TeamsAppPermissionPolicy>)taskList[1]).Result;
}
otherPolicies.AppsPermissionPolicy = teamsAppPermissionDocFromAAE.ToMTTeamsAppPermissionPolicy().ToMTPolicyDocument();
return otherPolicies;
}
else
{
return await _policyBackendProxy.GetTenantDefaultPoliciesAsync(tenantId, userPoliciesToFetch);
}
}
else
{
return await _policyBackendProxy.GetTenantDefaultPoliciesAsync(tenantId, userPoliciesToFetch);
}
}
The above method is similar to the first one except its taking diff parameters (string tenantId, IList<UserAssignedSettingType> userPoliciesToFetch) and returning diff type AcmsSettings.
I have to repeat this logic for every method in interface. I am certain there is a better and efficient way but not sure what is it.

How do I await this callback method to perform logic on the return data?

So I'm working in Silverlight right now unfortunately for the first time. I'm decently familiar with callbacks, but I'm not entirely sure how to convert this method to be synchronous to perform logic on the order data.
I've been frequently told that making this synchronous was ill-advised, but my goal is to check if certain fields have been modified in the XAML UI and are different from what exists in the database. Then prompt for a reason for the change. If there is a better way to go about this, I'd love to know.
I'm in Silverlight 5 with .Net Framework 4.0 in VS 2013
Thank you! Here's the async order provider:
public void GetOrder(string ordNumber, Action<Func<OrderLoadResults>> callback)
{
String exStr = String.Format("{0}.{1}() --> received an empty value for",
this.GetType().Name,
MethodBase.GetCurrentMethod().Name);
if (ordNumber == null)
{
throw new ArgumentNullException("ordNumber", exStr);
}
if (callback == null)
{
throw new ArgumentNullException("callback", exStr);
}
IOrderServiceAsync channel = CreateChannel();
AsyncCallback asyncCallback = ar => GetOrderCallback(callback, ar);
channel.BeginGetOrderByOrdNumber(ordNumber, asyncCallback.ThreadSafe(), channel);
}
And here's what I'm doing with it:
public List<ATMModifiedFieldModel> CheckForATMModifiedFields()
{
if (!_order.Stops.Items.Any(x => x.ModelState == ModelState.Modified))
{
return null;
}
List<StopModel> oldStop = new List<StopModel>();
Provider.OrderProvider orderProvider = new Provider.OrderProvider();
//Looking to convert this method to sync to pull the order info out to compare against
//orderProvider.GetOrder(_order.Item.OrdHdrNumber.ToString(),getResult => OnGetOrderComplete(getResult));
List<ATMModifiedFieldModel> modifiedFields = new List<ATMModifiedFieldModel>();
foreach (StopModel stop in _order.Stops.Items)
{
if (stop.ModelState == ModelState.Modified)
{
foreach (string ATMFieldName in Enum.GetNames(typeof(ATMFields)))
{
string beforeValue = "before value"; //Should check the value in the database
string afterValue = stop.GetType().GetProperty(ATMFieldName).GetValue(stop, null).ToString();
if (beforeValue != afterValue)
{
modifiedFields.Add(new ATMModifiedFieldModel(ATMFieldName, beforeValue, afterValue, stop.StpNumber, "Stop"));
}
}
}
}
return modifiedFields;
}

Firestore.Query.GetSnapshotAsync() never returns and doesn't give any errors - the program just doesn't stop

I'm building a game in Unity version 2019.4.1f1 (LTS) and i'm using Firebase Firestore database.
Basically when I try to fetch the user from the database, specifically when i'm calling GetSnapshotAsync(), it gets stuck.
The hierarchy is as follows:
StartState calls FetchUserState which calls DBGetObjectState which communicates with DatabaseManager (DatabaseManager contains the line where GetSnapshotAsync() gets called).
Would be really nice if someone could help me figure out this problem.
Thank you for helping!
class FetchUserState
//fetch user by userId
await stateManager.PushTempState(new DBGetObjectState<T>(StringConstants.DB_USERS_COLLECTION, //("Users")
field: "UserId", fieldEqualTo: _userId, withLoading: false));
class DBGetObjectState
Log("Getting Documents");
var documents = await DatabaseManager.GetDocuments<T>(_path, _limit, _orderBy, _orderByDescending, _field, _fieldEqualTo);
Log("Got something, checking..."); // doesn't get here
class DatabaseManager
public static async Task<Collection<T>> GetDocuments<T>(string path, int limit = -1,
string orderBy = null, string orderByDescending = null, string field = null, object fieldEqualTo = null)
{
var collectionRef = DBUtills.GetCollectionRef(path);
Query query = collectionRef;
if (field != null && fieldEqualTo != null)
query = query.WhereEqualTo(field, fieldEqualTo);
if (orderBy != null)
query = query.OrderBy(orderBy);
if (orderByDescending != null)
query = query.OrderByDescending(orderByDescending);
if (limit >= 0)
query = query.Limit(limit);
Debug.Log("Snapshotting!");
QuerySnapshot collectionSnapshot = await query.GetSnapshotAsync();
Debug.Log("got a snapshot, checking..."); // doesn't get here
//....
}

Thanks to anyone who tried to help, apparently it was a plugin that I used for handling tasks, I should've checked it first before even asking the question.
Anyways, if someone still experiences an issue, #derHugo's answer was definitely a solution, instead of awaiting the task, simply use ContinueWithOnMainThread() to create a callback that will be called after the task is finished, like that:
query.GetSnapshotAsync().ContinueWithOnMainThread(task =>
{
if(task.IsFaulted)
{
//....
}
QuerySnapshot collectionSnapshot = task.Result;
//....
}

Correct way to use Async/Await

I've been working for a few days on a performance problem.
Before I delve deeper I want to quickly explain how the specific service work.
I have a main Service that get a request and send requests to other micro-services but the single Entry Point for the user is to Main service, I thinks is more simple to understand with this image:
After the Main service get request from API he do some logic, query the Db and then get a list, every item on the list has Id, to get enrichment about every item the main service create request to one of the micro-service.
For example John request main service, main service get from Db a list of 90 items then the main service will create 90 calls to micro service and return to John single response that include 90 items.
Now the question is only about the right way to create async call to micro service.
This how I develop this part:
GetDetailsAsync(Id, result.Items, request.SystemComponentId);
private static void GetDetailsAsync(string Id, List<MainItem> items, int systemId)
{
var getDetailsTasks = new List<Task>();
foreach (MainItem single in items)
{
getDetailsTasks.Add(SetSingleDetailsAsync(Id, single, systemId));
}
Task.WhenAll(getDetailsTasks);
}
private static async Task SetSingleDetailsAsync(string Id, MainItem single, int systemId)
{
single.ActivityExtendedDetails = await ProcessItemDetailsRequest.GetItemDetailsAsync(Id, single.TypeId,
single.ItemId, systemId);
}
public static Task<JObject> GetItemDetailsAsync(string id, short type,
string itemId, int systemId)
{
var typeList = ActivityTypeDetails.GetActivityTypes();
var url = GetActivityUrl(id, type, itemId, typeList);
if (url == null)
{
throw new Failure($"No url defined for type {type}");
}
try
{
JObject res;
using (var stream = client.GetStreamAsync(url).Result)
using (var sr = new StreamReader(stream))
using (var reader = new JsonTextReader(sr))
{
var serializer = new JsonSerializer();
res = serializer.Deserialize<JObject>(reader);
}
return Task.FromResult(res);
}
catch(Exception ex)
{
Logger.Warn(
$"The uri {url} threw exception {ex.Message}.");
//[Todo]throw exception
return null;
}
}
This code run and the result is not good enough, the CPU rises very quickly and becomes very high, I think that I has a problem on GetItemDetailsAsync func because I use client.GetStreamAsync(url).Result
when using .Result it's block until the task is completed.
So I do some minor change on GetItemDetailsAsync to try to be really async:
public static async Task<JObject> GetItemDetailsAsync(string id, short type,
string itemId, int systemId)
{
var typeList = ActivityTypeDetails.GetActivityTypes();
var url = GetActivityUrl(id, type, itemId, typeList);
if (url == null)
{
throw new Failure($"No url defined for type {type}");
}
try
{
JObject res;
using (var stream = await client.GetStreamAsync(url))
using (var sr = new StreamReader(stream))
using (var reader = new JsonTextReader(sr))
{
var serializer = new JsonSerializer();
res = serializer.Deserialize<JObject>(reader);
}
return res;
}
catch(Exception ex)
{
Logger.Warn(
$"The uri {url} threw exception {ex.Message}.");
//[Todo]throw exception
return null;
}
}
But now I get null where I supposed to get the data that come from Async function.
I try to debugging and I noticed something weird, everything happen likes as I would expect: the methods was called, request to micro-service was executed and get response but the response from the End-Point(which is found on main-service) return before the async method return from micro-service, that cause that I get null instead of my expected data.
I thinks that maybe I don't use correctly async\await and would be happy if anyone could explain how this behavior happens

Caching Pattern for Fetching Collections

Is there a caching strategy/pattern that can be used for fetching multiple items, where only some may be cached? Does it make sense to use a cache in such scenarios?
More Detail
Where the cache would return a single (or known number of) result(s) caching's simple; if it's in the cache we return it, if not we fetch it, add it to the cache, then return it:
//using System.Runtime.Caching;
ObjectCache cache = MemoryCache.Default;
TimeSpan cacheLifetime = new TimeSpan(0, 20, 0);
SomeObjectFinder source = new SomeObjectFinder();
public SomeObject GetById(long id)
{
return GetFromCacheById(id) ?? GetFromSourceById(id);
}
protected SomeObject GetFromCacheById(long id)
{
return (SomeObject)cache.Get(id.ToString(),null);
}
protected SomeObject GetFromSourceById (long id)
{
SomeObject result = source.GetById(id);
return result == null ? null : (SomeObject)cache.AddOrGetExisting(id.ToString(), result, DateTimeOffset.UtcNow.Add(cacheLifetime), null);
}
However, where we don't know how many results to expect, if we fetched the results from the cache we wouldn't know that we'd fetched everything; only what's been cached. So following the above pattern, if none of or all of the results had been cached we'd be fine; but if half of them had been, we'd get a partial result set.
I was thinking the below may make sense; but I've not played with async/await before, and most of what I've read implies that calling async code from synchronous code is generally considered bad; so this solution's probably not a good idea.
public IEnumerable<SomeObject> GetByPartialName(string startOfName)
{
//kick off call to the DB (or whatever) to get the full result set
var getResultsTask = Task.Run<IList<SomeObject>>(async() => await GetFromSourceByPartialNameAsync(startOfName));
//whilst we leave that to run in the background, start getting and returning the results from the cache
var cacheResults = GetFromCacheByPartialName(startOfName);
foreach (var result in cacheResults)
{
yield return result;
}
//once all cached values are returned, wait for the async task to complete, remove the results we'd already returned from cache, then add the remaining results to cache and return those also
var results = getResultsTask.GetAwaiter().GetResult();
foreach (var result in results.Except(cacheResults))
{
yield return CacheAddOrGetExistingByName(result.Name, result);
}
}
protected async Task<IList<SomeObject>> GetFromSourceByPartialNameAsync(string startOfName)
{
return source.GetByPartialName(startOfName);
}
My assumption is that the answer's going to be "in this scenario either cache everything beforehand, or don't use cache"... but hopefully there's some better option.

You can use the Decorator and strategy pattern, here is the link to Steve Smith post regarding building a cached repository, and for individual items work as well, so you can use a decorator using redis and if not found it goes to the database and then save it to database, after that querying the data will be really fast.
Here is some sample code:
public class CachedAlbumRepository : IAlbumRepository
{
private readonly IAlbumRepository _albumRepository;
public CachedAlbumRepository(IAlbumRepository albumRepository)
{
_albumRepository = albumRepository;
}
private static readonly object CacheLockObject = new object();
public IEnumerable<Album> GetTopSellingAlbums(int count)
{
Debug.Print("CachedAlbumRepository:GetTopSellingAlbums");
string cacheKey = "TopSellingAlbums-" + count;
var result = HttpRuntime.Cache[cacheKey] as List<Album>;
if (result == null)
{
lock (CacheLockObject)
{
result = HttpRuntime.Cache[cacheKey] as List<Album>;
if (result == null)
{
result = _albumRepository.GetTopSellingAlbums(count).ToList();
HttpRuntime.Cache.Insert(cacheKey, result, null,
DateTime.Now.AddSeconds(60), TimeSpan.Zero);
}
}
}
return result;
}
}