We Keep Coding

MultiShardConnection and Entity Framework

Introduction. I have found one question and one issue related to the problem :
elastic-db-tools github issue. 2015
stackoverflow question. 2015
Actually I work with multi-tenant architecture. Using latest .net core, ef core and ElasticScale. Most of the time I work with only one tenant - usual case. ShardMapManager provide SqlConnection, and then I can create DbContext using it.
Also I have the following case - I need to request more than one shard at a time. Something like - looking for a Customer through all tenants.
ElasticScale has MultiShardConnection for that. I have to write usual ADO.NET queries to solve this problem. I don't like ADO.NET queries for real. The whole solution works with the ORM - EF, thus I want to use ORM everywhere! I was trying to find something like an adapter for the EF...
Okay. I wrote the following solution:
public sealed class TenantDataContextFactory : ITenantDataContextFactory
{
private readonly Lazy<string> _cachedConnectionString;
private readonly IShardingService _shardingService;
private readonly IConfigurationManager _configurationManager;
private readonly ILogger<TenantDataContextFactory> _logger;
//sp_set_session_context required MSSQL SERVER 2016 or above!!!
private const string SpSetSessionContextQuery = #"exec sp_set_session_context #key=N'TenantId', #value=#TenantId";
public async Task<TenantDataContext> CreateAsync(Guid tenantId)
{
SqlConnection sqlConnection = null;
try
{
sqlConnection = await _shardingService.ShardMap
.OpenConnectionForKeyAsync(key: tenantId,
connectionString: _cachedConnectionString.Value,
options: ConnectionOptions.Validate)
.ConfigureAwait(false);
var cmd = sqlConnection.CreateCommand();
cmd.CommandText = SpSetSessionContextQuery;
cmd.Parameters.AddWithValue("#TenantId", tenantId);
await cmd.ExecuteNonQueryAsync().ConfigureAwait(false);
return new TenantDataContext(tenantId, sqlConnection);
}
catch (Exception ex)
{
sqlConnection?.Dispose();
_logger.LogCritical(ex, $"Create(). Create {nameof(TenantDataContext)} for {tenantId} was ended with an error!");
throw;
}
}
}
public class MultiTenantConnectionFactory : IMultiTenantConnectionFactory
{
private readonly IShardingService _shardingService;
private readonly ITenantDataContextFactory _tenantDataContextFactory;
public async Task<IReadOnlyCollection<TenantDataContext>> GetContexts()
{
var shards = _shardingService.RegisteredTenants;
var connectionsTasks = shards.Select(x => _tenantDataContextFactory.CreateAsync(x));
return await Task.WhenAll(connectionsTasks).ConfigureAwait(false);
}
}
public sealed class MultiTenantRepository<T> : IMultiTenantRepository<T> where T : class, ITenantEntity
{
private readonly IMultiTenantConnectionFactory _multiTenantConnectionFactory;
public async Task<IList<T>> Find(Expression<Func<T, bool>> filter)
{
var dataContexts = await _multiTenantConnectionFactory.GetContexts().ConfigureAwait(false);
var tasksList = dataContexts.Select(async x =>
{
await using (x)
{
return await x.Set<T>().Where(filter).ToListAsync().ConfigureAwait(false);
}
});
var results = await Task.WhenAll(tasksList).ConfigureAwait(false);
return results.SelectMany(x => x).ToList();
}
}
But I don't like this solution! Managing connections, awaiting a lot of tasks... I think this solution is slow... Can anybody explain how should I work with the MultiShardConnection without ADO.NET?!
I'd like to test my solution and ADO.NET query performance. I think I will have to give up if I don't find another solution or use ADO.NET.
P.S. I know about ValueTask! I will change usual Task<T> to the ValueTask soon.

.Net Core Async critical section if working on same entity

I need to be sure that a method accessed via a web API cannot be accessed by multiple call at the same time if it work on the same object with the same id
I understand the use of SemaphoreSlim but a simple implemetation of that will lock the critical section for all. But I need that section locked only if it works on the same entity and not on 2 different
This is my scenario, an user start to work, the entity is created and is ready to be modified, then one or more user can manipulate this entity, but a part of this manipulation has to be in a critical section or it will lead to inconsistent data, when the work is finished, the entity will be removed from the work status and moved to and archive and can only be accessed readonly
The class which contains that function is injected as transient in the startup of the application
services.AddTransient<IWorkerService>(f => new WorkerService(connectionString));
public async Task<int> DoStuff(int entityId)
{
//Not Critical Stuff
//Critical Stuff
ReadObjectFromRedis();
ManipulateObject();
UpdateSqlDatabase();
SaveObjectToRedis();
//Not Critical Stuff
}
How can I achieve that?

Try this, I'm not sure if those objects are available in .net-core
class Controller
{
private static ConcurrentDictionary<int, SemaphoreSlim> semaphores = new ConcurrentDictionary<int, SemaphoreSlim>();
public async Task<int> DoStuff(int entityId)
{
SemaphoreSlim sem = semaphores.GetOrAdd(entityId, ent => new SemaphoreSlim(0, 1));
await sem.WaitAsync();
try
{
//do real stuff
}
finally
{
sem.Release();
}
}
}

This is not an easy problem to solve. I have a similar problem with cache: I want that when cache expires only one call is made to repopulate it. Very common approach for token e.g. that you have to renew every now and then.
A problem with an ordinary use of semaphore is that after you exit, all threads that were waiting will just go in and do the call again, that's why you need double check locking to fix it. If you can have some local state for you case I am not sure (I suppose you do since you have a reason for doing only one call and have state most likely), but here is how I solved it for token cache:
private readonly SemaphoreSlim _semaphore = new SemaphoreSlim(1);
public async Task<string> GetOrCreateAsync(Func<Task<TokenResponse>> getToken)
{
string token = Get();
if (token == null)
{
await _semaphore.WaitAsync();
try
{
token = Get();
if (token == null)
{
var data = await getToken();
Set(data);
token = data.AccessToken;
}
}
finally
{
_semaphore.Release();
}
}
return token;
}
Now I don't really know if it is bullet proof. If it were ordinary double check locking (not async), then it is not, though explanation why is really hard and goes to how processor do multithreading behind the scenes and how they reorder instructions.
But in cache case if there is a double call once in a blue moon is not that big of a problem.
I have not found a better way to do that and this is an example provided by e.g. Scott Hanselman and found few places on Stack Overflow as well.

Use of a semaphore is overkill for this. A named mutex will suffice.
class Foo
{
public void Bar(int id)
{
using var mutex = new Mutex(false, id.ToString(), out var createdNew);
if (createdNew)
{
// Business logic here.
}
}
}

Asp.net Core DI: Using SemaphoreSlim for write AND read operations with Singleton

I am re-tooling an ASP.NET CORE 2.2 app to avoid using the service locator pattern in conjunction with static classes. Double bad!
The re-tooling is involving the creation and injection of Singleton object as a repository for some global data. The idea here to avoid hits to my SQL server for some basic/global data that gets used over and over again in requests. However, this data needs to be updated on an hourly basis (not just at app startup). So, to manage the situation I am using SemaphoreSlim to handle one-at-a-time access to the data objects.
Here is a paired down sketch of what what I'm doing:
namespace MyApp.Global
{
public interface IMyGlobalDataService
{
Task<List<ImportantDataItem>> GetFilteredDataOfMyList(string prop1);
Task LoadMyImportantDataListAsync();
}
public class MyGlobalDataService: IMyGlobalDataService
{
private MyDbContext _myDbContext;
private readonly SemaphoreSlim myImportantDataLock = new SemaphoreSlim(1, 1);
private List<ImportantDataItem> myImportantDataList { get; set; }
public async Task<List<ImportantDataItem>> GetFilteredDataOfMyList(string prop1)
{
List<ImportantDataItem> list;
myImportantDataLock.WaitAsync();
try
{
list = myImportantDataList.Where(itm => itm.Prop1 == prop1).ToList();
}
finally
{
myImportantDataLock.Release();
}
return list;
}
public async Task LoadMyImportantDataListAsync()
{
// this method gets called when the Service is created and once every hour thereafter
myImportantDataLock.WaitAsync();
try
{
this.MyImportantDataList = await _myDbContext.ImportantDataItems.ToListAsync();
}
finally
{
myImportantDataLock.Release();
}
return;
}
public MyGlobalDataService(MyDbContext myDbContext) {
_myDbContext = myDbContext;
};
}
}
So in effect I am using the SemaphoreSlim to limit to one-thread-at-a-time access, for both READ and UPDATING to myImportantDataList. This is really uncertain territory for me. Does this seem like an appropriate approach to handle my injection of a global data Singleton throughout my app? Or should I expect insane thread locking/blocking?

The problem with using SemaphoreSlim is scalability.
As this is in a web application, it's fair to assume that you want to have the potential for more than one reader to access the data simultaneously. However, you are (understandably) limiting the number of requests for the semaphore that can be generated concurrently to 1 (to prevent concurrent read and write requests). This means you will serialize all reads too.
You need to use something like ReaderWriterLockSlim to allow multiple threads for reading, but ensure exclusive access for writing.

Creyke's answer hit the nail on the head for me: using ReaderWriterLockSlim. So I've marked it as the accepted answer. But I am posting my revised solution in case it might be helpful to anyone. Important to note that I'm using the following package to provide async functionality to ReaderWriterLockSlim: https://www.nuget.org/packages/Nito.AsyncEx/
using Nito.AsyncEx;
using System;
using System.Collections.Generic;
using System.Text;
namespace MyApp.Global
{
public interface IMyGlobalDataService
{
Task<List<ImportantDataItem>> GetFilteredDataOfMyList(string prop1);
Task LoadMyImportantDataListAsync();
}
public class MyGlobalDataService : IMyGlobalDataService
{
private MyDbContext _myDbContext;
private readonly AsyncReaderWriterLock myImportantDataLock = new AsyncReaderWriterLock();
private List<ImportantDataItem> myImportantDataList { get; set; }
public async Task<List<ImportantDataItem>> GetFilteredDataOfMyList(string prop1)
{
List<ImportantDataItem> list;
using (await myImportantDataLock.ReaderLockAsync())
{
list = myImportantDataList.Where(itm => itm.Prop1 == prop1).ToList();
}
return list;
}
public async Task LoadMyImportantDataListAsync()
{
// this method gets called when the Service is created and once every hour thereafter
using (await myImportantDataLock.WriterLockAsync())
{
this.MyImportantDataList = await _myDbContext.ImportantDataItems.ToListAsync();
}
return;
}
public MyGlobalDataService(MyDbContext myDbContext)
{
_myDbContext = myDbContext;
};
}
}

Azure App Services throws database not found when doing a retrieval

I have Azure App services in my Xamarin application, and everything was working 100% until this morning.
After a while, I found, what looks like the SDK removing my database from local storage, as soon as I do the .GetSyncTable() call.
My service looks like this:
private readonly IMobileServiceSyncTable<User> _table;
private readonly IMobileServiceClient _client;
private readonly MobileServiceSQLiteStore _store;
private readonly ICryptography _cryptography;
private readonly ISettings _settings;
public UserService(
IMobileServiceClient client,
ICryptography cryptography)
{
_client = client;
_cryptography = cryptography;
_settings = new Settings(cryptography) as ISettings;
_store = new MobileServiceSQLiteStore(Settings.SyncDb);
_store.DefineTable<User>();
_client.SyncContext.InitializeAsync(_store);
_table = _client.GetSyncTable<User>();
}
and then I do an "All()", which looks like this:
public async Task<List<User>> All()
{
try
{
var users = await _table.ToListAsync(); // <- this throws "table" not defined
return users;
}
catch (SQLiteException sqlex)
{
Log.Warning("UserService", sqlex.Message);
}
catch (Exception ex)
{
Log.Warning("UserService", ex.Message);
}
}
I have been trying to figure this out, and I'm nowhere closer to a solution.
During my debug, if I land my debugger on the await, and interrogate the _store variable, my "user" table is defined, then a few seconds later, _store no longer contains my table.
I used ADB to download the local store, and viewed it in a SQLite manager, and the table is indeed defined, just, for some reason, it gets "lost"?
Any ideas would be greatly appreciated.

Per my understanding, the issue may caused by the InitializeAsync has not been initialized correctly. You just called _client.SyncContext.InitializeAsync(_store); in the constructor of UserService class. And you need to invoke await _client.SyncContext.InitializeAsync(_store); for initializing the store.
The common Offline Sync Initialization would look as follows:
async Task InitializeAsync()
{
// Short circuit - local database is already initialized
if (Client.SyncContext.IsInitialized)
return;
// Create a reference to the local sqlite store
var store = new MobileServiceSQLiteStore("offlinecache.db");
// Define the database schema
store.DefineTable<TodoItem>();
// Actually create the store and update the schema
await Client.SyncContext.InitializeAsync(store);
}
Here is a similar issue about wrapping the operations against Azure Cloud Table. Moreover, you could follow adrian hall's book about An Offline Client.

How to call some async code in an ASP.NET application_start

In our application_startup, we seed up our database with some fake data, if no data exists.
To do this, we're using the Async methods to store the data. Great. Only problem is, we're not sure how to do this in the application_startup because that's not an async method.
I've spent soooo much time trying to understand #StevenCleary's tutorials and I'm always getting deadlocks. I totally grok what he consistently says:
As a general rule, you should use "async all the way down"; that is, don't block on async code
but I just don't get how I can do that, in this case :(
Lets imagine this is the code I'm trying to play with...
protected void Application_Start()
{
var someFakeData = LoadSomeFakeData();
var documentStore = new DocumentStore();
await documentStore.InitializeAsync(someFakeData);
...
// Registers this database as a singleton.
Container.Register(documentStore);
}
and later on .. some code that uses this documentStore. It is injected via construction injection ...
public SomeController(IDocumentStore documentStore)
{
_documentStore = documentStore;
}
public ViewModel GetFoos()
{
using (var session = _documentStore.OpenSession())
{
... db code goes in here ...
}
}
Clarification
I'm not trying to do some async code in here. I'm actually trying to call this async method, synchronously. Sure, i loose the benefits of async blah blah de blah.. but i'm happy with that. This is start up and I'm happy to block on startup.

In this case, you're asynchronously initializing a shared resource. So, I recommend that you either save the Task itself, or introduce an asynchronous wrapper type.
Using Task:
protected void Application_Start()
{
var someFakeData = LoadSomeFakeData();
var documentStore = new DocumentStore();
var documentStoreTask = documentStore.InitializeAsync(someFakeData);
...
// Registers this database task as a singleton.
Container.Register(documentStoreTask);
}
That may be too awkward, though, depending on Container. In that case, you can introduce an asynchronous wrapper type:
public sealed class DocumentStoreWrapper
{
private readonly Task<DocumentStore> _documentStore;
public DocumentStoreWrapper(Data data)
{
_documentStore = CreateDocumentStoreAsync(data);
}
private static async Task<DocumentStore> CreateDocumentStoreAsync(Data data)
{
var result = new DocumentStore();
await documentStore.InitializeAsync(data);
...
return result;
}
public Task<DocumentStore> DocumentStoreTask { get { return _documentStore; } }
}
protected void Application_Start()
{
var someFakeData = LoadSomeFakeData();
var documentStoreWrapper = new DocumentStoreWrapper(someFakeData);
...
// Registers this database wrapper as a singleton.
Container.Register(documentStoreWrapper);
}
Or, you could use AsyncLazy<T>, which does much the same thing but uses a background thread to execute the initialization code.

You can use of Task.Run(() => YourAsyncMethod()); inside of none async method like:
protected void Application_Start()
{
Task.Run(() => MyAsyncMethod(true));
}

This is an old topic, but it's popped up in my search and maybe it will for others.
For what the OP has requested (ie. To run an async method in a synchronous way from inside a synchronous method, and block until it's finished), is there some reason that the use of Task.WaitAll would not be a simple and adequate way of addressing this?
protected void Application_Start()
{
Task.WaitAll(MyAsyncMethod(true));
}

public static class AsyncHelper
{
private static readonly TaskFactory MyTaskFactory = new
TaskFactory(CancellationToken.None,
TaskCreationOptions.None,
TaskContinuationOptions.None,
TaskScheduler.Default);
public static TResult RunSync<TResult>(Func<Task<TResult>> func)
{
return MyTaskFactory
.StartNew(func)
.Unwrap()
.GetAwaiter()
.GetResult();
}
public static void RunSync(Func<Task> func)
{
MyTaskFactory
.StartNew(func)
.Unwrap()
.GetAwaiter()
.GetResult();
}
}
then use as
AsyncHelper.RunSync(ProcessAsync);
private async Task ProcessAsync(){ ....