We Keep Coding

Entity Framework - duplicates handling

I am using Entity Framework 6, C#, and MySQL InnoDB as our db engine.
I have the following code to "insert on duplicate update" a record:
public async Task AddHostIdToGroup(HostsToGroup hostToGroup)
{
using (var context = new MaintDbContext())
{
HostsToGroup htg = context.HostsToGroup.SingleOrDefault(hs => hs.HostId == hostToGroup.HostId);
if (htg != null)
{
htg.GroupId = hostToGroup.GroupId;
await context.SaveChangesAsync();
}
else
{
context.HostsToGroup.Add(hostToGroup);
await context.SaveChangesAsync();
}
}
}
The code itself looks fine for insert on duplicate update.
Still, on our production server I occasionally see duplicate errors.
My initial thought was that it's a race condition issue.
What can I do to prevent these errors, or how should I handle them?

Your problem is not necessarily the code but what triggers that code. Say the code is triggered by an ASP.net app button, if a user double clicks on the button then it could shoot off two simultaneous requests that results in double entity creation.
So either fix your front end/entry point to eliminate the double create scenario, push everything into a synchronous service/queue that will allow you to deduplicate or use Where instead of Single and handle the duplication in code.

ASP-Net Core Sql command concurrency issue

The back end of my application runs on NetCore as a Web API. The front end can be considered unrelated.
Most of my application runs on Entity framework. However I've recently had to implement some custom filtering into one of our DB calls. This Db call is called up to 3 times at once when a page on the web UI is loaded.
I'm having issues with the following code, which is throwing the following error:
System.InvalidOperationException: 'The connection was not closed. The connection's current state is open.'
The code:
foreach (var filter in dbFilters)
{
var cacheKey = $"matching_filters_id:{matchingConfigId}_filter:{filter.FilterFunctionName}";
var ids = new List<int>();
if (!_cache.TryGetValue(cacheKey, out ids))
{
ids = new List<int>();
using (var conn = _dbContext.Database.GetDbConnection())
{
conn.Open();
using (var command = conn.CreateCommand())
{
command.CommandText =
SqlHelper.BuildGetMatchingFilterIdsForIntersect(filter.FilterFunctionName);
using (var result = command.ExecuteReader())
{
while (!result.IsClosed && result.Read())
{
ids.Add((int) result["ItemId"]);
}
}
}
conn.Close();
}
var cacheEntryOptions = new MemoryCacheEntryOptions()
.SetSlidingExpiration(TimeSpan.FromSeconds(30));
_cache.Set(cacheKey, ids, cacheEntryOptions);
}
// Joins filtered list against each filter to make the list smaller.
firstFilter = firstFilter.Join(ids, o => o.SourceKey, id => id, (o, id) => o).ToArray();
}
What i'm attempting to do, is filter the list firstFilter which is a POCO containing a few properties. I'm attempting to filter this by joining it to one or more list of filtered ItemIds being returned from multiple different SQL queries. Not the most elegant solution but its what i have to work with.
What i understand to be happening is that as the client is calling this method more then once, the methods are executing concurrently, causing the DB connection to be opened twice at once, causing the connection to fail and the error to be thrown.
I've attempted to stop this from happening by removing the async keyword form my method: public JsonResult GetMatches(int matchingConfigId...... ) but without success. I've also tried to intorduce session into the application, which I've read might cause calls to the same method from one client 'session' to happen in sequence rather then concurrently. But this has not seemed to have any effect.
Of course i might be barking up the entirely wrong tree.. but I've been bashing my head against said tree for most of the day now and I've reached my limit. Any help would be greatly appreciated

So it turns out that the variable dbFilters, which came from entity framework, had not been resolved with a ToList() or ToListAsync() yet, so the connection to the DB for that call was still open, causing the subsequent calls in the above piece of code to fail.

Trade-offs between longer data context or transaction outside of context?

I am writing a Web API Rest service that does operations on a distinct set of entities. I have broken them down like this:
db = new DBEntities();
using (var dbContextTransaction = db.Database.BeginTransaction())
{
try
{
ProcessClient();
ProcessClientPerson();
ProcessGuardian();
ProcessAddress();
ProcessEmail();
ProcessPhones();
ProcessChildren();
dbContextTransaction.Commit();
}
catch (Exception ex)
{
dbContextTransaction.Rollback();
etc.
Following the advice that data contexts should live as short as possible, each of the methods creates its own data context, calls SaveChanges(), and disposes of it at the end:
private ProcessClient()
{
db = new DBEntities();
....
This obviously does not work - a transaction context created this way is tied to the data context. If something goes wrong in one of the entity operations, only that operation is rolled back (implicitly), but the overarching transaction is not.
I found this approach for creating a transaction outside of EF, but I am wondering if I should follow it or if I should just let my data context live for the duration of the transaction and keep the transaction inside of EF!?
I am not looking for an opinion, but for data around stability, performance, etc.

There is no immediate need to keep contexts short-lived. You can do that but you don't have to.
Over time entities will accumulate in a context. If you risk running out of memory it can be necessary to let go of a context.
Otherwise, the usual procedure is to keep the context alive for the duration of the logical unit of work. Here, that UOW is all those methods in their entirety.
This also makes transaction management easier (as you already found out).
dbContextTransaction.Rollback();
This is an anti-pattern. Simply don't commit in case of error.

I have mixed feelings about this. I am working against a legacy database that has no foreign key constraints, and I am inserting, updating, and deleting between 20 and 30 objects in one of these service calls.
The problem is that I need to call SaveChanges() frequently to get the identity column values that will become foreign keys.
On the other hand, I have to be able to roll back everything if there is a problem three layers down, so a single large transaction is needed.
For some reason that I have not been able to determine, calling SaveChanges repeatedly on the same data context would result in errors that the connection state is open. So I ended up giving each method its own data context anyway:
var scope = new TransactionScope(TransactionScopeOption.RequiresNew,
new TransactionOptions() { IsolationLevel = IsolationLevel.ReadUncommitted);
using (scope)
{
try
{
ProcessClient();
ProcessClientPerson();
ProcessGuardian();
ProcessAddress();
ProcessEmail();
ProcessPhones();
ProcessChildren();
scope.Complete();
}
catch (System.Data.Entity.Validation.
DbEntityValidationException ex)
{
[...] handle validation errors etc [...]
}
}
with each section doing basically this, once stripped down to the bare essentials:
private void ProcessClient() {
{
using (MyDBEntities db = new MyDBEntities())
{
[...] doing client stuff [...]
aClient.LastUpdated = DateTime.Now;
db.AddOrUpdate(db, db.Clients, aClient, aClient.ClientID);
db.SaveChanges();
ClientId = aClient.ClientID; // now I can use this to form FKs
}
}
Mixed feelings about locking, because on my development VM the transaction runs for 1-2 seconds and this is a production database with office staff and online customers doing CRUD transactions through web applications at the same time.
Unrelated, but helpful for my AddOrUpdate method was this blog post.

Wrapping methods using db connection in MVC

I was following the MVC music store tutorial and I came to the part where they are using database connections (DbConnection is a child of DbContext). I was taught to create methods like this (wrapping with using):
public class StoreManagerController : Controller
{
//
// GET: /StoreManager/
public ActionResult Index()
{
using(var db = new DbConnection())
{
var albums = db.Albums.Include(a => a.Genre).Include(a => a.Artist);
return View(albums.ToList());
}
}
...
}
but Visual Studio generated me a controller which looked like this:
public class StoreManagerController : Controller
{
private DbConnection db = new DbConnection();
//
// GET: /StoreManager/
public ActionResult Index()
{
var albums = db.Albums.Include(a => a.Genre).Include(a => a.Artist);
return View(albums.ToList());
}
...
}
I assume, Visual Studio isn't wrong, but why was I told to wrap each method with using to make the connections as short as possible and also the users to use separate connections?

I assume, Visual Studio isn't wrong, but why was I told to wrap each method with using
using(var db = new DbConnection())
{
var albums = db.Albums.Include(a => a.Genre).Include(a => a.Artist);
return View(albums.ToList());
}
The scope of db remains only within the curly braces. This is perhaps another purpose that using keyword serves in C#. It defines the scope of a variable, here in the above case it is your db object.
Now, if you debug the code that visual studio generated for you, then you notice that there is a Dispose method being invoked each and every time an object of a controller class is made or in other words, an action method is called within the corresponding Controller.
The DBContext instance is always disposed because of the following -
As you load more objects and their references into memory, the memory consumption of the context may increase rapidly. This may cause performance issues.
If an exception causes the context to be in an unrecoverable state, the whole application may terminate.
The chances of running into concurrency-related issues increase as the gap between the time when the data is queried and updated grows.
For more info - Reference

This might depend on the usability of your app; whether or not you need a persistent connection, and the cost of creating one (and a myriad of other factors).
But for starters, you should always dispose of the connection (as in the first pattern, not the one suggested by Visual Studio) and then move to other patterns based on new requirements or performance-related issues.
The biggest issue I see with the Visual Studio-suggested options is that you have no way of controlling the lifetime of the DbConncetion object, and are leaving it up to the garbage collector to eventually dispose of it. This could leave the connection resources in use for an rather undetermined period of time.

Spring.net TransactionPropagation NotSupported

I'm in troubles with TransactionPropagation.NotSupported. I believed that this propagation causes that the code is executed within no transaction. Means that when I marked the specific method, the current transaction will be suspended and the code will be executed without any transaction.
The current version of spring.net creates new transaction instead. See the following code:
[Test]
public void A() {
TransactionTemplate template = new TransactionTemplate(TransactionManager) {
PropagationBehavior = TransactionPropagation.NotSupported
};
template.Execute(delegate {
Assert.AreEqual(0,
SessionFactory.GetCurrentSession().Linq<XXX>().
Where(t => t.Id.Equals(YYY)).ToList().Count);
return null;
});
}
I hoped that this notation causes that linq query is executed without transaction and it'll throw the new exception. But the log showed that it creates both new session and transaction automatically.
I've find out this issue when I marked any method by mentioned annotation and despite the annotation the LINQ query inside was correctly executed.
The question is: how can I mark the method to it's doesn't use the transaction at all? I don't want to use propagation never as I want the current transaction would be suspended.
My project has the business code flow, there is transaction handling, and I want to mark any parts to be certainly non-transactional.

You mention being able to tell from the log that a new transaction is started. What log, the database or the application? What database are you using? Some databases won't let you run a query outside a transaction at all, so would just start one internally for you...
Update:
Your issues looks similar to this one:
https://jira.springframework.org/browse/SPRNET-1307?page=com.atlassian.jira.plugin.ext.bamboo%3Abamboo-build-results-tabpanel#issue-tabs
I would make sure you are running the version of Spring.NET that has this fix in it (looks like v 1.3.1 or greater?)
Also, you could try manually suppressing the transaction and see if that fixes the behavior:
using(var tx = new TransactionScope(TransactionScopeOption.Suppress))
{
// make DB call...
}