I am using Entity Framework 6, C#, and MySQL InnoDB as our db engine.
I have the following code to "insert on duplicate update" a record:
public async Task AddHostIdToGroup(HostsToGroup hostToGroup)
{
using (var context = new MaintDbContext())
{
HostsToGroup htg = context.HostsToGroup.SingleOrDefault(hs => hs.HostId == hostToGroup.HostId);
if (htg != null)
{
htg.GroupId = hostToGroup.GroupId;
await context.SaveChangesAsync();
}
else
{
context.HostsToGroup.Add(hostToGroup);
await context.SaveChangesAsync();
}
}
}
The code itself looks fine for insert on duplicate update.
Still, on our production server I occasionally see duplicate errors.
My initial thought was that it's a race condition issue.
What can I do to prevent these errors, or how should I handle them?
Your problem is not necessarily the code but what triggers that code. Say the code is triggered by an ASP.net app button, if a user double clicks on the button then it could shoot off two simultaneous requests that results in double entity creation.
So either fix your front end/entry point to eliminate the double create scenario, push everything into a synchronous service/queue that will allow you to deduplicate or use Where instead of Single and handle the duplication in code.
Related
We are using the ExecutionStrategy and have this helper method in our db context:
public Task<T> ExecuteWithinTransactionAsync<T>(Func<IDbContextTransaction, Task<T>> operation, string operationInfo)
{
int counter = 0;
return Database.CreateExecutionStrategy().ExecuteAsync(RunOperationWithinTransaction);
async Task<T> RunOperationWithinTransaction()
{
counter++;
if (counter > 1)
{
Logger.Log(LogLevel.Warn, $"Executing ({counter}. time) transaction for {operationInfo}.");
ClearChangeTracker();
}
using (var transaction = await Database.BeginTransactionAsync(IsolationLevel.Serializable))
{
return await operation.Invoke(transaction);
}
}
}
We than use ExecuteWithinTransactionAsync when calling complex/fragile business logic which should be executed in a serializable transaction reliably. We are using Postgres so it can happen that our transaction will be aborted due to serialization issues. The execution strategy detects it and retries the operation. That works nicely. But EF still keeps the old cache from the previous execution. That's why we introduced ClearChangeTracker which looks like this:
private void ClearChangeTracker()
{
ChangeTracker.DetectChanges();
foreach (var entity in ChangeTracker.Entries().ToList())
{
entity.State = EntityState.Detached;
}
}
And this seemed to have worked properly, until we found a case where it didn't work anymore. When we add new entities to a navigation property list, these entities won't be removed on the next try. For instance
var parent = context.Parents.FirstOrDefault(p => p.Id == 1);
if (parent.Children.Any())
{
throw new Exception("Parent already has a child"); // This exception is thrown on the second try
}
parent.Children.Add(new Child());
context.SaveChangesAsync();
So if the last line context.SaveChangesAsync() fails, and the whole operation is re-run, parent.Children already contains the new child added in parent.Children.Add(new Child()); and I didn't find any way to remove that item from EF.
However, if we remove the check (if (parent.Children.Any())), if the item already exists or not, and just try adding it a second time, it's only stored once in the DB.
I was trying to figure out how to clear the DbContext properly, but most of the time, the answer was just to create a new DbContext. However, that's not an option, since the DbContext is needed for the ExecutionStrategy. That's why I wanted to know, what's the suggested way to used the ExecutionStrategy and having a clean DbContext on every retry.
Further technical details
EF Core version: 1.1.2
Database Provider: Npgsql.EntityFrameworkCore.PostgreSQL (1.1.1)
Operating system: Windows 10, Dockerized in Linux
In ef-core 2.0.0, this new feature DbContext pooling was introduced. For it to work properly, DbContext instances are now able to reset their internal state, so they can be handed out as "new". The reset method can be called like so (inside your DbContext):
((IDbContextPoolable)this).ResetState();
So if you can upgrade to ef-core 2.0.0, go for it. Not only to benefit from this new feature, it's more mature in many ways.
Disclaimer: this method is intended for internal use, so the API may change in the future.
The back end of my application runs on NetCore as a Web API. The front end can be considered unrelated.
Most of my application runs on Entity framework. However I've recently had to implement some custom filtering into one of our DB calls. This Db call is called up to 3 times at once when a page on the web UI is loaded.
I'm having issues with the following code, which is throwing the following error:
System.InvalidOperationException: 'The connection was not closed. The connection's current state is open.'
The code:
foreach (var filter in dbFilters)
{
var cacheKey = $"matching_filters_id:{matchingConfigId}_filter:{filter.FilterFunctionName}";
var ids = new List<int>();
if (!_cache.TryGetValue(cacheKey, out ids))
{
ids = new List<int>();
using (var conn = _dbContext.Database.GetDbConnection())
{
conn.Open();
using (var command = conn.CreateCommand())
{
command.CommandText =
SqlHelper.BuildGetMatchingFilterIdsForIntersect(filter.FilterFunctionName);
using (var result = command.ExecuteReader())
{
while (!result.IsClosed && result.Read())
{
ids.Add((int) result["ItemId"]);
}
}
}
conn.Close();
}
var cacheEntryOptions = new MemoryCacheEntryOptions()
.SetSlidingExpiration(TimeSpan.FromSeconds(30));
_cache.Set(cacheKey, ids, cacheEntryOptions);
}
// Joins filtered list against each filter to make the list smaller.
firstFilter = firstFilter.Join(ids, o => o.SourceKey, id => id, (o, id) => o).ToArray();
}
What i'm attempting to do, is filter the list firstFilter which is a POCO containing a few properties. I'm attempting to filter this by joining it to one or more list of filtered ItemIds being returned from multiple different SQL queries. Not the most elegant solution but its what i have to work with.
What i understand to be happening is that as the client is calling this method more then once, the methods are executing concurrently, causing the DB connection to be opened twice at once, causing the connection to fail and the error to be thrown.
I've attempted to stop this from happening by removing the async keyword form my method: public JsonResult GetMatches(int matchingConfigId...... ) but without success. I've also tried to intorduce session into the application, which I've read might cause calls to the same method from one client 'session' to happen in sequence rather then concurrently. But this has not seemed to have any effect.
Of course i might be barking up the entirely wrong tree.. but I've been bashing my head against said tree for most of the day now and I've reached my limit. Any help would be greatly appreciated
So it turns out that the variable dbFilters, which came from entity framework, had not been resolved with a ToList() or ToListAsync() yet, so the connection to the DB for that call was still open, causing the subsequent calls in the above piece of code to fail.
I have Like button, when it pressed - controller action should add field to my dbTable("Likes"),
and if button pressed again - it should remove Like from dbTable("Likes").
Here code(simplified):
var like = db.Likes.Find(id);
if (like == null)
{
Like like = new Like("SomeData", UserPosedLike);
db.Add(like);
}
else
{
db.Likes.Remove(like);
}
db.SaveChanges();
It's work fine at 95% of times, but if i press button very fast 2-3 times, it throws error – “Null ref”, some data in Like is NULL (but this data can’t be null, because it references to other dbField(User)). Also, sometimes it adds two likes, but it shouldn’t.
I think it happens because of:
First action(press) – read db, start processing.
Second action (second press) – read db, start processing. (first action not saved data yet).
First action(press) – save data.
Second action(pess) – save data.
Here we got 2 likes. Probably with “null error” similar problem.
I received advice to lock this block of code, but I think it’s wrong. Also I thinking about Optimistic Concurrency, currently reading about it.
I’m not good with db, any help will be great - good book, code, article or advice.
Thank you!
You need to read and write in transaction.
using (var db = new YourContext())
{
using (var t = db.Database.BeginTransaction())
{
try
{
var like = db.Likes.Find(id);
if (like == null)
{
Like like = new Like("SomeData", UserPosedLike);
db.Add(like);
}
else
{
db.Likes.Remove(like);
}
db.SaveChanges();
t.Commit();
}
catch
{
t.Rollback();
}
}
}
I received advice to lock this block of code, but I think it’s wrong.
It is wrong.
Also as someone has suggested in comments, you can prevent users from clicking a button multiple times. You could disable the button and add a spinner.
Context: web application / ASP.NET 4.0 / EF 4.0 / MSSQLEXPRESS2012
I'm trying to do a simple delete as follows:
if (someObject != null)
{
if (!context.IsAttached(someObject))
context.SomeObjects.Attach(someObject);
context.DeleteObject(someObject);
context.SaveChanges();
}
The code executes without problem and SaveChanges returns the expected number of rows. But when I try to read the table afterwards it times out (this happens in mssms as well as in code).
It also seems to corrupt other processes, returning '8501 MSDTC on server is unavailable' and "No process is on the other end of the pipe" on subsequent site access. Which of the three errors happens in the application seems somewhat random, but the timeout in mssms always happens.
Inserts and updates on the same table execute without problem.
Apart from reading through a zillion posts, I tried wrapping it in try/catch and transaction with SaveChanges(System.Data.Objects.SaveOptions.None) and manual AcceptAllChanges; the debugger shows the correct object just before delete - everything appears normal until table read or accessing another page.
I suspect somehow the lock is not released, but other objects in different tables successfully delete with the exact same logic. I'm not sure where best to look next - any ideas are greatly appreaciated!
must have made a mistake with transaction before - the following worked:
var transactionScope = new TransactionScope
(TransactionScopeOption.RequiresNew,
new TransactionOptions()
{
IsolationLevel = System.Transactions.IsolationLevel.ReadUncommitted
}
);
try
{
using (transactionScope)
{ ...etc
the following link explains more about the whys of explicit transactions
http://blogs.u2u.be/diederik/post/2010/06/29/Transactions-and-Connections-in-Entity-Framework-40.aspx
still not sure why it's working without transaction for the other deletes though
I'm in troubles with TransactionPropagation.NotSupported. I believed that this propagation causes that the code is executed within no transaction. Means that when I marked the specific method, the current transaction will be suspended and the code will be executed without any transaction.
The current version of spring.net creates new transaction instead. See the following code:
[Test]
public void A() {
TransactionTemplate template = new TransactionTemplate(TransactionManager) {
PropagationBehavior = TransactionPropagation.NotSupported
};
template.Execute(delegate {
Assert.AreEqual(0,
SessionFactory.GetCurrentSession().Linq<XXX>().
Where(t => t.Id.Equals(YYY)).ToList().Count);
return null;
});
}
I hoped that this notation causes that linq query is executed without transaction and it'll throw the new exception. But the log showed that it creates both new session and transaction automatically.
I've find out this issue when I marked any method by mentioned annotation and despite the annotation the LINQ query inside was correctly executed.
The question is: how can I mark the method to it's doesn't use the transaction at all? I don't want to use propagation never as I want the current transaction would be suspended.
My project has the business code flow, there is transaction handling, and I want to mark any parts to be certainly non-transactional.
You mention being able to tell from the log that a new transaction is started. What log, the database or the application? What database are you using? Some databases won't let you run a query outside a transaction at all, so would just start one internally for you...
Update:
Your issues looks similar to this one:
https://jira.springframework.org/browse/SPRNET-1307?page=com.atlassian.jira.plugin.ext.bamboo%3Abamboo-build-results-tabpanel#issue-tabs
I would make sure you are running the version of Spring.NET that has this fix in it (looks like v 1.3.1 or greater?)
Also, you could try manually suppressing the transaction and see if that fixes the behavior:
using(var tx = new TransactionScope(TransactionScopeOption.Suppress))
{
// make DB call...
}