In case of this method:
public void Delete(int id)
{
using (var connection = GetOpenConnection())
{
connection.Execute($"DELETE FROM MyTable WHERE Id = {id}");
}
}
Or just:
GetOpenConnection().Execute($"DELETE FROM MyTable WHERE Id = {id}");
I wonder if the second is the best option to ease the maintenance and simplify.
First option gives you predictability: connection object returned from GetOpenConnection() will be disposed as soon as connection.Execute finishes.
On the other hand, if you use second approach, you can hope that the connection would be closed at some time in the future, but you have absolutely no certainty of when, and even if, it is going to happen.
Therefore one should prefer the first approach.
Note: Consider parameterizing your query. Even though in your situation insertion of the id into the query is non-threatening because id's type is int, it is a good idea to use parameters consistently throughout your code.
Answering this requires an understanding of how Sql Server (and other databases) use connections, and how ADO.Net uses connection pooling.
Database servers tend to only be able to handle a limited a number of active connections at a time. It has partly to do with the limited available ephemeral ports on a system, but other factors can come into play, as well. This is means it's important to make sure connections are either always closed promptly, or that we carefully limit connection use. If we want a database to scale to a large number of users, we have to do both.
.Net addresses this situation in two ways. First, the ADO.Net library you use for database access (System.Data and company) includes a feature called Connection Pooling. This feature pools and caches connections for you, to make it efficient to quickly open and close connections as needed. The feature means you should not try to keep a shared connection object active for the life of an application or session. Let the connection pool handle this, and create a brand new connection object for most trips to the database.
The other way it addresses the issue is with the IDisposable pattern. IDisposable provides an interface with direct support in the runtime via the using keyword, such that you can be sure unmanaged resources for an object — like that ephemeral port on the database server your connection was holding onto — are cleaned up promptly and in a deterministic way, even if an exception is thrown. This feature makes sure all those short-lived connections you create because of the connection pooling feature really are as short-lived as they need to be.
In other words, the using block in the first sample serves an important function. It's a mistake to omit it. On a busy system it can even lead to a denial of service situation for your database.
You get a sense of this in the question title itself, which asks, "Which is better to dispose the object?" Only one of those two samples disposes the object at all.
You could approach the design in this manner.
using(var context = new CustomerFactory().Create())
return context.RetrieveAll();
Then inside your CustomerContext you would have the dispose logic, the database connection, and your query. But you could create inherit a DbConnectionManager class, which will deal with the connection. But the entire class will be disposed, which would also salvage the connection manager.
public interface ICustomerRepository : IDisposable
{
IEnumerable<Customer> RetrieveAll();
}
public interface ICustomerFactory
{
ICustomerRepository Create();
}
public class CustomerFactory : ICustomerFactory
{
public ICustomerRepository Create() => new CustomerContext();
}
public class CustomerContext : ICustomerRepository
{
public CustomerContext()
{
// Instantiate your connection manager here.
}
public IEnumerable<Customer> RetrieveAll() => dbConnection.Query<Customer>(...);
}
That would be if you want to stub out an expressive call, kind of representing your fluid syntax in option two, without the negative impact.
Related
I do something like:
public class MyDbContext : DbContext {
public MyDbContext(bool readOnlyFlag) {
// Monitor.Enter(theLock); // needed ??
this.readOnlyFlag = readOnlyFlag;
// Database.EnsureCreated(); // needed ??
}
public DbSet<MyData> MyData { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder) {
string connectionString = "Data Source=C:\\mydb.db;";
if (readOnlyFlag) connectionString += "Mode=ReadOnly;";
optionsBuilder.UseSqlite(connectionString);
}
public override void Dispose() {
// Database.CloseConnection(); // needed ??
base.Dispose();
// Monitor.Exit(theLock); // needed ??
}
readonly bool readOnlyFlag;
// readonly object theLock = new object(); // needed ??
}
and then:
using (var dbc = new MyDbContext(true)) {
dbc.MyData.Where( ... code
}
I call such code this from multiple concurrent threads to run different queries.. (in a .Net Core 3.0 console App)
Questions:
If I understand correctly the database file will be opened when the using block starts and closed when it ends. Closing and opening a file on each query seems really inefficient but I could not find any reference to whether or not it's OK to keep a singleton MyDbContext (ie in the Program class) and reuse it ?
If I can reuse MyDbContext should I then use a lock around queries ?
In general do I need to use ie the Monitor remarked above to make sure queries don't run concurrently ? I've seen posts saying Sqlite needs this ?
Do I need to call Database.CloseConnection() ? seems to work fine without it but I've seen posts where it was called like above remarked ?
is Database.EnsureCreated() needed for Sqlite ?
Thanks!
Are you sure that you are the only user of the data? In other words, are you sure that the data does not change between two usages of your dbContext?
Furthermore: are you sure that your dbContext will always be used this way, or might it be that in future this dbContext might be connected to a real database?
If your thread will be the one and only user, now and in future, there is not much harm in reusing the DbContext. However, keep in mind that it is not guaranteed that data is really written before you Dispose the dbContext. Furthermore: your dbContext will keep all fetched data in local memory, so after a while you will have your complete database in local memory.
Consider using a repository pattern, where you hide how the data is persisted, the repository pattern knows a bit more about what your repository is used for and can make smarter decisions about what data to keep in memory and what data to query from your database by a fresh dbContext.
For instance, if you have a database with Schools, Students, and Teachers, and you frequently query their data, but seldom query data of retired Teachers and data of graduated students, your repository could keep all fetched non-retired/graduated Teachers / Students in memory and only create a fresh dbContext to fetch unknown data, fetch retired / graduated data or update the database
interface IRepositorySet<Tentity> : IEnumerable<Tentity>
where Tentity : class, new()
{
Tentity Add(Tentity entity);
Tentity Update(Tentity entity);
Tentity Delete(Tentity entity);
}
interface ISchoolsRepository
{
// for simple queries / add / update / remove only
IRepositorySet<School> Schools {get;}
IRepositorySet<Teacher> Teachers {get;}
IRepositorySet<Student> Students {get;}
}
The RepositorySet knows which dbContext to create when it needs data. All frequently fetched items will be kept in memory in a Dictionary with primary Key.
Upon creation the Dictionary is filled with all primary keys, and value null, indicating that the item is not fetched yet.
When data is requested, the RepositorySet first fetches data from the dictionary. All items that have still a null value will be fetched from a fresh dbContext and put in the dictionary.
Note that this won't work for huge amounts of data. Only consider this solution if you think you can keep all fetched data in memory. But then again: keeping your dbContext open will also keep all fetched data in memory.
You can use DbContext with Sqlite multithreading. Normally, you should use DbContext as instance per request, because of DbContext is not thread safe, one commit should not affect the others.
As mentioned on sqlite's site, it supports mutltithreading:
SQLite supports three different threading modes:
Single-thread. In this mode, all mutexes are disabled and SQLite is
unsafe to use in more than a single thread at once.
Multi-thread. In this mode, SQLite can be safely used by multiple
threads provided that no single database connection is used
simultaneously in two or more threads.
Serialized. In serialized mode, SQLite can be safely used by multiple
threads with no restriction.
The threading mode can be selected at compile-time (when the SQLite
library is being compiled from source code) or at start-time (when the
application that intends to use SQLite is initializing) or at run-time
(when a new SQLite database connection is being created). Generally
speaking, run-time overrides start-time and start-time overrides
compile-time. Except, single-thread mode cannot be overridden once
selected.
The default mode is serialized.
https://www.sqlite.org/threadsafe.html
Also I suggest you to take a look at this SQLite Concurrent Access and this Can I read and write to a SQLite database concurrently from multiple connections? .
According to above posts, sqlite writes locks the entire file even for reads. And in the internet some of the users suggests to taking locks in code explicitly for writes.
But new version of sqlite has a feature called WAL.
The second advantage of WAL-mode is that writers do not block readers
and readers to do not block writers. This is mostly true. But there
are some obscure cases where a query against a WAL-mode database can
return SQLITE_BUSY, so applications should be prepared for that
happenstance.
Sqlite itself says concurrent access even for multiple process can handled by sqlite.
And according to sqlite.org/faq
If your application has a need for a lot of concurrency, then you
should consider using a client/server database. But experience
suggests that most applications need much less concurrency than their
designers imagine.
When SQLite tries to access a file that is locked by another process,
the default behavior is to return SQLITE_BUSY.
It might be need to handled in application itself.
I'm currently using TransactionScope to manage transactions in my data layer, but I've been running into issues with nested transactions and async whereby the connection seems to close during the nested transaction or the transaction is promoted to MSDTC. I've not found the exact problem but after reading around it looks like this scenario isn't particuarly well supported and that I should be using Database.BeginTransaction() instead.
My problem is that I can't find information on how Database.BeginTransaction() works with nested transactions, particularly in my scenario where i'm wanting to use the ambient transaction rather than create a new one. My suspicion is that it isn't intended to work this way and if I want to manage nested transactions I should abstract out transaction management to give me more control.
Not wanting to add in unnecessary layers of abstractions I wanted to know if anyone has experience in this area and could confirm the behavior of Database.BeginTransaction() when nested inside another transaction?
Additional information about my DAL: Based on CQS pattern, I tend to encapsulate Db related code in command or query handlers, so a simplified/contrived example of how this nesting occurs would be:
public class AddBlogPostHandler
{
private readonly MyDbContext _myDbContext;
public AddBlogPostHandler(MyDbContext myDbContext)
{
_myDbContext = myDbContext;
}
public async Task ExecuteAsync(AddBlogPostCommand command)
{
using (var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
// .. code to create and add a draft blog post to the context
await _myDbContext.SaveChangesAsync();
var publishBlogPostCommand = new PublishBlogPostCommand();
// ..set some variables on the PublishBlogPostCommand
await PublishBlogPostAsync(command);
scope.Complete();
}
}
}
public class PublishBlogPostHandler
{
private readonly MyDbContext _myDbContext;
public PublishBlogPostHandler(MyDbContext myDbContext)
{
_myDbContext = myDbContext;
}
public async Task ExecuteAsync(PublishBlogPostCommand command)
{
using (var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
// .. some code to do one set of update
await _myDbContext.SaveChangesAsync();
// .. some other db updates that need to be run separately
await _myDbContext.SaveChangesAsync();
scope.Complete();
}
}
}
There is no such thing as nested transactions in the sense that the inner one can commit or rollback independently. Nested transactions really only maintain a ref count. At the last commit we get a physical commit. At the first rollback we get a physical rollback. Just making sure you are aware of that.
It is important to avoid MSDTC usage. This is possible both with TransactionScope and with BeginTransaction. With the former you need to explicitly Open the connection inside the scope so that EF does not open new connections all the time.
As you have read in the issue this is a flaw in EF (which L2S did not have). Please take the time to comment on the issue to make sure the team is aware that customers are running into this problem.
particularly in my scenario where i'm wanting to use the ambient transaction rather than create a new one.
This is perfect for TransactionScope. I think your switch to BeginTransaction is based on a misunderstanding. Maybe you can clarify in the comments.
confirm the behavior of Database.BeginTransaction() when nested inside another transaction
Explained in the first paragraph.
Additional information about my DAL: Based on CQS pattern, I tend to encapsulate Db related code in command or query handlers, so a simplified/contrived example of how this nesting occurs would be:
The code looks fine except for the missing db.Connection.Open() call (as explained above).
This pattern will support executing multiple queries and commands in the same transaction. Just wrap another scope around it. Make sure to not open connections twice, e.g. check conn.State before taking action.
Apparently (and quite possibly) there's a flaw in my current UnitOfWork implementation, because I have connection errors when doing many calls at once.
Exception:
The underlying provider failed on Open.
Inner Exception:
The connection was not closed. The connection's current state is
connecting.
This results in a HTTP 500 response on the client side.
UnitOfWork implementation
public class ScopedUnitOfWork : IUnitOfWork
{
public Entities Context { get; set; }
public UnitOfWorkState State { get; set; }
public ScopedUnitOfWork(IEnvironmentInformationProvider environmentInformationProvider)
{
this.Context = new Entities(environmentInformationProvider.ConnectionString);
this.State = UnitOfWorkState.Initialized;
}
public UowScope GetScope()
{
this.State = UnitOfWorkState.Working;
return new UowScope(this);
}
public SaveResult Save()
{
if (this.State != UnitOfWorkState.Working)
throw new InvalidOperationException("Not allowed to save out of Scope. Request an UowScope instance by calling method GetScope().");
this.Context.SaveChanges();
this.State = UnitOfWorkState.Finished;
return new SaveResult(ResultCodes.Ok);
}
}
Working on a single UowScope would solve the issue but that's not possible given the current circumstance, because each request is completely separate. De facto each request IS using an UoWScope, but apparently it goes wrong when the UoW receives many calls at once.
The UoW is injected through Unity IoC, so I suppose it's a singleton in effect.
The question
Is there a way to adapt the UoW so that separate high-frequency requests are not an issue?
Preferably I'd solve this server side, not client side, any tips? Thanks!
Disclaimer
I don't claim I fully understand UoW, so my implementation may need improvement, be gentle :). Any improvements on that are certainly welcome!
UPDATE
I -know- the EF Context is an UoW, I use mine at Domain level to enable transactional processing of data that is functionality related. And it's also by customer demand, I have no choice.
The issue you have is that the unit of work object is effectively a singleton as your IoC framework is keeping it around for the duration of your application. This means that your context is also being kept as a singleton as it's inside the UoW. So you will almost certainly get multiple concurrent calls to your context which will throw exceptions.
However, I think you are misusing the concept of what a UoW supposed to do. A UoW is there to provide a container for a group of transactions. For example lets say you have an eCommerce platform. When you create an order, you will insert a row in the orders table, then as part of the same transaction you will also insert rows into the order items table, update a users loyalty points etc. So you should do all this inside a single unit of work, commit it, then destroy it. Let the IoC framework (Unity in this case) create your unit of work for each session.
In the Business Logic Layer of an Entity Framework-based application, all methods acting on DB should (as I've heard) be included within:
using(FunkyContainer fc = new FunkyContainer())
{
// do the thing
fc.SaveChanges();
}
Of course, for my own convenience often times those methods use each other, for the sake of not repeating myself. The risk I see here is the following:
public void MainMethod()
{
using(FunkyContainer fc = new FunkyContainer())
{
// perform some operations on fc
// modify a few objects downloaded from DB
int x = HelperMethod();
// act on fc again
fc.SaveChanges();
}
}
public int HelperMethod()
{
using(FunkyContainer fc2 = new FunkyContainer())
{
// act on fc2 an then:
fc2.SaveChanges();
return 42;
}
}
I doesn't look good to me, when the container fc2 is created, while fc is still open and has not been saved yet. So this leads to my question number one:
Is having multiple containers open at the same time and acting on them carelessly an acceptable practice?
I came to a conclusion, that I could write a simple guard-styled object like this:
public sealed class FunkyContainerAccessGuard : IDisposable
{
private static FunkyContainer GlobalContainer { get; private set; }
public FunkyContainer Container // simply a non-static adapter for syntactic convenience
{
get
{
return GlobalContainer;
}
}
private bool IsRootOfHierarchy { get; set; }
public FunkyContainerAccessGuard()
{
IsRootOfHierarchy = (GlobalContainer == null);
if (IsRootOfHierarchy)
GlobalContainer = new FunkyContainer();
}
public void Dispose()
{
if (IsRootOfHierarchy)
{
GlobalContainer.Dispose();
GlobalContainer = null;
}
}
}
Now the usage would be as following:
public void MainMethod()
{
using(FunkyContainerAccessGuard guard = new FunkyContainerAccessGuard())
{
FunkyContainer fc = guard.Container;
// do anything with fc
int x = HelperMethod();
fc.SaveChanges();
}
}
public int HelperMethod()
{
using(FunkyContainerAccessGuard guard = new FunkyContainerAccessGuard())
{
FunkyContainer fc2 = guard.Container;
// do anything with fc2
fc2.SaveChanges();
}
}
When the HelperMethod is called by MainMethod, the GlobalContainer is already created, and its used by both methods, so there is no conflict. Moreover, HelperMethod can be also used separately, and then it creates its own container.
However, this seems like a massive overkill to me; so:
Has this problem been already solved in form of some class (IoC?) or at least some nice design pattern?
Thank you.
Is having multiple containers open at the same time and acting on them carelessly an acceptable practice?
Generally this is perfectly acceptable, sometimes even necessary, but you have to be caucious with that. To have multiple containers at the same time is especially handy when doing multithreading operations. Because of how db works generally each thread should have its own DbContext that should not be shared with other threads. Downside to using multiple DbContext at the same time is that each of them will use separate db connection, and sometimes they are limited, what may lead to application occasionally being unable to connect to database. Other downside is the fact that entity generated by one DbContext may not be used with entity generated by other DbContext. In your example HelperMethod returns primitive type, so this is perfectly safe, but if it would return some entity object that in MainMethod you would like to assign for instance to some navigation property of entity created by MainMethod DbContext then you will receive an exception. To overcome this in MainMethod you would have to use Id of entity returned by HelperMethod to retrieve that entity once more, this time with fc context. On the other hand there is an advantage of using multiple contexts - if one context have some troubles, for instance it tried to save something that violated index constaint, then all next trials of saving changes will result in the same exception as the faulty change will still be pending. If you use multiple DbContexts then if one would fail, then second will operate independently - this is why DbContexts should not live long. So generally I would say the best usage rule would be:
Each thread should use a separate DbContext
All methods that executes on the same thread should share the same DbContext
Of course the above applies if the job to be done is short. DbContext should not live long. The best example would be web applications - there each server request is handled by separate thread and the operations to generate response generally do not take long. In such case all methods executed to generate one response should share for convenience the same DbContext. But each request should be served by separate DbContext.
Has this problem been already solved in form of some class (IoC?) or at least some nice design pattern?
What you need to assure is that your DbContext class is singleton per thread, but each thread has its own instance of that class. In my opinion best way to assure this is with IoC. For instance in Autofac in web applications I register my DbContext with the following rule:
builder
.RegisterType<MyDbContext>()
.InstancePerHttpRequest();
This way autofac IoC generates one DbContext per request and share existing instance within the request serving thread. You do not need to care here for disposing your DbContext. Your IoC will do this when your thread is over.
Working in multiple connections at the same time is not the right approach most of the time because:
You can get distributed deadlocks that SQL Server cannot resolve.
You might not see data that was previously written but not yet committed.
You can't share entities across context boundaries (here: methods).
More resource usage.
No ability to transact across context boundaries (here: methods).
These are very severe disadvantages. Usually, the best model is to have one context, connection and transaction for the request that the app is processing (HTTP or WCF request). That's very simple to set up and avoids a lot of issues.
EF is supposed to be used as a live object model. Do not cripple it by reducing it to CRUD.
static FunkyContainer GlobalContainer
That does not work. You shouldn't share a context across requests. Super dangerous. Consider storing a context in HttpContext.Items or whatever is the per-request store in your app.
I have a number of static methods that perform simple operations like insert or delete a record. All these methods follow this template of using:
public static UserDataModel FromEmail(string email)
{
using (var db = new MyWebAppDataContext())
{
db.ObjectTrackingEnabled = false;
return (from u in db.UserDataModels
where u.Email == email
select u).Single();
}
}
I also have a few methods that need to perform multiple operations that use a DataContext:
public static UserPreferencesDataModel Preferences(string email)
{
return UserDataModel.Preferences(UserDataModel.FromEmail(email));
}
private static UserPreferencesViewModel Preferences(UserDataModel user)
{
using(var db = new MyWebAppDataContext())
{
var preferences = (from u in db.UserDataModels
where u == user
select u.Preferences).Single();
return new UserPreferencesViewModel(preferences);
}
}
I like that I can divide simple operations into faux-stored procedures in my data models with static methods like FromEmail(), but I'm concerned about the cost of having Preferences() invoking two connections (right?) via the two using DataContext statements.
Do I need to be? Is what I'm doing less efficient than using a single using(var db = new MyWebAppDataContext()) statement?
If you examine those "two" operations, you might see that they could be performed in 1 database roundtrip. Minimizing database roundtrips is a major performance objective (second to minimizing database io).
If you have multiple datacontexts, they view the same record differently. Normally, ObjectTracking requires that the same instance is always used to represent a single record. If you have 2 DataContexts, they each do their own object tracking on their own instances.
Suppose the record changes between DC1 observing it and and DC2 observing it. In this case, the record will not only have 2 different instances, but those different instances will have different values. It can be very challenging to express business logic against such a moving target.
You should definately retire the DataContext after the UnitOfWork, to protect yourself from stale instances of records.
Normally you should use one context for one logical unit of work. So have a look at the unit of work pattern, ex. http://dotnet.dzone.com/news/using-unit-work-pattern-entity
Of cause there is some overhead in creating a new DataContext each time. But its a good practice to do as Ludwig stated: One context per unit of work.
Its using connection pooling so its not a too expensive operation.
I also think creating a new DataContext each time is the correct way but this link explains different approaches for handling the data context. Linq to SQL DataContext Lifetime Management
I developed a wrapper component that uses an interface like:
public interface IContextCacher {
DataContext GetFromCache();
void SaveToCache(DataContext ctx);
}
And use a wrapper to instantiate the context; if it exists in cache, it's pulled from there, otherwise, a new instance is created and pushed to the Save method, and all future implementations would get the value from the getter.
Depending on the type of application would be the actual caching mechanism. Say for instance, an ASP.NET web application. This could store the context in the items collection, so its alive for the request only. For a windows app, it could pull it from some singleton collection. It could be whatever you wanted under the scenes.