I am working on a C# ASP.NET MVC application that uses Entity Framework 6 (code first). We have a model that has multiple child objects, including one that inherits from a base class. There is a web page that generates a view from this model (after mapping it to a view model). The data is fetched via a call like this:
_context.TableName.Include(includes..).FirstOrDefaultAsync(i => i.Id == id);
There are multiple calls to .Include. Due to the complexity of the model Entity Framework generates a large query.
The first time the code runs, it takes about 4.5 to 5 seconds. Subsequent views of the page result in the data access logic running in around 450 to 550 milliseconds (using the Stopwatch class to profile).
However if I add a loop to repeatedly pull back the exact same data after the initial fetch each call takes around 12 to 15 milliseconds.
I thought this was simply because these calls were obviously returning the object from Entity Framework's cache. That is, no call to the database or materialization required. So I added logic to dispose of the context between each iteration of the loop, with a new context being created each time.
Running this under SQL Profiler showed a torrent of calls to the database from Entity Framework. However the result was the same.
This makes me think there some opportunity to optimize performance that we are missing. Does anyone have any idea as to why there is this discrepancy between how long the data access logic takes to run when the web page is viewed, and when that exact same logic is run (from within the same web page call) in a loop?
Does creating a new context not completely 'reset' Entity Framework? There must be some caching or something else going on to explain the massive performance difference.
Huge thanks in advance.
Related
I recently exposed to Entity Framework 6 caching mechanism.
As we might figure from this article, it does it in First-Lever manner.
Our system uses EF 6 (code first) along with MemoryCache to improve performance.
The main reason we use MemoryCache is because we need to execute an intense query on every page request. We execute this query x3 times (in the worst case) on every page request since there are client call backs.
I wonder if we still need to use the MemoryCache mechanism if the EF 6 already use one.
It is worth saying that we don't use any special caching feature or cache dependencies. Just a simple MemoryCache with timeouts.
The fact that EF caches entities in context is in no way a replacement for "real" cache, for various reasons:
You should not reuse EF context for more that one logical operation, because EF context represents unit of work, and so should be used according to this pattern. Also, even if you for some reason reuse context in multiple operations - you absolutely cannot do that in multi-threaded environment, like web server application.
It does not prevent you from making multiple queries for the same data to your database, for example:
var entity1 = ctx.Entities.Where(c => c.Id == 1).First();
var entity2 = ctx.Entities.Where(c => c.Id == 1).First();
This will still execute two queries to your database, despite the fact that query is the same and returns the same entity. So nothing is really "cached" in usual sense here. Note however, that both queries will return the same entity, even if database row has been changed between two queries. That is what is meant by EF context "caching". It will execute database query two times, but second time, while evaluating the result, it will notice that there is already entity with the same key attached to the context. So it will return this existing ("cached") entity instead, and will ignore new values (if any) returned by the second query. That behaviour is additional reason to not reuse the context between multiple operations (though you should not do it anyway).
So if you want to reduce load on your database - you have to use second-level caching using whatever suits your needs (from simple InMemoryCache to caching EF provider to distributed memcached instance).
EF only implements what is called first level cache for entities, It stores the entities which have been retrieved during the life time of a context so when you ask for that entity the second time it returns the entity from context. What you need is a second level cache but EF dosen't implants this features. NCache for example implements a wonderful caching architecture and a out of the box a second level cache provider for EF. Not in its open source version.
I'm using Entity Framework 6 on a SQL Server database to query an existing database (database first, so there's an EDMX in my project).
I've noticed that the first time I request an entity, it can take up to thirty seconds for the query to be executed. Subsequent queries to the same object then get completed in a matter of milliseconds. The actual SQL being executed is very fast so it's not a slow query.
I've found that Entity Framework generates views on the background and that this is the most likely culprit. What I haven't found, however, is a good solution for this. There's a NuGet package that can handle the View Generation (EFInteractiveViews), but it hasn't been updated since 2014 and I hardly seem to find any information on how to use it.
What options do I have nowadays? I've tried initializing Entity Framework on Application_Start by doing a few queries, but this doesn't seem to help much at all, and also it's quite difficult to perform the real queries on Application_Start, because most queries use data from the current user (who is not yet logged on at Application_Start) so it's difficult to run these in advance.
I've thought about creating an ashx file that constantly polls the application by calling the API and keep it alive. I've also set the Application Pool to "AlwaysRunning" so that EF doesn't restart when the app pool is recycled.
Does anyone have any tips or ideas on how I can resolve this or things I can try?
Thanks a lot in advance. I've spent the better part of two days already searching for a viable solution.
There are many practices to speed up Entity Framework, I will mention some of them
Turn off the LazyLoading (EDMX => open the file right click anywhere => properties => Lazy Loading Enabled set it to false )
Use AsNoTracking().ToList() and when you want to update, use Attach and update object state to EntityState.Modified
Use Indexes on your table
Use Paging, do not load all the data at once
Split your Edmx into many smaller, only include the ones you need in your page, ( this will effect the performance in good way)
If you want to load related objects "be eager and not lazy", use Include, you might include using System.Data.Entity to use the lambda include features
Example for splitting your Edmx
If you have the following objects for a rent a car app : Country, City , Person, Car, Rent, Gender, Engine, Manufacturers,..etc.
Now
If you are working on a screen to Manage (CRUD) person, this means you don't need Car,Rent,Manufacturer, so create ManagePerson.edmx contains ( Country, City, Person, Gender)
If you are working on managing (CRUD) Car then you don't need (Person,City, Gender,Rent), so you can create ManageCar.edmx containing ( Car, Manufacturer,Country, Engine)
Entity Framework must first compile and translate your LINQ queries into SQL, but after this it then caches them. The first hit to a query is always going to take a long time, but as you mention after that the query will run very quickly.
When I first used EF it was constantly an issue brought up by testers, but when the system went live and was used frequently (and queries were cached) it wasn't an issue.
See Hadi Hassans answer for general speed up tips.
I had a model having approximately 500 entity types. Now I have added approximately 2500 entity types for future usage. Therefore now I have approximately 3000 entity types.
At this moment my program does the same as in the situation, where I had only 500 entities. My program is just building a graph of entities, i.e. instantiating a lot of entities and connecting them via references.
Unfortunately my program takes approximately 20 time longer to run as before I added the new entity types, even despite I don't deal with instances of the new entity types.
Is it correct, that there is substantial overhead in Entity Framework and it grows very significantly in the number of entities in the model, even despite the majority of the model will not be used during the lifetime of a DbContext?
EF does a lot of reflection work at startup (which takes considerable time) over all entities defined, regardless if they are actually used or not. So if you see the startup (much) longer then it's somehow normal.
If you encounter this delay over the next queries and operations then you probably have another issue and you would need to provide more information for a solution.
You can try to do the generate the 'EF Views' at compile-time, instead of run-time.
(not exactly sure what it is, but it is something that EF does at start-up time)
See here
Generate Views - Generates pre-compiled views used by the EF runtime to improve start-up performance.
I'm trying to root cause the issue with slow speed upon sign in in my EF6 web app hosted in azure. With all the research I've done I still don't quite understand why my app is behaving the way it is.
When I first deploy the app, and attempt to sign in, it's slow, this I understand why, and it's acceptable. Subsequent sign-ins and calls are relatively quick. What I don't understand is why if I don't interact with the application for maybe 5 minutes (even though my Azure Web App has Always On set to Enabled and is a Standard pricing tier Web App) the next login will be back down to taking 20+ seconds.
I don't quite understand what "First Run" means when anyone says it in regards to Entity being slow. Does it only mean, "the first time the web app is accessed by ANYONE", or does it mean something like, "When the dbContext is instantiated by ONE SPECIFIC client for the first time, that is THEIR specific first run, and their instance of the app/dbcontext whatever is now warmed up and ready"
The latter doesn't seem to make sense, because I can sign in on one machine, and move to another machine and it will be relatively quick as well.
"First time" means first dbContext use (query) after the application starts. But when app is iddle for some time, app pool is restarted and next time you enter the site it will start again. That's why EF takes time when there is no activity for some time.
Have a look at this post about app pool restart in azure
"First Run" would refer to the first time an EF query is run after the Entity Framework assemblies are loaded in to an app domain.
EF is a fairly large set of assemblies and they take a bit of time to load initially. On the first query they will also do a lot of work verifying the model. A lot of this time can be reduced by Pre-caching the views for the model (MSDN). For databases with a lot of mapped tables and stored procs this can be a quite a long time. I've had projects that could take up to 3 minutes to start up. Precaching reduced that to about 10 seconds. It does add quite a bit of complexity to managing schema changes though.
Before the Entity Framework can execute a query or save changes to the
data source, it must generate a set of mapping views to access the
database. These mapping views are a set of Entity SQL statement that
represent the database in an abstract way, and are part of the
metadata which is cached per application domain. If you create
multiple instances of the same context in the same application domain,
they will reuse mapping views from the cached metadata rather than
regenerating them. Because mapping view generation is a significant
part of the overall cost of executing the first query, the Entity
Framework enables you to pre-generate mapping views and include them
in the compiled project.
EDIT :
This question is not about compiled queries, it is about generating the EF database view at compile time.
From the ADO.NET team blog: Exploring the Performance of the ADO.NET Entity Framework - Part 1:
View Generation 56%– A big part of creating an abstracted view of the database is providing the actual view for queries and updates in the store’s native language. During this step, the store views are created. The good news is there is a way of making view generation part of the build process so that this step can be avoided at run time.
The first database call in my web app takes about 2.5 seconds instead of ~30 ms for subsequent identical calls.
I generated a precompiled view source file using the T4 template from the ADO.NET team blog, but it has made no detectable difference.
The T4 template takes about 2.5 seconds to run and the generated code compiles.
What am I missing?
Fixed it!
The Generated View derived from EntityViewContainer must be in the assembly that contains the STOs Self-Tracking Objects, not the one that contains the edmx model.
Your first call to your application after a build, the web server will have unloaded the application, and the first call "starts" the application again, and everything that is associated with that. This has nothing to do with pre-compiled views. Therefore, the first call will always take longer as the application is loaded.
Incidentally, the same thing will happen on your production server. An idle worker-pool might well unload your application, and the next call will load the application again, taking significantly longer to perform than usual requests.