I want to ask about the execution timing of a LinqToSql query.
From my understanding refer to this MSDN blog. It seems that a LinqToSql query will only execute when
IQueryable's property is accessed
IQueryable's function (which is not returning IQueryable/IEnumeration type) is called
However, I did an experiment like that:
var ents = from ent in dal.ents
select ent;
string s1 = ents.first().Value1; // I got 1 here
Console.ReadLine(); // When the system is waiting for my input. I change the same record manually in DB, I change Value2 of ent from 2 to 3 here.
string s2 = ents.first().Value2 // I got 2 here.
Why am I still getting "2" for s2?
From my understanding, ents.first().Value2 should connect to the DB again and get the new Value2. Why am I still getting the old value?
As soon as you get Value1 on this line, the call is made to the db
string s1 = ents.first().Value1;
Then it keeps the object in memory (along with Value2). It doesn't call the database again when you try to access Value2.
Finally, I think I found the working principal behind.
L2S is really working like
LinqToSql query will only CONNECTION TO DB AND EXECUTE when
IQueryable's property being accessed
IQueryable's function (which is not returning IQueryable / IEnumeration type) being called
But in additions, after L2S fetch data from DB for each record at the first time. It will cache the record by its PK.
Eventually, on each further fetch. It will check if the record has been fetched before.
yes, It will use the cached version of the record instead of the DB version.
If no, It will use the DB version.
P.S. The lifetime of the cached records will last until the DBContext being released.
Related
I am debugging a piece of code which is using EF as ORM. Now, I am seeing somewhat interesting behavior from the application:
This is the code where I'm calling a stored procedure:
List<RequestListEntity> results = new List<RequestListEntity>();
var temp = System.Data.Object.ObjectContext.ExecuteFunction<T>("storedProcedure", param);
foreach (var item in temp)
{
results.Add(item);
}
Observations:
When I run the stored procedure on the server, it is very fast. It has joins with tables, but it returns 1000 records within a second
When I call the stored procedure from C# using the code shown above, it also returns within a second and returns objectResult<T> with a total of 1000 entries.
Now when I try to iterate through the result OR try to convert the result to a List, it is hell of a lot slower.
Now this raises a lot of questions:
If it returning from DB so fast then why mere conversion of 1000 records is taking so much time? Or may be it is still going back to DB for conversion?
Is there anything I can do to make it fast? When it calls the function, it returns very fast.
The performance issue is because of lazy loading and object tracking. when this method called result maps to an entity type, two things happen that don't happen when context.Database.SqlQuery is executed :
The entities are tracked by the context's change tracker.
The entities perform lazy loading.
I have the following method in my code;
Invoice GetInvoice()
{
return entities.Invoices.First(i => i.InvoiceNo == invoiceData.InvoiceId);
}
I have read about first level caching but I'm not sure what happens inside EF6 if I do something like this;
Scenario 1: Get the invoice and hold a local reference;
var invoice = GetInvoice();
invoice.UpdatedBy = "Pete"
invoice.UpdatedTime = DateTime.Now;
Scenario 2: Call GetInvoice every time I want to update the invoice;
GetInvoice().UpdatedBy = "Pete"
GetInvoice().UpdatedTime = DateTime.Now;
In the second scenario does the entity framework query the database a second time or does it just return a cached instance of the invoice (checking for cached instances first and only querying the database if it finds none)?
Using .Single, .First, .Where, etc will not cache the results unless you are using second-level caching.
There will be multiple data calls to the database when you invoke your GetInvoice();
The performance consideration of EF here
if you indent to have caching read more about second-level caching out here
It will both query database second time AND return you a cached instance. When you query your invoice second time, it will execute new query but then discard all results and return you already existing instance, because context already contains mapped entity of the same type with the same primary key. If for example your invoice would be externally modified between first and second call - you won't see that changes, despite that second database call was made.
I use two stored procedures that return the data with the same structure (list of records of the same type).
I call my method Execute(ISession session) twice. First time for the first stored procedure (it returns correct list of 6 rows). Second time - for the second stored procedure (it returns list of 11 rows, but first 6 rows are from the first request that overwrite the correct rows).
I found
Impact on NHibernate caching for searches with results including calculated value mapped as a formula (e.g. rank)
But I can't use it for IQuery
Any ideas or links how it can be fixed ?
public dynamic Execute(ISession session)
{
var query = session.GetNamedQuery(QueryName)
.SetCacheable(false)
.SetCacheMode(CacheMode.Ignore)
.SetReadOnly(true);
var results = query.List<T>();
return results;
}
I'm going to take a stab at answering this, because I think I have a hunch of what's going on, and I want to set you on the right track. I've made a lot of assumptions here, so please don't be too harsh on me if I was completely wrong with my guesses.
It feels like you're trying to use NHibernate as a tool to simply translate rows into objects. Instead NHibernate is a tool that translates between your object oriented domain model and your relational database domain model. It does a lot more that just turn rows into objects. In particular, the NHibernate feature that you're tripping over here is how NHibernate ensures that within a single NHibernate session, a single row in the database which represents a single entity will correspond to a single instance of an object. It uses its first-level cache to accomplish this.
Let's say you have two queries, QueryA and QueryB. These queries have been constructed so that they each pull from separate tables, TableA and TableB, so really they represent separate entities. However, the queries have also somehow been built so that the result look to NHibernate like the same entity. If QueryA and QueryB happen to return some of the same ids, then NHibernate will combine them into the same instance, so you would see some of the results from QueryA repeated when you run QueryB.
So how do we fix it?
The quick and dirty fix would be to use different sessions for each of those two queries, or throw a session.Clear() in-between them. The more appropriate fix would be to change these named queries so that they actually do return two different entities.
I have the same problem, in first place I resolved the problem with session.Clear() but this solution lead to another bug. I read the response of Daniel and this response I served to detect that the issue is in the stored procedure, the stored procedure did not return an unique identifier and this produced the error when I mapped the ID with nhibernate.
I'm using BatchDelete found on the answer to this question: EF Code First Delete Batch From IQueryable<T>?
The method seems to be wasting too much time building the delete clause from the IQueryable. Specifically, deleting 20.000 elements using the IQueryable below is taking almost two minutes.
context.DeleteBatch(context.SomeTable.Where(x => idList.Contains(x.Id)));
All the time is spent on this line:
var sql = clause.ToString();
The line is part of this method, available on the original question linked above but pasted here for convenience:
private static string GetClause<T>(DbContext context, IQueryable<T> clause) where T : class
{
const string Snippet = "FROM [dbo].[";
var sql = clause.ToString();
var sqlFirstPart = sql.Substring(sql.IndexOf(Snippet, System.StringComparison.OrdinalIgnoreCase));
sqlFirstPart = sqlFirstPart.Replace("AS [Extent1]", string.Empty);
sqlFirstPart = sqlFirstPart.Replace("[Extent1].", string.Empty);
return sqlFirstPart;
}
I imagine making context.SomeTable.Where(x => idList.Contains(x.Id)) into a compiled query could help, but AFAIK you can't compile queries while using DbContext on EF 5. In thesis they should be cached but I see no sign of improvement on a second execution of the same BatchDelete.
Is there a way to make this faster? I would like to avoid manually building the SQL delete statement.
The IQueryable isn't cached and each time you evaluate it you're going out to SQL. Running ToList() or ToArray() on it will evaluate it once and then you can work with the list as the cached version.
If you want to preserve you're interfaces, you'd use ToList().AsQueryable() and this would pass in a cached version.
Related post.
How do I cache an IQueryable object?
It seems there is no way to cache the IQueryable in this case, because the query contains a list of ids to check against and the list changes in every call.
The only way I found to avoid the two minute delay in building the query every time I had to mass-delete objects was to use ExecuteSqlCommand as below:
var list = string.Join("','", ids.Select(x => x.ToString()));
var qry = string.Format("DELETE FROM SomeTable WHERE Id IN ('{0}')", list);
context.Database.ExecuteSqlCommand(qry);
I'll mark this as the answer for now. If any other technique is suggested that doesn't rely on ExecuteSqlCommand, I'll gladly change the answer.
There is a EF pattern that works Ok.
it uses projection. to return ONLY keys from DB. (projections are not added to context,
So this is pretty quick.
Then You build the context with KEY only stub POCOs, and light the fuse....
basically.
var deleteMagazine = Context.Set<DeadMeat>.Where(t=>t.IhateYou == true).Select(t=>t.THEKEY).toList
//Now instantiate a dummy POCO with KEY only for the list,
foreach ( var bullet in deleteMagazine)
{
context.Set<deadmeat>.attach(bullet);
context.set<deadmeat>.remove(bullet);
// consider saving chnages every 1000 records .... performance, trial different values
if (magazineisEmpty) // your counter logic here :-)
context.SaveChanges
}
// shoot anyone still moving
context.SaveChanges
check SQL server profiler....
I'm using C#, .NET (4.0) and Entity Framework to connect to SQL CE 4.0.
I query some objects with specific properties, but the query returns only objects that meet search criteria only if that data was already saved to database, which is not that problematic, bigger problem is that if data is changed, but not yet saved to database it will still meet search criteria.
Example:
var query = from location in mainDBContext.Locations
where location.InUse == true
select location;
This query returns also objects where location.InUse = false if InUse was true when loaded from DB and then changed later on in code.
This is screen capture from one of the query results objects.
I really don't understand why it does this. I would understand if this query would always query database and I would get the older version of this object (thus InUse would be true).
Thank you for your time and answers.
That is how EF works internally.
Every entity uniquely identified by its key can be tracked by the context only once - that is called identity map. So it doesn't matter how many times did you execute the query. If the query is returning tracked entities and if it is repeatedly executed on the same context instance it will always return the same instance.
If the instance was modified in the application but not saved to the database your query will be executed on the database where persisted state will be evaluated but materialization process will by default use the current data from the application instead of data retrieved from the database. You can force the query to return state from the database (by setting mainDBContext.Locations.MergeOption = MergeOption.OverwriteChagens) but because of identity map your current modifications will be lost.
I'm not really sure what exactly your problem is, but I think you have to know this:
That kind of query always return data that is submitted into DB. When you change some entities in your code but they are not submitted into database the LINQ query will query the data from database, without your in-code changes.
LINQ queries use Deferred Execution, so your 'query' variable is not a list of results, it's just a query definition that is evaluated each time results are needed. You should add .ToList() to evaluate that query and get a list of results in that certain line of code.
An example for .ToList():
var query = (from location in mainDBContext.Locations
where location.InUse == true
select location).ToList();
I just ran into the same thing myself. It's a bit messy, but another option is to examine the Local cache. You can do this, for example:
var query = from location in mainDBContext.Locations.Local
where location.InUse == true
select location;
This will only use the local cache not saved to the database. A combination of local and database queries should enable you to get what you want.