I have spent 2 days bashing my head against this problem, and I can't seem to crack it (the problem that is). The same code was working fine until I added database relationships, and I have since read a lot about lazy-loading.
I have two database tables with a 1:1 relationship between them. PromoCode table tracks codes, and has a PK column named id. CustomerPromo table has a column PromoId which is linked to the PromoCode table id. These two tables have no other relationships. I generated all this in SQL Server Management Studio, then generated the model from the database.
To make matters slightly more complicated, I'm doing this inside a WCF data service, but I don't believe that should make a difference (it worked before database relationships were added). After enabling logging, I always get an Exception in the log file with text:
DataContext accessed after Dispose.
My function currently returns all entries from the table:
using (MsSqlDataContext db = new MsSqlDataContext())
{
// This causes issues with lazy-loading
return db.PromoCodes.ToArray();
}
I have read numerous articles/pages/answers and they all say to use the .Include() method. But this doesn't work for me:
return db.PromoCodes.Include(x => x.CustomerPromos).ToArray();
I've tried the "magic string" version as well:
return db.PromoCodes.Include("CustomerPromos").ToArray();
The only code I've managed to get to work is this:
PromoCode[] toReturn = db.PromoCodes.ToArray();
foreach (var p in toReturn)
p.CustomerPromos.Load();
return toReturn;
I've tried added a .Where() criteria to the query, I've tried .Select(), I've tried moving the .Include() after the .Where() (this answer says to do it last, but I think that's only due to nested queries). I've read about scenarios where .Include() will silently fail, and after all this I'm no closer.
What am I missing? Syntax problem? Logic problem? Once I get this "simple" case working, I also need to have nested Includes (i.e. if CustomerPromo table had a relationship to Customer).
Edit
Including all relevant code. The rest is either LINQ to SQL, or WCF Data Services configuration. This is all there is:
[WebGet]
[OperationContract]
public PromoCode[] Test()
{
using (MsSqlDataContext db = new MsSqlDataContext())
{
return db.PromoCodes.Include(x => x.CustomerPromos).ToArray();
}
}
If I call that through a browser directly (e.g. http://<address>:<port>/DataService.svc/Test) I get a reset connection message and have to look up the WCF logs to find out "DataContext accessed after Dispose.". If I make the same query through an AJAX call in a webpage I get an AJAX error with status error (that's all!).
I prematurely posted the previous answer when I didn't actually have any child data to fetch. At the time I was only interested in fetching parent data, and that answer worked.
Now when I actually need child data as well I find it didn't work completely. I found this article which indicates that .Include() (he says Including() but I'm not sure if that's a typo) has been removed, and the correct solution is to use DataLoadOptions. In addition, I also needed to enable Unidirectional Serialisation.
And to top it off, I no longer need DeferredLoadingEnabled. So now the final code looks like this:
using (MsSqlDataContext db = new MsSqlDataContext())
{
DataLoadOptions options = new DataLoadOptions();
options.LoadWith<PromoCode>(p => p.CustomerPromos);
db.LoadOptions = options;
return db.PromoCodes.ToArray();
}
After setting Unidirectional Serialisation it will happily return a parent object without having to load the child, or explicitly set DeferredLoadingEnabled = false;.
Edit: This did not solve the problem entirely. At the time of testing there wasn't any child data, and I wasn't trying to use it. This only allowed me to return the parent object, it doesn't return child objects. For the full solution see this answer.
Contrary to everything I've read, the answer is not to use .Include() but rather to change the context options.
using (MsSqlDataContext db = new MsSqlDataContext())
{
db.DeferredLoadingEnabled = false; // THIS makes all the difference
return db.PromoCodes.ToArray();
}
This link posted in the question comments (thanks #Virgil) hint at the answer. However I couldn't find a way to access LazyLoadingEnabled for LINQ to SQL (I suspect it's for EntityFramework instead). This page indicated that the solution for LINQ to SQL was DeferredLoadingEnabled.
Here is a link to the MSDN documentation on DeferredLoadingEnabled.
Related
Some previous code I had written used the Find() method to retrieve single entities by their primary key:
return myContext.Products.Find(id)
This worked great because I had this code tucked into a generic class, and each entity had a different field name as its primary key.
But I had to replace the code because I noticed that it was returning cached data, and I need it to return data from the database each call. Microsoft's documentation confirmed this is the behavior of Find().
So I changed my code to use SingleOrDefault or FirstOrDefault. I haven't found anything in documentation that states these methods return cached data.
Now I am executing these steps:
Save an entity via EF.
Execute an UPDATE statement in SSMS to update the recently saved
record's Description field.
Retrieve the entity into a new entity variable using SingleOrDefault
or FirstOrDefault.
The entities being returned still have the old value in the Description field.
I have run a SQL trace, and verified that the data is being queried during step 3. This baffles me - if EF is making a round trip to the database, why is it returning cached data?
I've searched online, and most answers apply to the Find() method. Furthermore, they suggest some solutions that are merely workarounds (dispose the DbContext and instantiate a new one) or solutions that won't work for me (use the AsNoTracking() method).
How can I retrieve my entities from the database and bypass the EF cache?
The behaviour you're seeing is described in Microsoft's How Queries Work article under point 3:
For each item in the result set
a. If this is a tracking query, EF checks if the data represents an entity already in the change tracker for the context instance
If so, the existing entity is returned
It's described a little better in this blog post:
It turns out that Entity Framework uses the Identity Map pattern. This means that once an entity with a given key is loaded in the context’s cache, it is never loaded again for as long as that context exists. So when we hit the database a second time to get the customers, it retrieved the updated 851 record from the database, but because customer 851 was already loaded in the context, it ignored the newer record from the database (more details).
All of this is saying that if you make a query, it checks the primary key first to see if it already has it in the cache. If so, it uses what's in the cache.
How do you avoid it? The first is to make sure you're not keeping your DbContext object alive too long. DbContext objects are only designed to be used for one unit of work. Bad things happen if you keep it around too long, like excessive memory consumption.
Do you need to retrieve data to display to the user? Create a DbContext to get the data and discard that DbContext.
Do you need to update a record? Create a new DbContext, update the record and discard that DbContext.
This is why, when you use EF Core with dependency injection in ASP.NET Core, it is created with a scoped lifetime, so any DbContext object only lives for the life of one HTTP request.
In the rare case you really do need to get fresh data for a record you already have an object for, you can use EntityEntry.Reload()/EntityEntry.ReloadAsync like this:
myContext.Entry(myProduct).Reload();
That doesn't help you if you only know the ID though.
If you really really need to reload an entity that you only have the ID for, you could do something weird like this:
private Product GetProductById(int id) {
//check if it's in the cache already
var cachedEntity = myContext.ChangeTracker.Entries<Product>()
.FirstOrDefault(p => p.Entity.Id == id);
if (cachedEntity == null) {
//not in cache - get it from the database
return myContext.Products.Find(id);
} else {
//we already have it - reload it
cachedEntity.Reload();
return cachedEntity.Entity;
}
}
But again, this should only be used in limited cases, when you've already addressed any cases of long-living DbContext object because unwanted caching isn't the only consequence.
Ok, I have the same problem and finally found the answer,
You doing everything right, that's just how EF works.
You can use .AsNoTracking() for your purposes:
return myContext.Products.AsNoTracking().Find(id)
make sure you addedusing Microsoft.EntityFrameworkCore; at the top.
It works like a magic
I am new to neo4j and neo4jclient so please excuse if my question is not on the right track.
With mutating cypher queries it is possible to create multiple nodes and relationships in one call. I want to create a query similar to the movie example data that comes with neo4j so that it creates multiple nodes and relationships in one post to the REST API.
Is it possible to write a query like this using neo4jclinet?
I've had a look at the documentation below but I couldn't see an example of the type of query I was after. Im quite new to this so perhaps I missed it.
https://github.com/Readify/Neo4jClient/wiki/cypher-examples
Usually there are two ways of doing this. First is to use batch with multiple queries, which isn't supported by the neo4jclient. Looking at the source code shows that there is some kind of batch support (like the GraphClient.ExecuteBatch method), but those are private.
The second is to build a Cypher query by adding each node as a parameter, which should result in request as described here. The code would look like this:
var queryBuilder = client.Cypher.
Create ("(movie:Movide {newMovie})");
foreach (Movie movie in movies) {
queryBuilder = queryBuilder.WithParam ("newMovie", movie);
}
queryBuilder.ExecuteWithoutResults ()
This however throws an ArgumentException: "A parameter with the given key is already defined in the query." if there is more then one element on the list.
So you are probably stuck with one of those nasty workarounds:
building a query with multiple (indexed) parameters
executing one query for one node
a manual query, which is discouraged.
This seems to me like an issue worth reporting to Readify.
I have this issue when declaring a IQueryable<T> object inside the:
select new { };
part when I want to call a none LINQ method like Tostring() or any other functions that I have as follows:
select new
{
TFPricep = CurrancyHelper.DecimalToCurrency(TFPrice),
TFDatep = TFDate.Tostring()
};
I get this famous error message that says:
LINQ to Entities does not recognize the method 'foo'.
So what happens is that I perform a ToList() and get the data from database and after that I will Have to perform a foreach loop on the list and call functions like ToString() and etc on each list record which has performance issues . as far as I have tested there was no such issue in LINQ to SQL but this issue still exists in Entity Framework
I wonder if there is any way to call none LINQ methods when querying and prevent performing such for each loop ?
When I faced this issue I found two possible solutions but finally decided to avoid using anything that Linq to Entities could not translate.
I found two options:
1) If you are using an .edmx file look at this:
http://msdn.microsoft.com/en-us/library/vstudio/dd456857(v=vs.110).aspx
2) This other option is a little uglier but does not require an .edmx file:
http://damieng.com/blog/2009/06/24/client-side-properties-and-any-remote-linq-provider
https://www.nuget.org/packages/Microsoft.Linq.Translations/
https://github.com/peschuster/PropertyTranslator
One final comment: I never tried these options so I'm not 100% sure that they even work but they seem legit.
No
Currently our website is facing a problem with slow response times (more than 1 min) when we query CRM from our website. We are using CRM 2011 though a web service. When we investigated we found that the time was spent at the point of querying CRM.
We have used the CrmSvcUtil.exe to generate our proxy classes that map to CRM entities. Then we create an instance of context and query CRM using LINQ with C#.
When we query, We load our parent object with LINQ to CRM and then we use LoadProperty to load the related children.
I would like to know if anyone out there using a different method of querying CRM, and if you have come across issues like this in your implementation.
I’ve included a simplified sample query below.
public void SelectEventById(Guid id)
{
var crmEventDelivery = this.ServiceContext.EventDeliverySet.FirstOrDefault(eventDelivery => eventDelivery.Id == id);
if (crmEventDelivery != null)
{
this.SelectCrmEventDeliveryWithRelationships(crmEventDelivery);
}
}
private void SelectCrmEventDeliveryWithRelationships(EventDelivery crmEventDelivery)
{
// Loading List of Venue Delivery on parent crmEventDelivery thats been passed
this.ServiceContext.LoadProperty(crmEventDelivery, Attributes.EventDelivery.eventdelivery_venuedelivery);
foreach (var venueDelivery in crmEventDelivery.eventdelivery_venuedelivery)
{
// Loading Venue on each Venue Delivery
ServiceContext.LoadProperty(venueDelivery, Attributes.VenueDelivery.venue_venuedelivery);
}
// Loading List of Session Delivery on parent crmEventDelivery thats been passed
this.ServiceContext.LoadProperty(crmEventDelivery, Attributes.EventDelivery.eventdelivery_sessiondelivery);
foreach (var sessionDelivery in crmEventDelivery.eventdelivery_sessiondelivery)
{
// Loading Presenters on each Session Delivery
ServiceContext.LoadProperty(sessionDelivery, Attributes.SessionDelivery.sessiondelivery_presenterbooking);
}
}
Like mentioned on the other answers your main problem is the number of web service calls. What no one mentioned is that you can retrieve many objects with a single call using query joins. So you could try something like:
var query_join = (from e in ServiceContext.EventDeliverySet
join v in ServiceContext.VenueDeliverySet on e.EventDeliveryId equals v.EvendDeliveryId.Id
join vn in ServiceContext.VenueSet on v.VenueDeliveryId equals vn.VenueDeliveryId.Id
join s in ServiceContext.SessionDeliverSet on e.EventDeliveryId equals s.EvendDeliveryId.Id
where e.EventDeliveryId == id // *improtant (see below)
select new { EventDelivery = e, VenueDelivery = v, Venue = vn, SessionDeliver = s }).ToList();
Then you can run a foreach on query_join and put it together.
***improtant: do not use the base Id property (e.Id), stick with e.EntityNameId.Value (don't know why but it took a while for me to figure it out. Id returns default Guid value "00000..").
Based on what you've provided this looks like a standard lazy-load issue, except my guess is that each lazy load is resulting in a web service call. This would be called a "chatty" service architecture. Your goal should be to make as few service calls as possible to retrieve data for a single request.
Calling to fill in details can seem like a good idea because you can re-use the individual service methods for cases where you only want data 1 or 2 levels deep, or all the way down, but you pay a steep performance penalty.
You would be better off defining a web service call that returns a complete object graph in scenarios like this. I don't know if/what you're using for an ORM layer within the CRM but if you make a specific call to fetch a complete graph of Deliveries then the ORM can eager-fetch the data into fewer SQL statements. Fewer calls to the web service (and subsequently fewer calls into the CRM's data store) should noticeably improve your performance.
So I can see why this might take a while. I think as everyone else have commented you are making quite a few web service calls. If you get a moment it would be interesting to know if the individual calls are slow or its just because you are making so many, I would suggest profiling this.
In any case I suspect you would get better performance by not using the strongly type entities.
I would suggest using a FetchXml query, this will allow you to build a Sql Xml-Style query. Basically you should be able to replace your many we bservice calls with a single call. The MSDN has an example, also check out the Stunnware FetchXml designer, Products > Stunnware Tools > Download and Evaluation. It was built for Crm 4 but supports virtually all the features you will need.
If you dont fancy that, you could also try a QueryExpression or OData, both of which should allow you to get your data in one hit.
After trying all the suggested tips in the other answers and doing further profiling, in our particular scenario with our use of CRM, and how it was set up - we decided to simply bypass it.
We ended up using some of the in-built views, this is not a recommended approach in the CRM documentation, but we really needed to achieve higher performance and the CRM approach in this instance was just in our way.
To anyone else reading this, see the other answers too.
Because the query does not know what fields will be needed later, all columns are returned from the entity when only the entity is specified in the select clause. In order to specify only the fields you will use, you must return a new object in the select clause, specifying the fields you want to use.
So instead of this:
var accounts = from acct in xrm.AccountSet
where acct.Name.StartsWith("Test")
select acct;
Use this:
var accounts = from acct in xrm.AccountSet
where acct.Name.StartsWith("Test")
select new Account()
{
AccountId = acct.AccountId,
Name = acct.Name
};
Check out this post more details.
To Linq or not to Linq
I have just started a new project using a Linq to Sql model and I'm implementing our first many to many relationship. I have found this blog that gives great info on how implement this:
http://blogs.msdn.com/mitsu/archive/2008/03/19/how-to-implement-a-many-to-many-relationship-using-linq-to-sql-part-ii-add-remove-support.aspx
When I try to add some child objects in and then remove one before saving I get an error,
System.InvalidOperationException:
Cannot remove an entity that has not
been attached.
Any ideas? Someone has already commented on this to the author of the blog, but there has been no response.
Much appreciated!
You are calling DeleteOnSubmit, on an entity that has not been sucked into the DataContext.
To check for this (in a rather hacky way), do the following:
var e = some_entity;
var cs = dc.GetChangeSet();
if (cs.Inserts.Any( x => x == e))
{
dc.SomeTable.DeleteOnSubmit(e);
}
dc.SubmitChanges();
I suspect that's why the author mentions that this particular post is working 'in memory' and not with linq to sql. Another approach would be to modify the onAdd event to Insert the added object into the context, that should at least alleviate the error you are seeing.
Take a look at the way PLINQO handles many to many relationships. There is a List of each entity that relates to another entity through a many to many relationship. You can add and remove without receiving the error you mention here.