We Keep Coding

Problem with EF Core updating nested entities when using automapper

I am maintaining an application which uses EF Core to persist data to a SQL database.
I am trying to implement a new feature which requires me to retrieve an object from the database (Lets pretend its an order) manipulate it and some of the order lines which are attached to it and save it back into the database. Which wouldn't be a problem but I have inherited some of this code so need to try to stick to the existing way of doing things.
The basic process for data access is :
UI -> API -> Service -> Repository -> DataContext
The methods in the repo follow this pattern (Though I have simplified it for the purposes of this question)
public Order GetOrder(int id)
{
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
The service is where business logic and mapping to DTOs are applied, this is what the GetOrder method would look like :
public OrderDTO GetOrder(int id)
{
var ord = _repo.GetOrder(id);
return _mapper.Map<OrderDto>(ord);
}
So to retrieve and manipulate an order my code would look something like this
public void ManipulateAnOrder()
{
// Get the order DTO from the service
var order = _service.GetOrder(3);
// Manipulate the order
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
_service.SaveOrder(order);
}
And the method in the service which allows this to be saved back to the DB would look something like this:
public void SaveOrder(OrderDTO order)
{
// Get the original item from the database
var original = _repo.GetOrder(order.Id);
// Merge the original and the new DTO together
_mapper.Map(order, original);
_repo.Save(original);
}
Finally the repositories save method looks like this
public void Save(Order order){
_context.Update(order)
_context.SaveChanges();
}
The problem that I am encountering is using this method of mapping the Entities from the context into DTOs and back again causes the nested objects (in this instance the OrderLines) to be changed (or recreated) by AutoMapper in such a way that EF no longer recognises them as being the entities that it has just given to us.
This results in errors when updating along the lines of
InvalidOperationException the instance of ProductLine cannot be tracked because another instance with the same key value for {'Id'} is already being tracked.
Now to me, its not that there is ANOTHER instance of the object being tracked, its the same one, but I understand that the mapping process has broken that link and EF can no longer determine that they are the same object.
So, I have been looking for ways to rectify this, There are two ways that have jumped out at me as being promising,
the answer mentioned here EF & Automapper. Update nested collections
Automapper.Collection
Automapper.collection seems to be the better route, but I cant find a good working example of it in use, and the implementation that I have done doesn't seem to work.
So, I'm looking for advice from anyone who has either used automapper collections before successfully or anyone that has any suggestions as to how best to approach this.
Edit, I have knocked up a quick console app as an example, Note that when I say quick I mean... Horrible there is no DI or anything like that, I have done away with the repositories and services to keep it simple.
I have also left in a commented out mapper profile which does work, but isn't ideal.. You will see what I mean when you look at it.
Repo is here https://github.com/DavidDBD/AutomapperExample

Ok, after examining every scenario and counting on the fact that i did what you're trying to do in my previous project and it worked out of the box.
Updating your EntityFramework Core nuget packages to the latest stable version (3.1.8) solved the issue without modifying your code.

AutoMapper in fact "has broken that link" and the mapped entities you are trying to save are a set of new objects, not previously tracked by your DbContext. If the mapped entities were the same objects, you wouldn't have get this error.
In fact, it has nothing to do with AutoMapper and the mapping process, but how the DbContext is being used and how the entity states are being managed.
In your ManipulateAnOrder method after getting the mapped entities -
var order = _service.GetOrder(3);
your DbContext instance is still alive and at the repository layer it is tracking the entities you just retrieved, while you are modifying the mapped entities -
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
Then, when you are trying to save the modified entities -
_service.SaveOrder(order);
this mapped entities reach the repository layer and DbContext tries to add them to its tracking list, but finds that it already has entities of same type with same Ids in the list (the previously fetched ones). EF can track only one instance of a specific type with a specific key. Hence, the complaining message.
One way to solve this, is when fetching the Order, tell EF not to track it, like at your repository layer -
public Order GetOrder(int id, bool tracking = true) // optional parameter
{
if(!tracking)
{
return _context.Orders.Include(o=>o.OrderLines).AsNoTracking().FirstOrDefault(x=>x.Id == id);
}
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
(or you can add a separate method for handling NoTracking calls) and then at your Service layer -
var order = _repo.GetOrder(id, false); // for this operation tracking is false

EF 6 Lazy Loading Disabled but Child Record Loads Anyway

I'm using EF6 code first. There are two tables, Lesson and LessonSections. The LessonSections table has a foreign key to Lesson.Id
Here is the Lesson class with none important fields removed:
public partial class Lesson
{
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2214:DoNotCallOverridableMethodsInConstructors")]
public Lesson()
{
LessonSections = new HashSet<LessonSection>();
}
[StringLength(50)]
public string Id { get; set; }
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2227:CollectionPropertiesShouldBeReadOnly")]
public virtual ICollection<LessonSection> LessonSections { get; set; }
}
Here is how I'm initiating my data model:
var db = new Touch_Type_Trainer_DB.DataModel();
db.Configuration.ProxyCreationEnabled = false;
db.Configuration.LazyLoadingEnabled = false;
Just after my first call to the database to retrieve the first lesson in the database, the resulting object has no LessonSections
Then I make a second call to retrieve the sections into a separate object. (They must be in a separate objects since I want to serialize them to a JSON string and the serializer halts on the circular reference between Lesson and LessonSections if I use the standard EF LazyLoading.)
Now my original object has two sections loaded from the database even though I never accessed the LessonSections property and even though LazyLoadingEnabled is set to False!
Why do the LessonSections get loaded?
Edit:
I'm using Newtonsoft to serialize my object into a JSON string. Maybe there is a configuration setting in Newtonsoft that I should be setting so it doesn't get caught in the circular reference problem?
Also, I do want LazyLoading enabled for the majority of the code, just not for the serializing part.

is this a problem for you, or are you just curious as to why its happening?
The DBContext keeps track of all references for you. When you load the sections, it knows that the lessons have references to them, and wires it up for you.
you could stop this by disconnecting the objects , or by loading the sections from a different dbcontext
myDbContext.Entry(someLesson).State=Detached;
For the serialization issue, see this Q&A How Do You "Really" Serialize Circular Referencing Objects With Newtonsoft.Json?
or
http://johnnycode.com/2012/04/10/serializing-circular-references-with-json-net-and-entity-framework/

This is, in my opinion, a bad behaviour of EF.
EF works in this way (like probably you already noted): when you disable lazy loading relationships are not resolved (properties are not loaded even if you access the property).
In your case you can access to les.LessonSections and you see that is null or (in your case that you initialize it in Lesson constructor) empty.
If you, with the same context access to an object or to a collection not loaded with lazy load (build a query and materialize it) EF automatically try to solve relationships like during lazy load (also if the object does not have a proxy). This is your behavior, in a totally different query you access to LessonSections and EF solves relationships in Lesson.
At first look is a good behaviour but at the end you can have an inconsistent context (some objects with relationships resolved and some other not) and it can result in a buggy app. IMHO, the behaviour could be more consistent if EF (with LazyLoad disabled) doesn't resolve relationships at all and if you need to solve it you do it by hand. But is just my opinion.
About your question you can detach the object like suggested in another answer or use two different contexts (with the same or a different connection) in the same unit of work. I prefere (usually I use) this second approach disposing the context with Lazy Load disabled as soon as possible.

Linq to entities lazy loading

I have the following class generated by entity framework:
public partial class Branch
{
public short Id { get; set; }
public short CompanyId { get; set; }
public string Code { get; set; }
public string Title { get; set; }
public virtual Company Ts_Companies { get; set; }
}
I have the following method which takes all of the branches out of the database:
public Branch[] LoadBranches(int companyId, int page, int limit, string search, string sort, string sortOrder)
{
using (var dbContext = new TimeShedulerEntities())
{
var _branches = (from ct in dbContext.Branches
where ct.Title.Contains(search) || ct.Code.Contains(search)
select ct).OrderBy(c => c.Title).Skip((page - 1) * limit).Take(limit);
return _branches.ToArray();
}
}
In my model designer I see that the Lazy Loading is set to true, but when I iterate over the branches, the property Ts_Companies is null. Also I get the following exception:
An exception of type 'System.ObjectDisposedException' occurred in
EntityFramework.dll but was not handled in user code
Additional information: The ObjectContext instance has been disposed
and can no longer be used for operations that require a connection.
Am I forgetting something?

You created and disposed of the context during your function since it was inside the using statement. Each entity happens to know from which context it was created so that lazy loading is possible.
When you accessed the Ts_Companies property, the entity realized that it had not yet loaded that property since it is probably a navigation property and attempted to ask its ObjectContext (TimeShedulerEntities) to load that property. However, the context had been disposed and so that it what caused that exception.
You need to modify your query as follows to 'pre-load' the Ts_Companies:
var _branches = (from ct in dbContext.Branches.Include("Ts_Companies")
where ct.Title.Contains(search) || ct.Code.Contains(search)
select ct).OrderBy(c => c.Title).Skip((page - 1) * limit).Take(limit);
It will take possibly quite a bit longer to load depending on the size of the Ts_Companies object and how many you end up bringing back at once, but the entity will stop asking its object context to load the Ts_Companies since you would have already loaded them.
A side note: I have found that creation and disposal of object context on a per-method basis causes problems when the entities are passed outside the function. If you want to create and destroy the object context in every function, you probably want to have the function return something that is not an entity. In other words, have an object that can be constructed from an entity and has the properties you need, but don't have it reference the entity. In java these are often called Data Transfer Objects (DTOs). You lose the read-write ability of entity framework, but you don't have unexpected ObjectDisposedExceptions flying all over the place.
The problem comes when you ask an entity to be associated with another (for example, adding on entity to a ICollection property of another entity) when they come from different objectcontexts. This will cause headaches for you since you would have to manually attach the objects to the same context before performing that operation. Additionally, you lose the ability to save changes to those entities without manually attaching them to a different context.
My opinion on how I would do it:
I've found it easier to either have an object containing all of these database access functions control the lifetime of the context (i.e. have your containing object be IDisposable and during disposal, destroy the context) or simply not return entities and have the datastore be read-old, write-new essentially without any modification ability.
For example, I have my object (I will call it my data access object) with a bunch of methods for getting database objects. These methods return entities. The data access object also has a SaveChanges method which simply calls the context's SaveChanges method. The data access object contains the context in a protected property and keeps it around until the data access object itself is disposed. Nobody but the data access object is allowed to touch its context. At that point, the context is disposed by manually calling 'Dispose'. The data access object could then used inside a using statement if that is your use case.
In any case, it is probably best to avoid passing entities attached to a context outside the scope in which their context exists since entity framework keeps references to that context all over the place in the individual entities

But you didn't load your Ts_Companies, use Eager Loading instead:
var _branches = dbContext.Branches
.Where(b => b.Title.Contains(search) || b.Code.Contains(search))
.Include("Ts_Companies")
.OrderBy(c => c.Title)
.Skip((page - 1) * limit)
.Take(limit);
And I came across the same issue before System.ObjectDisposedException, in my MVC project and I didn't use using blocks,instead I define my context on class level.If I need to return and use an array (in my View) I use that context.If I need to just update some information then I have used using blocks.I hope this helps.

EF 4.1 DBContext and Navigation Properties

We've been using EF STEs for a while, but our application has grown quite a bit and we decided to sue the new 4.1 DbContext so we can "evolve" a separate business layer on top of our data layer without having to use different types for it.
In the elementary evaluation for the DbContext way of doing things, I am facing a little problem.
I am used to query and preload required related data like:
return context.Orders.Include("Detail").SingleOrDefault(ord => ord.ID == ID);
And then send the returned object to the UI for modification, and when returned from the UI save the changes to the database.
From what I read so far, doing the "change saving" in DbContext is easily done using code like this:
context.Entry(order).State = EntityState.Modified;
The problem with this code is that it actually marks all properties in the object as modified, a thing that's not allowed for some properties in my model (a business rule).
I resorted to the following solution (which seems to require a lot of code for a relatively small requirement! BTW, changing a modified property state to Unchanged is not supported):
context.Orders.Attach(order);
DbEntityEntry<Order> ordEntity = context.Entry(order);
string[] arr =
{
ordEntity.Property(ord => ord.ID).Name,
ordEntity.Property(ord => ord.ClientID).Name,
};
foreach (string prop in ordEntity.OriginalValues.PropertyNames)
{
if (!arr.Contains(prop))
{
ordEntity.Property(prop).IsModified = true;
}
}
context.SaveChanges();
The problem I am facing with this code is that the "Attach" statement is throwing an exception saying that there is some sort of conflict in the navigation properties in the attached object, even if no changes were made to anything at all! (saving the object exactly as it was retrieved from the database).
The error message is something like:
"Conflicting changes to the role 'Detail' of the relationship 'OrdersDatamodel.FK_Order_Detail' have been detected."
The questions are:
Is there a more "elegant" way for preventing the modification of certain object properties?
Does anybody know what's going on with the exception raised when attaching the object to the context?
Thanks.

From what I read so far, doing the "change saving" in DbContext is easily done using code like this:
context.Entry(order).State = EntityState.Modified;
You rarely need to explicitly set the state. When you modify properties, assuming they are virtual, the state will automatically change to Modified without you having to set it. Otherwise, DetectChanges will pick this up during your call to SaveChanges.

Entity Framework 4 (Assocations configured with filters)

I have a user entity that contains a collection of survey entities. i would like the assocation to include a filter on the relationship, such as 'IsCompleted', so whenever i eager load (or lazy load for that matter) the collection, this filtering happens.
Is this something we have control over?
thanks!

If you are using a DB back-end that supports views, you might consider using the view as the source for the collection of survey entities. Leverage the power of the DB to do that filtering for you.

Loading of associations for an entity always just gets them all, whether because you used Include during the initial query, called Load after the fact, or lazy-loading caused it. The concept of the navigation property kind of assumes this behavior.
E.J. Brennan's answer would work well. If you're not concerned about loading all surveys behind the scenes (because of performance/memory reasons or something) then you might also consider creating a separate property via a partial class definition on your entity that returns the filtered list.
public partial class User
{
public ICollection<Survey> CompletedSurveys
{
get { return Surveys.Where(s => s.IsCompleted); }
}
}