We Keep Coding

Problem with EF Core updating nested entities when using automapper

I am maintaining an application which uses EF Core to persist data to a SQL database.
I am trying to implement a new feature which requires me to retrieve an object from the database (Lets pretend its an order) manipulate it and some of the order lines which are attached to it and save it back into the database. Which wouldn't be a problem but I have inherited some of this code so need to try to stick to the existing way of doing things.
The basic process for data access is :
UI -> API -> Service -> Repository -> DataContext
The methods in the repo follow this pattern (Though I have simplified it for the purposes of this question)
public Order GetOrder(int id)
{
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
The service is where business logic and mapping to DTOs are applied, this is what the GetOrder method would look like :
public OrderDTO GetOrder(int id)
{
var ord = _repo.GetOrder(id);
return _mapper.Map<OrderDto>(ord);
}
So to retrieve and manipulate an order my code would look something like this
public void ManipulateAnOrder()
{
// Get the order DTO from the service
var order = _service.GetOrder(3);
// Manipulate the order
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
_service.SaveOrder(order);
}
And the method in the service which allows this to be saved back to the DB would look something like this:
public void SaveOrder(OrderDTO order)
{
// Get the original item from the database
var original = _repo.GetOrder(order.Id);
// Merge the original and the new DTO together
_mapper.Map(order, original);
_repo.Save(original);
}
Finally the repositories save method looks like this
public void Save(Order order){
_context.Update(order)
_context.SaveChanges();
}
The problem that I am encountering is using this method of mapping the Entities from the context into DTOs and back again causes the nested objects (in this instance the OrderLines) to be changed (or recreated) by AutoMapper in such a way that EF no longer recognises them as being the entities that it has just given to us.
This results in errors when updating along the lines of
InvalidOperationException the instance of ProductLine cannot be tracked because another instance with the same key value for {'Id'} is already being tracked.
Now to me, its not that there is ANOTHER instance of the object being tracked, its the same one, but I understand that the mapping process has broken that link and EF can no longer determine that they are the same object.
So, I have been looking for ways to rectify this, There are two ways that have jumped out at me as being promising,
the answer mentioned here EF & Automapper. Update nested collections
Automapper.Collection
Automapper.collection seems to be the better route, but I cant find a good working example of it in use, and the implementation that I have done doesn't seem to work.
So, I'm looking for advice from anyone who has either used automapper collections before successfully or anyone that has any suggestions as to how best to approach this.
Edit, I have knocked up a quick console app as an example, Note that when I say quick I mean... Horrible there is no DI or anything like that, I have done away with the repositories and services to keep it simple.
I have also left in a commented out mapper profile which does work, but isn't ideal.. You will see what I mean when you look at it.
Repo is here https://github.com/DavidDBD/AutomapperExample

Ok, after examining every scenario and counting on the fact that i did what you're trying to do in my previous project and it worked out of the box.
Updating your EntityFramework Core nuget packages to the latest stable version (3.1.8) solved the issue without modifying your code.

AutoMapper in fact "has broken that link" and the mapped entities you are trying to save are a set of new objects, not previously tracked by your DbContext. If the mapped entities were the same objects, you wouldn't have get this error.
In fact, it has nothing to do with AutoMapper and the mapping process, but how the DbContext is being used and how the entity states are being managed.
In your ManipulateAnOrder method after getting the mapped entities -
var order = _service.GetOrder(3);
your DbContext instance is still alive and at the repository layer it is tracking the entities you just retrieved, while you are modifying the mapped entities -
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
Then, when you are trying to save the modified entities -
_service.SaveOrder(order);
this mapped entities reach the repository layer and DbContext tries to add them to its tracking list, but finds that it already has entities of same type with same Ids in the list (the previously fetched ones). EF can track only one instance of a specific type with a specific key. Hence, the complaining message.
One way to solve this, is when fetching the Order, tell EF not to track it, like at your repository layer -
public Order GetOrder(int id, bool tracking = true) // optional parameter
{
if(!tracking)
{
return _context.Orders.Include(o=>o.OrderLines).AsNoTracking().FirstOrDefault(x=>x.Id == id);
}
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
(or you can add a separate method for handling NoTracking calls) and then at your Service layer -
var order = _repo.GetOrder(id, false); // for this operation tracking is false

Entity Framework not changing EnityState on Update

I have a working Update method , which is simply changing a property value and calls SaveChanges() on db context:
public void Update(int id, string name)
{
var entity = context.Entities.Single(x => x.Id == id);
entity.Name = name;
context.SaveChanges();
}
this way changes do indeed get applied , however the EnityState remains "Unchanged". Any thoughts as to why? I am trying to avoid having to tell EF what's happening explicitly e.g. using context.Entry(entity).State = EntityState.Modified;
the problem is I am using the state in the overriden SaveChanges method:
public override int SaveChanges()
{
var context = ((IObjectContextAdapter)this).ObjectContext;
var objectStateEntries =
context.ObjectStateManager
.GetObjectStateEntries(EntityState.Added | EntityState.Modified);
...
return base.SaveChanges();
}
..when debugging, i can see that the state of my entity is Unchanged.

If you haven't disabled the change tracking of EF or proxy creation, then you shouldn't have problem with that update. EF by default tracks automatically changes when you entities meet the requirements you can find in this msdn page. If you meet those requirements and check later the type of you entity once is returned by Single extension method you will see that is a proxy class, not your real class. So, first check if you're meeting all those requirements that EF needs to track your changes automatically, you'll be fine with that code.
For either of these proxies to be created:
A custom data class must be declared with public access.
A custom data class must not be sealed
A custom data class must not be abstract .
A custom data class must have a public or protected constructor that
does not have parameters. Use a protected constructor without
parameters if you want the CreateObject method to be used to create a
proxy for the POCO entity. Calling the CreateObject method does not
guarantee the creation of the proxy: the POCO class must follow the
other requirements that are described in this topic.
The class cannot implement the IEntityWithChangeTracker or
IEntityWithRelationships interfaces because the proxy classes
implement these interfaces.
The ProxyCreationEnabled option must be set to true.
For change tracking proxies:
Each property that is mapped to a property of an entity type in the
data model must have non-sealed,
public, and virtual get and set
accessors.
A navigation property that represents the "many" end of a
relationship must return a type that implements ICollection, where T
is the type of the object at the other end of the relationship.
If you want the proxy type to be created along with your object, use
the CreateObject method on the ObjectContext when creating a new
object, instead of the new operator.

Linq ToList() fails to trigger Immediate Execution

I am having a very peculiar problem: the ToList() extension method is failing to convert results to a list. here is my code, standard boilerplate linq query, I converted ToList() twice for good measure
var assets = new List<Asset>();
using (var ctx = new LeaseContext())
{
assets = ctx.Assets.OrderBy(o => o.Reference).Where(w => w.Status == AssetStatus.Active).ToList();
assets.ToList();
}
return assets;
yet the assets are still a list of System.Data.Entities.DynamicProxies....
I've never had this problem before.

The reason is lazy loading. When lazy loading is enabled in EF (by default), then (again, by default) EF creates dynamic proxies for each entity. It's required for loading related entities. Dynamic proxy will be inherited from entity class. So in your case it will be inherited from Asset. But dynamic proxy will have reference to the context which created its instance. And it will override navigation properties (which are virtual) to query entities via context which is stored in dynamic proxy.
And it's completely legal to add instances of derived types to list of base type.
If you don't want dynamic proxies, then just disable lazy loading and proxy creation:
ctx.Configuration.LazyLoadingEnabled = false; // turn-off loading on-demand
ctx.Configuration.ProxyCreationEnabled = false; // turn-off wrapper class generation
Technically you can just turn-off proxy generation and lazy loading will not work. But I prefer to turn-off both settings explicitly.

Entity Framework 6 - DataServiceContext Detect Has Changes

I have a WCF server application running Entity Framework 6.
My client application consumes OData from the server via a DataServiceContext, and in my client code I want to be able to call a HasChanges() method on the context to see if any data in it has changed.
I tried using the following extension method:
public static bool HasChanges(this DataServiceContext ctx)
{
// Return true if any Entities or links have changes
return ctx.Entities.Any(ed => ed.State != EntityStates.Unchanged) || ctx.Links.Any(ld => ld.State != EntityStates.Unchanged);
}
But it always returns false, even if an entity it is tracking does have changes.
For instance, given that I have a tracked entity named Customer, the following code always returns before calling SaveChanges().
Customer.Address1 = "Fred"
if not ctx.HasChanges() then return
ctx.UpdateObject(Customer)
ctx.SaveChanges()
If I comment out the if not ctx.HasChanges() then return line of code, the changes are saved successfully so I'm happy that the entity has received the change and is able to save it.
It seems that the change is getting tracked by the context, just that I can't determine that fact from my code.
Can anyone tell me how to determine HasChanges on a DataServiceContext?

Far out. I just read through DataServiceContext.UpdateObjectInternal(entity, failIfNotUnchanged), which is called directly from UpdateObject(entity) with a false argument.
The logic reads like:
If already modified, return; (short-circuit)
If not unchanged, throw if failIfNotUnchanged; (true only from ChangeState())
Else set state to modified. (no data checks happened)
So by the looks of it, UpdateObject doesn't care about/check the internal state of the entity, just the State enum. This makes updates feel a little inaccurate when there are no changes.
However, I think your problem is then in the OP 2nd block of code, you check your extension HasChanges before calling UpdateObject. The entities are only glorified POCOs (as you can read in your Reference.cs (Show Hidden Files, then under the Service Reference)). They have the obvious properties and a few On- operations to notify about changing. What they do not do internally is track state. In fact, there is an EntityDescriptor associated to the entity, which is responsible for state-tracking in EntityTracker.TryGetEntityDescriptor(entity).
Bottom line is operations actually work very simply, and I think you just need to make your code like
Customer.Address1 = "Fred";
ctx.UpdateObject(Customer);
if (!ctx.HasChanges()) return;
ctx.SaveChanges();
Though as we know now, this will always report HasChanges == true, so you may as well skip the check.
But don't despair! The partial classes provided by your service reference may be extended to do exactly what you want. It's totally boilerplate code, so you may want to write a .tt or some other codegen. Regardless, just tweak this to your entities:
namespace ODataClient.ServiceReference1 // Match the namespace of the Reference.cs partial class
{
public partial class Books // Your entity
{
public bool HasChanges { get; set; } = false; // Your new property!
partial void OnIdChanging(int value) // Boilerplate
{
if (Id.Equals(value)) return;
HasChanges = true;
}
partial void OnBookNameChanging(string value) // Boilerplate
{
if (BookName == null || BookName.Equals(value)) return;
HasChanges = true;
}
// etc, ad nauseam
}
// etc, ad nauseam
}
But now this works great and is similarly expressive to the OP:
var book = context.Books.Where(x => x.Id == 2).SingleOrDefault();
book.BookName = "W00t!";
Console.WriteLine(book.HasChanges);
HTH!

Is it possible you're not adding/editing your entities properly? MSDN states that you must use AddObject, UpdateObject, or DeleteObject to get change tracking to fire on the client (https://msdn.microsoft.com/en-us/library/gg602811(v=vs.110).aspx - see Managing Concurrency). Otherwise your extension method looks good.

In order for this to work, automatic change tracking must be enabled. You can find this setting in
ctx.Configuration.AutoDetectChangesEnabled
All the entity objects must also be tracked by the context ctx.
This means that they must be returned by one of ctx's methods or explicitly added to the context.
It also means that they must be tracked by the same instance of DataServiceContext. Are you somehow creating more than one context?
The model must also be configured correctly. Perhaps Customer.Address1 is not mapped to a database column. In that case, EF will not detect changes to the column.

I doubt that the datacontext in client is not the same one.so the changes is always is false.
You must be sure the Datacontext is the same one(instance), for every changes of the Datacontext. Then to detect the changes is meaningful.
Another way ,you must tracked the changes by yourself.simply using the Trackable Entities to help you tracking the changes of entities in the datacontext.
BTW. I Use the Code ' ctx.ChangeTracker.HasChanges()' to detect the changes of DataContext.
public bool IsContextDirty(DataServiceContext ctx)
{
#if DEBUG
var changed = ctx.ChangeTracker.Entries().Where(t => t.State != EntityState.Unchanged).ToList();
changed.ForEach(
(t) => Debug.WriteLine("entity Type:{0}", t.Entity.GetType()));
#endif
return ctx != null && ctx.ChangeTracker.HasChanges();
}

Nhibernate C# making XML files for every query

I am trying to learn how to use the power of Nhibernate and after watching a turtorial i am starting to geta good grip on it.
However there is something that is bothering me.
Take for instance the following query:
var query = "SELECT * " +
"from DAGE_I_KS WHERE DATO in (:orderYear));";
var session = mySessionFactory.OpenSession();
var result = session.CreateSQLQuery(query)
.AddEntity(typeof(DAGE_I_KS))
.SetString("orderYear", "2012")
.List<DAGE_I_KS>();
Now for this to work i have to do the following steps:
Create an xml file that maps the object i want to use
Create the object class file
Create the query and execute it and insert the result into the object (or a list)
Now i have to do this for every single query i want to do. In a program you will most likely do 10+ query (if not more) which means that i have to do these steps 10 times.
For me it seems ilogical. And i am having a hard time understanding how this can save me some time and trouble.
My question is, is this correct that i have to do this every time or is there something ive missed and is there a better way to do use Nhibernate when you have costum querys.

Runtime NH revolves around the concept of a session (ISession). As you know, you get the session from the session factory.
The session is the central object through which you interact with NH to execute queries and save data. The session has a flush mode property.
It is important to know that all the entities that you get from the session by using a query are linked to that session. So, if you modify an entity which is fetched, even if you do not explicitly call session.Update(obj), when the session is flushed these changes will be persisted. Thus it follows that the only methods of the session which you actually must call are session.saveOrUpdate(obj) and session.delete(obj), because you must register newly created entities with the session, and you must register entities for deletion. When the session is flushed, these changes will be persisted.
Try to read about the session flush modes, and also the ITransaction interface.
Now, how is this related to queries?
Well, I have learned that it is best to encapsulate queries into query objects. Is not my idea, but something I picked up from the blog of Fabio Maulo, or Ayende not sure but searching for the term query object in conjuncture with NH should be informative, and if you find out who to credit with the idea I will update the answer :)
It is clear that the query object needs the session to work. So, you must write the plumbing to give it one. In my implementation, a query object would be simply:
public class ShiftByNameWithOccurences : AbstractQueryObject<IList<Shift>>
{
private string Name;
public ShiftByNameWithOccurences(string name)
{
this.Name = name;
}
public override IList<Shift> GetResult()
{
var list =
(
from shift in this.Session.Query<Shift>()
where shift.Name == this.Name
select shift
)
.Fetch(p => p.Occurrences)
.ToList();
return list;
}
}
Obviously, it is easy to implement a generic by name query for all your db entities.
In this example, the occurrences of the shift are eagerly loaded, to avoid the N+1 select problem.
It is easy to see how having encapsulated queries can benefit your OO driven app design.
Also, since you have the ISession instance, you can use what ever query method of NH that you choose.
For reference:
public abstract class AbstractQueryObject<TResult> : IQueryObject<TResult>
{
protected ISession Session;
public void Configure(object parameter)
{
if (!(parameter is ISession))
throw new ArgumentException(String.Format("Argument of wrong type. Expecting {0} but got {1}.", typeof(ISession), parameter != null ? parameter.GetType().ToString() : "null"), "parameter");
this.Session = parameter as ISession;
}
public abstract TResult GetResult();
}
public interface IQueryObject<TResult>
{
void Configure(object parameter);
TResult GetResult();
}
Even, this interface is imagined to not be linked to NH, but also so that you can use it with EF, or some other future ORM if needed. The abstract query object is in the x.NH namespace :)
Edit:
For the task of logging SQL of each query I can suggest to use interceptors to log sql.
Register a per session interceptor when you create a session. For this task you should register only one interceptor object with the session. Then, when you instantiate the query object pass in the interceptor object as a constructor argument, or in the configure method whichever you like. When the query is being executed (in get result method) - at the start tell to the interceptor object that you want to start listening. In the implementation of your interceptor create code which will then forward all the sql to the listening query object. In the query object create a custom xml using the sql. When the query is finished, unregister the query object from the interceptor object.
But keep in mind that if additional lazy loading sql statements are executed after the query object exits the get result method will not be logged.
E.g.:
public class UseCase
{
public void Method()
{
//when instantiating a session pass the interceptor to it.
//then, also pass this sniffer to the query objects you create.
//make the query objects listeners.
//when the query object is to be executed (start of the get result method)
//call the set active listener method on the sniff notifier given to the q.o.
//in the on prepare statement method of the q.o. do whatever with the sql.
SqlSniffer myInterceptor = new SqlSniffer();
var session = this.SessionFactory.OpenSession(myInterceptor);
}
}
public interface ISqlSniffListener
{
void OnPrepareStatement(string sql);
}
public interface ISqlSniffNotifier
{
void SetActiveListener(ISqlSniffListener listener);
}
public class SqlSniffer : EmptyInterceptor, ISqlSniffNotifier
{
private ISqlSniffListener ActiveListener;
public void SetActiveListener(ISqlSniffListener listener)
{
this.ActiveListener = listener;
}
public override NHibernate.SqlCommand.SqlString OnPrepareStatement(NHibernate.SqlCommand.SqlString sql)
{
if (this.ActiveListener != null)
this.ActiveListener.OnPrepareStatement(sql.ToString());
return base.OnPrepareStatement(sql);
}
}
Or even better, it is probably better to implement a listener which is not integrated in the query object (to honor SRP) .. but you would have to signal to it which query is being executed then...which can be easy if you had an object responsible for executing the query objects, which provides them the session, and also possibly creates it ;)
P.S.
It was Ayende who I read discussing the q.o., but the idea seems older.