We Keep Coding

.Net Core Lazyloading issue when model built outside the context

Lazy loading of associated entity objects does not work when model is built outside of OnModelCreating and the associated object despite keeping all the methods virtual.
Eg.,
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
if (!optionsBuilder.IsConfigured)
{
optionsBuilder
.UseLazyLoadingProxies()
.UseModel(new ModelBuilderService().GetOrCreateCompiledModel())
.UseSqlServer(#"connectionstring",
sqlOption => sqlOption.UseNetTopologySuite());
}
}
public class ModelBuilderService
{
private static IModel GetOrCreateCompiledModel(IEnumerable<string> modelSupplyingAssemblyPatterns)
{
var conventions = SqlServerConventionSetBuilder.Build();
var modelBuilder = new ModelBuilder(conventions);
var modelBuilderType = typeof(ModelBuilder);
var entityMethod = modelBuilderType.GetMethod("Entity", modelBuilderType.GetGenericArguments());
var pathToUse = AppDomain.CurrentDomain.BaseDirectory;
if (!AppDomain.CurrentDomain.BaseDirectory.Contains("bin"))
{
pathToUse = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "bin");
}
var entitiesAdded = new HashSet<string>();
if (entityMethod == null)
{
throw new NullReferenceException("Cannot find Entity method on DbModelBuilder");
}
foreach (var assemblyPattern in modelSupplyingAssemblyPatterns)
{
var dataProviderModels = Directory.EnumerateFiles(pathToUse, assemblyPattern, SearchOption.AllDirectories);
foreach (var dll in dataProviderModels)
{
var assembly = Assembly.LoadFrom(dll);
modelBuilder.ApplyConfigurationsFromAssembly(assembly);
var typesToRegister = assembly.GetTypesInheritingFrom<BaseObject>();
foreach (var entity in typesToRegister)
{
if (entitiesAdded.Add(entity.FullName))
{
entityMethod.MakeGenericMethod(entity)
.Invoke(modelBuilder, new object[] { });
}
}
}
}
return modelBuilder.Model;
}
}
Trying to find a workaround for this issue since I have a generic solution and the data entities are built outside the Context using "UseModel" method but the lazyloading support goes away for this way and proxy objects are not created for entities fetched from database.

The problem is that lazy loading proxies package uses convention which is executed after the model is built and modifies it. While the external model is built without that convention being in place, so the functionality is not activated at all.
The following workaround is for the latest official EF Core release 2.2.4 at the time of writing. Most likely it would need to be updated accordingly if you upgrade to a newer EF Core version (3.0+) or removed if they fix it.
The EF Core 2.2.4 code of SqlServerConventionSetBuilder.Build() method you are using looks like this:
public static ConventionSet Build()
{
var serviceProvider = new ServiceCollection()
.AddEntityFrameworkSqlServer()
.AddDbContext<DbContext>(o => o.UseSqlServer("Server=."))
.BuildServiceProvider();
using (var serviceScope = serviceProvider.GetRequiredService<IServiceScopeFactory>().CreateScope())
{
using (var context = serviceScope.ServiceProvider.GetService<DbContext>())
{
return ConventionSet.CreateConventionSet(context);
}
}
}
As you can see, it uses some trickery, with the most important being the own DbContextOptionsBuilder. So all we need is to add UseLazyLoadingProxies() call to that builder.
In order to do that, create a private static method with the modified code:
static ConventionSet BuildSqlServerConventionSet()
{
var serviceProvider = new ServiceCollection()
.AddEntityFrameworkSqlServer()
.AddDbContext<DbContext>(o => o.UseSqlServer("Server=.").UseLazyLoadingProxies()) // <--
.BuildServiceProvider();
using (var serviceScope = serviceProvider.GetRequiredService<IServiceScopeFactory>().CreateScope())
{
using (var context = serviceScope.ServiceProvider.GetService<DbContext>())
{
return ConventionSet.CreateConventionSet(context);
}
}
}
and use it in place of SqlServerConventionSetBuilder.Build() call, e.g.
var conventions = BuildSqlServerConventionSet();
// ... the rest
Update: Also note that the ModelBuilder.Model property returns pending mutable model during the building. In order to get the finalized model "ready for use by the runtime", replace
return modelBuilder.Model;
with
return modelBuilder.FinalizeModel();
This method is executed by EF Core infrastructure "automatically when using OnModelCreating".

One workaround for lazy-loading with EFCore 2.2.4 in this scenario is by injecting the ILazyLoader service into an entity. This method doesn't require entity types to be inherited from or navigation properties to be virtual, and allows entity instances created with new to lazy-load once attached to a context. However, it requires a reference to the ILazyLoader service, which is defined in the Microsoft.EntityFrameworkCore.Abstractions package. Example code for lazyloading a Datamodel with many-many relationship will be as follows:
`public partial class PersonOrganisation
{
private Person person;
private Organisation organisation;
private ILazyLoader LazyLoader { get; set; }
private PersonOrganisation(ILazyLoader lazyLoader)
{
LazyLoader = lazyLoader;
}
public PersonOrganisation()
{
}
public Guid? PersonId { get; set; }
public Guid? OrganisationId { get; set; }
public virtual Organisation Organisation {
get => LazyLoader.Load(this, ref organisation);
set => organisation = value;
}
public virtual Person Person {
get => LazyLoader.Load(this, ref person);
set => person = value;
}
}`

Update an existing entry with Entity Framework

I want to update an existing entry with entity framework and here is my current code:
[HttpPost]
public IActionResult Edit(Product product)
{
if (ModelState.IsValid)
{
var result = _productRepository.Query().FirstOrDefault(x => x.Id == product.Id);
if (result == null)
return RedirectToAction("Index", "Products");
_productRepository.Update(product);
//result = product;
_productRepository.Save();
return View("Edit", result);
}
What have I tried:
result = product; doesn't seem to update the row in db.
public void Update(T item)
{
_context.Entry(item).CurrentValues.SetValues(item);
}
doesn't seem to update the row in db.
result.Title = product.Title - works, but then I have to do this for each field, is there a way to update a row simply by replacing the values with another object?

Edit
Actually, I realized that the code below will not work because you are already tracking the same entity, this is caused by this line:
var result = _productRepository.Query().FirstOrDefault(x => x.Id == product.Id);
So you need to either remove that line and just use the Update method below with the product object as the parameter or use the result object and update it based on the data from your product class and then save:
result.Name = product.Name;
[...]
In this case you don't need to call _repository.update, just _repository.save
Using product to update
Assuming your Product class is an object of the same class as your Product entity class you need to make sure that it is being tracked by entity framework and to mark it as modified before it can be saved:
To do that, modify your update method as follows:
public void Update(T item)
{
if (!_context.Set<T>().Local.Any(e => e == item))
{
_context.Set<T>().Attach(item);
}
_context.Entry(item).State = EntityState.Modified
}
Then just save it and it should work:
_productRepository.Update(product);
_productRepository.Save();
A better approach?
Instead of sending entity framework entities back and forth to and from your views, you could create a model class specifically for your view and then retrieve and update your database entity models as needed:
For example if your Product database entity looks like this:
public class Product
{
public int Id { get; set; }
public string ProductName { get; set; }
public int InternalId { get; set; }
}
In your view you don't need / want to use the InternalId field, so you would have a model in your Website assembly that could look like the following:
public class ProductModel
{
public int Id { get; set; }
public string ProductName { get; set; }
}
And then in your controller, this is what you will use:
[HttpPost]
public IActionResult Edit(ProductModel product)
{
if (!ModelState.IsValid)
{
return View(product);
}
var dbProduct = _productRepository.Query().FirstOrDefault(x => x.Id == product.Id);
if (dbProduct == null)
{
//Product doesn't exist, create one, show an error page etc...
//In this case we go back to index
return RedirectToAction("Index", "Products");
}
//Now update the dbProduct using the data from your model
dbProduct.ProductName = product.ProductName;
If you have a lot of fields, you don't want to do this manually, there are some libraries that will do this for you, for example, AutoMapper or my personal favorite (faster, easier to use) ValueInjecter
Using ValueInjecter you would do something like this to assign all the common properties automatically
dbProduct.InjectFrom(product);
Finally, just call save, this time you don't need to change the state because EF is already tracking your entity:
_productRepository.Save();

How do I insert entities with a 1:n relationship in Azure App Service

I need a 1:n relationship using the Azure App Service. I followed this tutorial for the entities.
My client entities (same as tutorial, except I added a constructor):
public abstract class Entity {
// defines id, version, createdAt, updatedAt, deleted
}
public class Animal : Entity {
public string Name { get; set; }
}
public class Zoo : Entity {
public Zoo() { Animals = new List<Animal>(); }
public string Name { get; set; }
public List<Animal> Animals { get; set; }
}
My server entities (same as tutorial, except I added a constructor):
public class Animal : EntityData {
public string Name { get; set; }
}
public class Zoo : EntityData {
public Zoo() { Animals = new List<Animal>(); }
public string Name { get; set; }
public virtual ICollection<Animal> Animals { get; set; }
}
The ZooController table controller (stock scaffolded code, except I added attributes):
[HttpPatch]
public Task<Zoo> PatchZoo(string id, Delta<Zoo> patch) {
return UpdateAsync(id, patch);
}
[HttpPost]
public async Task<IHttpActionResult> PostZoo(Zoo item) {
Zoo current = await InsertAsync(item);
return CreatedAtRoute("Tables", new { id = current.Id }, current);
}
There is a similar table controller for Animal.
Now for the problem. I perform an insert and an update from the client:
// create animal (child)
var animal = new Animal() { Name = "Dumbo" };
await animalTable.InsertAsync(animal);
// create zoo (parent)
var zoo = new Zoo() { Name = "Tokyo National Zoo" };
await zooTable.InsertAsync(zoo);
// add animal to zoo
zoo.Animals.Add(animal);
await zooTable.UpdateAsync(zoo);
// push
await zooTable.MobileServiceClient.SyncContext.PushAsync();
This throws exceptions. On the server, it throws with StatusCode: 409, ReasonPhrase: 'Conflict', and on the client it throws with "The operation failed due to a conflict: 'Violation of PRIMARY KEY constraint 'PK_dbo.Animals'. Cannot insert duplicate key in object 'dbo.Animals'. Basically, the server is saying that I'm trying to insert the same Animal record twice.
What am I doing wrong?

I'm pretty sure you're running into the Entity Framework "detatched entities" problem. See e.g. Many to Many Relationships not saving. The problem is that Entity Framework has not loaded the child item into its context, so it thinks that it needs to insert the child item as well as the parent. (There have been long-standing feature requests in Entity Framework to solve this, but the feature has never been added.)
On the server, you need to check if the referenced child entity already exists in the database, and if it does, just add a reference to the existing item, rather than inserting.

Please try as shown below.
// create animal (child)
var animal = new Animal() { Name = "Dumbo" };
// create zoo (parent)
var zoo = new Zoo() { Name = "Tokyo National Zoo" };
zoo.Animals.Add(animal);
await zooTable.InsertAsync(zoo);
Note : Above code will automatically insert a record to the animalTable too due to FK relationship. You don't need to do it manually.

For anyone else struggling with this. Here is my solution based on #lindydonna's feedback.
[HttpPost]
public async Task<IHttpActionResult> PostZoo(Zoo item) {
// replace child entities with those from the context (so they are tracked)
if (item.Animals.Any()) {
// find animals which are already in the db
var ids = item.Animals.Select(x => x.Id);
var oldItems = context.Animals.Where(x => ids.Contains(x.Id)).ToList();
var newItems = item.Animals.Where(x => !oldItems.Select(y => y.Id).Contains(x.Id));
// add new animals if there are any
if (newItems.Any()) {
foreach (var i in newItems) {
context.Animals.Add(i);
}
await context.SaveChangesAsync();
}
// replace
item.Animals.Clear();
foreach (var i in oldItems) item.Animals.Add(i);
foreach (var i in newItems) item.Animals.Add(i);
}
// now add the Zoo (which contains tracked Animal entities)
Zoo current = await InsertAsync(item);
return CreatedAtRoute("Tables", new { id = current.Id }, current);
}
This works. Though it's probably not performant.
You should probably also add:
modelBuilder.Entity<Zoo>().HasMany(e => e.Animals);
...though for some reason for me it works without as well.
EDIT
Better way, which doesn't allow new child entities to be sent together with the parent entity. If that is allowed, then they won't have id, createdAt, etc. Those child entities should be inserted into the client-side context to populate those properties. Then, when performing a push, the SDK seems to be smart enough to POST them first before POSTing the parent.
[HttpPost]
public async Task<IHttpActionResult> PostZoo(Zoo item) {
// replace child entities with those from the context (so they are tracked)
if ((item.Animals != null) && item.Animals.Any()) {
// find animals which are already in the db
var ids = item.Animals.Select(x => x.Id);
var oldAnimals = _context.Animals.Where(x => ids.Contains(x.Id)).ToList();
var newAnimals = item.Animals.Where(x => !oldAnimals.Select(y => y.Id).Contains(x.Id));
// don't allow new animal entities
if (tagsNew.Any()) {
var response = new HttpResponseMessage(HttpStatusCode.BadRequest);
var body = new JObject(new JProperty("error", "New Animals must be added first"));
response.Content = new StringContent(body.ToString(), Encoding.UTF8, "application/json");
throw new HttpResponseException(response);
}
// replace
item.Animals.Clear();
foreach (var i in oldAnimals) item.Animals.Add(i);
}
// now add the Zoo (which contains tracked Animal entities)
Zoo current = await InsertAsync(item);
return CreatedAtRoute("Tables", new { id = current.Id }, current);
}

Caching and lazy loading with entity framework

let's say I have an application, for example a web site, where my objectcontext leaves during the time of a request. Some datas I load with EF should be cached to avoid to read in DB and improve performance.
Ok, I read my datas with EF, I put my object in cache (says AppFabric, not in memory cache), but related datas that can be lazy loaded are now null (and access to this property results in a nullreferenceexception). I don't want to load everything in one request, because it's going to be too long, so I want to keep the loading on demand and as soon as it's read, I would like to complete the cache with the new fetched datas.
Note :
only read operations, no create/update/delete.
Don't want to use second level cache like "EF Provider Wrappers" made by Jarek Kowalski
How can I do that ?
EDIT : I've built this samples with northwind database, it's working :
class Program
{
static void Main(string[] args)
{
// normal use
List<Products> allProductCached = null;
using (NORTHWNDEntities1 db = new NORTHWNDEntities1())
{
allProductCached = db.Products.ToList().Clone<DbSet<Products>>();
foreach (var product in db.Products.Where(e => e.UnitPrice > 100))
{
Console.WriteLine(product.ProductName + " => " + product.Suppliers.CompanyName);
}
}
// try to use cache, but missing Suppliers
using (NORTHWNDEntities1 db = new NORTHWNDEntities1())
{
foreach (var product in allProductCached.Where(e => e.UnitPrice > 100))
{
if (product.Suppliers == null)
product.Suppliers = db.Suppliers.FirstOrDefault(s => s.SupplierID == product.SupplierID).Clone<Suppliers>();
Console.WriteLine(product.ProductName + " => " + product.Suppliers.CompanyName);
}
}
// try to use full cache
using (NORTHWNDEntities1 db = new NORTHWNDEntities1())
{
foreach (var product in allProductCached.Where(e => e.UnitPrice > 100))
{
Console.WriteLine(product.ProductName + " => " + product.Suppliers.CompanyName);
}
}
}
}
public static class Ext
{
public static List<Products> Clone<T>(this List<Products> list)
{
return list.Select(obj =>
new Products
{
ProductName = obj.ProductName,
SupplierID = obj.SupplierID,
UnitPrice = obj.UnitPrice
}).ToList();
}
public static Suppliers Clone<T>(this Suppliers obj)
{
if (obj == null)
return null;
return new Suppliers
{
SupplierID = obj.SupplierID,
CompanyName = obj.CompanyName
};
}
}
The problem is that I have to copy everything (without missing a property) and test everywhere if the property is null and load the needed property. My code is of course more and more complex, so that will be a problem if I miss something. No other solution ?

You cannot access the database in EF without an ObjectContext or a DbContext.
You can still use caching effectively, even if you don't have the original context any more.
Maybe your scenario is something like this... Imagine that you have some reference data that you use frequently. You do not want to hit the database each time you need it, so you store it in a cache. You also have per-user data that you don't want to cache. You have navigation properties from your user data to your reference data. You want to load your user data from the database, and have EF automatically "fix up" the navigation properties to point to the reference data.
For a request:
Create a new DbContext.
Retrieve reference data from the cache.
Make a deep copy of the reference objects. (You probably don't want to have the same entities attached to multiple contexts simultaneously.)
Attach each of the reference objects to the context. (e.g. with DbSet.Attach())
Execute whatever queries are required to load the per-user data. EF will automatically "fix up" the references to the reference data.
Identify newly loaded entities that could be cached. Ensure that they contain no references to entities that should not be cached, then save them to the cache.
Dispose of the context.
Cloned Objects and Lazy Loading
Lazy loading in EF is usually accomplished using dynamic proxies. The idea is that you make all properties that could potentially be loaded dynamically virtual. Whenever EF creates an instance of your entity type, it actually substitutes a derived type instead, and that derived type has the lazy loading logic in its overridden version of your properties.
This is all well and good, but in this scenario you are attaching entity objects to the context that were not created by EF. You created them, using a method called Clone. You instantiated the real POCO entity, not some mysterious EF dynamic proxy type. That means you won't get lazy loading on these entities.
The solution is simple. The Clone method must take an additional argument: the DbContext. Don't use the entity's constructor to create a new instance. Instead, use DbSet.Create(). This will return a dynamic proxy. Then initialize its properties to create a clone of the reference entity. Then attach it to the context.
Here is the code you might use to clone a single Products entity:
public static Products Clone(this Products product, DbContext context)
{
var set = context.Set<Products>();
var clone = set.Create();
clone.ProductName = product.ProductName;
clone.SupplierID = product.SupplierID;
clone.UnitProce = product.UnitPrice;
// Initialize collection so you don't have to do the null check, but
// if the property is virtual and proxy creation is enabled, it should get lazy loaded.
clone.Suppliers = new List<Suppliers>();
return clone;
}
Code Sample
namespace EFCacheLazyLoadDemo
{
using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations.Schema;
using System.Data.Entity;
using System.Linq;
class Program
{
static void Main(string[] args)
{
// Add some demo data.
using (MyContext c = new MyContext())
{
var sampleData = new Master
{
Details =
{
new Detail { SomeDetail = "Cod" },
new Detail { SomeDetail = "Haddock" },
new Detail { SomeDetail = "Perch" }
}
};
c.Masters.Add(sampleData);
c.SaveChanges();
}
Master cachedMaster;
using (MyContext c = new MyContext())
{
c.Configuration.LazyLoadingEnabled = false;
c.Configuration.ProxyCreationEnabled = false;
// We don't load the details here. And we don't even need a proxy either.
cachedMaster = c.Masters.First();
}
Console.WriteLine("Reference entity details count: {0}.", cachedMaster.Details.Count);
using (MyContext c = new MyContext())
{
var liveMaster = cachedMaster.DeepCopy(c);
c.Masters.Attach(liveMaster);
Console.WriteLine("Re-attached entity details count: {0}.", liveMaster.Details.Count);
}
Console.ReadKey();
}
}
public static class MasterExtensions
{
public static Master DeepCopy(this Master source, MyContext context)
{
var copy = context.Masters.Create();
copy.MasterId = source.MasterId;
foreach (var d in source.Details)
{
var copyDetail = context.Details.Create();
copyDetail.DetailId = d.DetailId;
copyDetail.MasterId = d.MasterId;
copyDetail.Master = copy;
copyDetail.SomeDetail = d.SomeDetail;
}
return copy;
}
}
public class MyContext : DbContext
{
static MyContext()
{
// Just for demo purposes, re-create db each time this runs.
Database.SetInitializer(new DropCreateDatabaseAlways<MyContext>());
}
public DbSet<Master> Masters { get { return this.Set<Master>(); } }
public DbSet<Detail> Details { get { return this.Set<Detail>(); } }
}
public class Master
{
public Master()
{
this.Details = new List<Detail>();
}
public int MasterId { get; set; }
public virtual List<Detail> Details { get; private set; }
}
public class Detail
{
public int DetailId { get; set; }
public string SomeDetail { get; set; }
public int MasterId { get; set; }
[ForeignKey("MasterId")]
public Master Master { get; set; }
}
}
Here is a sample model, different from yours, that shows how to get this working in principle.

Entity framework creates new record when SaveChanges is called

I have two entities:
public class Order:Entity
{
public virtual User User { get; set; }
...
}
public class User:Entity
{
public virtual ICollection<Order> Orders { get; set; }
...
}
Next, I create order:
var order = _orderService.CreateTransientOrder(orderNumbers, PcpSession.CurrentUser);
PcpSession.Order = order;
this is CreateTransientOrder. It's only create Order, but not save into database:
public Order CreateTransientOrder(string orderNumbers, User currentUser)
{
...fill fields
order.User = currentUser;
return order;
}
Now it's all ok. Next, I save order to the database:
_orderService.CreateOrder(PcpSession.Order);
This is CreateOrder:
public void CreateOrder(Order order)
{
order.OrderDate = DateTime.Now;
_repository.Save(order);
_repository.SaveChanges();
}
This is my Save method of repository:
public void Save<T>(T entity) where T : class, IEntity
{
_context.Set<T>().Add(entity);
}
When the SaveChanges is called in the database creates new user with new ID and order have new User_Id. In the debugger in the CreateOrder method, Id is equal current user. Where is a problem?
Thanks.

User is probably not being tracked by the context. When you add order to the context it also adds the related entities and then on save changes creates a new user (or attempts to). Attach() the user to the context before you call _context.Set<T>().Add(entity);.

I guess the problem is not related with the code you have provided. It seems to be related to where you are initializing PcpSession.CurrentUser.
It seems PcpSession.CurrentUser object is not attached to the context. Either fetch this entity to the context before making you Order related calls or attach it.

You need attach your Entity if not attach in context.
for exemple in Repository Generic
> public void Add(T entity)
> {
> entity.Create = DateTime.Now;
> db.Set<T>().Attach(entity); // Add Line
> db.Set<T>().Add(entity);
> Save();
> }
I do not know if it's clean but it regulates the problem