We have an application with a fairly complex entity model where high-performance and low-latency are essential but we have no need for horizontal scalability. The application has a number of event sources in addition to a self-hosted ASP.NET Web API 2. We use Entity Framework 6 to map from POCO classes to the database (we use the excellent Reverse POCO Generator to generate our classes).
Whenever an event arrives the application has to make some adjustment to the entity model and persist this delta adjustment to the database via EF. At the same time read or update requests may arrive via the Web API.
Because the model involves many tables and FK relationships, and reacting to an event usually requires all relationships under the subject entity to be loaded, we have elected to maintain the entire set of data in an in-memory cache rather than load the entire object graph for each event. The image below shows a simplified version of our model:-
At program start-up we load all the interesting ClassA instances (and their associated dependency graph) via a temporary DbContext and insert into a Dictionary (ie our cache). When an event arrives we find the ClassA instance in our cache and attach it to a per-event DbContext via DbSet.Attach(). The program is written using the await-async pattern throughout and multiple events can be processed at the same time. We protect the cached objects from being accessed concurrently by the use of locks so we guarantee that a cached ClassA can be loaded into a DbContext only one-at-a-time. So far so good, the performance is excellent and we are happy with the mechanism. But there is a problem. Although the entity graph is fairly self-contained under ClassA, there are some POCO classes representing what we consider to be read-only static-data (shaded in orange in the image). We have found that EF sometimes complains
An entity object cannot be referenced by multiple instances of IEntityChangeTracker.
when we attempt to Attach() two different instances of ClassA at the same time (even though we are attaching to different Dbcontexts) because they share a reference to the same ClassAType. This is demonstrated by the code snippet below:-
ConcurrentDictionary<int,ClassA> theCache = null;
using(var ctx = new MyDbContext())
{
var classAs = ctx.ClassAs
.Include(a => a.ClassAType)
.ToList();
theCache = new ConcurrentDictionary<int,ClassA>(classAs.ToDictionary(a => a.ID));
}
// take 2 different instances of ClassA that refer to the same ClassAType
// and load them into separate DbContexts
var ctx1 = new MyDbContext();
ctx1.ClassAs.Attach(theCache[1]);
var ctx2 = new MyDbContext();
ctx2.ClassAs.Attach(theCache[2]); // exception thrown here
Is there any way to inform EF that ClassAType is read-only/static and we don't want it to ensure that each instance can be loaded into only one DbContext? So far the only way around the problem I've found is to modify the POCO generator to ignore these FK relationships, so they are not part of the entity model. But this complicates the programming because there are processing methods in ClassA which need access to the static data.
I think the key to this question is what exactly the exception means:-
An entity object cannot be referenced by multiple instances of IEntityChangeTracker.
It occurred to me that perhaps this exception is Entity Framework complaining that an instance of an object has been changed in multiple DbContexts rather than simply being referenced by objects in multiple DbContexts. My theory was based on the fact that the generated POCO classes have reverse FK navigation properties, and that Entity Framework would naturally attempt to fix-up these reverse navigation properties as part of the process of attaching the entity graph to the DbContext (see a description of the fix-up process)
To test out this theory I created a simple test project where I could enable and disable the reverse navigation properties. To my great joy I discovered that the theory was correct, and that EF is quite happy for the objects to be referenced multiple times so long as the objects themselves don't change - and this includes navigation properties being changed by the fix-up process.
So the answer to the question is simply follow 2 rules:-
Ensure the static data objects are never changed (ideally they should have no public setter properties) and
Do not include any FK reverse navigation properties pointing back to the referring classes. For users of the Reverse POCO Generator I have made a suggestion to Simon Hughes (the author) to add an enhancement making this a configuration option.
I've included the test classes below:-
class Program
{
static void Main(string[] args)
{
ConcurrentDictionary<int,ClassA> theCache = null;
try
{
using(var ctx = new MyDbContext())
{
var classAs = ctx.ClassAs
.Include(a => a.ClassAType)
.ToList();
theCache = new ConcurrentDictionary<int,ClassA>(classAs.ToDictionary(a => a.ID));
}
// take 2 instances of ClassA that refer to the same ClassAType
// and load them into separate DbContexts
var classA1 = theCache[1];
var classA2 = theCache[2];
var ctx1 = new MyDbContext();
ctx1.ClassAs.Attach(classA1);
var ctx2 = new MyDbContext();
ctx2.ClassAs.Attach(classA2);
// When ClassAType has a reverse FK navigation property to
// ClassA we will not reach this line!
WriteDetails(classA1);
WriteDetails(classA2);
classA1.Name = "Updated";
classA2.Name = "Updated";
WriteDetails(classA1);
WriteDetails(classA2);
}
catch(Exception ex)
{
Console.WriteLine(ex.Message);
}
System.Console.WriteLine("End of test");
}
static void WriteDetails(ClassA classA)
{
Console.WriteLine(String.Format("ID={0} Name={1} TypeName={2}",
classA.ID, classA.Name, classA.ClassAType.Name));
}
}
public class ClassA
{
public int ID { get; set; }
public string ClassATypeCode { get; set; }
public string Name { get; set; }
//Navigation properties
public virtual ClassAType ClassAType { get; set; }
}
public class ClassAConfiguration : System.Data.Entity.ModelConfiguration.EntityTypeConfiguration<ClassA>
{
public ClassAConfiguration()
: this("dbo")
{
}
public ClassAConfiguration(string schema)
{
ToTable("TEST_ClassA", schema);
HasKey(x => x.ID);
Property(x => x.ID).HasColumnName(#"ID").IsRequired().HasColumnType("int").HasDatabaseGeneratedOption(System.ComponentModel.DataAnnotations.Schema.DatabaseGeneratedOption.Identity);
Property(x => x.Name).HasColumnName(#"Name").IsRequired().HasColumnType("varchar").HasMaxLength(50);
Property(x => x.ClassATypeCode).HasColumnName(#"ClassATypeCode").IsRequired().HasColumnType("varchar").HasMaxLength(50);
//HasRequired(a => a.ClassAType).WithMany(b => b.ClassAs).HasForeignKey(c => c.ClassATypeCode);
HasRequired(a => a.ClassAType).WithMany().HasForeignKey(b=>b.ClassATypeCode);
}
}
public class ClassAType
{
public string Code { get; private set; }
public string Name { get; private set; }
public int Flags { get; private set; }
// Reverse navigation
//public virtual System.Collections.Generic.ICollection<ClassA> ClassAs { get; set; }
}
public class ClassATypeConfiguration : System.Data.Entity.ModelConfiguration.EntityTypeConfiguration<ClassAType>
{
public ClassATypeConfiguration()
: this("dbo")
{
}
public ClassATypeConfiguration(string schema)
{
ToTable("TEST_ClassAType", schema);
HasKey(x => x.Code);
Property(x => x.Code).HasColumnName(#"Code").IsRequired().HasColumnType("varchar").HasMaxLength(12);
Property(x => x.Name).HasColumnName(#"Name").IsRequired().HasColumnType("varchar").HasMaxLength(50);
Property(x => x.Flags).HasColumnName(#"Flags").IsRequired().HasColumnType("int");
}
}
public class MyDbContext : System.Data.Entity.DbContext
{
public System.Data.Entity.DbSet<ClassA> ClassAs { get; set; }
public System.Data.Entity.DbSet<ClassAType> ClassATypes { get; set; }
static MyDbContext()
{
System.Data.Entity.Database.SetInitializer<MyDbContext>(null);
}
const string connectionString = #"Server=TESTDB; Database=TEST; Integrated Security=True;";
public MyDbContext()
: base(connectionString)
{
}
protected override void OnModelCreating(System.Data.Entity.DbModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Configurations.Add(new ClassAConfiguration());
modelBuilder.Configurations.Add(new ClassATypeConfiguration());
}
}
I think this might work: try using AsNoTracking in those entities DbSets when selecting them at program start-up:
dbContext.ClassEType.AsNoTracking();
This will disable the change tracking for them, so EF will not try to persist them.
Also, the POCO class for those entities should only have read-only properties (with no set method).
Related
I have a list of DTOs and want to map this list to a list of entites. The entity itself has one property which comes from another source. Can I map this property to all items of the list with one map.
My classes:
Entity:
public class Account
{
public int Id {get;set;}
public string Name {get;set;}
public Guid ExternalId {get;set;}
}
DTO:
public class ExternalAccountDto
{
public int Id {get;set;}
public string Name {get;set;}
}
My Service:
public class AccountService
{
public async Task AddExternalAccounts(Guid externalId, List<ExternalAccountDto> accounts)
{
var entities = _mapper.Map(accounts);
// TODO: Map 'externalId' to all entities
// _mapper.Map(externalId, entities); // DOES NOT WORK!
_context.Create(entities);
}
}
Mapping
public class AccountProfile: Profile
{
public AccountProfile()
{
CreateMap<ExternalAccountDto, Account>();
// TODO: CreateMap for Guid on every Account
}
}
Can anyone give me some advice!
You should use the AfterMap function to do some postprocessing on the mapped items.
There are two ways to go about this. One is using something statically defined in the mapping profile. But in your case, you have something that's dynamic at runtime, like the ExternalId. Doing the aftermap in your AccountService then makes perfect sense.
I've found these kind of constructions very useful, especially when I want to consult other injected services for additional information.
public void AddExternalAccounts(Guid externalId, List<ExternalAccountDto> accounts)
{
var entities = _mapper.Map<List<ExternalAccountDto>, List<Account>>(accounts,
options => options.AfterMap((source, destination) =>
{
destination.ForEach(account => account.ExternalId = externalId);
}));
}
Two more cents regarding the AccountProfile class:
You can check upon creation of the mapping profile if the mapping profile is correct. This will save you a headache running into this problem later at runtime. You'll know immediately that there is a problem with the configuration.
var config = new MapperConfiguration(cfg =>
{
cfg.AddProfile<MappingProfile>();
cfg.AllowNullDestinationValues = false;
});
// Check that there are no issues with this configuration, which we'll encounter eventually at runtime.
config.AssertConfigurationIsValid();
_mapper = config.CreateMapper();
This also notified me that an .Ignore() on the ExternalId member of the Account class was required:
CreateMap<ExternalAccountDto, Account>().ForMember(d => d.ExternalId, a => a.Ignore());
Imagine that I want to add an IsDeleted colum or some auditing columns to all of my entities. I could create a base class from which all of my entities will inherit and this will solve my problem, however I cannot specify the order in which the column will be created so I will end up with all the auditing fields before the fields of my entity, which I do not want. I want them to be at the end of the table.
In the standard version of entity framework we can do this by using annotations that specify the order of the columns. However, such a thing does not exist for EF core at the moment.
I could do it with the fluent api on the OnModelCreating() method, the problem is that I only know how to do it individually for each of my entities, which means I would have to write the same code for every entity I have.
Is there any way I can do it generically for all of my entities? Some sort of for loop that iterates through all the entities registered in the DbSets on my dbcontext?
Your question title is about adding the same properties to multiple entities. However, you actually know how to achieve this (use a base type) and your actual question is how to ensure that these properties come last in the generated tables' columns.
Although column order shouldn't really matter nowadays, I'll show an alternative that you may like better than a base type and also positions the common properties at the end of the table. It makes use of shadow properties:
Shadow properties are properties that are not defined in your .NET entity class but are defined for that entity type in the EF Core model.
Most of the times, auditing properties don't need much visibility in the application, so I think shadow properties is exactly what you need. Here's an example:
I have two classes:
public class Planet
{
public Planet()
{
Moons = new HashSet<Moon>();
}
public int ID { get; set; }
public string Name { get; set; }
public virtual ICollection<Moon> Moons { get; set; }
}
public class Moon
{
public int ID { get; set; }
public int PlanetID { get; set; }
public string Name { get; set; }
public Planet Planet { get; set; }
}
As you see: they don't have auditing properties, they're nicely mean and lean POCOs. (By the way, for convenience I lump IsDeleted together with "audit properties", although it isn't one and it may require another approach).
And maybe that's the main message here: the class model isn't bothered with auditing concerns (single responsibility), it's all EF's business.
The audit properties are added as shadow properties. Since we want to do that for each entity we define a base IEntityTypeConfiguration:
public abstract class BaseEntityTypeConfiguration<T> : IEntityTypeConfiguration<T>
where T : class
{
public virtual void Configure(EntityTypeBuilder<T> builder)
{
builder.Property<bool>("IsDeleted")
.IsRequired()
.HasDefaultValue(false);
builder.Property<DateTime>("InsertDateTime")
.IsRequired()
.HasDefaultValueSql("SYSDATETIME()")
.ValueGeneratedOnAdd();
builder.Property<DateTime>("UpdateDateTime")
.IsRequired()
.HasDefaultValueSql("SYSDATETIME()")
.ValueGeneratedOnAdd();
}
}
The concrete configurations are derived from this base class:
public class PlanetConfig : BaseEntityTypeConfiguration<Planet>
{
public override void Configure(EntityTypeBuilder<Planet> builder)
{
builder.Property(p => p.ID).ValueGeneratedOnAdd();
// Follows the default convention but added to make a difference :)
builder.HasMany(p => p.Moons)
.WithOne(m => m.Planet)
.IsRequired()
.HasForeignKey(m => m.PlanetID);
base.Configure(builder);
}
}
public class MoonConfig : BaseEntityTypeConfiguration<Moon>
{
public override void Configure(EntityTypeBuilder<Moon> builder)
{
builder.Property(p => p.ID).ValueGeneratedOnAdd();
base.Configure(builder);
}
}
These should be added to the context's model in OnModelCreating:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.ApplyConfiguration(new PlanetConfig());
modelBuilder.ApplyConfiguration(new MoonConfig());
}
This will generate database tables having columns InsertDateTime, IsDeleted and UpdateDateTime at the end (independent of when base.Configure(builder) is called, BTW), albeit in that order (alphabetical). I guess that's close enough.
To make the picture complete, here's how to set the values fully automatically in a SaveChanges override:
public override int SaveChanges()
{
foreach(var entry in this.ChangeTracker.Entries()
.Where(e => e.Properties.Any(p => p.Metadata.Name == "UpdateDateTime")
&& e.State != Microsoft.EntityFrameworkCore.EntityState.Added))
{
entry.Property("UpdateDateTime").CurrentValue = DateTime.Now;
}
return base.SaveChanges();
}
Small detail: I make sure that when an entity is inserted the database defaults set both fields (see above: ValueGeneratedOnAdd(), and hence the exclusion of added entities) so there won't be confusing differences caused by client clocks being slightly off. I assume that updating will always be well later.
And to set IsDeleted you could add this method to the context:
public void MarkForDelete<T>(T entity)
where T : class
{
var entry = this.Entry(entity);
// TODO: check entry.State
if(entry.Properties.Any(p => p.Metadata.Name == "IsDeleted"))
{
entry.Property("IsDeleted").CurrentValue = true;
}
else
{
entry.State = Microsoft.EntityFrameworkCore.EntityState.Deleted;
}
}
...or turn to one of the proposed mechanisms out there to convert EntityState.Deleted to IsDeleted = true.
You can always generate an initial migration for the model and manually rearrange the column order in the Migration.
Here is the open issue tracking support for explicit column ordering in EF Core: https://github.com/aspnet/EntityFrameworkCore/issues/10059
Also see this question and answer on using Shadow Properties and Query Filters for soft deletes. EF Core: Soft delete with shadow properties and query filters
So I created a new simple project just to help a friend.
So I made a class Customer which has a list of Stuff
So far so good, now with the mapping and storing the relationsship. I went to map in accordance with fluent nhibernate class maps and ended up with the following
public class CustomerMap : ClassMap<Customer> {
Id(p => p.Id).GenerateBy.Guid();
HasMany(p => p.Stuff).Access.CamelCaseField().KeyColumn("Id").Inverse().Cascade.AllDeleteOrphan();
}
public class StuffMap : ClassMap<Stuff> {
Id(p => p.Id).GeneratedBy.Guid();
Reference(p => p.Customer).Column("CustomerId).Not.Nullable();
}
and my classes
public class Customer {
private ISet<Stuff> stuff = new HashSet<Stuff>()
public virtual IEnumerable<Stuff> Stuff => stuff;
public void AddStuff(Stuff newstuff) {
newstuff.Customer = this;
stuff.Add(stuff);
}
}
public class Stuff {
public virtual Customer Customer { get; set; }
}
All this works good and when I create a new Customer and add one of more Stuff elements into the collection using the method AddStuff and commits the transaction it gets correctly written to the database.
However now the strange begins, when I make a test like the following
[TestMethod]
public void TestStuffAndCustomer() {
var customer = session.Add(new Customer());
customer.AddStuff(new Stuff());
session.Flush();
var customer = session.Query<Customer>().Single();
customer.Stuff.Should().HaveCount(1);
}
The assertion of the collection fails with reason that the count of the collection is 0. However if I debug the test and check the collection it contains one element. The assertion fails regardless however.
So what is wrong with this setup?
I think you add new Customer and Stuff to Customer on session, but without saving them you flush the session.
I have two POCO classes, each containing a collection of the other. I am attempting to configure a many-to-many relationship between these classes. However, when I load the context, EF does not populate any of the collections of each object. How can this be done automatically?
public class POCO1
{
public int ID { get; set; }
public virtual ICollection<POCO2> POCO2s { get; set; }
}
public class PCOO2
{
public int ID { get; set; }
public virtual ICollection<POCO1> POCO1s { get; set; }
}
public class POCOContext : DbContext
{
public DbSet<POCO1> POCO1s { get; set; }
public DbSet<POCO2> POCO2s { get; set; }
modelBuilder.Entity<POCO1>()
.HasKey(p => p.ID);
modelBuilder.Entity<POCO1>()
.HasMany(p => p.POCO2s).WithMany(p => p.POCO1s);
modelBuilder.Entity<POCO2>()
.HasKey(p => p.ID);
modelBuilder.Entity<POCO2>()
.HasMany(p => p.POCO1s).WithMany(p => p.POCO2s);
}
public class Test
{
static void Main(string[] args)
{
using (var context = new POCOContext())
{
context.Database.Initialize(true);
var p1 = context.POCO1s.Create();
var p2 = context.POCO2s.Create();
p1.POCO2s.Add(p2);
p2.POCO1s.Add(p1);
context.saveChanges();
}
// reload context from db
using (var context = new POCOContext())
{
context.POCO1s.ToList()[0].POCO2s.Count(); // == 0, but should be 1
context.POCO2s.ToList()[0].POCO1s.Count(); // == 0, but should be 1
}
}
}
Edit to include more info: from what I have read it should not be necessary to include annotations or even specify keys using Fluent API, and using these conventions EF 5.0 should configure the relations correctly. Indeed this appears to be the case, because the database tables are constructed correctly using code first (there is a POCO1 table, a POCO2 table, and an intersection table, all of which are populated correctly during testing). The problem is when reading the data back out, it does not seem to populate the many to many relationships. Other posts on SO and various tutorials and MSDN documentation suggest that using the proper conventions, lazy loading, and virtual collections should work, but that is not the behavior I'm seeing.
In my case the problem was due to disabled proxy creation on the context (ie context.Configuration.ProxyCreationEnabled = false;) in my actual code. The "automatic collection population" feature described in the original question is known as "lazy loading" and it requires proxy creation to be enabled. This MSDN article has more information.
Also, as mentioned in the original question, this feature works by convention without specifying keys or relationships (using annotations or fluent API).
I am using Entity Framework to model an existing database. One of the database tables contains a column with the same name as the table, AnonymousUID.
I use the Entity Framework Power Tools function Reverse Engineer Code First to generate the model classes and mappings. The reverse engineering procedure automatically renames the AnonymousUID class member (to AnonymousUID1) to avoid that a member name is the same as the class name. The generated model class thus looks like this:
public partial class AnonymousUID
{
public string UID { get; set; }
public string AnonymousUID1 { get; set; }
}
and the EF mapping constructor is implemented like this:
public AnonymousUIDMap()
{
// Primary Key
this.HasKey(t => t.UID);
// Properties
this.Property(t => t.UID).IsRequired().HasMaxLength(64);
this.Property(t => t.AnonymousUID1).IsRequired().HasMaxLength(64);
// Table & Column Mappings
this.ToTable("AnonymousUID");
this.Property(t => t.UID).HasColumnName("UID");
this.Property(t => t.AnonymousUID1).HasColumnName("AnonymousUID");
}
The database context class is implemented like this:
public partial class MyDbContext : DbContext
{
// Constructors...
public DbSet<AnonymousUID> AnonymousUIDs { get; set; }
...
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Configurations.Add(new AnonymousUIDMap());
...
}
}
This is all good and well, and the code builds without problems. But when I try to access arbitrary contents of the database:
using (var context = new MyDbContext())
{
var foos = from foo in context.Foos select foo;
...
}
the following exception is nonetheless thrown:
System.Data.Entity.ModelConfiguration.ModelValidationException :
One or more validation errors were detected during model generation:
\tAnonymousUID: Name: Name 'AnonymousUID' cannot be used in type
CodeFirstDatabaseSchema.AnonymousUID'. Member names cannot be the
same as their enclosing type.
There is obviously some additional mapping build-up going on in CodeFirstDatabaseSchema, and this procedure is not able to avoid the class/member name clash.
Why does this error occur? After all, the reverse engineering procedure managed to circumvent the naming issue.
Without modifying the schema of the already established database, is there some way I can avoid this exception from being thrown?
I am using Entity Framework 6.0 (pre-release) from Nuget in a .NET Framework 4 project.
As I guessed at in the comment above, change the column name to start with a lowercase 'a'
...HasColumnName("anonymousUID");
Let's hope this is a pre release defect and is fixed in the RTM ;-)
Is the code you displayed the only place you're using the AnonymousUID name? What is the name of your DbSet in the generated DbContext?
I built a test app, and didn't get any errors, here it in it's entirety:
public partial class AnonymousUID
{
[Key]
public int UID { get; set; }
public string AnonymousUID1 { get; set; }
}
public class Model : DbContext
{
public DbSet<AnonymousUID> AnonymousUIDs { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<AnonymousUID>()
.Property(a => a.AnonymousUID1)
.HasColumnName("AnonymousUID");
modelBuilder.Entity<AnonymousUID>()
.ToTable("AnonymousUID");
}
}
class Program
{
static void Main(string[] args)
{
var model = new Model();
var a = new AnonymousUID();
a.AnonymousUID1 = "hello world";
model.AnonymousUIDs.Add(a);
model.SaveChanges();
var applications = model.AnonymousUIDs.ToList();
Console.WriteLine(applications.Count);
Console.ReadLine();
}
}
Created this database:
It was able to create, insert, and then display the count of the table.
I had a similar issue, but I could not rename the column because it would break legacy code. The type names that Power Tools generated are all lowercase. I changed the capitalization on the conflicting types, and it worked.
I suspect that CodeFirstDatabaseSchema is maintaining lowercase property names, but I am not sure. All I know is that changing my lowercase type names to properly capitalized type names fixed it. I hope that is helpful for someone.