We Keep Coding

How to add the same column to all entities in EF Core?

Imagine that I want to add an IsDeleted colum or some auditing columns to all of my entities. I could create a base class from which all of my entities will inherit and this will solve my problem, however I cannot specify the order in which the column will be created so I will end up with all the auditing fields before the fields of my entity, which I do not want. I want them to be at the end of the table.
In the standard version of entity framework we can do this by using annotations that specify the order of the columns. However, such a thing does not exist for EF core at the moment.
I could do it with the fluent api on the OnModelCreating() method, the problem is that I only know how to do it individually for each of my entities, which means I would have to write the same code for every entity I have.
Is there any way I can do it generically for all of my entities? Some sort of for loop that iterates through all the entities registered in the DbSets on my dbcontext?

Your question title is about adding the same properties to multiple entities. However, you actually know how to achieve this (use a base type) and your actual question is how to ensure that these properties come last in the generated tables' columns.
Although column order shouldn't really matter nowadays, I'll show an alternative that you may like better than a base type and also positions the common properties at the end of the table. It makes use of shadow properties:
Shadow properties are properties that are not defined in your .NET entity class but are defined for that entity type in the EF Core model.
Most of the times, auditing properties don't need much visibility in the application, so I think shadow properties is exactly what you need. Here's an example:
I have two classes:
public class Planet
{
public Planet()
{
Moons = new HashSet<Moon>();
}
public int ID { get; set; }
public string Name { get; set; }
public virtual ICollection<Moon> Moons { get; set; }
}
public class Moon
{
public int ID { get; set; }
public int PlanetID { get; set; }
public string Name { get; set; }
public Planet Planet { get; set; }
}
As you see: they don't have auditing properties, they're nicely mean and lean POCOs. (By the way, for convenience I lump IsDeleted together with "audit properties", although it isn't one and it may require another approach).
And maybe that's the main message here: the class model isn't bothered with auditing concerns (single responsibility), it's all EF's business.
The audit properties are added as shadow properties. Since we want to do that for each entity we define a base IEntityTypeConfiguration:
public abstract class BaseEntityTypeConfiguration<T> : IEntityTypeConfiguration<T>
where T : class
{
public virtual void Configure(EntityTypeBuilder<T> builder)
{
builder.Property<bool>("IsDeleted")
.IsRequired()
.HasDefaultValue(false);
builder.Property<DateTime>("InsertDateTime")
.IsRequired()
.HasDefaultValueSql("SYSDATETIME()")
.ValueGeneratedOnAdd();
builder.Property<DateTime>("UpdateDateTime")
.IsRequired()
.HasDefaultValueSql("SYSDATETIME()")
.ValueGeneratedOnAdd();
}
}
The concrete configurations are derived from this base class:
public class PlanetConfig : BaseEntityTypeConfiguration<Planet>
{
public override void Configure(EntityTypeBuilder<Planet> builder)
{
builder.Property(p => p.ID).ValueGeneratedOnAdd();
// Follows the default convention but added to make a difference :)
builder.HasMany(p => p.Moons)
.WithOne(m => m.Planet)
.IsRequired()
.HasForeignKey(m => m.PlanetID);
base.Configure(builder);
}
}
public class MoonConfig : BaseEntityTypeConfiguration<Moon>
{
public override void Configure(EntityTypeBuilder<Moon> builder)
{
builder.Property(p => p.ID).ValueGeneratedOnAdd();
base.Configure(builder);
}
}
These should be added to the context's model in OnModelCreating:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.ApplyConfiguration(new PlanetConfig());
modelBuilder.ApplyConfiguration(new MoonConfig());
}
This will generate database tables having columns InsertDateTime, IsDeleted and UpdateDateTime at the end (independent of when base.Configure(builder) is called, BTW), albeit in that order (alphabetical). I guess that's close enough.
To make the picture complete, here's how to set the values fully automatically in a SaveChanges override:
public override int SaveChanges()
{
foreach(var entry in this.ChangeTracker.Entries()
.Where(e => e.Properties.Any(p => p.Metadata.Name == "UpdateDateTime")
&& e.State != Microsoft.EntityFrameworkCore.EntityState.Added))
{
entry.Property("UpdateDateTime").CurrentValue = DateTime.Now;
}
return base.SaveChanges();
}
Small detail: I make sure that when an entity is inserted the database defaults set both fields (see above: ValueGeneratedOnAdd(), and hence the exclusion of added entities) so there won't be confusing differences caused by client clocks being slightly off. I assume that updating will always be well later.
And to set IsDeleted you could add this method to the context:
public void MarkForDelete<T>(T entity)
where T : class
{
var entry = this.Entry(entity);
// TODO: check entry.State
if(entry.Properties.Any(p => p.Metadata.Name == "IsDeleted"))
{
entry.Property("IsDeleted").CurrentValue = true;
}
else
{
entry.State = Microsoft.EntityFrameworkCore.EntityState.Deleted;
}
}
...or turn to one of the proposed mechanisms out there to convert EntityState.Deleted to IsDeleted = true.

You can always generate an initial migration for the model and manually rearrange the column order in the Migration.
Here is the open issue tracking support for explicit column ordering in EF Core: https://github.com/aspnet/EntityFrameworkCore/issues/10059
Also see this question and answer on using Shadow Properties and Query Filters for soft deletes. EF Core: Soft delete with shadow properties and query filters

Intermittent Issue With Entity Framework Creating Unwanted Rows

I have an MVC application that uses Entity Framework v6. We have a class
public class ChildObject
{
public string Name { get; set; }
....
}
that maps to a table in the database. This table has 6 rows that are never changed. Neither will there ever be any additions. We have a second class defined along the lines of the following:
public class ParentClass
{
public int ChildObjectId { get; set; }
public ChildObject ChildObject { get; set; }
....
}
Whenever a ParentClass object is created or updated the logic only references the ChildObjectId property. The ChildObject property is only referenced when data is pulled back for viewing. However about once per month an extra row appears in the ChildObject table that is a duplicate of an existing row. This obviously causes issues. However I can't see how this could happen seeing as we only ever save using the Id value. Any thoughts on how this could be occurring would be very much appreciated.

The typical culprit for behavior like you describe is when a new child entity is composed based on existing data and attached to the parent rather than the reference associated to the context. An example might be that you load child objects as a set to select from, and send the data to your view. The user wants to change an existing child reference to one of the 6 selections. The call back to the server passes a child object model where there is code something like:
parent.ChildObject = new ChildObject{ Name = model.Name, ... }
rather than:
var child = context.Children.Single(x => x.Id = model.ChildObjectId);
parent.ChildObject = child;
Depending on how your domain is set up you may run into scenarios where the EF context creates a new child entity when a navigation property is set. Check with a FindUsages on the ChildObject property and look for any use of the setter.
In general you should avoid combining the use of FK properties (ChildObjectId) with navigation properties (ChildObject) because you can get confusing behavior between what is set in the navigation reference vs. the FK. Entities should be defined with one or the other. (Though at this time EF Core requires both if Navigation properties are used.)
A couple notables from your example:
Mark the navigation property as virtual - This ensures that EF assigns a proxy and recognizes it.
Option A - Remove the FK child ID property. For the parent either use an EntityTypeConfiguration or initialize the DbContext to map the FK column:
EntityTypeConfiguration:
public class ParentClassConfiguration : EntityTypeConfiguration<ParentClass>
{
public ParentClassConfiguration()
{
ToTable("ParentTable");
HasKey(x => x.ParentObjectId)
.Property(x => x.ParentObjectId)
.HasDatabaseGeneratedOption(DatabaseGeneratedOption.Identity);
HasRequired(x => x.ChildObject)
.WithMany()
.Map(x => x.MapKey("ChildObjectId"));
.WillCascadeOnDelete(false);
}
}
or on context model generation: (Inside your DbContext)
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<ParentObject>().HasRequired(x => x.ChildObject).WithMany().Map(x => x.MapKey("ChildObjectId")).WillCascadeOnDelete(false);
}
or Option B - Ensure the FK is linked to the reference, and take measures to ensure that the two are always kept in sync:
public class ParentClassConfiguration : EntityTypeConfiguration<ParentClass>
{
public ParentClassConfiguration()
{
ToTable("ParentTable");
HasKey(x => x.ParentObjectId)
.Property(x => x.ParentObjectId)
.HasDatabaseGeneratedOption(DatabaseGeneratedOption.Identity);
HasRequired(x => x.ChildObject)
.WithMany()
.HasForeignKey(x => x.ChildObjectId));
.WillCascadeOnDelete(false);
}
}
or on context model generation: (Inside your DbContext)
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<ParentObject>().HasRequired(x => x.ChildObject).WithMany().HasForeignKey(x => x.ChildObjectId)).WillCascadeOnDelete(false);
}
Option B is the only one currently available with EF Core to my knowledge, and it may help mitigate your issue but you still have to take care to avoid discrepancies between the navigation property and the FK. I definitely recommend option A, though it will likely require a bit of change if your code is commonly accessing the FK column.

How can I implement "Soft Deletes" with "Entity Framework Core" (aka EF7)?

I'm trying to implement a "Soft Delete" using EF7. My Item table has a field named IsDeleted of type bit. All of the examples that I see around SO and elsewhere are using something like this:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Item>().Map(m => m.Requires("IsDeleted").HasValue(false));
}
but Map() is no longer a method of ModelBuilder.
EDIT: Let me clarify. I'm mostly only interested in reading the data right now. I want EF to automatically filter out all records in my Item table where IsDeleted == 1 (or true). I do not want to require an && x.IsDeleted == false at the end of every query.

It's 2021, and it occurred to me to add a more modern, standard, built-in solution that pertains to current versions of EF Core.
With global query filters you can ensure that certain filters are always applied to certain entities. And you can define your soft deletion properties via an interface, which facilitates programmatically adding the filter to all relevant entities. See:
...
public interface ISoftDeletable
{
public string DeletedBy { get; }
public DateTime? DeletedAt { get; }
}
...
// Call it from DbContext.OnModelCreating()
private static void ConfigureSoftDeleteFilter(ModelBuilder builder)
{
foreach (var softDeletableTypeBuilder in builder.Model.GetEntityTypes()
.Where(x => typeof(ISoftDeletable).IsAssignableFrom(x.ClrType)))
{
var parameter = Expression.Parameter(softDeletableTypeBuilder.ClrType, "p");
softDeletableTypeBuilder.SetQueryFilter(
Expression.Lambda(
Expression.Equal(
Expression.Property(parameter, nameof(ISoftDeletable.DeletedAt)),
Expression.Constant(null)),
parameter)
);
}
}
Then, to make sure this flag is used during deletion instead of hard deletion (alternative to e.g. repositories setting the flag instead of deleting the entity):
public override Task<int> SaveChangesAsync(bool acceptAllChangesOnSuccess, CancellationToken cancellationToken = default)
{
foreach (var entry in ChangeTracker.Entries<ISoftDeletable>())
{
switch (entry.State)
{
case EntityState.Deleted:
// Override removal. Unchanged is better than Modified, because the latter flags ALL properties for update.
// With Unchanged, the change tracker will pick up on the freshly changed properties and save them.
entry.State = EntityState.Unchanged;
entry.Property(nameof(ISoftDeletable.DeletedBy)).CurrentValue = _currentUser.UserId;
entry.Property(nameof(ISoftDeletable.DeletedAt)).CurrentValue = _dateTime.Now;
break;
}
}
return base.SaveChangesAsync(acceptAllChangesOnSuccess, cancellationToken);
}
Caveat 1: Cascade Delete Timing
One crucial aspect is to take into account the cascade deletion of related entities, and either disable cascade delete, or understand and control the cascade delete timing behavior of EF Core. The default value of the CascadeDeleteTiming setting is CascadeTiming.Immediate, which causes EF Core to immediately flag all navigation properties of the 'deleted' entity as EntityState.Deleted, and reverting the EntityState.Deleted state only on the root entity won't revert it on the navigation properties. So if you have navigation properties which don't use soft deletion, and you want to avoid them being deleted, you must handle their change tracker state too (instead of just handling it for e.g. ISoftDeletable entities), or change the CascadeDeleteTiming setting as shown below.
The same is true for owned types used on the soft-deleted entities. With the default deletion cascade timing EF Core also flags these owned types as 'deleted', and in case they are set as Required/non-nullable, you will encounter SQL update failures when trying to save the soft-deleted entities.
public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options) : base(options)
{
ChangeTracker.CascadeDeleteTiming = CascadeTiming.OnSaveChanges;
}
Caveat 2: Effect on other root entities
If you define a global query filter this way, EF Core will diligently hide all other entities that reference a soft-deleted entity.
For example if you've soft-deleted a Partner entity, and you have Order entities where each of them references a partner through a (required) navigation property, then, when you retrieve the list of orders and you include the partner, all orders that reference a soft-deleted Partner will be missing from the list.
This behavior is discussed at the bottom of the documentation page.
Sadly, the global query filters as of EF Core 5 don't provide an option to limit them to root entities, or to disable only one of the filters. The only available option is to use the IgnoreQueryFilters() method, which disables ALL filters. And since the IgnoreQueryFilters() method takes an IQueryable and also returns an IQueryable, you cannot use this method to transparently disable the filter inside your DbContext class for an exposed DbSet.
Though, one important detail is that this occurs only if you Include() the given navigation property while querying. And there is an interesting solution for getting a result set that has query filters applied to certain entities but doesn't have them applied to other entities, relying on a lesser known feature of EF, relational fixup. Basically, you load a list of EntityA that has navigation property EntityB (without including EntityB). And then you separately load the list of EntityB, using IgnoreQueryFilters(). What happens is that EF automatically sets the EntityB navigation property on EntityA to the loaded EntityB instances. This way the query filter was applied to EntityA itself, but wasn't applied to the EntityB navigational property, so you can see EntityAs even with soft-deleted EntityBs. See this answer on another question. (Of course this has performance implications, and you still can't encapsulate it in DbContext.)

Disclaimer: I'm the owner of the project Entity Framework Plus
As you will see in #Adem link, our library supports query filtering.
You can easily enable/disable a global/instance filter
QueryFilterManager.Filter<Item>(q => q.Where(x => !x.IsDeleted));
Wiki: EF Query Filter
Edit: Answer sub question
Care to explain how this works behind the scene?
Firstly, you can either initialize filter globally or by instance
// Filter by global configuration
QueryFilterManager.Filter<Customer>(q => q.Where(x => x.IsActive));
var ctx = new EntitiesContext();
// TIP: You can also add this line in EntitiesContext constructor instead
QueryFilterManager.InitilizeGlobalFilter(ctx);
// Filter by instance configuration
var ctx = new EntitiesContext();
ctx.Filter<Post>(MyEnum.EnumValue, q => q.Where(x => !x.IsSoftDeleted)).Disable();
Under the hood, the library will loop on every DbSet of the context and checks if a filter can be applied to the generic type.
In this case, the library will filter the original/filtered query from the DbSet using the filter then modify the current internal query for the new filtered query.
In summary, we changed some DbSet internal value to use the filtered query.
The code is FREE & Open Source if you want to learn about how it works.
Edit: Answer sub question
#jonathan will this filter included navigation collections too?
For EF Core, it's not supported yet since Interceptor is not available yet. But starting from EF Core 2.x, the EF Team has implemented Global query filters which should allow this.

If you can migrate to EF Core 2.0 you can use
Model-level query filters
https://learn.microsoft.com/en-us/ef/core/what-is-new/index
If you use EF Core 1.0
You can make some trick with available EF Core features:
Inheritance
https://learn.microsoft.com/en-us/aspnet/core/data/ef-mvc/inheritance
Shadow properties
https://learn.microsoft.com/en-us/ef/core/modeling/shadow-properties
public class Attachment : AttachmentBase
{}
public abstract class AttachmentBase
{
public const string StatePropertyName = "state";
public Guid Id { get; set; }
}
public enum AttachmentState
{
Available,
Deleted
}
public class AttachmentsDbContext : DbContext
{
public AttachmentsDbContext(DbContextOptions options)
: base(options)
{
}
public DbSet<Attachment> Attachments { get; set; }
public override Task<int> SaveChangesAsync(CancellationToken cancellationToken = default(CancellationToken))
{
IEnumerable<EntityEntry<Attachment>> softDeletedAttachments = ChangeTracker.Entries<Attachment>().Where(entry => entry.State == EntityState.Deleted);
foreach (EntityEntry<Attachment> softDeletedAttachment in softDeletedAttachments)
{
softDeletedAttachment.State = EntityState.Modified;
softDeletedAttachment.Property<int>(AttachmentBase.StatePropertyName).CurrentValue = (int)AttachmentState.Deleted;
}
return base.SaveChangesAsync(cancellationToken);
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<AttachmentBase>()
.HasDiscriminator<int>(AttachmentBase.StatePropertyName)
.HasValue<Attachment>((int)AttachmentState.Available);
modelBuilder.Entity<AttachmentBase>().Property<int>(AttachmentBase.StatePropertyName).Metadata.IsReadOnlyAfterSave = false;
modelBuilder.Entity<Attachment>()
.ToTable("available_attachment");
modelBuilder.Entity<AttachmentBase>()
.ToTable("attachment");
base.OnModelCreating(modelBuilder);
}
}

Entity framework map entity to multiple tables in cascade

I'm facing a problem using EF.
I have the following situation:
From this database schema i'd like to generate the following entity by merge tables data:
// Purchases
public class Purchase
{
//Fields related to Purchases
public int IdPurchase { get; set; }
public string CodPurchase { get; set; }
public int IdCustomer { get; set; }
public decimal Total { get; set; }
//Fields related to Customers table
public string CodCustomer { get; protected set; }
public string CompanyTitle { get; protected set; }
public string CodType { get; protected set; }
//Fields related to CustomersType table
public string DescrType { get; protected set; }
}
As you can see, in my context i don't want 3 separated entities for each table. I want a single one with the fields related to all tables. All fields of Customers and CustomersType tables must be readonly (so i've set the relative setters protected) and the others must be editables so that EF can track changes. In particular, i'd like to have the ability to change the "IdCustomer" field and let EF to automatically update "CodCustomer", "CompanyTitle", "DescrType"....and so on by doing cross table select.
To do that, i wrote this configuration class:
internal class PurchaseConfiguration : EntityTypeConfiguration<Purchase>
{
public PurchaseConfiguration(string schema = "dbo")
{
ToTable(schema + ".Purchases");
HasKey(x => x.IdPurchase);
Property(x => x.IdPurchase).HasColumnName("IdPurchase").IsRequired().HasDatabaseGeneratedOption(DatabaseGeneratedOption.Identity);
Property(x => x.IdCustomer).HasColumnName("IdCustomer").IsRequired();
Property(x => x.Total).HasColumnName("Total").IsRequired().HasPrecision(19, 4);
Map(mc =>
{
mc.Properties(n => new
{
n.CodCustomer,
n.CompanyTitle,
n.CodType
});
mc.ToTable("Customers");
});
Map(mc =>
{
mc.Properties(n => new
{
n.DescrType,
});
mc.ToTable("CustomersType");
});
}
}
I've tested it but it doesn't work as expected. I always get this message:
Properties for type 'Purchase' can only be mapped once. The non-key
property 'CodCustomer' is mapped more than once. Ensure the
Properties method specifies each non-key property only once.
Maybe there's something wrong or i forget something (for example the join fields of Map<> that i don't know where to specify them).
How can i accomplish in the correct way this task?
I don't want to have "Customers" and "CustomersType" DBSets in my context.
Is there a way to avoid it?
I even thought to add into the "IdCustomer" setter a custom query to update manually "Customers" and "CustomersType" related fields, but i don't want to do that for 2 reasons:
I don't have any DbConnection avaiable into the "Purchases" class, so i can't create a DbCommand to read data from DB.
I want entity class to be persistent-ignorant
EF seems to be a powerfull tool that can do these sort of things and i don't want to reinvent the wheel by writing custom procedures.
I've uploaded the example C# source and the tables CREATE scripts (MS SQLServer) here.
All entities are autogenerated by the "EF reverse POCO generator" T4 template (the T4 template is disabled, to activate it set CustomTool = TextTemplatingFileGenerator).
Do not forget to update the ConnectionString in the app.config.
Thanks in advance.

Not the right mapping
I'm afraid the bad news is that this mapping is not possible with this table structure. What you're trying to achieve here is known as entity splitting. However, entity splitting requires 1:1 associations, because sets of records in the involved tables represent one entity. With this mapping, you can't have a Customer belonging to more than one Purchase. That would mean that you could modify multiple Purchase entities by modifying a Customer property of only one of them.
Maybe the news isn't that bad, because I think you actually want to have 1-n associations. But then you can't have these "flattened" properties in Purchase.
As an alternative you could create delegated properties like so:
public string CodCustomer
{
get { return this.Customer.CodCustomer; }
set { this.Customer.CodCustomer = value; }
}
You'd have to Include() Customers and CustomersTypes when you fetch Purchases.
Another alternative is to use a tool like AutoMapper to map Purchase to a DTO type having the flattened properties.
But what does the exception tell me?
You map the Purchase entity to the Purchases table. But you don't specify which properties you want to map to this table. So EF assumes that all properties should be mapped to it. So that's the first (implicit) mapping of CodCustomer. The second one is the one in the mc.ToTable statement. (EF only reports the first problem.)
To fix this, you should add a mapping statement for the left-over Purchase properties:
Map(mc =>
{
mc.Properties(n => new
{
n.IdPurchase,
n.CodPurchase,
n.IdCustomer,
n.Total,
});
mc.ToTable("Purchases");
});
By the way, you should also remove the mapping configuration classes of Customer and CustomersType, they're redundant.
But, as said, the database schema doesn't match the required structure. If you try to save a Purchase you will get a foreign key constraint exception. This is because EF expects the following table structure:
Where the columns IdPurchase in Customer and CustomersType are both primary key and foreign key to Purchase. I don't think this is what you had in mind when designing the database.

How can you add a navigation property on a view in EF 6.1 CodeFirst

Let's make a case to explain my problem.
MyTable1
+id
+myTable2Id
MyTable2
+id
MyView1
+id
+myTable2Id
MyView1 exists in the case, from data from the MyTable1. Now i want to create a Navigation property from my EF6.1 Code first approach in my View to MyTable2.
I know that it was possible from the database first approach, but is it also possible from the code-first approach and how?
EDIT:
I search some on internet, but due many meanings of the word View, it's very hard to find information on it.
Also with the approaches in codes that i tried, i always get an error that the migration can't be completed. Because the Migration tries to add an foreign key to the view, which isn't possible.
EDIT2:
To elaborate a bit more on my explanation. I want to be able to approach it in code the following way:
Guid table2Id = context.MyView1.FirstOrDefault().MyTable2.id;
EDIT3:
I will eleborate a bit more, to see if i can get my problem better explained.
When i added the following to my view Entity:
public virtual MyTable2 Table2 { get; set;}
EF will automaticly generate the following migration:
public override void Up() {
CreateIndex("MyView1", "MyTable2Id");
AddForeignKey("MyView1", "MyTable2Id", "MyTable2", "id")
}
Which on running update-database gives the following error :
"Cannot create index on view 'MyView1' because the view is not schema bound"
EDIT4:
With help of the comment that the migration aren't of stone.. and are changeable i made it.
I used the following fluentAPI:
// Map one-to-zero or one relationship
modelBuilder.Entity<MyTable2>()
.HasRequired(t => t.MyTable1)
.WithOptional(t => t.MyTable2);
modelBuilder.Entity<MyTable1>()
.HasOptional(t => t.MyTable2);
And changing my tables to this: (The FK to the MyTable2 and removed from the view)
MyTable1
+id
MyTable2
+id
+myTable1
MyView1
+id
Which in the end is better because this way i have less Null values in my model.

In EF you can use a database views and map it to an entity and reference it just as you do with tables.
For code first process you have to create the View in Up and drop it in Down methods from migration class:
public partial class AddView : DbMigration
{
public override void Up()
{
this.Sql(#"CREATE VIEW MyView1 AS ....");
}
public override void Down()
{
this.Sql(#"DROP VIEW MyView1....");
}
}
EDIT:
public long myTable2Id { get; set; }
[ForeignKey( "myTable2Id" )]
public virtual MyTable2 Table2 {get;set;}