Problem
I have a situation whereby I need to use Entity Framework 6, Code First, with a legacy database structure which cannot be changed. The database has a very generic table which stores text based data alongside some non key data which can be used to relate the record back to another table.
To illustrate:
Assume the Notes table has a model as follows:
[Table("Notes")]
public class Notes
{
[Key]
public int RecordId { get; set; }
[Required]
public string RelatedTableName { get; set; }
[Required]
public int RelatedTableRecordId { get; set; }
[Required]
public string NotesText { get; set; }
}
I then have another model which could look like so:
[Table("Drivers")]
public class Drivers
{
[Key]
public int RecordId { get; set; }
[Required]
public string DriverName { get; set; }
public ICollection<Notes> DriverNotes { get; private set; }
}
There is no foreign key which links the tables. The Drivers table is linked to the Notes table by way of the RelatedTableName and RelatedTableRecordId fields.
I do not have a problem reading data from the database and hydrating the models using entity framework.
The problem I have is that I want to be able to save a new Driver and its newly created Notes in one transaction and have the RelatedTableRecordId field set to the primary key of the Driver.
If a foreign key existed entity framework would know to back fill the property but in this case it doesn't know about the relationship.
Key Points
Database Structure must not change.
Must use Entity Framework 6 Code First
Must be able to use an Execution Strategy.
Require a relationship between non key fields.
Need to be able to persist all data in a single transaction.
What I've Tried
I had a similar issue with Audit type data and solved it by doing something similar to the following (note that this is very pseudo here):
public override int SaveChanges()
{
int changes = 0;
//Disable the current execution strategy as the default ones do not support user instantiated transactions.
this.ContextConfiguration.SuspendExecutionStrategy();
try
{
//Wrap a whole transaction inside an execution strategy so that auditing can be combined with regular saving of changes.
this.ExecutionStrategy.Execute(
() =>
{
using (var transaction = this.Database.BeginTransaction())
{
//Reset the change count so that it doesn't increase each time the transaction fails.
changes = 0;
//Remove any audit records created by previous failed transactions.
this.AuditTableChanges.Local.Clear();
//Evaluate the change tracker to identify entities which will potentially require an audit trail.
var insertedEntities = this.ChangeTracker.Entries().Where(entryEntity => entryEntity.State == EntityState.Added).ToList();
//Save all changes to get identities.
changes = base.SaveChanges();
//Create the audit trail for inserted entities. This step must occur after the initial call to SaveChanges() so that the identities are set.
foreach (DbEntityEntry entryEntity in insertedEntities)
{
//For each inserted record, get the audit record entries and add them
foreach (AuditTableChange auditTableChange in GetAuditRecords(entryEntity, AuditTableChangeType.Insert).Result)
this.AuditTableChanges.Add(auditTableChange);
}
//Save the audit trail for inserted entities.
changes += base.SaveChanges();
//Commit all changes to the database
transaction.Commit();
}
});
}
finally
{
//Re-enable the execution strategy so that other calls can benefit from the retry policy.
this.ContextConfiguration.UnSuspendExecutionStrategy();
}
return changes;
}
This worked fine for the Audit data as the implementation was hidden away in the framework. I do not want my development team to have to do all of the above each time they persist records.
In its simplistic form this is as much as I'd want people to be doing:
public void CreateDriver()
{
using (MyContext context = new MyContext())
{
Drivers driver = new Drivers();
driver.DriverName = "Joe Bloggs";
Notes driverNote = new Notes();
driverNote.RelatedTableName = "Drivers";
driverNote.NotesText = "Some very long text";
driver.DriverNotes.Add(driverNote);
context.Drivers.Add(driver);
context.SaveChanges();
}
}
In a way I want a foreign key which exists in code but not in the database so that entity framework knows to fill in the RelatedTableRecordId field. I've read some articles on hacking the EDMX but this project is purely Code First only.
There are older questions on stack overflow which are similar but relate to older versions of entity framework and don't help much or have as much detail as the above.
I'm hoping that someone may have experienced a similar problem and has an answer which may involve perhaps some custom mapping/metadata or some overrides to entity framework logic.
Any help would be appreciated.
Thanks,
Greg
Related
I am using Entity Framework Core 6.0 and I am currently looking for a way on how to persist data audit-proof (immutably).
Concretely, any intended modification of an existing database entry (SQL UPDATE statement) shall be translated into a new version/ revision of the existing entity (SQL INSERT statement) and, thus, result in a new database entry which supersedes the existing one (correlation among related entities is established by CorrelationId). Only the metadata of an entry are allowed to be updated, such that predecessor and successor relate to each other by ID (see PredecessorId and SucessorId, respectively).
Of course, this should not only work for entity relations (one-to-one, one-to-many, many-to-many) on first level but all along a complete object graph (arbitrary level).
I've added sample code for entity modeling for a contrived example to illustrate the idea:
Once I have placed an Order Order any subsequent changes of an ordered product Product must not have any influence on the order placed (not the names, not the prices). Instead a change in a product shall result in a new database entry with the same CorrelationId and metadata info on current and previous entry to relate to each other.
The example is contrived and the data modeling in the example should not be considered from a domain perspective.
I already looked into Entity Framework Plus and its audit functionality but from what I understood, it does only provide a separate table for the changes made to entities. However, an update on an entity results in a SQL UPDATE and in the example above a change in product price would affect the order already placed (Total would be different). The old state of the entity does not exist in its form any longer but rather one can reconstruct and comprehend who, when and how the entity changed with the audit information.
So, is there any clever solution on how to handle this in Entity Framework Core? I already tried myself in overriding the SaveChanges(Async) methods in the DbContext but I was not successful in preserving the entity relations:
case 1: principal changed, dependent(s) unchanged
-> new principal entity with reference to unchanged dependent(s)
case 2: principal unchanged, dependent(s) changed
-> no new principal entity with reference to new dependent(s)
case 3: principal changed, dependent(s) changed
-> new principal entity with reference to new dependent(s)
case 4: principal unchanged, dependent(s) unchanged
-> no action required
using System;
using System.Linq;
using System.Collections.Generic;
public abstract record BaseEntity
{
public long Id { get; init; }
public long? PredecessorId { get; private init; }
public long? SuccessorId { get; private init; }
public DateTimeOffset CreatedTimestamp { get; private init; }
public bool IsDeleted { get; private init; }
public DateTimeOffset DeletedTimestamp { get; private init; }
public Guid CorrelationId { get; init; }
}
public record Product : BaseEntity
{
public string Name { get; init; }
public double Price { get; init; }
}
public record Order : BaseEntity
{
public IReadOnlyCollection<Product> Products { get; init; }
public double Total => Products.Sum(o => o.Price);
}
I'm using ASP.NET Zero. Project Version: 5.1.0 and .NET Core 2.0 template. I'm trying to enable Entity History for my entity so that I can see the deleted and old column values for the table.
Entity class:
[Table("TestingEntity")]
[Audited]
public class TestingEntity : AuditedEntity , IMayHaveTenant
{
public int? TenantId { get; set; }
public virtual string Code { get; set; }
}
ApplicationModule class:
public class MyCompanyApplicationModule : AbpModule
{
public override void PreInitialize()
{
// ...
Configuration.EntityHistory.IsEnabledForAnonymousUsers = true;
Configuration.EntityHistory.Selectors.Add(new NamedTypeSelector("Abp.AuditedEntities", type => typeof(IAudited).IsAssignableFrom(type)));
}
// ...
}
Running the following queries give no results.
SELECT * FROM [AbpEntityChangeSets]
SELECT * FROM [AbpEntityPropertyChanges]
SELECT * from [AbpEntityChanges]
Reference: https://aspnetboilerplate.com/Pages/Documents/Entity-History
Update
It is not giving proper results when I'm deleting an entity item.
It's inserting records for each property with old and new values the same in [AbpEntityPropertyChanges] table.
And there is no clear information that this entity item is deleted, its deletion time, and DeletedBy.
Is this due to using AuditedEntity in my entity class? I'm using hard delete, so I thought not to add these columns to the table: is deleted, its deletion time, and DeletedBy.
Entity History is disabled in ASP.NET Zero. You can enable it:
Configuration.EntityHistory.IsEnabled = true;
Update
It is not giving proper results when I'm deleting an entity item.
It's inserting records for each property with old and new values the same in [AbpEntityPropertyChanges] table.
That has been resolved in PR #2977, which will be released with ABP v3.5.
And there is no clear information that this entity item is deleted, its deletion time, and DeletedBy.
Is this due to using AuditedEntity in my entity class? I'm using hard delete, so I thought not to add these columns to the table: is deleted, its deletion time, and DeletedBy.
You won't find those in AbpEntityPropertyChanges table, since those aren't property changes.
information that this entity was deleted: EntityChange.ChangeType
its deletion time: EntityChangeSet.CreationTime
DeletedBy: EntityChangeSet.UserId
Additional information
relationship between AbpEntityChangeSets and AbpEntityChanges tables: EntityChange.cs
public class EntityChange : Entity<long>, IMayHaveTenant
{
/// <summary>
/// Gets/sets change set id, used to group entity changes.
/// </summary>
public virtual long EntityChangeSetId { get; set; }
// ...
}
possible values of EntityChange.ChangeType: EntityChangeType.cs
public enum EntityChangeType : byte
{
Created = 0,
Updated = 1,
Deleted = 2
}
Do we have a plan to add UI for this feature? So that we can see Entity History from UI.
This has been added in ASP.NET Zero 5.4.0.
I have solved the issue by doing the below change in file ProjectName.EntityFrameworkCore\EntityFrameworkCore\ProjectNameEntityFrameworkCoreModule.cs by setting the following value to true, You need to enable Entity History.
Configuration.EntityHistory.IsEnabled = true;
You can refer https://github.com/aspnetzero/aspnet-zero-core/issues/818#issuecomment-365250173.
Just recently tried OrmLite for MySql in a C# console project:
I'm loving it, since it's easy to implement with MySql, in contrast to Entity Framework. However I'm trying to enable automatic updates of the database schema from my model. I extended the code given in the answers in:
With OrmLite, is there a way to automatically update table schema when my POCO is modified?
And came up with a solution that can add and delete columns in your database schema, depending on the model. Works pretty good, and utilizes a lot of the features OrmLite.ModelDefinition gives.
See the solution here:
https://github.com/contradel/Extending-NServiceKit.OrmLite
However I can't get it to automatically create foreign keys with GetDialectProvider.ToAddForeignKeyStatement(). And I can't find any documentation. Does anyone know how it works?
I would like to be able to do this:
public class Order
{
//Try to add or comment out properties here, SQL will be generated automatic
[AutoIncrement]
public int Id { get; set; } //pk
public DateTime? OrderDate { get; set; }
[References(typeof(Customer))] //Creates Foreign Key
public int CustomerId { get; set; }
}
And then:
foreach (FieldDefinition field in missingOnDb)
{
//if (field != ForeignKey)
var addSql = string.Format(db.GetDialectProvider().ToAddColumnStatement(typeof(T), field));
//else
//addSql = db.GetDialectProvider().ToAddForeignKeyStatement(??,??,??,??,??);
Console.WriteLine(addSql);
db.ExecuteSql(addSql);
}
I am using Entity Framework 4.3 Code First, and I have problem with updating many-to-many relationships.
I defined the following classes:
public abstract class Entity
{
[Column(Order = 0)]
[Key]
public int Id { get; set; }
[Timestamp]
[Column(Order = 1)]
public byte[] Version { get; set; }
}
public class Video : Entity
{
public string Title { get; set; }
public string Description { get; set; }
public TimeSpan Length { get; set; }
public virtual ICollection<Coworker> Coworkers { get; set; }
}
public class Coworker : Entity
{
public string FirstName { get; set; }
public string LastName { get; set; }
public virtual ICollection<Video> Videos { get; set; }
}
When the database is created, the schema look right:
There is a Videos, Coworkers and VideoCoworkers table too, without
I use repository pattern in an N-Tier application to access database, my Insert and Update method looks like this:
public T Insert(T entity)
{
//Creates database context. When it disposes, it calls context.SaveChanges()
using (var session = new DatabaseSession())
{
session.Context.Set<T>().Add(entity);
}
}
public T Update(T entity)
{
//Creates database context. When it disposes, it calls context.SaveChanges()
using (var session = new DatabaseSession())
{
entity = session.Context.Set<T>().Attach(entity);
session.Context.Entry(entity).State = EntityState.Modified;
}
return entity;
}
When I update an entity, I create the entity object from a DTO, that's why use DbSet.Attach instead of selecting it and updating the properties one-by-one.
When I initialize the database, I add some test data:
Create 3 Coworkers, where I set first and last name. (A, B, C)
Create 3 Videos, where I set title, description and length, and also set some coworkers. First video has A,B, second has B,C and third has A,C.
When I list the Videos from code, I can see that Video.Coworkers collection is filled with good values, and when I query the link table (VideoCoworkers) in SQL Server Management Studio, it also looks good.
My problem is
when I update for example the title of the Video, it works. But when I try to delete from Video2 the existing coworkers (B and C), and try to add coworker A, then the relationship is not updated. It also does not work when I only try to add new coworker, or only try to delete one. I create the entity which is used as the parameter of the Update() method by creating a new Video entity with a new collection of Coworkers (which are selected from the database with Find() method by Id).
What is the correct way to update many-to-many relationships?
But when I try to delete from Video2 the existing coworkers (B and C),
and try to add coworker A, then the relationship is not updated.
Without using a generic repository the correct procedure would be:
using (var session = new DatabaseSession())
{
video2 = session.Context.Set<Video>().Include(v => v.Coworkers)
.Single(v => v.Id == video2Id);
coworkerA = new Coworker { Id = coworkerAId };
session.Context.Set<Coworker>().Attach(coworkerA);
video2.Coworkers.Clear();
video2.Coworkers.Add(coworkerA)
session.Context.SaveChanges();
}
The essential part is that you must load or attach the entity in its original state, change the entity, i.e. remove and add children, and then save the changes. EF's change detection will create the necessary INSERT and DELETE statements for the link table entries. The simple procedure to set the state to Modified you are trying in your generic Update method is suited only for updating scalar properties - like changing the video title - but won't work for updating relationships between entities.
For solve this problem:
attach the entity to context
load the collection(the collection is not loaded, because )
change the state of entity to modified
save changes
So your code for update should be like this:
public Video Update(Video entity)
{
//Creates database context. When it disposes, it calls context.SaveChanges()
using (var session = new DatabaseSession())
{
entity = session.Context.Set<Video>().Attach(entity);
session.Context.Entry(entity).Collection(p => p.Coworkers).Load();
session.Context.Entry(entity).State = EntityState.Modified;
}
return entity;
}
Please refer here to see how to save master detail in asp.net mvc with database first. Hopefully it will give you the idea about the code first. You may also have a look at knokout.js example
I have a pretty deep object hierarchy in my application, and I am having trouble saving the entities. Depending on the order I do things, I either one of two errors:
[OptimisticConcurrencyException: Store update, insert, or delete statement affected an unexpected number of rows (0). Entities may have been modified or deleted since entities were loaded. Refresh ObjectStateManager entries.]
or
[DbUpdateException: An error occurred while saving entities that do not expose foreign key properties for their relationships. The EntityEntries property will return null because a single entity cannot be identified as the source of the exception. Handling of exceptions while saving can be made easier by exposing foreign key properties in your entity types. See the InnerException for details.]
Here is the classes I am working with:
public class SpecialEquipment : Entity
{
public Equipment Equipment { get; set; }
public virtual ICollection<AutoclaveValidation> Validations { get; set; }
}
public class Equipment : Entity
{
public string Model { get; set; }
public string SerialNumber { get; set; }
public Location Location { get; set; }
public EquipmentType EquipmentType { get; set; }
public ICollection<Identifier> Identifiers { get; set; }
}
public class Identifier : Entity
{
public IdentifierType Type { get; set; }
public string Value { get; set; }
}
public class Location : Entity
{
public Building Building { get; set; }
public string Room { get; set; }
}
What I was trying to do was populate one SpecialEquipment object based on form inputs and already existing objects in the database and then save the special equipment to push all changes through, it looks like this:
Building building = buildingService.GetExistingOrNew(viewModel.BuildingCode)); //checks to see if building exists already, if not, create it, save it, and re-query
Location location = locationService.GetExistingOrNew(viewModel.Room, building); //checks to see if location exists already, if not, create it, save it, and re-query
EquipmentType equipmentType = equipmentTypeService.GetOne(x => x.Name == EquipmentTypeConstants.Names.Special);
Equipment equipment = new Equipment{ EquipmentType = equipmentType, Location = location };
equipment.Identifiers = new Collection<Identifier>();
foreach (FormIdentifier formIdentifier in identifiers)
{
FormIdentifier fIdentifier = formIdentifier;
IdentifierType identifierType = identifierTypeService.GetOne(x => x.Id == fIdentifier.Key);
equipment.Identifiers.Add(new Identifier { Type = identifierType, Value = fIdentifier.Value });
}
EntityServiceFactory.GetService<EquipmentService>().Save(equipment);
SpecialEquipment specialEquipment = new SpecialEquipment();
specialEquipment.Equipment = equipment;
specialEquipmentService.Save(specialEquipment);
This code returns Store update, insert, or delete statement affected an unexpected number of rows (0). If I comment out the foreach identifiers OR put the foreach identifiers after the equipment save and then call equipment save after the loop the code works. If I comment out the foreach identifiers and the save equipment line, I get : The INSERT statement conflicted with the FOREIGN KEY constraint "SpeicalEquipment_Equipment". The conflict occurred in database "xxx", table "dbo.Equipments", column 'Id'.
So how can I make these errors not occur but still save my object? Is there a better way to do this? Also I don't like saving my equipment object, then associating/saving my identifiers and/or then my special equipment object because if there is an error occurring between those steps I will have orphaned data. Can someone help?
I should mention a few things that aren't inheritly clear from code, but were some answers I saw for similar questions:
My framework stores the context in the HttpContext, so all the service methods I am using in my API are using the same context in this block of code. So all objects are coming from/being stored in one context.
My Entity constructor populates ID anytime a new object is created, no entities have a blank primary key.
Edit: At the request of comments:
My .Save method calls Insert or Update depending on if the entity exists or not (in this example insert is called since the specialEquipment is new):
public void Insert(TClass entity)
{
if (Context.Entry(entity).State == EntityState.Detached)
{
Context.Set<TClass>().Attach(entity);
}
Context.Set<TClass>().Add(entity);
Context.SaveChanges();
}
public void Update(TClass entity)
{
DbEntityEntry<TClass> oldEntry = Context.Entry(entity);
if (oldEntry.State == EntityState.Detached)
{
Context.Set<TClass>().Attach(oldEntry.Entity);
}
oldEntry.CurrentValues.SetValues(entity);
//oldEntry.State = EntityState.Modified;
Context.SaveChanges();
}
GetExistingOrNew for Building and location both are identical in logic:
public Location GetExistingOrNew(string room, Building building)
{
Location location = GetOne(x => x.Building.Code == building.Code && x.Room == room);
if(location == null)
{
location = new Location {Building = building, Room = room};
Save(location);
location = GetOne(x => x.Building.Code == building.Code && x.Room == room);
}
return location;
}
Get one just passes that where predicate to the context in my repository with singleOrDefault. I am using a Service Layer/Repository Layer/Object Layer format for my framework.
Your Insert method does not seem to be correct:
public void Insert(TClass entity)
{
if (Context.Entry(entity).State == EntityState.Detached)
Context.Set<TClass>().Attach(entity);
Context.Set<TClass>().Add(entity);
Context.SaveChanges();
}
specialEquipment is a new entity and the related specialEquipment.Equipment as well (you are creating both with new)
Look what happens if you pass in the specialEquipment into the Insert method:
specialEquipment is detached because it is new
So, you attach it to the context
Attach attaches specialEquipment and the related specialEquipment.Equipment as well because both were detached from the context
Both are in state Unchanged now.
Now you add specialEquipment: This changes the state of specialEquipment to Added but not the state of specialEquipment.Equipment, it is still Unchanged.
Now you call SaveChanges: EF creates an INSERT for the added entity specialEquipment. But because specialEquipment.Equipment is in state Unchanged, it doesn't INSERT this entity, it just sets the foreign key in specialEquipment
But this FK value doesn't exist (because specialEquipment.Equipment is actually new as well)
Result: You get the FK constraint violation.
You are trying to fix the problem with calling Save for the equipment but you have the same problem with the new identifiers which will finally throw an exception.
I think your code should work if you add the specialEquipment (as the root of the object graph) at the end once to the context - without attaching it, so that the whole graph of new objects gets added, basically just:
context.Set<SpecialEquipment>().Add(specialEquipment);
context.SaveChanges();
(BTW: Your Update also doesn't look correct, you are just copying every property of entity to itself. The context won't detect any change and SaveChanges won't write any UPDATE statement to the database.)
My guess? It can't have an ID if you haven't saved it and that's the root of the problem (since it works if you save first).
Pop everything in a transaction, so if anything goes wrong all is rolled back. Then you don't have orphans.
See http://msdn.microsoft.com/en-us/library/bb738523.aspx for how to use transactions with EF.