We Keep Coding

Json into EF with duplicates

I have following sample JSON (simplified):
{
"programmes":[
{
"programmeID":"163",
"title":"Programme 1",
"color":"#ff5f57",
"moderators":[
{
"moderatorID":"27",
"name":"Moderator 1",
}
]
},
{
"programmeID":"153",
"title":"Programme 2",
"color":"#ff5f57",
"moderators":[
{
"moderatorID":"27",
"name":"Moderator 1",
}
]
},
]
}
I want to convert into following objects:
public class Programme
{
[Key]
public string programmeID { get; set; }
public string title { get; set; }
//Navigational Property
public virtual ICollection<Moderator> moderators { get; set; }
}
public class Moderator
{
[Key]
public string moderatorID { get; set; }
public string name { get; set; }
//Navigational Property
public virtual ICollection<Programme> programmes { get; set; }
}
Everything is fine, objects looks great, populated correctly, but when trying to save into db a get this error:
The instance of entity type 'Moderator' cannot be tracked because
another instance with the key value '{moderatorID: 27}' is already
being tracked. When attaching existing entities, ensure that only one
entity instance with a given key value is attached.
This error happens when calling .Add with the list of Programmes:
List<Programme> programmes;
programmes = JsonSerializer.Deserialize<List<Programme>>(JSONcontent);
//... db init
db.AddRange(programmes);
I understand that there is a duplicity for Moderator (ID=27), but I think that it's quite a normal pattern in JSON. I can do some workaround, but wondering if there is any best practice how to handle this in EF directly?

you can not add children you are created from deserialization as an existed ones. One of the solutions is finding the existed children at first
List<Programme> programmes = JsonSerializer.Deserialize<List<Programme>>(JSONcontent);
foreach (var program in programmes)
{
var moderators=new List<Moderator>();
foreach (var moderator in program.moderators)
{
var existingModerator=db.Moderators.FirstOrDefault(i=>
i.moderatorId=moderator.moderatorId);
moderators.Add(existingModerator);
}
programm.moderators=moderators;
}
db.AddRange(programmes);
but I usually prefer to create an explicit class for many to many relations. It will make all your code much more simple.

LazyLoading not behaving as expected in EFCore

Here are 2 (minimized classes):
[Index(nameof(Name), IsUnique = true)]
public partial class ParticipantList
{
public ParticipantList()
{
Emails = new HashSet<Email>();
}
[Key]
[Column("id")]
public long Id { get; set; }
//...
[InverseProperty(nameof(Email.ParticipantList))]
public virtual ICollection<Email> Emails { get ; set; }
}
public partial class Email
{
[Key]
[Column("id", TypeName = "integer")]
public long Id { get; set; }
[Column("participant_list_id", TypeName = "integer")]
public long? ParticipantListId { get; set; }
/...
[ForeignKey(nameof(ParticipantListId))]
[InverseProperty("Emails")]
public virtual ParticipantList ParticipantList { get; set; }
}
And the DbContext OnConfiguring method contains:
optionsBuilder
.UseLazyLoadingProxies()
.UseSqlite(...);
And the DBContext OnModelCreating method contains:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Email>(entity =>
{
entity.HasOne(d => d.ParticipantList)
.WithMany(p => p.Emails)
.HasForeignKey(d => d.ParticipantListId)
.OnDelete(DeleteBehavior.ClientSetNull);
});
This is what works:
var email = _db.Emails.Find(1);
// email.ParticipantList (this lazy loads just fine).
What doesn't work:
var plist = _db.ParticipantLists.Find(1);
// plist.Emails.Count() == 0 (this doesn't lazy load at all)
// plist.Emails.Count == 0 (again no lazy loading)
// plist.Emails.ToList() (empty list, no lazy loading)
I should insert a rant here about the stupidity of having both Count() and Count with different meanings here. I mean, seriously, who is going to catch that during a code review????
This is most likely due to:
public ParticipantList()
{
Emails = new HashSet<Email>();
}
Which was created by the ef-scaffold app. I mean it makes sense, the HashSet is empty, but it seems like the model should override the access to the HashSet and do the lazy loading.
The problem is that if you remove this:
public ParticipantList()
{
// Emails = new HashSet<Email>();
}
And then call this code:
// plist.Emails.Count() == 0 (Now you get a null reference exception
// because guess what: Emails is not initialized!)
// plist.Emails.ToList() (Also gives a null reference exception)
// plist.Emails.Count == 0 (Again, a null reference exception)
The documentation shows code like this (note that Posts is NOT initialized in the example at https://learn.microsoft.com/en-us/ef/core/querying/related-data/lazy ):
public class Blog
{
public int Id { get; set; }
public string Name { get; set; }
public virtual ICollection<Post> Posts { get; set; }
}
public class Post
{
public int Id { get; set; }
public string Title { get; set; }
public string Content { get; set; }
public virtual Blog Blog { get; set; }
}
Which will purportedly lazy load the Posts from the Blog. But when I follow that pattern and remove the ctor that initializes Emails/Posts I get a null reference exception.
To add to the mystery/madness is this:
var email = _db.Emails.Find(1);
var plist = _db.ParticipantLists.Find(1);
// plist.Emails.Count() == 1 (It knows about the relation and added it to
// the Emails HashSet on the ParticipantList!)
So, how do I get Emails to LazyLoad. I know about the 1 + N problem, and I'm not worried about that aspect, but I'd like the models to load when they are requested.

EF Core 3.1 does not allow Contains search on Enum property

I'm trying to do a contains search on enum property in my DbSet and EF Core 3.1 throws the below error
The LINQ expression 'DbSet .Where(d =>
d.Position.ToString().Contains("acc"))' could not be translated.
Either rewrite the query in a form that can be translated, or switch
to client evaluation explicitly by inserting a call to either
AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync()
Entity:
public class DemoEntity
{
[Key]
public int Id { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public Position Position { get; set; }
}
Enum - Position:
public enum Position
{
[Display(Name = "Accountant")]
Accountant,
[Display(Name = "Chief Executive Officer (CEO)")]
ChiefExecutiveOfficer,
[Display(Name = "Integration Specialist")]
IntegrationSpecialist,
[Display(Name = "Junior Technical Author")]
JuniorTechnicalAuthor,
[Display(Name = "Pre Sales Support")]
PreSalesSupport,
[Display(Name = "Sales Assistant")]
SalesAssistant,
[Display(Name = "Senior Javascript Developer")]
SeniorJavascriptDeveloper,
[Display(Name = "Software Engineer")]
SoftwareEngineer
}
DbContext:
public class DemoDbContext : DbContext
{
public DemoDbContext(DbContextOptions options)
: base(options) { }
public DbSet<DemoEntity> Demos { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder
.Entity<DemoEntity>()
.Property(e => e.Position)
.HasConversion<string>();
}
}
When I query the table as follows I'm getting the error
try
{
var test = await _context.Demos.Where(x => x.Position.ToString().Contains("acc")).ToListAsync();
}
catch (System.Exception e)
{
//throw;
}
The Position is of type NVARCHAR(MAX) in my database.
This is not possible? If so please can you help me with explanation?

This issue has been logged in efcore github repo and here is the workaround for this now. The enum property needs to be casted into object and then to string
var test = await _context.Demos.Where(x => ((string) object)x.Position).Contains("acc")).ToListAsync();
Hope this helps someone out there.

Disconnected Entities in EF Core

I have been looking to log changes on entities in a disconnected scenario.
Original Problem: I call DbContext.Update(Entity) on an updated complex entity and even though nothing has changed, everything is marked as Changed by the ChangeTracker and my concurrency RowVersion column counts up.
POST User with no changes
"userSubPermissions": [],
"fidAdresseNavigation": {
"idAdresse": 1,
"fidPlzOrt": 3,
"strasse": "Gerold Str.",
"hausnr": "45",
"rowVersion": 10,
"isDeleted": 0,
"fidPlzOrtNavigation": {
"idPlzOrt": 3,
"plz": "52062",
"ort": "Aachen",
"rowVersion": 9,
"isDeleted": 0
}
},
"idUser": 35,
"fidAnrede": null,
"fidAdresse": 1,
"fidAspnetuser": "a7ab78be-859f-4735-acd1-f06cd832be7e",
"vorname": "Max",
"nachmname": "Leckermann",
"eMail": "kunde#leasing.de",
"rowVersion": 11,
"isDeleted": 0
POST Returned user
"userSubPermissions": [],
"fidAdresseNavigation": {
"idAdresse": 1,
"fidPlzOrt": 3,
"strasse": "GeroldPenis Str.",
"hausnr": "45",
"rowVersion": 11,
"isDeleted": 0,
"fidPlzOrtNavigation": {
"idPlzOrt": 3,
"plz": "52062",
"ort": "Aachen",
"rowVersion": 10,
"isDeleted": 0
}
},
"idUser": 35,
"fidAnrede": null,
"fidAdresse": 1,
"fidAspnetuser": "a7ab78be-859f-4735-acd1-f06cd832be7e",
"vorname": "Max",
"nachmname": "Leckermann",
"eMail": "kunde#leasing.de",
"rowVersion": 12,
"isDeleted": 0
The RowVersion logic is in the DBContext and only changes the row version when the Entity State is Modified.
foreach (var entity in ChangeTracker.Entries().Where(e => e.State == EntityState.Modified))
{
var saveEntity = entity.Entity as ISavingChanges;
saveEntity.OnSavingChanges();
}
From my investigation over the past 2 days there seem to be two alternatives for EF Core.
1.- As mentioned on this and many similar posts you can use the TrackGraph https://www.mikesdotnetting.com/article/303/entity-framework-core-trackgraph-for-disconnected-data
Problem: This solution marks the entire entity as changed when the ID of the entity is present even though nothing has changed. I get the exact same result as described above, the RowVersion counts up meaning my Entities an all of its Properties have been marked as Modified.
2.- As mentioned in this post
https://blog.tonysneed.com/2017/10/01/trackable-entities-for-ef-core/ you can download a NuGet package and implement the IMergeable and the ITrackable
interfaces on my Entities.
Problem: The Client needs to track the changes in the Model and pass them to the API, which I want to avoid since Angular does not seem to offer a good solution for this.
3.- There is another solution in the book Programming Entity Framework on how EF 6 handles disconnected scenarios, by recording original values Chapter 4: Recording Original Values.
When an Update comes in, the context reads the entity from the DB and then compares the incoming Entity with the DB Entity to see if any changes have been made on the properties and then marks them as Modified.
https://www.oreilly.com/library/view/programming-entity-framework/9781449331825/ch04.html
Problem: Some of the code described in the book is not implementable in EF Core one example is.
((IObjectContextAdapter)this).ObjectContext
.ObjectMaterialized += (sender, args) =>
{
var entity = args.Entity as IObjectWithState;
if (entity != null)
{
entity.State = State.Unchanged;
entity.OriginalValues =
BuildOriginalValues(this.Entry(entity).OriginalValues);
}
};
There must be a way to implement this clever solution of handling changes on the API instead of the Client in Disconnected scenarios, otherwise, this might be a very useful Feature to have in EF Core since most Developers are using it for Disconnected scenarios.

My response is using TrackGraph
Synopsis
I want to Modify only few columns of the entity as well as Add\Modify nested child entities
Here, I am updating Scenario entity and modifying only ScenarioDate.
In its child entity, i.e. navigation property TempScenario, I am adding a new record
In the nested child entity, Scenariostation, I am adding as well modifying the records
public partial class Scenario
{
public Scenario()
{
InverseTempscenario = new HashSet<Scenario>();
Scenariostation = new HashSet<Scenariostation>();
}
public int Scenarioid { get; set; }
public string Scenarioname { get; set; }
public DateTime? Scenariodate { get; set; }
public int Streetlayerid { get; set; }
public string Scenarionotes { get; set; }
public int? Modifiedbyuserid { get; set; }
public DateTime? Modifieddate { get; set; }
public int? Tempscenarioid { get; set; }
public virtual Scenario Tempscenario { get; set; }
public virtual ICollection<Scenario> InverseTempscenario { get; set; }
public virtual ICollection<Scenariostation> Scenariostation { get; set; }
}
public partial class Scenariostation
{
public Scenariostation()
{
Scenariounit = new HashSet<Scenariounit>();
}
public int Scenariostationid { get; set; }
public int Scenarioid { get; set; }
public int Stationid { get; set; }
public bool? Isapplicable { get; set; }
public int? Createdbyuserid { get; set; }
public int? Modifiedbyuserid { get; set; }
public DateTime? Modifieddate { get; set; }
public virtual Scenario Scenario { get; set; }
public virtual Station Station { get; set; }
}
public partial class Station
{
public Station()
{
Scenariostation = new HashSet<Scenariostation>();
}
public int Stationid { get; set; }
public string Stationname { get; set; }
public string Address { get; set; }
public NpgsqlPoint? Stationlocation { get; set; }
public int? Modifiedbyuserid { get; set; }
public DateTime? Modifieddate { get; set; }
public virtual ICollection<Scenariostation> Scenariostation { get; set; }
}
With EF Core, data update in a disconnected scenario is tricky if you don't want to make 2 database round trips.
Even though 2 database trips seems not significant, it can hamper performance if the data table has millions of records.
Also, if there are only few columns to be updated, including columns of nested child entities, Usual Approach will not work
Usual approach
public virtual void Update(T entity)
{
if (entity == null)
throw new ArgumentNullException("entity");
var returnEntity = _dbSet.Attach(entity);
_context.Entry(entity).State = EntityState.Modified;
}
But the problem here for a disconnected EF Core update, if you use this DbContext.Entry(entity).EntityState = EntityState.IsModified, all the columns will updated.
and so some columns will be updated to its default value i.e. null or default data type value.
Further more, some records of ScenarioStation won't be updated at all because the entity state will be UnChanged.
So inorder to update only the columns which are sent from the client, somehow EF Core needs to be told.
Using ChangeTracker.TrackGraph
Recently I found this DbConetxt.ChangeTracker.TrackGraph method which can be used to mark Added, UnChanged state for the entities.
difference is that with TrackGraph, you can add custom logic, as it iteratively navigates through Navigation properties of the entity.
My custom logic using TrackGraph
public virtual void UpdateThroughGraph(T entity, Dictionary<string, List<string>> columnsToBeUpdated)
{
if (entity == null)
throw new ArgumentNullException("entity");
_context.ChangeTracker.TrackGraph(entity, e =>
{
string navigationPropertyName = e.Entry.Entity.GetType().Name;
if (e.Entry.IsKeySet)
{
e.Entry.State = EntityState.Unchanged;
e.Entry.Property("Modifieddate").CurrentValue = DateTime.Now;
if (columnsToBeUpdated.ContainsKey(navigationPropertyName))
{
foreach (var property in e.Entry.Properties)
{
if (columnsToBeUpdated[e.Entry.Entity.GetType().Name].Contains(property.Metadata.Name))
{
property.IsModified = true;
}
}
}
}
else
{
e.Entry.State = EntityState.Added;
}
});
}
With this approach, I am able to easily handle only required column updates as well as new additions/modifications for any of the nested child entities and its columns.

Many to Many relationship Entity Framework creating new rows

I'm building a feedback functionality. The feedback is supposed to have multiple categories and it should be possible to find feedback based on a category so there is a many to many relationship.
I've set up the following code for this, its designed as code first.
The feeback item:
public class FeedbackItem
{
public FeedbackItem()
{
}
[Key]
public long Id { get; set; }
public virtual ICollection<FeedbackCategory> Categorys { get; set; }
//public
public string Content { get; set; }
public bool Notify { get; set; }
public string SubscriptionUserName { get; set; }
public string SubscriptionUserEmail { get; set; }
public long SubscriptionId { get; set; }
}
The feedback category:
public class FeedbackCategory
{
[Key]
public int Id { get; set; }
public string Value { get; set; }
public virtual ICollection<FeedbackItem> Feedbacks { get; set; }
}
The Database Context:
public class FeedbackContext : DbContext, IFeedbackContext
{
public FeedbackContext() : base("DefaultConnection")
{
//Database.SetInitializer<FeedbackContext>(new FeedbackContextDbInitializer());
}
public DbSet<FeedbackItem> FeedbackItems { get; set; }
public DbSet<FeedbackCategory> Categories { get; set; }
}
And the Initializer
class FeedbackContextDbInitializer : DropCreateDatabaseAlways<FeedbackContext>
{
protected override void Seed(FeedbackContext context)
{
IList<FeedbackCategory> categories = new List<FeedbackCategory>()
{
new FeedbackCategory() { Value = "Android" },
new FeedbackCategory() { Value = "API" }
};
foreach (var feedbackCategory in categories)
{
context.Categories.Add(feedbackCategory);
}
base.Seed(context);
}
}
The code above generates three tables when ran. These being FeedbackCategories, FeedbackCategoryFeedbackItems and FeedbackItems
The table FeedbackCategories is seeded with some already existing categories. The trouble comes when I try to create a new FeedbackItem that has one or more categories.
The Json i provide is the following:
{
"categorys": [
{
"$id": "1",
"Feedbacks": [],
"Id": 1,
"Value": "Android"
}
],
"subscriptionUserName": "name",
"subscriptionUserEmail": "my#email.com",
"content": "this is a feedback item",
"notify": false,
"subscriptionId": 2
}
This is converted into a FeedbackItem and handled by the following code
public class FeedbackSqlRepository : IFeedbackSqlRepository
{
public int Create(FeedbackItem feedback)
{
if (feedback == null)
{
throw new ArgumentNullException("feedback", "FeedbackItem cannot be empty.");
}
using (var context = new FeedbackContext())
{
context.FeedbackItems.Add(feedback);
return context.SaveChanges();
}
}
}
The thing that happens here is that EF creates a new FeedbackItem, a new FeedbackCategory an maps the created feedback item to the newly created feedback category in the FeedbackCategoryFeedbackItems table.
this is not the working i want
I want the following:
Create a new FeedbackItem and reverence an existing FeedbackCategory in the FeedbackCategoryFeedbackItems table. My knowledge of EF is too little to understand what's going wrong here and what to do to get the preferred working.
========
Fixed the issue with the following code inside the Create method from the FeedbackSqlRepository:
foreach (FeedbackCategory feedbackCategory in feedback.Categories)
{
context.Entry(feedbackCategory).State = EntityState.Unchanged;
}
context.FeedbackItems.Add(feedback);
return context.SaveChanges();

Entity Framework will not examine entity contents and determine for you if they are new or added.
DbSet.Add() causes ALL entities in an object graph to be marked as added and to generate inserts when you call SaveChanges().
DbSet.Attach() leaves all entities marked as Unmodified.
If some of your entities are new, some are modified and some are just references then you should use either Add() or Attach() and then manually set entity states where necessary before calling SaveChanges().
DbContext.Entry(entity).State = EntityState.Unmodified