We Keep Coding

Data Context error when saving data to database .NET 5.0 [duplicate]

This question already has answers here:
DbContextOptionsBuilder.EnableSensitiveDataLogging Doesn't Do Anything
(3 answers)
Closed last year.
I have a little problem saving data in the database. In a given method, I make a request to an API, serialize the data, and try to save to the database, as shown in the image, but I end up getting an error referring to the application's DataContext, saying:
System.InvalidOperationException: The instance of entity type 'Launch ' cannot be tracked because another instance with the same key value for {'id'} is already being tracked. When attaching existing entities, ensure that only one entity instance with a given key value is attached.
Consider using 'DbContextOptionsBuilder.EnableSensitiveDataLogging' to see the conflicting key values.
Any tips on how to resolve this issue?
Save Database Method
using System.Collections.Generic;
using System.Threading.Tasks;
using Newtonsoft.Json;
using RestSharp;
using SpaceFlightApiChallenge.Data.Context;
using SpaceFlightApiChallenge.Domain.Entities;
namespace SpaceFlightApiChallenge.Data
{
public class DatabaseImport
{
private readonly SpaceFlightContext _context;
public DatabaseImport(SpaceFlightContext context)
{
_context = context;
}
public async Task ImportData()
{
string url = "https://api.spaceflightnewsapi.net/v3/";
var client = new RestClient(url);
var apiRequest = new RestRequest("articles", Method.Get);
apiRequest.AddHeader("Accept", "application/json");
var response = await client.ExecuteAsync(apiRequest);
var content = response.Content;
var articles = JsonConvert.DeserializeObject<List<Article>>(content);
_context.Articles.AddRange(articles);
await _context.SaveChangesAsync();
}
}
}
Article Class
using System.Collections.Generic;
namespace SpaceFlightApiChallenge.Domain.Entities
{
public class Article
{
public int id { get; set; }
public bool featured { get; set; }
public string title { get; set; }
public string url { get; set; }
public string imageUrl { get; set; }
public string newsSite { get; set; }
public string summary { get; set; }
public string publishedAt { get; set; }
public List<Launch> launches { get; set; }
public List<Event> events { get; set; }
}
}
Launch Class
namespace SpaceFlightApiChallenge.Domain.Entities
{
public class Launch
{
public string id { get; set; }
public string provider { get; set; }
}
}
Event Class
namespace SpaceFlightApiChallenge.Domain.Entities
{
public class Event
{
public string id { get; set; }
public string provider { get; set; }
}
}
EDIT: I believe the whole problem is with the id's. All objects that the external API returns me (Article, event, launch) have their own id, but when they enter the database, EF Core wants to assign a new id, due to the identity property. I don't need to change the id's, the data must be saved in the database as it comes from the API, so that later I can consult them. Here's the API link from where I'm getting the data: https://api.spaceflightnewsapi.net/v3/documentation

Maybe your list contains duplicates. EF does not allow to track entites with same key. If you have that error, try
_context.Entry(<entity>).State = EntityState.Detached;

I can see in your Article class
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public int id { get; set; }
Try to change it
[Key]
public int id { get; set; }
I think the promblem is a primary key, but for testing pourposes try to add one by one , to see what which item has the problem.Try it using a debugger, and see is the first item will get an exception or not
foreach(var article in articles)
{
context.Articles.Add (article);
context.SaveChanges();
}

I'd say it's because the external service is giving you repeated IDs, and your JSON deserialize routine is just making new objects with the same ID and attaching them to the object graph. This means EF ends up receiving something like:
{
id: 13479,
title: Engineers taking more time ...
events: [ { id: 482 } ] //object at memory address 1234
},
{
id: 13477,
title: NASA takes break in JWST deployment ...
events: [ { id: 482 } ] //object at memory address 4567
}
Two different Event objects with the same ID; EF would expect those to be the same object instance
Whether you can get the JSON deser routine to fix this up (by e.g. keeping a dictionary of seen-before objects based on ID) without a custom deser, I don't know*..
..but it should be possible to fix the objects up so the graph entities are unique:
var events = articles.SelectMany(a => a.Events).ToLookup(e => e.Id);
var launches = articles.SelectMany(a => a.Launches).ToLookup(l=> l.Id);
foreach(var a in articles){
a.Events = a.Events.Select(e => events[e.Id].First()).ToList();
a.Launches = a.Launches.Select(l => launches[l.Id].First()).ToList();
}
Turning the events into a Lookup on id (something like a Dictionary<int, List<Event>>) will group up all the different event objects into enumerations accessible by id. This can then be used to recreate an event list where all the events within refer to a common Event. If we enumerate the event list, pull the id out of the Event we find in the list and use that to lookup the First event in the lookup, it points everything to the same event instance for that id
It means you have a lookup called events that is:
events -> lookup of [ { id:482 /*mem address 1234*/ }, {id: 482 /*mem address 4567*/ } ]
And where you once had an Article 13477 whose Events had one Event (with id 482, at memory address 4567), that list is replaced with an event list whose event is again 482, but this time pointing to the Event at memory address 1234
{
id: 13479,
title: Engineers taking more time ...
events: [ { id: 482 } ] //object at memory address 1234: first element in lookup 482
},
{
id: 13477,
title: NASA takes break in JWST deployment ...
events: [ { id: 482 } ] //object at memory address 1234: first element in lookup 482
}
Now.. I hope that these events aren't repeated across invocations of the download (i.e. next week there will be another different article that also has id 482), because if they do you'll probably get a primary key violation.
The solution to that one is probably going to be to extend the loop that is patching up the object graph so that it first looks up in the DB the event with 482, and put that one into the Article (and then fall back to a new event 482 if the DB doesn't know it)
Note; you can use a [JsonProperty("json name here")] attribute to declare the name in json vs the name in C#, so you don't have to have C# that violates normal PascalCase convention for its props
Here's a bunch of code generated from http://quicktype.io (no affiliation) that is C# naming convention square, and deser's your JSON. You can blend it with your entities if you want (my code block above uses PascalCase syntax derived from this set of deser classes, or if you don't plan to change your deser classes you can adjust the code above to camel case) :
// <auto-generated />
//
// To parse this JSON data, add NuGet 'Newtonsoft.Json' then do:
//
// using SpaceX;
//
// var article = Article.FromJson(jsonString);
namespace SpaceX
{
using System;
using System.Collections.Generic;
using System.Globalization;
using Newtonsoft.Json;
using Newtonsoft.Json.Converters;
public partial class Article
{
[JsonProperty("id")]
public long Id { get; set; }
[JsonProperty("title")]
public string Title { get; set; }
[JsonProperty("url")]
public Uri Url { get; set; }
[JsonProperty("imageUrl")]
public Uri ImageUrl { get; set; }
[JsonProperty("newsSite")]
public string NewsSite { get; set; }
[JsonProperty("summary")]
public string Summary { get; set; }
[JsonProperty("publishedAt")]
public DateTimeOffset PublishedAt { get; set; }
[JsonProperty("updatedAt")]
public DateTimeOffset UpdatedAt { get; set; }
[JsonProperty("featured")]
public bool Featured { get; set; }
[JsonProperty("launches")]
public List<Launch> Launches { get; set; }
[JsonProperty("events")]
public List<Event> Events { get; set; }
}
public partial class Event
{
[JsonProperty("id")]
public long Id { get; set; }
[JsonProperty("provider")]
public string Provider { get; set; }
}
public partial class Launch
{
[JsonProperty("id")]
public Guid Id { get; set; }
[JsonProperty("provider")]
public string Provider { get; set; }
}
public partial class Article
{
public static List<Article> FromJson(string json) => JsonConvert.DeserializeObject<List<Article>>(json, SpaceX.Converter.Settings);
}
public static class Serialize
{
public static string ToJson(this List<Article> self) => JsonConvert.SerializeObject(self, SpaceX.Converter.Settings);
}
internal static class Converter
{
public static readonly JsonSerializerSettings Settings = new JsonSerializerSettings
{
MetadataPropertyHandling = MetadataPropertyHandling.Ignore,
DateParseHandling = DateParseHandling.None,
Converters =
{
new IsoDateTimeConverter { DateTimeStyles = DateTimeStyles.AssumeUniversal }
},
};
}
}
* I expect it would be possible but I've never deser'd directly to a DB entity (nor would i recommend to do so; I'd generally have one set of Dtos for API shuttling and another set of entities for DB work)
Edit:
Actually, try this, which is a slight rethink of the above:
//your code
var articles = JsonConvert.DeserializeObject<List<Article>>(content);
//my code
foreach(var a in articles){
a.Events = a.Events.Select(e => _context.Events.Find(e.Id) ?? e).ToList();
a.Launches = a.Launches.Select(l => _context.Launches.Find(l.Id) ?? l).ToList();
_context.Articles.Add(a);
}
//your code
//_context.Articles.AddRange(articles); - not needed now
Ok, so that above is what I mean with the tweaks : after you deserialize the json you have articles with repeated same-id events and launches. For each article we'll rebuild the events. This code also incorporates the idea of "look up the event by id in the db first" - using Find should be fairly quick as it will first look locally for the entity- if it has been downloaded before then it is returned from local cache in the context. If Find doesn't get the item locally it hits the db, if that also returns null then we use the event/launch that we're currently processing. This should then become the event that Find will find next time (because we add to the context soon after) which resolves the duplicate entity problem by reusing the one the context knows about
If that doesn't work as expected, this one should:
var events = articles.SelectMany(a => a.Events).ToLookup(e => e.Id);
var launches = articles.SelectMany(a => a.Launches).ToLookup(l=> l.Id);
foreach(var a in articles){
a.Events = a.Events.Select(e => _context.Events.Find(e.Id) ?? events[e.Id].First()).ToList();
a.Launches = a.Launches.Select(l => _context.Launches.Find(l.Id) ?? launches[l.Id].First()).ToList();
_context.Articles.Add(a);
}
//your code
//_context.Articles.AddRange(articles); - not needed now

How do you construct an Entity class to fetch another Entity from repository?

This is a C# Question, using .NET framework built on Asp.NET Boilerplate.
Again, to re-emphasis the question being asked is "HOW...", so if an answer that was provided was a url link or a descriptive explanation on how something was supposed to be done, i would very much appreciate it. (Dont answer questions on how to tie shoelace by showing a picture of a tied shoe, nor do you answer "how to fish" by showing a recording of someone fishing...)
Since the question is pretty basic (i don't need to rephrase/repeat the header again), i'll give an example.
If i have a Forum service, and i create a class to load a Thread. Inside that thread class should be some sort of collection, array, list, or even a dbset of Post that is pulled on construct.
[Table("Thread", Schema = "dbo")]
public class ThreadModel
{
[Key]
public long Id { get; set; }
public string Title { get; set; }
//Idea 1
//Value should automatically be pulled and cached the moment class connects to database
public Post[] Posts { get; set; }
//Idea 2
//Post has a constructor to return all post that matches a thread id. While new tag keeps the return value constantly refreshed.
public Post[] Posts { get { return new Post(this.Id) } }
//Idea 3
//Not sure how collection is supposed to work. Does it automatically just pull or will i need to make a method to request?
public virtual ICollection<Post> Posts { get; set; }
//Example constructor
//When connected to database key-value pairs that match database labels will automatically get stored in class
protected ThreadModel()
{
//Idea 1-A
//Should be a value of null or empty if database yields no results
Posts = new Post();
}
public ThreadModel(int threadid) : this()
{
//Idea 1-A
Id = threadid;
//new Post => returns all posts in db
//Posts default value is all post in db
Posts = Posts.Select(post => post.threadid == this.id)
//Idea 3-A
Posts = Posts.Get(post => post.threadid == this.id)
//Idea 4
Posts = new Posts().GetThread(threadid);
}
}
Side questions
If all entities are created by inheriting Entity then at what point am i exposed to EntityFramework and DbContext?
I love this example here, submitted by a user as they attempt to connect ABP to their database. But their example doesn't show parent/child resources. I'm unable to find the guide they used to create that, and how it relates back to using ABP to fetch EntityFramework's DbContext example
How does this work? I'm unable to find instructions or explanation for this? (What am i to enter into google to get answers on these mechanics?)
[Table("AbpItems")]
public class Item : Entity
{
[ForeignKey("PostId")]
public Post Post { get; set; }
public int PostId { get; set; }
}
How does this integrate into/with abp's EntityFramework?
Where am i supposed to be creating my Database Table/Class? The project follows the Core.csproj, Application.csproj, and EntityFramework.csproj assembly layout. But it seems like every example is creating the classes at different stages or locations of the solution.

use GetAllIncluding. See https://github.com/aspnetboilerplate/aspnetboilerplate/issues/2617
Here's a complete solution ;
namespace EbicogluSoftware.Forum.Threads
{
[Table("Threads")]
public class Thread : FullAuditedEntity
{
[Required]
[StringLength(500)]
public virtual string Title { get; set; }
[Required]
[StringLength(2000)]
public virtual string Text { get; set; }
public virtual List<Post> Posts { get; set; }
public Thread()
{
Posts = new List<Post>();
}
}
[Table("Posts")]
public class Post : FullAuditedEntity
{
[Required]
[StringLength(2000)]
public virtual string Text { get; set; }
}
public class ThreadDto
{
public string Title { get; set; }
public string Text { get; set; }
public List<PostDto> Posts { get; set; }
public ThreadDto()
{
Posts = new List<PostDto>();
}
}
public class PostDto
{
public string Text { get; set; }
}
public class ThreadAppService : IApplicationService
{
private readonly IRepository<Thread> _threadRepository;
public ThreadAppService(IRepository<Thread> threadRepository)
{
_threadRepository = threadRepository;
}
public async Task<List<TenantListDto>> GetThreads()
{
var threads = await _threadRepository.GetAllIncluding(x => x.Posts).ToListAsync();
return threads.MapTo<List<TenantListDto>>();
}
}
}
Where am i supposed to be creating my Database Table/Class?
You can create them in YourProject.Core.proj

Is it possible to have extra (ignored) properties in C#?

I have a repository for a DocumentDb database. My documents all have a set of common properties so all documents implement the IDocumentEntity interface.
public interface IDocumentEntity {
[JsonProperty("id")]
Guid Id { get; set; }
[JsonProperty("documentClassification")]
DocumentClassification DocumentClassification { get; set; }
}
public class KnownDocument : IDocumentEntity {
[JsonProperty("id")]
Guid Id { get; set; }
[JsonProperty("documentClassification")]
DocumentClassification DocumentClassification { get; set; }
[JsonProperty("knownProperty")]
string KnownProperty { get; set; }
}
public class BaseDocumentRepository<T> where T : IDocumentEntity {
public Set(T entity) {
// ... stuff
}
}
This works fine with a KnownDocument where I know all of the properties. But, of course, what's great about a Document Db is that I don't need to know all of the properties (and in many cases I won't).
So my client submits something like this-
{unknownProperty1: 1, unknownProperty2: 2}
And I want to upsert this using my document repository.
public OtherDocumentService() {
_otherDocumentService = new OtherDocumentRepository();
}
public UpsertDocument(dynamic entity) {
entity.id = new Guid();
entity.documentClassification = DocumentClassification.Other;
_otherDocumentRepository.Set(entity);
}
But I get an InvalidCastException from dynamic to IDocumentEntity. I assume it's because of the extra properties that exist on the dynamic object but not on the IDocumentEntity interface?
What I'm trying to do is leave my document entities open to be dynamic, but rely on a few properties being there to maintain them.

Entity parameter passed to the UpsertDocument should explicitly implement IDocumentEntity in order do make the code works, it is not enough just have a Id property.
Some options:
1) Proxy may be applied:
public class ProxyDocumentEntity : IDocumentEntity
{
public dynamic Content { get; private set; }
public ProxyDocumentEntity(dynamic #content)
{
Content = #content;
}
public Guid Id
{
get { return Content.Id; }
set { Content.Id = value; }
}
}
... using
public void UpsertDocument(dynamic entity)
{
entity.Id = new Guid();
repo.Set(new ProxyDocumentEntity(entity));
}
The stored document will have nested Object property, which may be not acceptable
2)There is a lib https://github.com/ekonbenefits/impromptu-interface which creates a proxy dynamically
and does not make extra property like solution above.
Drawback will be in performance.
Technically it could be 2 methods:
public void UpsertDocument(IDocumentEntity entity){...}
public void UpsertDocument(dynamic entity){...}
so the first (fast) will work for the objects which implement IDocumentEntity and second(slow) for the rest of the objects.
But this is a speculation a bit , as I dunno the details of the whole code base of the project you have.

If you have some flexibility as to how to name those dynamic properties, you could stuff them into a Dictionary property on your object:
public Dictionary<string, dynamic> extra { get; set; }

Converting a string into a List and then using Entity Framework to insert into Database

I am converting a string into a list and then trying to use entity framework to insert it into a DB. The issue that I am having is that I don't know how to save the changes to the DB.
This is the code that I am trying to use and is where the string is converted to a list:
if (intCounter == 0)
{
return JsonConvert.DeserializeObject<List<foo>>(jsonString).Cast<T>().ToList();
}
Then in a seperate class below.
ConvertJson.Convert<CouncilEvent>(strResponseJSONContent, intCounter);
The Entity Framework Model that I am trying to use for the list.
namespace foo.Models
{
public partial class foo
{
public foo()
{
this.EventDates = new List<EventDate>();
}
public System.Guid foo_PK { get; set; }
public string EntityID { get; set; }
public string Address1 { get; set; }
public string Address2 { get; set; }
public string Address3 { get; set; }
public bool Adult { get; set; }
}
}
The class foo contains properties that match those in the string.
It is this foo that I am then trying to insert into the DB. foo is also part of my Entity Framework model.
I have never used a list in this situation before and I thought it would just be a matter of using db.SaveChanges() but that doesn't seem to work. Where would I place the necessary lines of code such as using (db = new contextFoo) and db.SaveChanges(). Also do I need to add the items? I haven't because I thought I was already adding them to the class and therefore didn't need to do this manually?

db.SaveChanges() will only 'update' your database to what was changed. So, you need to add something to the database, and then call SaveChanges() for it to work.

You can loop the list to add the objects to the context, then call SaveChanges()...
var councilEvents = ConvertJson.Convert<CouncilEvent>(strResponseJSONContent, intCounter);
using (var db = new contextFoo())
{
foreach (var councilEvent in councilEvents)
{
db.CouncilEvents.Add(councilEvent);
}
db.SaveChanges();
}

Many to Many relationship Entity Framework creating new rows

I'm building a feedback functionality. The feedback is supposed to have multiple categories and it should be possible to find feedback based on a category so there is a many to many relationship.
I've set up the following code for this, its designed as code first.
The feeback item:
public class FeedbackItem
{
public FeedbackItem()
{
}
[Key]
public long Id { get; set; }
public virtual ICollection<FeedbackCategory> Categorys { get; set; }
//public
public string Content { get; set; }
public bool Notify { get; set; }
public string SubscriptionUserName { get; set; }
public string SubscriptionUserEmail { get; set; }
public long SubscriptionId { get; set; }
}
The feedback category:
public class FeedbackCategory
{
[Key]
public int Id { get; set; }
public string Value { get; set; }
public virtual ICollection<FeedbackItem> Feedbacks { get; set; }
}
The Database Context:
public class FeedbackContext : DbContext, IFeedbackContext
{
public FeedbackContext() : base("DefaultConnection")
{
//Database.SetInitializer<FeedbackContext>(new FeedbackContextDbInitializer());
}
public DbSet<FeedbackItem> FeedbackItems { get; set; }
public DbSet<FeedbackCategory> Categories { get; set; }
}
And the Initializer
class FeedbackContextDbInitializer : DropCreateDatabaseAlways<FeedbackContext>
{
protected override void Seed(FeedbackContext context)
{
IList<FeedbackCategory> categories = new List<FeedbackCategory>()
{
new FeedbackCategory() { Value = "Android" },
new FeedbackCategory() { Value = "API" }
};
foreach (var feedbackCategory in categories)
{
context.Categories.Add(feedbackCategory);
}
base.Seed(context);
}
}
The code above generates three tables when ran. These being FeedbackCategories, FeedbackCategoryFeedbackItems and FeedbackItems
The table FeedbackCategories is seeded with some already existing categories. The trouble comes when I try to create a new FeedbackItem that has one or more categories.
The Json i provide is the following:
{
"categorys": [
{
"$id": "1",
"Feedbacks": [],
"Id": 1,
"Value": "Android"
}
],
"subscriptionUserName": "name",
"subscriptionUserEmail": "my#email.com",
"content": "this is a feedback item",
"notify": false,
"subscriptionId": 2
}
This is converted into a FeedbackItem and handled by the following code
public class FeedbackSqlRepository : IFeedbackSqlRepository
{
public int Create(FeedbackItem feedback)
{
if (feedback == null)
{
throw new ArgumentNullException("feedback", "FeedbackItem cannot be empty.");
}
using (var context = new FeedbackContext())
{
context.FeedbackItems.Add(feedback);
return context.SaveChanges();
}
}
}
The thing that happens here is that EF creates a new FeedbackItem, a new FeedbackCategory an maps the created feedback item to the newly created feedback category in the FeedbackCategoryFeedbackItems table.
this is not the working i want
I want the following:
Create a new FeedbackItem and reverence an existing FeedbackCategory in the FeedbackCategoryFeedbackItems table. My knowledge of EF is too little to understand what's going wrong here and what to do to get the preferred working.
========
Fixed the issue with the following code inside the Create method from the FeedbackSqlRepository:
foreach (FeedbackCategory feedbackCategory in feedback.Categories)
{
context.Entry(feedbackCategory).State = EntityState.Unchanged;
}
context.FeedbackItems.Add(feedback);
return context.SaveChanges();

Entity Framework will not examine entity contents and determine for you if they are new or added.
DbSet.Add() causes ALL entities in an object graph to be marked as added and to generate inserts when you call SaveChanges().
DbSet.Attach() leaves all entities marked as Unmodified.
If some of your entities are new, some are modified and some are just references then you should use either Add() or Attach() and then manually set entity states where necessary before calling SaveChanges().
DbContext.Entry(entity).State = EntityState.Unmodified