I have a user season model in my Entity Framework Core project:
public class UserSeason
{
public int Id { get; set; }
public int SeasonId { get; set; }
public int ProfileId { get; set; }
}
SeasonId and ProfileId a foreign keys to corresponding tables.
Pair (SeasonId, ProfileId) has IsUnuque constrain on the database level.
I also have a bit list of ProfileIds. For some of them already have a UserSeason row, but some don't.
What is the most efficient way to create such rows?
My current approach is
var alreadyExistProfileIds = await _db.UserSeasons.Where(x => x.SeasonId == seasonId && profileIds.Contains(x.ProfileId)).Select(x => x.ProfileId).ToArrayAsync();
var missedProfileIds = profileIds.Except(alreadyExistProfileIds).ToArray();
_db.UserSeasons.AddRange(missedProfileIds.Select(i => new StarPassUserSeasonDbModel{ProfileId = i, SeasonId = seasonId}));
await _db.SaveChangesAsync();
However, I don't like it, because I need to load missedProfiledId into the memory and it is big number.
Is there are a way to do that in database level with Entity Framework core?
Related
The minimal project sources to reproduce the issue is here :
https://wetransfer.com/downloads/8d9325ce7117bb362bf0d61fc7c8571a20220708100401/326add
===================
This error is a classic; In layman's terms it is usually caused by a "bad" insertion when a navigation is not properly taken in account, causing a faulty Ef state somewhere.
Many solutions have been posted along the years but I fail to see how my specific scenario could cause the issue!
My schema is a many-to-many between Groups and Users. The middle entity is named GroupUser.
There's a twist : Each GroupUser has an owned entity containing extra data, DataPayload. This choice was made for versatility -- we wanted that payload to be stored in its own table.
Schema:
public class User {
public Guid Id { get; private set; }
public IList<GroupUser> GroupUsers { get; private set; } = new List<GroupUser>();
public User(Guid id) { Id = id; }
}
public class Group {
public Guid Id { get; private set; }
public Group(Guid id) { Id = id; }
}
public class GroupUser {
public Guid Id { get; private set; }
public Guid GroupId { get; private set; }
public Guid UserId { get; private set; }
// Navigation link
public Group? Group { get; private set; }
public DataPayload? Data { get; private set; }
public GroupUser(Guid groupId, Guid userId, DataPayload data) {
Id = Guid.NewGuid(); //Auto generated
UserId = userId;
GroupId = groupId;
Data = data;
}
// This extra constructor is only there to make EF happy! We do not use it.
public GroupUser(Guid id, Guid groupId, Guid userId) {
Id = id;
UserId = userId;
GroupId = groupId;
}
}
public class DataPayload {
//Note how we did not defined an explicit Id; we let EF do it as part of the "Owned entity" mechanism.
///// <summary>foreign Key to the Owner</summary>
public Guid GroupUserId { get; private set; }
public int DataValue { get; private set; }
public DataPayload(int dataValue) {
DataValue = dataValue;
}
public void SetDataValue(int dataValue) {
DataValue = dataValue;
}
}
To make it all work, we configure the navigations like this :
// --------- Users ---------
builder
.ToTable("Users")
.HasKey(u => u.Id);
// --------- Groups ---------
builder
.ToTable("Groups")
.HasKey(g => g.Id);
// --------- GroupUsers ---------
builder
.ToTable("GroupUsers")
.HasKey(gu => gu.Id);
builder
.HasOne<User>() //No navigation needed
.WithMany(u => u.GroupUsers)
.HasForeignKey(gu => gu.UserId);
builder
.HasOne<Group>(gu => gu.Group) //Here, we did define a navigation
.WithMany()
.HasForeignKey(gu => gu.GroupId);
builder
.OwnsOne(gu => gu.Data,
navBuilder => {
navBuilder.ToTable("PayloadDatas");
navBuilder.Property<Guid>("Id"); //Note: Without this EF would try to use 'int'
navBuilder.HasKey("Id");
//Configure an explicit foreign key to the owner. It will make our life easier in our Unit Tests
navBuilder.WithOwner().HasForeignKey(d => d.GroupUserId);
}
);
//.OnDelete(DeleteBehavior.Cascade) // Not needed (default behaviour for an owned entity)
Now, you know how everything is defined.
Basic setup : works!
var group = new Group(groupId);
await dbContext.Groups.AddAsync(group);
await dbContext.SaveChangesAsync();
var user = new User(userId);
await dbContext.Users.AddAsync(user);
await dbContext.SaveChangesAsync();
Follow-up scenario : fails!
var groupUser = new GroupUser(groupId, userId, new DataPayload(dataValue: 777777));
user.GroupUsers.Add(groupUser);
await dbContext.SaveChangesAsync(); // Crash happens here!!!
Error:
Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException : The database operation was expected to affect 1 row(s), but actually affected 0 row(s); data may have been modified or deleted since entities were loaded.
I suspect that EF gets confused by the addition of two entities at once, where it has to compute some Ids itself : the new GroupUser and the DataPayload it contains. I'm not sure how it's supposed to know that it needs to give an Id to the GroupUser first and then use that Id as the foreign key in PayloadData. But that's just me; it might or might not be related to the issue.
But what do I need to change?
The mistake was in GroupUser's id-less constructor:
Id = Guid.NewGuid();
The code needs to let EF manage the keys when it comes to owned entities such as DataPayload which rely on a foreign key (GroupUserId) that's still in the making at the time of saving.
If you set a key value (Guid.NewGuid()) yourself, then EF gets confused between:
linking the new DataPayload entity to the GroupUser entity where you've shoehorned an Id value,
OR
just expecting an empty value (foreign key) and setting all the keys (both the GroupUser's Id and DataPayload's GroupUserId) itself.
All in all, EF feels like you announced that you were about to let it create 1 entity, but you've pulled the rug under its feet and done it yourself, so it ends up with 0 entity to work with. Hence the error message.
It should have been :
Id = Guid.Empty;
With Guid.Empty, EF clearly identifies that this entity is new and that has to be the same one as the one you told it to create and link to the new PayloadData -- that is, the instance that you've set in GroupUser.Data.
So I'm using Entity Framework Core to build a database of Guilds (Another name for Discord Servers) and Users, with the Discord.NET Library. Each Guild has many users, and each user can be in many guilds. First time using EF and I'm having some teething issues. The two classes are:
public class Guild
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
public ulong Snowflake { get; set; }
public DateTimeOffset CreatedAt { get; set; }
public string Name { get; set; }
public ICollection<User> Users { get; set; }
}
public class User
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
public ulong Snowflake { get; set; }
public string Username { get; set; }
public ushort DiscriminatorValue { get; set; }
public string AvatarId { get; set; }
public ICollection<Guild> Guilds { get; set; }
public DateTimeOffset CreatedAt { get; set; }
}
With the goal of having 3 tables: Guild, Users, and GuildUsers. This is my current function for getting the guilds:
using var context = new AutomataContext();
var discordGuilds = this.client.Guilds.ToList();
var dbGuilds = context.Guilds;
List<Guild> internalGuilds = discordGuilds.Select(g => new Guild
{
Snowflake = g.Id,
Name = g.Name,
CreatedAt = g.CreatedAt,
Users = g.Users.Select(gu => new User
{
Id = context.Users.AsNoTracking().FirstOrDefault(u => u.Snowflake == gu.Id)?.Id ?? default(int),
}).ToList(),
}).ToList();
// Upsert Guilds to db set.
foreach (var guild in internalGuilds)
{
var existingDbGuild = dbGuilds.AsNoTracking().FirstOrDefault(g => g.Snowflake == guild.Snowflake);
if (existingDbGuild != null)
{
guild.Id = existingDbGuild.Id;
dbGuilds.Update(guild); // Hits the second Update here and crashes
}
else
{
dbGuilds.Add(guild);
}
}
await context.SaveChangesAsync();
I should note, a 'snowflake' is a unique ID that discord uses, but I wanted to keep my own unique ID for each table.
High level overview, guilds are collected into Discord.NET models. These are then transformed into internalGuilds (my guild class, which includes the list of users). Each of these is looped through and upserted to the database.
The issue arises in the second guild loop, where an error is thrown in the "Update" that a User ID is already being tracked (Inside the guild). So the nested ID is already being tracked? Not sure what's going on here, any help would be appreciated. Thanks.
This exception is most likely occurring because you are loading Users without tracking then looping through and potentially trying to update or insert guilds /w the same user reference, especially using the Update method.
I would suggest removing the use of AsNoTracking. Working with detached entity references via AsNoTracking is more of a performance tweak for when reading large amounts of data. You can pre-fetch all of the User references by their snowflake:
using (var context = new AutomataContext())
{
var discordGuilds = this.client.Guilds.ToList();
// Get the user snowflakes from the guilds, and pre-fetch them.
var userSnowflakes = discordGuilds.SelectMany(g => g.Users.Select(u => u.Id)).ToList();
var users = await context.Users
.Where(x => userSnowflakes.Contains(x.Snowflake))
.ToListAsync();
// We need to add references for any New user snowflakes.
var existingSnowflakes = users.Select(x => x.Snowflake).ToList();
// If more detail is needed for new user records, it will need to be fetched from the passed in Guild.User.
var newUsers = userSnowflakes.Except(existingSnowFlakes)
.Select(x => new User { SnowflakeId = x }).ToList();
if(newUsers.Any())
users.AddRange(newUsers);
List<Guild> internalGuilds = discordGuilds.Select(g => new Guild
{
Snowflake = g.Id,
Name = g.Name,
CreatedAt = g.CreatedAt,
Users = g.Users
.Select(gu => users.Single(u => u.Snowflake == gu.Id))
.ToList(),
}).ToList(),
// Upsert Guilds to db set.
foreach (var guild in internalGuilds)
{
var existingGuildId = context.Guilds
.Where(x => x.Snowflake == guild.Snowflake)
.Select(x => x.Id)
.SingleOrDefault();
if (existingGuildId != 0)
{
guild.Id = existingGuildId;
dbGuilds.Update(guild);
}
else
{
dbGuilds.Add(guild);
}
}
await context.SaveChangesAsync();
This should help ensure that the User references for existing users are pointing at the same instances, whether existing users or new user references that will be associated to the DbContext when first referenced.
Ultimately I don't recommend using Update for "Upsert" scenarios, instead since the Db Record needs to be fetched anyways, updating values on the fetched instance or inserting a new one. Update will want to send all fields from an entity to the database each time, rather than just sending what has changed. It means enforcing a bit more control over what can possibly be changed vs. what should not be.
I'm trying to use bulkinsert from EFCore.BulkExtensions in .NET Core 2.1 with one to one relationships. I have 2 Database Entities
public partial class Entity1
{
public int ID { get; set; }
public virtual Entity2 Entity2Obj { get; set; }
}
public partial class Entity2
{
public int ID { get; set; }
public int Entity1_ID{ get; set; }
public virtual Entity1 Entity1Obj { get; set; }
}
Then I insert a list of Entity1 Items
_context.BulkInsert(Entity1ItemsList);
This only works and inserts Entity1 objects in my database. The associated Entity2 items are not inserted. Is there a way to achieve that?
The navigation properties are automatically created during Scaffold-DbContext (database first)
entity.HasOne(d => d.Entity1 )
.WithOne(p => p.Entity2)
.HasForeignKey<Entity2>(d => d.Entity1_ID)
.OnDelete(DeleteBehavior.ClientSetNull)
.HasConstraintName("FK_Entity1_Entity2");
After searching around I think I found the solution. First of all set the following config
var bulkConfig = new BulkConfig()
{
SetOutputIdentity = true,
PreserveInsertOrder = true
};
Run the initial bulkconfig to insert the Entities1List and get the unique IDs from SQL Server
_context.BulkInsert(Entity1ItemsList, bulkConfig);
Now the unique primary keys are assigned so to set their values to their related entities use
foreach (var item in Entity1ItemsList)
{
item.Entity2Obj.Entity1_ID = item.ID;
}
This will set the correct Foreign Key values. Then I BulkInsert the Entities2List
var Entities2List = Entity1ItemsList.Select(u => u.Entity2Obj).ToList();
_context.BulkInsert(Entities2List);
I was hoping for something simpler by using only one BulkInsert which will automatically add all the navigation properties instead of 2. But this works and it's fast.
This is probably what you're looking for since you also want to insert child properties.
context.BulkInsert(Entity1ItemsList, options => options.IncludeGraph = true);
You can read further here:
I have a Windows Forms application and want to get information from multiple tables of my database in the single dataGridView area.
I am trying to do it without SqlConnection and SqlDataAdapter, so the connection will be proceed through Entity Framework
DBContext db = new DBContext
Is it possible to do so?
I have three tables:
User:
UserID,
Name
System:
SysID,
SysType
Activities:
ActivID,
UserID (FK)
SysID (FK)
Date,
Version,
Changes
My code:
using DBContext db = new DBCntext())
{
dataGridView.DataSource = db.Table.ToList<Table>();
}
So I would write in case of only one table, but would it be possible to concatenate two or more tables without do it right in the database?
At the end I want to get User+System+Activities tables within one dataGridView.
If your class is properly defined like this:
public class Activity
{
public int ActivityID { get; set; }
public string Version{ get; set; }
public DateTime Date { get; set; }
public string Changes { get; set; }
public int UserID { get; set; }
public User User { get; set; } // navigation property for User
public int SystemID { get; set; }
public System System { get; set; } // navigation property for System
}
Now you can write a query like:
using DBContext db = new DBCntext())
{
dataGridView.DataSource = db.Activitys
.Include(a => a.User)
.Include(a => a.System)
.Select(a => new MyModel {
MyModel.ActivityID = a.ActivityID,
MyModel.Version= a.Version,
MyModel.Date = a.Date ,
MyModel.Changes = a.Changes,
MyModel.UserName = a.User.Name,
MyModel.SysType= a.System.SysType
})
.ToList();
}
If you have not defined navigation properties, then you could use a LINQ Join.
I'm using EF Core 2.1, database first approach. I'm trying to Include() a foreign key entity when fetching my target entity collection, but something strange is happening.
The entity structure is Job -> JobStatus. I'm fetching some Job entities, and want to include the Job's JobStatus foreign key property. The issue is that the JobStatus entity has a ICollection[Job] property that is populating every single Job from the database. This is causing the payload to be gigabytes in size.
When I include the JobStatus on the Job, I'd like to satisfy one of the following solutions. I'm also open to other solutions or workarounds I haven't thought of.
*how can I prevent the JobStatus' ICollection property from populating?
*Or can I prevent Entity Framework from generating that property in the first place?
I've already explored Ignoring the ReferenceLoopHandling
services.AddMvc().AddJsonOptions(options => {
options.SerializerSettings.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore;
});
Here are the entities, automatically generated by Entity Framework.
public partial class Job
{
public long Id { get; set; }
public long StatusId { get; set; }
public JobStatus Status { get; set; }
}
public partial class JobStatus
{
public JobStatus()
{
Job = new HashSet<Job>();
}
public long Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public ICollection<Job> Job { get; set; }
}
Example code that is causing the problem
var jobs = _context.Set<Job>()
.Where(job => job.Id == 1)
.Include(job => job.Status);
One way to avoid the Job collection from being populated is to explicitly select the columns that you want returned, either through a defined or anonymous type:
var jobs = _context.Set<Job>()
.Where(job => job.Id == 1)
.Include(job => job.Status)
.Select(job => new
{
Id = job.Id,
StatusName = job.Status.Name
});
Add a "virtual" keyword. Any virtual ICollections will be lazy-loaded unless you specifically mark them otherwise.
public virtual ICollection<Job> Job { get; set; }