So I'm using Entity Framework Core to build a database of Guilds (Another name for Discord Servers) and Users, with the Discord.NET Library. Each Guild has many users, and each user can be in many guilds. First time using EF and I'm having some teething issues. The two classes are:
public class Guild
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
public ulong Snowflake { get; set; }
public DateTimeOffset CreatedAt { get; set; }
public string Name { get; set; }
public ICollection<User> Users { get; set; }
}
public class User
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
public ulong Snowflake { get; set; }
public string Username { get; set; }
public ushort DiscriminatorValue { get; set; }
public string AvatarId { get; set; }
public ICollection<Guild> Guilds { get; set; }
public DateTimeOffset CreatedAt { get; set; }
}
With the goal of having 3 tables: Guild, Users, and GuildUsers. This is my current function for getting the guilds:
using var context = new AutomataContext();
var discordGuilds = this.client.Guilds.ToList();
var dbGuilds = context.Guilds;
List<Guild> internalGuilds = discordGuilds.Select(g => new Guild
{
Snowflake = g.Id,
Name = g.Name,
CreatedAt = g.CreatedAt,
Users = g.Users.Select(gu => new User
{
Id = context.Users.AsNoTracking().FirstOrDefault(u => u.Snowflake == gu.Id)?.Id ?? default(int),
}).ToList(),
}).ToList();
// Upsert Guilds to db set.
foreach (var guild in internalGuilds)
{
var existingDbGuild = dbGuilds.AsNoTracking().FirstOrDefault(g => g.Snowflake == guild.Snowflake);
if (existingDbGuild != null)
{
guild.Id = existingDbGuild.Id;
dbGuilds.Update(guild); // Hits the second Update here and crashes
}
else
{
dbGuilds.Add(guild);
}
}
await context.SaveChangesAsync();
I should note, a 'snowflake' is a unique ID that discord uses, but I wanted to keep my own unique ID for each table.
High level overview, guilds are collected into Discord.NET models. These are then transformed into internalGuilds (my guild class, which includes the list of users). Each of these is looped through and upserted to the database.
The issue arises in the second guild loop, where an error is thrown in the "Update" that a User ID is already being tracked (Inside the guild). So the nested ID is already being tracked? Not sure what's going on here, any help would be appreciated. Thanks.
This exception is most likely occurring because you are loading Users without tracking then looping through and potentially trying to update or insert guilds /w the same user reference, especially using the Update method.
I would suggest removing the use of AsNoTracking. Working with detached entity references via AsNoTracking is more of a performance tweak for when reading large amounts of data. You can pre-fetch all of the User references by their snowflake:
using (var context = new AutomataContext())
{
var discordGuilds = this.client.Guilds.ToList();
// Get the user snowflakes from the guilds, and pre-fetch them.
var userSnowflakes = discordGuilds.SelectMany(g => g.Users.Select(u => u.Id)).ToList();
var users = await context.Users
.Where(x => userSnowflakes.Contains(x.Snowflake))
.ToListAsync();
// We need to add references for any New user snowflakes.
var existingSnowflakes = users.Select(x => x.Snowflake).ToList();
// If more detail is needed for new user records, it will need to be fetched from the passed in Guild.User.
var newUsers = userSnowflakes.Except(existingSnowFlakes)
.Select(x => new User { SnowflakeId = x }).ToList();
if(newUsers.Any())
users.AddRange(newUsers);
List<Guild> internalGuilds = discordGuilds.Select(g => new Guild
{
Snowflake = g.Id,
Name = g.Name,
CreatedAt = g.CreatedAt,
Users = g.Users
.Select(gu => users.Single(u => u.Snowflake == gu.Id))
.ToList(),
}).ToList(),
// Upsert Guilds to db set.
foreach (var guild in internalGuilds)
{
var existingGuildId = context.Guilds
.Where(x => x.Snowflake == guild.Snowflake)
.Select(x => x.Id)
.SingleOrDefault();
if (existingGuildId != 0)
{
guild.Id = existingGuildId;
dbGuilds.Update(guild);
}
else
{
dbGuilds.Add(guild);
}
}
await context.SaveChangesAsync();
This should help ensure that the User references for existing users are pointing at the same instances, whether existing users or new user references that will be associated to the DbContext when first referenced.
Ultimately I don't recommend using Update for "Upsert" scenarios, instead since the Db Record needs to be fetched anyways, updating values on the fetched instance or inserting a new one. Update will want to send all fields from an entity to the database each time, rather than just sending what has changed. It means enforcing a bit more control over what can possibly be changed vs. what should not be.
Related
I have a user season model in my Entity Framework Core project:
public class UserSeason
{
public int Id { get; set; }
public int SeasonId { get; set; }
public int ProfileId { get; set; }
}
SeasonId and ProfileId a foreign keys to corresponding tables.
Pair (SeasonId, ProfileId) has IsUnuque constrain on the database level.
I also have a bit list of ProfileIds. For some of them already have a UserSeason row, but some don't.
What is the most efficient way to create such rows?
My current approach is
var alreadyExistProfileIds = await _db.UserSeasons.Where(x => x.SeasonId == seasonId && profileIds.Contains(x.ProfileId)).Select(x => x.ProfileId).ToArrayAsync();
var missedProfileIds = profileIds.Except(alreadyExistProfileIds).ToArray();
_db.UserSeasons.AddRange(missedProfileIds.Select(i => new StarPassUserSeasonDbModel{ProfileId = i, SeasonId = seasonId}));
await _db.SaveChangesAsync();
However, I don't like it, because I need to load missedProfiledId into the memory and it is big number.
Is there are a way to do that in database level with Entity Framework core?
This is not a duplicate question as I have looked up many questions including this, which is the closest to what I want but didn't solve the challenge.
I have my table models relation set up this way:
public class User
{
public long UserId { get; set; }
public string Name { get; set; }
public IList<Transaction> Transactions { get; set; }
}
public class Transaction
{
public long TransactionId { get; set; }
public User User { get; set; }
public User Patient { get; set; }
}
fluent api setup for the entity
//some other modelbuilder stuff
modelBuilder.Entity<User>(entity =>
{
entity.HasMany(e => e.Transactions).WithOne(e => e.User);
//wanted to add another entity.HasMany(e => e.User).WithOne(e => e.Patient) but efcore didn't allow me.
});
This generates a Transaction table with UserUserId and PatientUserId and takes the right values on save.
But when I do a get with a user Id
User user = dbcontext.Set<User>().Include(t => t.Transactions).FirstOrDefault(u => u.UserId == userId);
user.Transactions have a list of transaction all with null Transaction.Patient
What exactly is going on here and how do I get past it?
Thanks.
You are nesting navigations. So, you have to use ThenInclude like this to add Patient which is a navigation property of Transaction.
User user = dbcontext.Set<User>().Include(t => t.Transactions).ThenInclude(p => p.Patient).FirstOrDefault(u => u.UserId == userId);
There is a problem with my Db which I figured only now, when I started to work at the web api. My USER entity:
public class User { get; set; }
{
public int UserId { get; set; }
public string Name { get; set; }
}
And this is ACTIVITY
public class Activity
{
public int ActivityId { get; set; }
public User User { get; set; }
}
I added an activity and checked in SSMS. Everything seems to be good, there is a field named UserId which stores the id. My problem is when I try to get a User from an Activity because I keep getting null objects. I didn't set anything special in my DbContext for this.
This is where I'm trying to get an User from an Activity object:
public ActionResult ActivityAuthor(int activityId)
{
Activity activityItem = unitOfWork.Activity.Get(activityId);
return Json(unitOfWork.User.Get(activityItem.User.UserId));
}
Relation between User and Activity
The User property of Activity class should be marked as virtual. It enables entity framework to make a proxy around the property and loads it more efficiently.
Somewhere in your code you should have a similar loading method as following :
using (var context = new MyDbContext())
{
var activity = context.Activities
.Where(a => a.ActivityId == id)
.FirstOrDefault<Activity>();
context.Entry(activity).Reference(a => a.User).Load(); // loads User
}
This should load the User object and you won't have it null in your code.
Check this link for more information msdn
my psychic debugging powers are telling me that you're querying the Activity table without Include-ing the User
using System.Data.Entity;
...
var activities = context.Activities
.Include(x => x.User)
.ToList();
Alternatively, you don't need Include if you select properties of User as part of your query
var vms = context.Activities
.Select(x => new ActivityVM() {UserName = x.User.Name})
.ToList();
When I add a new item to a virtual collection property of my Entity it'll be added to the database.
For example
Loan.Notes.Add(new Note {Subject = "Test"})
This will create a new row in Notes table and showing the loanId for the link.
I tried to replicate this exact working functionality for another table. I set a breakpoint right before it hits save and I can see the collection has the NEW entity but it won't create. I'm clueless towards why it's working in one scenario and not the other.
Notes saves completely fine, status history doesn't.
public void Save()
{
foreach (var noteDTO in Notes.Where(x => x.NewRecord))
{
Loan.Notes.Add(new Note
{
Subject = noteDTO.Subject,
Comments = noteDTO.Comments,
Active = noteDTO.Active,
Loan = Loan,
Loan_Id = Loan.Id
});
}
foreach (var status in StatusHistory.Where(x => x.Id == 0))
{
Loan.StatusHistory.Add(
new LoanStatusHistory
{
Loan_Id = Loan.Id,
Active = status.Active,
IsCurrent = status.Current,
Date = status.StatusDate.Value.Date,
StatusReason = status.Reason,
LoanStatus_Id = status.Status_Id.Value,
Order = status.Order.Value,
Created = DateTime.Now,
CreatedBy = CurrentUser.GetInstance().MyUser.Id,
Modified = DateTime.Now,
LastModifiedBy = CurrentUser.GetInstance().MyUser.Id
});
}
}
UnitOfWork.LoanRepository.Update(Loan);
UnitOfWork.Save();
}
In Notes and Status models I have set there to be a virtual relationship to loan.
public virtual Loan Loan { get; set; }
In Loan Model I have set there to be a virtual relationship to Notes and Status
public virtual ICollection<LoanStatusHistory> StatusHistory { get; set; }
public virtual ICollection<Note> Notes { get; set; }
I was wondering if anyone could give me suggestions or tips to fix this please.
Edit: Updated problem description based on testing - 12 Sep 2011.
I have this query that throws a NotSupportedException ("Specified method is not supported.") whenever I call .ToList().
IQueryable<FileDefinition> query = db
.FileDefinitions
.Include(x => x.DefinitionChangeLogs)
.Include(x => x.FieldDefinitions.Select(y => y.DefinitionChangeLogs)) // bad
.Include(x => x.FieldDefinitions.Select(y => y.FieldValidationTables)) // bad
.Where(x => x.IsActive);
List<FileDefinition> retval = query.ToList();
If I comment out either line that I have commented as "bad", then the query works. I have also tried including different nested entities in my object model with the same effect. Including any 2 will cause a crash. By nested, I mean a navigation property of a navigation property. I also tried using the .Include methods with a string path: same result.
My table structure looks like this:
This is using MySQL 5.1 (InnoDB tables obviously) as the database store with MySQL Connector/NET 6.3.4.
So my question is: Why doesn't this work?
Note: I can get it to work if I explicitly load the related entities like in this link. But I want to know why EF hates my data model.
ANSWER: MySQL Connector is apparently not capable of handling the 2nd nested entity include. It throws the NotSupportedException, not .NET EF. This same error was also present when I tried this using EF4.0, but my research at the time led me to believe it was self-tracking entities causing the issue. I tried upgrading to latest Connector, but it started causing an Out of Sync error. This is yet another reason for me to hate MySQL.
Maybe a little late to the party but i found the following workaround fairly useful in a current project:
IQueryable<FileDefinition> query = db.FileDefinitions
.Include(x => x.FieldDefinitions.Select(y => y.DefinitionChangeLogs.Select(z => z.FieldDefinition.FieldValidationTables)))
Where rather than using a second row of includes, use Select to get back to the original navigation property and another Select to go forwards to the property you need to include.
I have made a little console application to test your scenario and this test application works:
using System;
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
namespace EFIncludeTest
{
public class Parent
{
public int Id { get; set; }
public string Name { get; set; }
public ICollection<ChildLevel1> ChildLevel1s { get; set; }
}
public class ChildLevel1
{
public int Id { get; set; }
public string Name { get; set; }
public ICollection<ChildLevel2a> ChildLevel2as { get; set; }
public ICollection<ChildLevel2b> ChildLevel2bs { get; set; }
}
public class ChildLevel2a
{
public int Id { get; set; }
public string Name { get; set; }
}
public class ChildLevel2b
{
public int Id { get; set; }
public string Name { get; set; }
}
public class MyContext : DbContext
{
public DbSet<Parent> Parents { get; set; }
}
class Program
{
static void Main(string[] args)
{
// Create entities to test
using (var ctx = new MyContext())
{
var parent = new Parent
{
Name = "Parent",
ChildLevel1s = new List<ChildLevel1>
{
new ChildLevel1
{
Name = "FirstChildLevel1",
ChildLevel2as = new List<ChildLevel2a>
{
new ChildLevel2a { Name = "FirstChildLevel2a" },
new ChildLevel2a { Name = "SecondChildLevel2a" }
},
ChildLevel2bs = new List<ChildLevel2b>
{
new ChildLevel2b { Name = "FirstChildLevel2b" },
new ChildLevel2b { Name = "SecondChildLevel2b" }
}
},
new ChildLevel1
{
Name = "SecondChildLevel1",
ChildLevel2as = new List<ChildLevel2a>
{
new ChildLevel2a { Name = "ThirdChildLevel2a" },
new ChildLevel2a { Name = "ForthChildLevel2a" }
},
ChildLevel2bs = new List<ChildLevel2b>
{
new ChildLevel2b { Name = "ThirdChildLevel2b" },
new ChildLevel2b { Name = "ForthChildLevel2b" }
}
},
}
};
ctx.Parents.Add(parent);
ctx.SaveChanges();
}
// Retrieve in new context
using (var ctx = new MyContext())
{
var parents = ctx.Parents
.Include(p => p.ChildLevel1s.Select(c => c.ChildLevel2as))
.Include(p => p.ChildLevel1s.Select(c => c.ChildLevel2bs))
.Where(p => p.Name == "Parent")
.ToList();
// No exception occurs
// Check in debugger: all children are loaded
Console.ReadLine();
}
}
}
}
My understanding was that this basically represents your model and the query you are trying (taking also your comments to your question into account). But somewhere must be an important difference which is not visible in the code snippets in your question and which makes your model fail to work.
Edit
I have tested the working console application above with MS SQL provider (SQL Server 2008 R2 Express DB), not MySQL Connector. Apparently this was the "important difference".