I'm having a problem loading data from a second level entity using EF Core 3 but it works as expected with EF6 but I'm converting a project to try to migrate the lot to .NET Core, including EF.
I'm not sure if this is the right way to express this but in short, I have a list of companies that have users and these users have roles and for some reason, I can load the companies and their relevant users from a many to many relationship but the roles for each user are not being loaded.
I have the following Generic function:
protected virtual IQueryable<TEntity> GetQueryable(
Expression<Func<TEntity, bool>> filter = null,
Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
string includeProperties = null,
bool isCollection = false,
int? skip = null,
int? take = null)
{
IQueryable<TEntity> query = this.Context.Set<TEntity>();
if (filter != null)
{
query = query.Where(filter.Expand());
}
if (includeProperties != null)
{
query = includeProperties
.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries)
.Aggregate(query, (current, include) => current.Include(include));
}
if (orderBy != null)
{
query = orderBy(query);
}
if (skip.HasValue)
{
query = query.Skip(skip.Value);
}
if (take.HasValue)
{
query = query.Take(take.Value);
}
query = query.AsExpandableEFCore();
return query;
}
and I call it as follows:
var databaseCompanies = await this.UnitOfWork.Companies
.GetAllAsync(null, "Companies.User.Roles");
And my entities are defined as follows:
Company:
public class Company: Entity<Guid>
{
private ICollection<CompanyUsers> _companyUsers;
public virtual ICollection<CompanyUsers> CompanyUsers
{
get => this._companyUsers ?? (this._companyUsers = new HashSet<CompanyUsers>());
set => this._companyUsers = value;
}
}
User:
public class User : Entity<Guid>
{
private ICollection<CompanyUsers> _companyUsers;
private ICollection<UserRole> _roles;
public virtual ICollection<UserRole> Roles
{
get => this._roles ?? (this._roles = new HashSet<UserRole>());
set => this._roles = value;
}
public virtual ICollection<CompanyUsers> CompanyUsers
{
get => this._companyUsers ?? (this._companyUsers = new HashSet<CompanyUsers>());
set => this._companyUsers = value;
}
}
Role:
public class UserRole
{
public Guid UserId { get; set; }
public RoleType Role { get; set; }
public User User { get; set; }
}
Class to define the many to many relation:
public class CompanyUsers
{
public Guid UserId { get; set; }
public User User { get; set; }
public Guid CompanyId { get; set; }
public Company Company { get; set; }
}
As you can see, the Company and User classes have a many to many relation defined by CompanyUsers and there is a one to many relationship between the User the the UserRole classes. When calling:
var databaseCompanies = await this.UnitOfWork.Companies
.GetAllAsync(null, "Companies.User");
It only loads the companies and the users, so I tried this instead:
var databaseCompanies = await this.UnitOfWork.Companies
.GetAllAsync(null, "Companies.User.Roles");
or
var databaseCompanies = await this.UnitOfWork.Companies
.GetAllAsync(null, "Companies.User,Companies.User.Roles");
But none of them work and the roles for the users belonging to the companies never get loaded.
In the last example, it will add 2 .Include but when looking at the query variable but I noticed that expression has 2 arguments but they are different of different type:
1) {value(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[Data.Entities.Company])}
2) "Company.User.Roles"
Again, if I load only one level below the main entity such as load a specific user, it will load its roles accordingly:
var databaseUser = await this.UnitOfWork.Users
.GetFirstAsync(u => u.Username == username, null, "Roles", true);
Any ideas on how to resolve this?
Thanks.
UPDATE:
As mentioned below, I really wish I could swear right now!! The penny dropped after noticing that the query generated by EF Core was correct and returned the role as part of the query but they were all NULL.
I went to check my Roles table and that's when I realized that it had been wiped when I rolled back a migration regarding roles in order to fix the relationship and I never re-generated the dummy roles that were associated with all the users of the companies I had defined!!
Apologies to all of you who helped!!
You need to include related data using ThenInclude.
Look at this article: https://learn.microsoft.com/en-us/ef/core/querying/related-data
From link:
You can drill down through relationships to include multiple levels of
related data using the ThenInclude method. The following example loads
all blogs, their related posts, and the author of each post. C#
using (var context = new BloggingContext())
{
var blogs = context.Blogs
.Include(blog => blog.Posts)
.ThenInclude(post => post.Author)
.ToList();
}
In your GetQueryable method parameters replace string includeProperties = null with Func<IQueryable<T>, IIncludableQueryable<T, object>> includes == null and then in the method body update the code as follows:
if (includes != null)
{
query = includes(query);
}
Now when calling the GetQueryable method, pass the value of includes as follows:
sp => sp.Include(i => i.FirstLevel).ThenInclude(f => f.SecondLevel)
Job done! Now It should work as expected!
Related
So I'm using Entity Framework Core to build a database of Guilds (Another name for Discord Servers) and Users, with the Discord.NET Library. Each Guild has many users, and each user can be in many guilds. First time using EF and I'm having some teething issues. The two classes are:
public class Guild
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
public ulong Snowflake { get; set; }
public DateTimeOffset CreatedAt { get; set; }
public string Name { get; set; }
public ICollection<User> Users { get; set; }
}
public class User
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
public ulong Snowflake { get; set; }
public string Username { get; set; }
public ushort DiscriminatorValue { get; set; }
public string AvatarId { get; set; }
public ICollection<Guild> Guilds { get; set; }
public DateTimeOffset CreatedAt { get; set; }
}
With the goal of having 3 tables: Guild, Users, and GuildUsers. This is my current function for getting the guilds:
using var context = new AutomataContext();
var discordGuilds = this.client.Guilds.ToList();
var dbGuilds = context.Guilds;
List<Guild> internalGuilds = discordGuilds.Select(g => new Guild
{
Snowflake = g.Id,
Name = g.Name,
CreatedAt = g.CreatedAt,
Users = g.Users.Select(gu => new User
{
Id = context.Users.AsNoTracking().FirstOrDefault(u => u.Snowflake == gu.Id)?.Id ?? default(int),
}).ToList(),
}).ToList();
// Upsert Guilds to db set.
foreach (var guild in internalGuilds)
{
var existingDbGuild = dbGuilds.AsNoTracking().FirstOrDefault(g => g.Snowflake == guild.Snowflake);
if (existingDbGuild != null)
{
guild.Id = existingDbGuild.Id;
dbGuilds.Update(guild); // Hits the second Update here and crashes
}
else
{
dbGuilds.Add(guild);
}
}
await context.SaveChangesAsync();
I should note, a 'snowflake' is a unique ID that discord uses, but I wanted to keep my own unique ID for each table.
High level overview, guilds are collected into Discord.NET models. These are then transformed into internalGuilds (my guild class, which includes the list of users). Each of these is looped through and upserted to the database.
The issue arises in the second guild loop, where an error is thrown in the "Update" that a User ID is already being tracked (Inside the guild). So the nested ID is already being tracked? Not sure what's going on here, any help would be appreciated. Thanks.
This exception is most likely occurring because you are loading Users without tracking then looping through and potentially trying to update or insert guilds /w the same user reference, especially using the Update method.
I would suggest removing the use of AsNoTracking. Working with detached entity references via AsNoTracking is more of a performance tweak for when reading large amounts of data. You can pre-fetch all of the User references by their snowflake:
using (var context = new AutomataContext())
{
var discordGuilds = this.client.Guilds.ToList();
// Get the user snowflakes from the guilds, and pre-fetch them.
var userSnowflakes = discordGuilds.SelectMany(g => g.Users.Select(u => u.Id)).ToList();
var users = await context.Users
.Where(x => userSnowflakes.Contains(x.Snowflake))
.ToListAsync();
// We need to add references for any New user snowflakes.
var existingSnowflakes = users.Select(x => x.Snowflake).ToList();
// If more detail is needed for new user records, it will need to be fetched from the passed in Guild.User.
var newUsers = userSnowflakes.Except(existingSnowFlakes)
.Select(x => new User { SnowflakeId = x }).ToList();
if(newUsers.Any())
users.AddRange(newUsers);
List<Guild> internalGuilds = discordGuilds.Select(g => new Guild
{
Snowflake = g.Id,
Name = g.Name,
CreatedAt = g.CreatedAt,
Users = g.Users
.Select(gu => users.Single(u => u.Snowflake == gu.Id))
.ToList(),
}).ToList(),
// Upsert Guilds to db set.
foreach (var guild in internalGuilds)
{
var existingGuildId = context.Guilds
.Where(x => x.Snowflake == guild.Snowflake)
.Select(x => x.Id)
.SingleOrDefault();
if (existingGuildId != 0)
{
guild.Id = existingGuildId;
dbGuilds.Update(guild);
}
else
{
dbGuilds.Add(guild);
}
}
await context.SaveChangesAsync();
This should help ensure that the User references for existing users are pointing at the same instances, whether existing users or new user references that will be associated to the DbContext when first referenced.
Ultimately I don't recommend using Update for "Upsert" scenarios, instead since the Db Record needs to be fetched anyways, updating values on the fetched instance or inserting a new one. Update will want to send all fields from an entity to the database each time, rather than just sending what has changed. It means enforcing a bit more control over what can possibly be changed vs. what should not be.
I'm trying to defined a lambda query in a property of my code first EF model as seen below as, GetLatestTransaction :
public class ApplicationUser : IdentityUser
{
public string FirstName { get; set; }
public string LastName { get; set; }
public virtual List<TransactionModel> Transactions { get; set; }
public TransactionModel GetLatestTransaction {
get {
return Transactions.OrderByDescending(x => x.Created).FirstOrDefault();
}
}
}
The reason for this is that I don't want to have to retype this query in many places and by having it in one place reduce the chances of a bug.
I want to use this in a query like this:
var user = _DB.Users
.Select(u => new UserDetailsView()
{
Id = u.Id,
FirstName= u.FirstName,
LastName= u.LastName,
Balance = u.GetLatestTransaction.ValueResult
}).FirstOrDefault(x => x.Id == userId);
This is however resulting in this error:
System.NotSupportedException: 'The specified type member 'GetLatestTransaction' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.'
Is there some way to achieve this without storing another relation to the latest transaction on the user and having to update it every time there is a new transaction?
Edit: I would also like to do it as above to avoid making another query to the database, I want it all in one go to improve performance.
Your ApplicationUser class represents the table in the database. It does not represent the usage of the data in the table.
Quite a lot of people think it is good practice to separate the database structure from the usage of the data. This separation is quite often done using the repository pattern. The repository is an abstraction from the internal datastructure of the database. It allows you to add functionality to your classes without demanding this functionality in the control classes that communicate with the database.
There are numerous articles about the repository. This one helped me to understand what functionality I should put in my entity framework classes and which in the repository.
So you'll need a class that represents the elements in your database table and one that represents the applicationUsers with only their LatestTransaction
The class that represents the database table:
class ApplicationUser : IdentityUser
{
public int Id {get; set;}
public string FirstName { get; set; }
public string LastName { get; set; }
public virtual List<TransactionModel> Transactions { get; set; }
}
ApplicationUser with the latest transaction
class AppicationUserExt : <base class needed?>
{
public int Id {get; set;}
public string FirstName { get; set; }
public string LastName { get; set; }
public TransactionModel LatestTransaction { get; set; }
}
The function to get your extended ApplicationUser is an extension function of your ApplicationUser. Input: IQueryable<ApplicationUser output: IQueryable<ApplicationUserExt>
static class MyDbContextExtensions
{
// returns ne ApplicationUserExt for every ApplicationUser
public IQueryable<ApplicationUserExt> ToExtendedUsers(this IQueryable<ApplicationUser> applicationUsers)
{
return applicationUsers
.Select(user => new ApplicationUserExt()
{
Id = user.Id,
FirstName = user.FirstName,
LastName = user.LastName,
LatestTransaction = user.Trnasactions
.OrderByDescenting(transaction => transaction.CreationDate)
.FirstOrDefault(),
}
}
}
}
So whenever you have a query with the ApplicationUsers you want, you can use ToExtendedUsers() to get the extended suers
using (var dbContext = new MyDbContext(...))
{
// you wanted to have a query like:
var result dbContext.ApplicationUsers
.Where(user => user.FirstName = "John"
&& user.LastName = "Doe");
// you'll have to add ToExtendedUsers:
var result = dbContext.ApplicationUsers
.Where(user => user.FirstName = "John"
&& user.LastName = "Doe");
.ToExtendedUsers();
}
As the result is still an IQueryable, no query has been done yet. You can still add LINQ statements before the query is done:
var result2 = result
.Where(user.LatestTransaction.Year == 2018)
.GroupBy(user => user.LatestTransaction.Date)
.OrderBy(group => group.Key)
.Take(10)
.ToList();
You see, that you can still do all kinds of LINQ stuff as long as it is an ApplicationUser. As soon as you need the LatestTransaction you convert it to an ApplicationUserExt and continue concatenating your linq statements.
There is a problem with my Db which I figured only now, when I started to work at the web api. My USER entity:
public class User { get; set; }
{
public int UserId { get; set; }
public string Name { get; set; }
}
And this is ACTIVITY
public class Activity
{
public int ActivityId { get; set; }
public User User { get; set; }
}
I added an activity and checked in SSMS. Everything seems to be good, there is a field named UserId which stores the id. My problem is when I try to get a User from an Activity because I keep getting null objects. I didn't set anything special in my DbContext for this.
This is where I'm trying to get an User from an Activity object:
public ActionResult ActivityAuthor(int activityId)
{
Activity activityItem = unitOfWork.Activity.Get(activityId);
return Json(unitOfWork.User.Get(activityItem.User.UserId));
}
Relation between User and Activity
The User property of Activity class should be marked as virtual. It enables entity framework to make a proxy around the property and loads it more efficiently.
Somewhere in your code you should have a similar loading method as following :
using (var context = new MyDbContext())
{
var activity = context.Activities
.Where(a => a.ActivityId == id)
.FirstOrDefault<Activity>();
context.Entry(activity).Reference(a => a.User).Load(); // loads User
}
This should load the User object and you won't have it null in your code.
Check this link for more information msdn
my psychic debugging powers are telling me that you're querying the Activity table without Include-ing the User
using System.Data.Entity;
...
var activities = context.Activities
.Include(x => x.User)
.ToList();
Alternatively, you don't need Include if you select properties of User as part of your query
var vms = context.Activities
.Select(x => new ActivityVM() {UserName = x.User.Name})
.ToList();
I have three entities Role, RoleUser and User
I want to select each Compulsory Role and load the related User's where the RoleUser.RecordID in the join table is equal to a given value.
Using the UOW and GenericReposiity Get(...) method I would call ...
.Get(role => role.Compulsory == true, null, "RoleUser.User") to select all the compulsory role's and load all User for the navigation RoleUser.User.
How can I filter them, is it possible using the implemented Get() method?
Entities
public class Role
{
public int RoleID { get; set; }
public bool Compulsory { get; set; }
public virtual IList<RoleUser> RoleUser { get; set; }
}
public class RoleUser
{
public string UserID { get; set; }
public virtual User User { get; set; }
public Guid RecordID { get; set; }
public virtual Record Record { get; set; }
public int RoleID { get; set; }
public virtual Role Role { get; set; }
}
public class User
{
public string userID { get; set; }
}
Get
public virtual IList<TEntity> Get(
Expression<Func<TEntity, bool>> filter = null,
Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
string includeProperties = "")
{
IQueryable<TEntity> query = dbSet;
if (filter != null)
{
query = query.Where(filter);
}
foreach (var includeProperty in includeProperties.Split
(new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
{
query = query.Include(includeProperty);
}
if (orderBy != null)
{
return orderBy(query).ToList();
}
else
{
return query.ToList();
}
}
No , you cant filter the RoleUser from this Get method. if you want to filter RoleUser you want to project them into another list.
You can filter your Role based on the RoleUser table.
Include will always bring full data in the table unless you use IQuerable Projection.
var rolesWithAnyUserName = UnitOfWork.Role
.Get(role => role.Compulsory == true && role.RoleUser
.Any(g=>g.RoleUserName=="Name"), null, "RoleUser.User")
This will only filter the Role based on the RoleUser filter
If you need to filter Include tables write the query where you can send IQuerable to the database
Try something like this , this will filter both Role and RoleUser
var result = DataContext.Role.Where(role => role.Compulsory == true )
.Include(gl => gl.RoleUser)
.Select(x => new
{
role = x,
roleUsers= x.RoleUser.Where(g => g.Id == 1),
}).ToList();
I have an entity like below
public class Role
{
[Key, Required, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
[Required, StringLength(30)]
public string Name { get; set; }
public virtual ICollection<User> Users { get; set; }
public virtual ICollection<RolePermission> Permissions { get; set; }
public bool HasPermission(String code)
{
foreach (var p in this.Permissions)
{
if (p.Permission.Code.Equals(code))
return true;
}
return false;
}
}
and in controller, this code run fine:
for (var p in db.User.Where(u => u.UserId == 1).First().Role.Permissions) { PrintToDebug(); }
but:
User ur = db.User.Where(u => u.UserId == 1).First();
ur.Role.HasPermission("Some_Code_Were_Defined");
then the Permissions list in HasPermission always has a zero length, why and how to solve?
This is occurring because of Entity Framework Lazy Loading. In your first statement, you are specifically requesting the Permissions property, which causes Entity Framework to generate a query that loads that table from the database. In the second query, you are only asking Entity Framework to load the User table from the database, but the HasPermission method you are calling has no way of making another database call to load the Permissions table.
This is a common issue when working with Entity Framework. It can be resolved by using the Include() extension method from Entity Framework to eagerly load the related table in the second query, i.e. User ur = db.User.Where(u => u.UserId == 1).Include(u => u.Role.Permissions).First();