I have a Activities model as follows
public class Activity
{
// this is email
public string CreatedBy {set;get;}
// Relationship
public ApplicationUser User{set;get;}
}
Then I have the User model:
public class ApplicationUser
{
// ID from Identity
public string Id{set;get;}
public string Email {set;get;}
}
Of course I have the corresponding tables in the database.
What i need to find out is the users who didnt do any activity.
While the code below works, it is not efficient and times out. Because I have 500K activity in the Activities table.
var userz = _db.Users.AsNoTracking();
var groupedUsers = _db.Activities.AsNoTracking().GroupBy(x => x.CreatedBy).Select(group => new { CreatedBy = group.Key, Count = 1 }).Select(x=> x.CreatedBy);
var result = userz.Where(x => groupedUsers.Contains(x.Email) == false);
I tried the same Query for Navigation property, which is indexed, ie: User above. Yet the query times out.
Is there a more efficient solution for this using left join?
You should be better of with foreign keys but if this is really how your classes look you could try
_db.Users.Where(u => !_db.Activities.Any(u => a.ApplicationUser == u));
Related
I have ViewModel
UserGroup.cs
public class UserGroup
{
public User User { get; set; }
public List<Group> Groups { get; set; }
}
And I want to have the User and all related Groups. The problem is that the query returns duplicates of all users if there is more than 1 group related to him (and It's expected of course). The first thing that comes to my mind is to fetch all the users with single query and then foreach user to get the groups related to him and push it in the list. But I'm looking for a better way (if there is) to do that.
The relationship is made with junction table.
SQL Query
SELECT u.UserName, ug.GroupName FROM auth.UserUserGroup uug
INNER JOIN [auth].[User] u ON u.UserId = uug.UserId
INNER JOIN auth.UserGroup ug ON ug.UserGroupId = uug.UserGroupId
I'm sure someone will have a more elegant solution, but I've used Linq to construct a tiered object a number of times
var dbRecords = repository.Get(userId); // However you are getting your records, do that here
var result = dbRecords.GroupBy(x => x.UserId)
.Select(g => new UserGroup
{
User = new User { UserName = g.First().UserName },
Groups = g.Select(y => new Group { GroupName = y.GroupName }
}
);
I've tried to use your object names. Hopefully I didn't make any mistakes
I think I'm having a senior moment, but I also think I have not run into this situation before. I have two columns in my MVC5 Identity 2.1 Users table.
UserId | BannedBy (and also an IsBanned bool)
Both fields are userid guid strings. However, BannedBy refers to a different user in the same Users table.
When I display my view of banned users (a table and each row is one banned user), I don't want to show the BannedBy guid, I want to show the related UserName for that BannedBy guid. However, I can't seem to figure out what I need to do.
I've tried a ViewModel and method approach:
public ActionResult BannedUsers()
{
var bannedUsers = db.Users.Where(d => d.IsBanned);
var model = new BannedUsersViewModel
{
BannedUsers = bannedUsers,
BannedByUserName = GetUserName(bannedUsers.BannedBy)
};
return View(model);
}
Then like an outer approach to my viewmodel:
var model = new BannedUsersViewModel
{
BannedUsers = bannedUsers
};
model.BannedByUserName = GetUserName(model.bannedUsers.BannedBy);
However, it seems I can't use the bannedUsers.BannedBy (I also tried all that above with a capital B... BannedUsers.BannedBy) data before it's actually been rendered? And now I've scrapped the viewmodel and am trying to do like a related data join on my query:
db.Users.Join(d => d.BannedBy == d.UserId).Where(d => d.IsBanned);
(I'm sure this is way off, I'm just trying to give you an idea)
Does anyone know the proper way of doing this? I was also thinking about calling a method from my view, but seems like that would be breaking the MVC rules?
Thank you.
Update: Here is the GetUserName method:
public string GetUserName(string userId)
{
var result = db.Users.Find(userId);
return result.UserName;
}
Update #2: Here is the BannedUsersViewModel:
public class BannedUsersViewModel
{
public IEnumerable<ApplicationUser> BannedUsers { get; set; }
public string BannedByUserName { get; set; }
}
Update #3: A pic:
I am going to take a stab at this. We can modify it as needed (unless I am completely off base, in which case, I will delete this and we will all pretend it never happened). Does this get you in the ballpark:
public ActionResult BannedUsers()
{
var bannedUsers =
db.Users
.Where(d => d.IsBanned)
.Join(
db.Users,
bannee => bannee.BannedBy,
banner => banner.UserId,
(bannee, banner) => new BannedUser()
{
BannedByUserName = banner.UserName,
BannedUser = bannee
})
.AsEnumerable();
var model = new BannedUsersViewModel
{
BannedUsers = bannedUsers
};
return View(model);
}
public class BannedUsersViewModel
{
public IEnumerable<BannedUser> BannedUsers { get; set; }
}
public class BannedUser
{
public ApplicationUser BannedUser { get; set; }
public string BannedByUserName { get; set; }
}
The idea is that we get all of the banned users, join them to all of the users that banned those users and then group by the user that banned them. You end up with a collection of objects that have the user that banned other users and the users they banned.
So far, I went with #lazy's comment in doing a self join. I'm not sure how good I feel about it though. So essentially, I added this to my IdentityModels.cs under public class ApplicationUser : IdentityUser:
[ForeignKey("BannedBy")]
public virtual ApplicationUser BannedByUser { get; set; }
Then in my controller, changed my original query to a list:
var bannedUsers = db.Users.Where(d => d.IsBanned).ToList();
return View(bannedUsers);
(If I don't convert to list, it complains about having more than one data reader open.) Then in my View in my foreach loop:
#Html.DisplayFor(modelItem => item.BannedByUser.UserName)
And boom:
I'm a little worried of the performance impact?...especially for a page that isn't used that often and is not really that important. Is there an impact if the page isn't being called (like with the Index that was created and such)? I'm also a little leery since there seems to be some magic happening with Identity.
Anyway, I'm still open to other ideas...or thoughts about this one. Thanks again for everyone's help.
I have a Simple table like this:
User(Id, FirstName, ...) <===> Role(Id, Title, ...)
witch have N:M relation
what i want to do is to remove the relation in between them by having there Ids, so my method should be like this :
public void UnlinkUsersFromRoles(int[] roleIds, int[] userIds)
{
var myContext = new DefaultContext();
// ?? <= how to break the relation without loading unnecessary columns
}
If this is many-to-many relationship, there must be a join table in between: for these tables User and Role, let's say it's called UserRole, and is a simple join table (i.e., no other columns on that table, other than the FK ids to the other two tables):
public class UserRole
{
public int UserId { get; set; }
public int RoleId { get; set; }
}
With an explicitly defined join table such as this, UnlinkUsersFromRoles could be defined as follows:
public void UnlinkUsersFromRoles(int[] roleIds, int[] userIds)
{
using (var context = new DefaultContext())
{
foreach (var ur in context.UserRoles
.Where(u => userIds.Contains(u.UserId))
.Where(r => roleIds.Contains(r.RoleId))
.ToArray())
{
context.UserRoles.Remove(ur);
}
context.SaveChanges();
}
}
You're certainly going to wind up loading all the UserRole rows to be removed in when you call ToArray, but this is certainly better than loading in all the related User and Role rows.
Can you just do this:
var rolesToRemove = myContext.Roles.Where(r=> roleIds.Contains(r.Id)).ToArray();
foreach(var user in myContext.Users.Where(u=> userIds.Contains(u.Id)){
forearch(var var role in rolesToRemove) {
user.Roles.Remove(role);
}
}
myContext.SaveChanges();
When you say:
how to break the relation without loading unnecessary columns
You mean, the above code doesn't fullfill you requeriment?
EDIT:
If you have a explicity relationship class like UserInRoles, you should use John Castleman Answer, or just create one, map the ForeignKeys and use his solution.
I am trying to add new record into SQL database using EF. The code looks like
public void Add(QueueItem queueItem)
{
var entity = queueItem.ApiEntity;
var statistic = new Statistic
{
Ip = entity.Ip,
Process = entity.ProcessId,
ApiId = entity.ApiId,
Result = entity.Result,
Error = entity.Error,
Source = entity.Source,
DateStamp = DateTime.UtcNow,
UserId = int.Parse(entity.ApiKey),
};
_statisticRepository.Add(statistic);
unitOfWork.Commit();
}
There is navigation Api and User properties in Statistic entity which I want to load into new Statistic entity. I have tried to load navigation properties using code below but it produce large queries and decrease performance. Any suggestion how to load navigation properties in other way?
public Statistic Add(Statistic statistic)
{
_context.Statistic.Include(p => p.Api).Load();
_context.Statistic.Include(w => w.User).Load();
_context.Statistic.Add(statistic);
return statistic;
}
Some of you may have question why I want to load navigation properties while adding new entity, it's because I perform some calculations in DbContext.SaveChanges() before moving entity to database. The code looks like
public override int SaveChanges()
{
var addedStatistics = ChangeTracker.Entries<Statistic>().Where(e => e.State == EntityState.Added).ToList().Select(p => p.Entity).ToList();
var userCreditsGroup = addedStatistics
.Where(w => w.User != null)
.GroupBy(g => g.User )
.Select(s => new
{
User = s.Key,
Count = s.Sum(p=>p.Api.CreditCost)
})
.ToList();
//Skip code
}
So the Linq above will not work without loading navigation properties because it use them.
I am also adding Statistic entity for full view
public class Statistic : Entity
{
public Statistic()
{
DateStamp = DateTime.UtcNow;
}
public int Id { get; set; }
public string Process { get; set; }
public bool Result { get; set; }
[Required]
public DateTime DateStamp { get; set; }
[MaxLength(39)]
public string Ip { get; set; }
[MaxLength(2083)]
public string Source { get; set; }
[MaxLength(250)]
public string Error { get; set; }
public int UserId { get; set; }
[ForeignKey("UserId")]
public virtual User User { get; set; }
public int ApiId { get; set; }
[ForeignKey("ApiId")]
public virtual Api Api { get; set; }
}
As you say, the following operations against your context will generate large queries:
_context.Statistic.Include(p => p.Api).Load();
_context.Statistic.Include(w => w.User).Load();
These are materialising the object graphs for all statistics and associated api entities and then all statistics and associated users into the statistics context
Just replacing this with a single call as follows will reduce this to a single round trip:
_context.Statistic.Include(p => p.Api).Include(w => w.User).Load();
Once these have been loaded, the entity framework change tracker will fixup the relationships on the new statistics entities, and hence populate the navigation properties for api and user for all new statistics in one go.
Depending on how many new statistics are being created in one go versus the number of existing statistics in the database I quite like this approach.
However, looking at the SaveChanges method it looks like the relationship fixup is happening once per new statistic. I.e. each time a new statistic is added you are querying the database for all statistics and associated api and user entities to trigger a relationship fixup for the new statistic.
In which case I would be more inclined todo the following:
_context.Statistics.Add(statistic);
_context.Entry(statistic).Reference(s => s.Api).Load();
_context.Entry(statistic).Reference(s => s.User).Load();
This will only query for the Api and User of the new statistic rather than for all statistics. I.e you will generate 2 single row database queries for each new statistic.
Alternatively, if you are adding a large number of statistics in one batch, you could make use of the Local cache on the context by preloading all users and api entities upfront. I.e. take the hit upfront to pre cache all user and api entities as 2 large queries.
// preload all api and user entities
_context.Apis.Load();
_context.Users.Load();
// batch add new statistics
foreach(new statistic in statisticsToAdd)
{
statistic.User = _context.Users.Local.Single(x => x.Id == statistic.UserId);
statistic.Api = _context.Api.Local.Single(x => x.Id == statistic.ApiId);
_context.Statistics.Add(statistic);
}
Would be interested to find out if Entity Framework does relationship fixup from its local cache.
I.e. if the following would populate the navigation properties from the local cache on all the new statistics. Will have a play later.
_context.ChangeTracker.DetectChanges();
Disclaimer: all code entered directly into browser so beware of the typos.
Sorry I dont have the time to test that, but EF maps entities to objects. Therefore shouldnt simply assigning the object work:
public void Add(QueueItem queueItem)
{
var entity = queueItem.ApiEntity;
var statistic = new Statistic
{
Ip = entity.Ip,
Process = entity.ProcessId,
//ApiId = entity.ApiId,
Api = _context.Apis.Single(a => a.Id == entity.ApiId),
Result = entity.Result,
Error = entity.Error,
Source = entity.Source,
DateStamp = DateTime.UtcNow,
//UserId = int.Parse(entity.ApiKey),
User = _context.Users.Single(u => u.Id == int.Parse(entity.ApiKey)
};
_statisticRepository.Add(statistic);
unitOfWork.Commit();
}
I did a little guessing of your namings, you should adjust it before testing
How about make a lookup and load only necessary columns.
private readonly Dictionary<int, UserKeyType> _userKeyLookup = new Dictionary<int, UserKeyType>();
I'm not sure how you create a repository, you might need to clean up the lookup once the saving changes is completed or in the beginning of the transaction.
_userKeyLookup.Clean();
First find in the lookup, if not found then load from context.
public Statistic Add(Statistic statistic)
{
// _context.Statistic.Include(w => w.User).Load();
UserKeyType key;
if (_userKeyLookup.Contains(statistic.UserId))
{
key = _userKeyLookup[statistic.UserId];
}
else
{
key = _context.Users.Where(u => u.Id == statistic.UserId).Select(u => u.Key).FirstOrDefault();
_userKeyLookup.Add(statistic.UserId, key);
}
statistic.User = new User { Id = statistic.UserId, Key = key };
// similar code for api..
// _context.Statistic.Include(p => p.Api).Load();
_context.Statistic.Add(statistic);
return statistic;
}
Then change the grouping a little.
var userCreditsGroup = addedStatistics
.Where(w => w.User != null)
.GroupBy(g => g.User.Id)
.Select(s => new
{
User = s.Value.First().User,
Count = s.Sum(p=>p.Api.CreditCost)
})
.ToList();
So I have a table of (new) users and a table of groups. What I'm trying to do is add the users to the groups.
What I thought I'd do is :-
using(context = new MyEntity())
{
foreach(csvUser from csvSource)
{
User oUser = new User();
oUser.Firstname = csvUser.Firstname;
Group oGroup = new Group();
// Set the primary key for attach
oGroup.ID = csvUser.GroupID;
context.Group.Attach(oGroup);
oUser.Groups.Add(oGroup);
context.Users.Add(oUser);
}
context.saveChnages();
}
So bascially loop through all the new users, grab their group id from the CSV File (group already exists in db). So I would attach to the group and then add the group.
However I'm running into an error because as soon as a user with group id which has already been attached tries to attach it booms.
An object with a key that matches the key of the supplied object could
not be found in the ObjectStateManager. Verify that the key values of
the supplied object match the key values of the object to which
changes must be applied.
Which I can understand, its trying to re-attach an object its already attached to in memory. However is there a way around this? All I need to do is attach a new user to a pre-existing group from the database.
That error is usually associated with the ApplyCurrentValues in some form or shape - when it tries to update your (previously) detached entity.
It's not entirely clear why is that happening in your case - but maybe you have something else going on - or you're just 'confusing' EF with having attaching the same group over again.
The simplest way I think is to just use Find - and avoid Attach
context.Group.Find(csvUser.GroupID);
Which loads from cache if there is one - or from Db if needed.
If an entity with the given primary key values exists in the context,
then it is returned immediately without making a request to the store.
Otherwise, a request is made to the store for an entity with the given
primary key values and this entity, if found, is attached to the
context and returned. If no entity is found in the context or the
store, then null is returned
That should fix things for you.
You could also turn off applying values form Db I think (but I'm unable to check that at the moment).
It's seems like you're adding a new Group for each User you're iterating on, try this:
using(context = new MyEntity())
{
// fetch Groups and add them first
foreach(groupId in csvSource.Select(x => x.GroupID).Distinct())
{
context.Groups.Add(new Group { ID = groupId });
}
// add users
foreach(csvUser from csvSource)
{
User oUser = new User();
oUser.Firstname = csvUser.Firstname;
var existingGroup = context.Groups.Single(x => x.Id == csvUser.GroupID);
oUser.Groups.Add(existingGroup);
context.Users.Add(oUser);
}
context.saveChnages();
}
It seems like you have a many-to-many relationship between Users and Groups. If that is the case and you are using Code-First then you model could be defined like this...
public class User
{
public int Id { get; set; }
public string Firstname { get; set; }
// Other User properties...
public virtual ICollection<UserGroup> UserGroups { get; set; }
}
public class Group
{
public int Id { get; set; }
// Other Group properties...
public virtual ICollection<UserGroup> UserGroups { get; set; }
}
public class UserGroup
{
public int UserId { get; set; }
public User User { get; set; }
public int GroupId { get; set; }
public Group Group { get; set; }
}
Next, configure the many-to-many relationship...
public class UserGroupsConfiguration : EntityTypeConfiguration<UserGroup>
{
public UserGroupsConfiguration()
{
// Define a composite key
HasKey(a => new { a.UserId, a.GroupId });
// User has many Groups
HasRequired(a => a.User)
.WithMany(s => s.UserGroups)
.HasForeignKey(a => a.UserId)
.WillCascadeOnDelete(false);
// Group has many Users
HasRequired(a => a.Group)
.WithMany(p => p.UserGroups)
.HasForeignKey(a => a.GroupId)
.WillCascadeOnDelete(false);
}
}
Add the configuration in your DbContext class...
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Configurations.Add(new UserGroupsConfiguration());
...
}
Now your task is simpler...
foreach (var csvUser in csvSource)
{
User oUser = new User();
oUser.Firstname = csvUser.Firstname;
// Find Group
var group = context.Groups.Find(csvUser.GroupID);
if(group == null)
{
// TODO: Handle case that group is null
}
else
{
// Group found, assign it to the new user
oUser.UserGroups.Add(new UserGroup { Group = group });
context.Users.Add(oUser);
}
}
context.SaveChanges();