I have a query with three sub queries and my problem is that the sub queries are run for each Country (n+1).
I have boiled down my query to make it easier to read, since the main query is around 70 rows, and changed the domain to make it more understandable.
I've tried including Cities/Mountains/Rivers and running .ToList() on the sub queries, but to no avail.
// The CountryDto class I'm selecting to.
public class CountryDto
{
public string CountryName { get; set; }
public IEnumerable<CityDto> CityDtos { get; set; }
public IEnumerable<MountainDto> MountainDtos { get; set; }
public IEnumerable<RiverDto> RiverDtos { get; set; }
}
// The query
var query = _db.Countries
.Select(country => new CountryDto
{
CountryName = country.Name,
CityDtos = country.Citites
.Where(city => city.Population > 10000)
.Select(city => new CityDto
{
Name = city.Name,
}),
MountainDtos = country.Mountains
.Where(mountain => mountain.Height > 100)
.Select(mountain => new MountainDto
{
Name = mountain.Name,
}),
RiverDtos = country.Rivers
.Where(river => river.Length > 1000)
.Select(river => new RiverDto
{
Name = river.Name,
}),
})
.Where(c => c.CityDtos.Any() || c.MountainDtos.Any() || c.RiverDtos.Any());
var totalCount = query.Count();
var countries = await query.ToListAsync();
Entity Framework Core supports parts of the query being evaluated on the client and parts of it being pushed to the database. It is up to the database provider to determine which parts of the query will be evaluated in the database.
In your case, all .Any parts are evaluated in the client side I guess. You can configure your code to throw an exception for client evaluation.
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder
.UseSqlServer(#"Server=(localdb)\mssqllocaldb;Database=EFQuerying;Trusted_Connection=True;")
.ConfigureWarnings(warnings => warnings.Throw(RelationalEventId.QueryClientEvaluationWarning));
}
For more info https://learn.microsoft.com/en-us/ef/core/querying/client-eval
Ended up splitting the query into a .Where() part for the count and a .Select() part for the result, which removed my n+1 issue.
var query = await _db.Countries
.Include(c => c.Cities)
.Include(c => c.Mountains)
.Include(c => c.Rivers)
.Where(c => c.Cities.Any(city => city.Population > 10000)
|| c.Mountains.Any(mountain => mountain.Heigh > 1000)
|| c.River.Any(river => river.Length > 100000))
.Where(c => c.Cities.Any() || c.Mountains.Any() || c.Rivers.Any())
.ToListAsync();
var totalCount = query.Count();
var countries = query
.Select(country => new CountryDto
{
CountryName = country.Name,
CityDtos = country.Citites
.Select(city => new CityDto
{
Name = city.Name,
}),
MountainDtos = country.Mountains
.Select(mountain => new MountainDto
{
Name = mountain.Name,
}),
RiverDtos = country.Rivers
.Select(river => new RiverDto
{
Name = river.Name,
}),
})
.ToList();
Related
Here is my query as it stands now:
Goals = await (from p in _context.FixtureActivityTb
where p.ActivityType.Trim() == "G"
group p by p.PlayerId into x
join j in _context.PlayerTb on x.Key equals j.PlayerId
select new Stats
{
Name = j.GivenName,
pID = j.PlayerId,
TeamId = j.TeamId,
Count = x.Count()
})
.OrderByDescending(s => s.Count)
.ThenBy(s => s.Name)
.Take(10)
.ToListAsync();
As you can see this is a mix of method and query. I should be able to do a join, where, and add the data to a custom class all in Method, however so far I have not been able to put it all together. Any guidance will be appreciated.
I will include these other items, however, I think they are beside the point.
Variable Declaration:
public IList<Stats> Goals { get; set; }
Class:
public class Stats
{
public Guid pID { get; set; }
public string TeamId { get; set; }
public string Name { get; set; }
public int Count { get; set; }
}
If i understand your question (and the lack of coffee isn't affecting me), to get this all to a Linq chain method, it should be as simple as
Goals = await _context.FixtureActivityTb.Where(p => p.ActivityType.Trim() == "G")
.GroupBy(p => p.PlayerId)
.Join(_context.PlayerTb, x => x.Key, j => j.PlayerId, (x, j)
=> new Stats
{
Name = j.GivenName,
pID = j.PlayerId,
TeamId = j.TeamId,
Count = x.Count()
})
.OrderByDescending(s => s.Count)
.ThenBy(s => s.Name)
.Take(10)
.ToListAsync();
It's quite acceptable to mix the two, especially where you have a base query, and a paging part, as here. You can even compose the queries over multiple statements, eg:
var q = from p in _context.FixtureActivityTb
where p.ActivityType.Trim() == "G"
group p by p.PlayerId into x
join j in _context.PlayerTb on x.Key equals j.PlayerId
select new Stats
{
Name = j.GivenName,
pID = j.PlayerId,
TeamId = j.TeamId,
Count = x.Count()
};
Goals = await q.OrderByDescending(s => s.Count)
.ThenBy(s => s.Name)
.Take(10)
.ToListAsync();
Updated with other relevant classes
public class TrendItem
{
public string ItemTitle { get; set; }
public IEnumerable<string> ItemValues { get; set; }
}
public class TrendValue
{
public int Id { get; set; }
public TrendResultType TrendType { get; set; }
public string Value { get; set; }
public Trend Trend { get; set; } // contains DateRecorded property
}
Please see the function below that leverages on EF Core (2.1):
public async Task<List<TrendItem>> GetTrends(
int companyId,
TrendResultType type,
DateTimeOffset startDate,
DateTimeOffset endDate,
RatingResultGroup group
)
{
var data = _dataContext.TrendValues.Where(rr =>
rr.Trend.CompanyId == companyId &&
rr.TrendType == type &&
rr.Trend.DateRecorded >= startDate &&
rr.Trend.DateRecorded <= endDate);
return await data.GroupBy(rr => new { rr.Trend.DateRecorded.Year, rr.Trend.DateRecorded.Month })
.Select(g => new TrendItem() { ItemTitle = $"{g.Key.Year}-{g.Key.Month}", ItemValues = g.Select(rr => rr.Value) })
.ToListAsync();
}
I'm getting problems, specifically with the portion g.Select(rr => rr.Value), where I intended to select a collection of values (strings).
Whenever I try to change that to something else like g.Sum(rr => int.Parse(rr.Value)), it works fine. It's the retrieval of the collection that seems to be a problem.
I always get ArgumentException: Argument types do not match.
Is this due to the async function?
Hmm, I'd start by looking to simplify the data retrieval into an anonymous type to perform the grouping, then transform into the view model. It may be that EF is getting a bit confused working with the grouped items values.
It looks like you want to group by date (year-month) then have a list of the values in each group:
var query = _dataContext.TrendValues
.Where(rr =>
rr.Trend.CompanyId == companyId
&& rr.TrendType == type
&& rr.Trend.DateRecorded >= startDate
&& rr.Trend.DateRecorded <= endDate)
.Select(rr => new
{
TrendYear = rr.Trend.DateRecorded.Year,
TrendMonth = rr.Trend.DateRecorded.Month,
rr.Value
}).ToListAsync();
var data = query.GroupBy(rr => new { rr.TrendYear, rr.TrendMonth })
.Select(g => new TrendItem() { ItemTitle = $"{g.Key.Year}-{g.Key.Month}", ItemValues = g.ToList().Select(rr => rr.Value).ToList() })
.ToList();
return data;
which could be simplified to:
var query = _dataContext.TrendValues
.Where(rr =>
rr.Trend.CompanyId == companyId &&
rr.TrendType == type &&
rr.Trend.DateRecorded >= startDate &&
rr.Trend.DateRecorded <= endDate)
.Select(rr => new
{
TrendDate = rr.Trend.DateRecorded.Year + "-" + rr.Trend.DateRecorded.Month,
rr.Value
}).ToListAsync();
var data = query.GroupBy(rr => rr.TrendDate)
.Select(g => new TrendItem() { ItemTitle = $"{g.Key}", ItemValues = g.ToList().Select(rr => rr.Value).ToList() })
.ToList();
return data;
The selecting of the TrendDate in the query may need some work if Trend.Year and Trend.Month are numeric fields.
My model:
public class StatusDetailsViewModel
{
public string Status { get; set; }
public long CountNo { get; set; }
}
And my function to get the data:
public List<StatusDetailsViewModel> CheckMeetingStatus(long actionId)
{
List<StatusDetailsViewModel> statusDetails;
var statuses = _igniteDb.myTable.Where(a => a.actionId == actionId)
.GroupBy(a => new { a.Status, a.ElectionGroup }).GroupBy(c => new { c.Key.Status})
.Select(b => new { Status = b.Key.Status, CountNo = b.Count()}).ToList();
//How to Map statuses to statusDetails??
return statusDetails;
}
I've being trying to use auto mapper to achieve this. But I am not sure how to configure my mapper. Any ideas?
Why do you have to map at in your code. Instead of creating anonymous type, you can create the object of StatusDetailsViewModel. Change the select statement as Select(b => new StatusDetailsViewModel() { Status = b.Key.Status, CountNo = b.Count()}
If you want to use AutoMapper then you should define the source and target types while defining the Map/profile. You can not map the anonymous type with AutoMapper.
Your code to create the object of StatusDetailsViewModel (without using Automapper as you don't require for the code you have asked for)
public List<StatusDetailsViewModel> CheckMeetingStatus(long actionId)
{
List<StatusDetailsViewModel> statusDetails;
var statuses = _igniteDb.myTable.Where(a => a.actionId == actionId)
.GroupBy(a => new { a.Status, a.ElectionGroup }).GroupBy(c => new { c.Key.Status})
.Select(b => new StatusDetailsViewModel () { Status = b.Key.Status, CountNo = b.Count()}).ToList();
//How to Map statuses to statusDetails??
return statusDetails;
}
Just because you're returning a List<StatusDetailsViewModel>, you don't need AutoMapper here just project your data by using directly your view model like below:
List<StatusDetailsViewModel> statusDetails = _igniteDb.myTable.Where(a => a.actionId == actionId)
.GroupBy(a => new { a.Status, a.ElectionGroup })
.GroupBy(c => new { c.Key.Status})
.Select(b => new StatusDetailsViewModel { /* <--- Here you instantiate your view model */
Status = b.Key.Status,
CountNo = b.Count()}
).ToList();
I want to filter rows from table Users using Entity Framework. Users and Roles have a n-n relationship:
public class Users
{
public Users()
{
Roles = new HashSet<Role>();
}
public Int64 Id { get; set; }
public string UserName { get; set; }
public virtual ICollection<Roles> Roles { get; set; }
}
public class Roles
{
public Roles()
{
Users = new HashSet<Users>();
}
public string Id { get; set; }
public string Name { get; set; }
public string DisplayName { get; set; }
public virtual ICollection<Users> Users { get; set; }
}
AND:
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<Roles>()
.HasMany(e => e.Users)
.WithMany(e => e.Roles)
.Map(m => m.ToTable("UserRoles")
.MapLeftKey("RoleId")
.MapRightKey("UserId"));
...............
}
I want to filter Users based on UserName and their Roles. The number of users is large, so I want to execute the query only after applying the conditions and pagination.
I want to find users that their name includes input's UserName (if is not null or empty), and they have all roles of input's Roles list.
What code should I use to achieve this?
public Users FilterUsers(UserSearchDto inputParam, int pageSize, int pageNumber)
{
var skip = (pageNumber - 1) * pageSize;
var take = pageSize;
int total = 0;
var isNotUserNameFiltered = string.IsNullOrEmpty(inputParam.UserName);
var isNotRolesFiltered = inputParam.Roles?.Count <= 0; //Roles type is List<string>
var q = db.Users
.Include("Roles")
.AsQueryable();
q = q.Where(user =>
(isNotUserNameFiltered || user.UserName.Contains(inputParam.Item.UserName)) &&
(isNotRolesFiltered // || // what should I write here?
));
// get totalcount
total = q.Count();
// execute and get result from database
var data = q.Select(user => new Users
{
Id = user.Id,
UserName = user.UserName,
Roles = user.Roles.Select(role => role.DisplayName).ToList()
})
.Distinct()
.OrderBy(u => u.Id)
.Skip(skip)
.Take(take)
.ToList();
}
what should I write here?
The standard way is to use All method:
inputParam.Roles.All(roleName => user.Roles.Any(role => role.Name == roleName))
(you might need to use role.DisplayName depending of what is in inputParam.Roles)
But in EF6 you will get better SQL translation if you count the matches and compare with the count of the desired set:
user.Roles.Count(role => inputParam.Roles.Contains(role.Name)) == inputParam.Roles.Count
Just try like this;
(isNotRolesFiltered || user.Roles.All(r => inputParam.Roles.Contains(r))
You can simply use .Contains/Any/All/etc for simple-type collections in IQueryable.Where clause (roles.All(...) in code below)
Here is abstract code (based on your) with username/role filters:
var skip = (pageNumber - 1) * pageSize;
var take = pageSize;
var query = db.Users
.AsNoTracking()
.Include("Roles")
.AsQueryable();
if (!string.IsNullOrEmpty(username))
{
var username = inputParam.UserName.ToLower();
query = query.Where(u => u.UserName.ToLower().Contains(username));
}
if (inputParam.Roles?.Count > 0)
{
var roles = db.Roles
.AsNoTracking()
.Where(r => inputParam.Roles.Contains(r.Name))
.Select(r => r.Id)
.ToList();
query = query.Where(u => roles.All(inputRole => u.Roles.Any(userRole => userRole.Id == inputRole)));
}
var total = query.Count();
var result = query
.Skip(skip)
.Take(take)
.OrderBy(u => u.Id)
.Select(...)
.ToList();
Lines 8-12 – prepare query to filter by username (only in case
username not null and not empty)
Lines 14-22 – prepare query to filter by roles
Line 14 – get total number of filtered users
Lines 25-30 – take only users for specific page and transform it to
DTO
Notes:
Query not executed until you not called .AsEnumerable() or .ToList()
methods.
I preloaded Role Ids because SQL Server should make a lot of comparisons and comparison between Ids (int) faster than comparison between Role.Name (string/nvarchar). Lines 19-23
When I try to include an ICollection property, it works, but if I try to include Enumerable, it fails to do so. I'm trying to have Eager loading, may I ask how abouts do Eager load an IEnumerable.
Person.cs
public virtual ICollection<Ownership> Ownerships { get; set; }
public virtual IEnumerable<Business> Businesses { get { return Ownerships.Where(x => x.Active).Select(ownership => ownership.Business); } }
My current code setup(lazy loading) that works
Entity = UnitOfWork.PersonRepository.Get(x => x.Id == id).First();
Entity.Businesses.ForEach(
x =>
SelectedBusinesses.Add(
Businesses.FirstOrDefault(b => b.Id == x.Id) ?? new BusinessDTO { Id = x.Id, Name = x.Name }));
what I tried to do - crashes
Entity = UnitOfWork.PersonRepository.Get(x => x.Id == id, includeProperties: new[] { "Ownerships", "Businesses"}).First();
Entity.Businesses.ForEach(
x =>
SelectedBusinesses.Add(
Businesses.FirstOrDefault(b => b.Id == x.Id) ?? new BusinessDTO { Id = x.Id, Name = x.Name }));