I'm trying to replicate a SQL statement in EF Core but cant seem to find a way to do it, to set the scene I have the following table structure
Slot -> SlotInstance -> SlotInstanceUser
(a Slot can have many SlotInstances, a SlotInstance can have many SlotInstanceUsers)
When a user registers for a SlotInstance a record is created in SlotInstanceUsers storing the SlotInstanceId and UserId - all good there.
I'm able to write SQL to get a list of slot instances which the user has not registered for e.g.
SELECT
S.StartDate, S.EndDate, S.StartTime, S.EndTime, S.DayOfWeek,
SI.Date
FROM
Slot S WITH (NOLOCK)
INNER JOIN
SlotInstance SI WITH (NOLOCK) ON S.Id = SI.SlotId
WHERE
SI.ID not in (
SELECT
SlotInstanceId
FROM
SlotInstanceUser SIU WITH (NOLOCK)
WHERE
SIU.UserId = #UserID
)
ORDER BY
SI.Date
But I just cant seem to replicate this in EF core - what am I missing?
You can write the LINQ query pretty much the same way as the SQL query. Just remember that in LINQ select is last, variables (aliases) are mandatory, and the equivalent of SQL NOT IN is !Contains. e.g.
var query =
from s in db.Slots
join si in db.SlotInstances on s.Id equals si.SlotId
where !(from siu in db.SlotInstanceUsers
where siu.UserId == userId)
select siu.SlotInstanceId).Contains(si.Id)
orderby si.Date
select new
{
s.StartDate, s.EndDate, s.StartTime, s.EndTime, s.DayOfWeek,
si.Date
};
But in EF Core you have more options, especially for joins, since normally the relationships (and associated joins) are encapsulated with navigation properties. So the model you are describing with words in EF Core/C# terms is something like
public class Slot
{
public int Id { get; set; }
// Other properties...
public ICollection<SlotInstance> SlotInstances { get; set; }
}
public class SlotInstance
{
public int Id { get; set; }
// Other properties...
public Slot Slot { get; set; }
public ICollection<SlotInstanceUser> SlotInstanceUsers { get; set; }
}
public class SlotInstanceUser
{
public int Id { get; set; }
// Other properties...
public SlotInstance SlotInstance { get; set; }
}
and the query would be like
var query =
from s in db.Slots
from si in s.SlotInstances
where !si.SlotInstanceUsers.Any(siu => siu.UserId == userId)
orderby si.Date
select new
{
s.StartDate, s.EndDate, s.StartTime, s.EndTime, s.DayOfWeek,
si.Date
};
(this actually translates to SQL NOT EXISTS, but that's not essential).
And if you don't need projection, but simply slot instances (with slot info) which the user has not registered for, then it would be simply
var query = db.SlotInstances
.Include(si => si.Slot)
.Where(si => !si.SlotInstanceUsers.Any(siu => siu.UserId == userId))
Related
Using EF core 5 and ASP.NET Core 3.1, I am trying to get a filtered collection based on a condition on its grandchildren collection.
I have the following entities:
public class Organisation
{
public int Id { get; set; }
public int? OrganisationId { get; set; }
public IEnumerable<Customer> Customers { get; set; }
}
public partial class Customer
{
[Key]
public uint Id { get; set; }
public int? EmployerId { get; set; }
public int? OrganisationId { get; set; }
public List<TimecardProperties> TimecardsProperties { get; set; }
}
public partial class TimecardProperties
{
[Key]
public int Id { get; set; }
public int? EmployerId { get; set; }
public int? Week { get; set; }
public short? Year { get; set; }
}
The goal is to get all Organisations that have at least one customer and the customer has at least 1 timecard property that is in week=34 and year=2021.
So far I have tried the following:
////necessary join to get Organisations for user id
IQueryable<Organisation> ouQuery = (from cou in _dbContext.Organisations
join uou in _dbContext.table2 on cou.OrganisationId equals uou.OrganisationId
where uou.UsersId == int.Parse(userId)
select cou)
.Where(cou => cou.Customers.Where(c => c.TimecardsProperties.Count > 0).Any())
.Include(cou => cou.Customers.Where(c => c.TimecardsProperties.Count > 0))
.ThenInclude(c => c.TimecardsProperties.Where(tc => tc.tWeek == 34 && tc.Year > 2020))
;
This returns a organisation list that each have a customers list but some customers have a count of timecards 0. I don't want to have organisation in the returned list that does not have at least one item in the timecards collection.
Also, it is too slow, and if I try to filter the produced list its even
slower (over 15 seconds)
I have also tried a raw sql query on the organisation db context but it is again very slow:
select distinct count(id) from organisation a where organisation_id in (
select organisation_id from customers where employer_id in (select distinct employer_id from timecards a
inner join timecard_components b on a.id=b.timecards_id
where week IN(
34) and year in (2021,2021) and invoice !=0 and type = 'time'
group by employer_id, week)
);
In general, I want to know the the total
count of the returned organisation collection for pagination (so I don't need to include all attributes of each entity)
as well as return only a part of the correct results, which satisfy the conditions,
an organisation list that has at least 1 timecards in
their customers by executing the query in the end like so:
ouQuery.Skip((page - 1) * pageSize).Take(pageSize).ToListAsync();
I have also tried the EntityFramework.Plus and projection with no results.
How could I write this to achieve getting the total count of the organisation list and a part of these results (first 10) to display to the user?
Use navigation properties. This is the query you want:
var orgsQuery = dbContext.Organizations
.Where( o => o.Customers.Any( c =>
c.TimecardProperties.Any( tp =>
tp.Year = 2021
&& tp.Week = 34 ) ) );
Add includes and other predicates as needed
Hi good day I am new with Entity Framework. I just wanna to know if there is a way I could improve my implementation. Here are the codes.
public async Task<List<Record>> GetRecordsByBatchId(string batchId, string source)
{
List<string> idList = new List<string>();
//[1] Get all parent ID from table 1 with a filter of source and batchId
var parentIds= await _context.Set<FirstTable>()
.Where(a => a.IsActive
&& a.BatchId.Equals(batchId)
&& a.Source.Equals(source)).Select(b => b.ParentId).ToListAsync();
if (parentIds.Count() == 0)
{
return new List<Record>();
}
//[2] Query idNumber of each parentId from [1] to SecondTable
List<long> idNumber = await _context.Set<SecondTable>()
.Where(a => parentIds.Contains(a.Id))
.Select(b => b.IdNumber).ToListAsync();
//[3] Query Record/s that contains idNumber from previous query [2]. it is possible that 1 or
//more records has same idNumber
List<Risk> recordByIdNumber = await _context.Set<SecondTable>()
.Where(a => idNumber.Contains(a.IdNumber)).ToListAsync();
//[4] In this part I just want to group the records in [3] by Id number and sort each group
//by its endorsementNumber in descending order and return the record with highest endorsement
//number for each group
return (from record in recordByIdNumber
group record by record.IdNumber into g
orderby g.Key
select g.OrderByDescending(risk =>risk.EndorsementNumber).FirstOrDefault()).ToList();
}
}
The model for the FirstTable
public class FirstTable
{
public Guid? ParentId{ get; set; }
public string BatchId { get; set; }
public string Source { get; set; }
public bool IsActive { get; set; }
}
The model for the SecondTable
public class SecondTable
{
public Guid Id{ get; set; }
public int EndorsementNumber { get; set; }
public long IdNumber { get; set; }
}
Note: I just include the necessary properties in the model.
This approach is working as expected. I just wanna know if there is a possibility that these queries could be optimized that there is only 1 query for the SecondTable table.
Any help would be greatly appreciated, thanks in advance.
Yes, queries 1-3 can and should be combined. In order to do that you need, to have navigation properties in your model. It seems that there is one-to-many relationship between FirstTable and SecondTable. Let's use Customer and Order instead.
class Customer {
int CustomerId
string BatchId
ICollection<Order> Orders
}
class Order {
int OrderId
int CustomerId
Customer Customer
Risk Risk
}
in which case you just write third query as
List<Risk> = await _context.Orders.Where(o => o.Customer.BatchId == batchId)
.Select(o => o.Risk).ToListAsync();
Obviously, I am only guessing the structure and the relationship. But hopefully, this can get you started. For me Contains() is "code smell". There is a high chance that there will be large list out of your first query, and contains() will produce a huge IN clause in the database, that can easily crash the system
var parentIds = _context.Set<FirstTable>()
.Where(a => a.IsActive
&& a.BatchId.Equals(batchId)
&& a.Source.Equals(source)).Select(b => new { b.parentId });
var risks = await (from s in _context.Set<SecondTable>()
join p in parentIds on s.Id equals p.parentId
join r in _context.Set<SecondTable>() on s.IdNumber equals r.IdNumber
select r).GroupBy(r=>r.IdNumber)
.Select(r=> r.OrderByDescending(risk =>risk.EndorsementNumber).FirstOrDefault())
.ToArrayAsync();
return risks;
You can have 1 query instead of 3. It will perform better as the number of the rows from the first query grows.
EDIT: As #SvyatoslavDanyliv mentioned in the comments, group-take operations may not work depending on the version of the EF and the provider you use. You may need to separate the query and the group by operation like below :
var result = await (from s in _context.Set<SecondTable>()
join p in parentIds on s.Id equals p.parentId
join r in _context.Set<SecondTable>() on s.IdNumber equals r.IdNumber
select r).ToArrayAsync();
var risks = result.GroupBy(r=>r.IdNumber)
.Select(r=> r.OrderByDescending(
risk =>risk.EndorsementNumber).FirstOrDefault())
.ToArray();
return risks;
I'm using Automapper version 10.0 with EF Core 5 and .NET 5.
It would seem that automapper causes a query that selects every property separately:
SELECT FALSE, a11."Avatar", ((
SELECT a0."Id"
FROM "AspNetUserRoles" AS a
INNER JOIN "AspNetRoles" AS a0 ON a."RoleId" = a0."Id"
WHERE a11."Id" = a."UserId"
ORDER BY a0."Order" DESC
LIMIT 1) IS NULL), (
SELECT a2."Color"
FROM "AspNetUserRoles" AS a1
INNER JOIN "AspNetRoles" AS a2 ON a1."RoleId" = a2."Id"
WHERE a11."Id" = a1."UserId"
ORDER BY a2."Order" DESC
LIMIT 1), (
SELECT a4."Id"
FROM "AspNetUserRoles" AS a3
INNER JOIN "AspNetRoles" AS a4 ON a3."RoleId" = a4."Id"
WHERE a11."Id" = a3."UserId"
ORDER BY a4."Order" DESC
LIMIT 1), (
SELECT a6."IsStaff"
FROM "AspNetUserRoles" AS a5
INNER JOIN "AspNetRoles" AS a6 ON a5."RoleId" = a6."Id"
WHERE a11."Id" = a5."UserId"
ORDER BY a6."Order" DESC
LIMIT 1), (
SELECT a8."Name"
FROM "AspNetUserRoles" AS a7
INNER JOIN "AspNetRoles" AS a8 ON a7."RoleId" = a8."Id"
WHERE a11."Id" = a7."UserId"
ORDER BY a8."Order" DESC
LIMIT 1), COALESCE((
SELECT a10."Order"
FROM "AspNetUserRoles" AS a9
INNER JOIN "AspNetRoles" AS a10 ON a9."RoleId" = a10."Id"
WHERE a11."Id" = a9."UserId"
ORDER BY a10."Order" DESC
LIMIT 1), 0), a11."Title", a11."UserName", t."Body", t."CommentsThreadId", t."DateTime", t."Id"
FROM (
SELECT c."Id", c."AuthorId", c."Body", c."CommentsThreadId", c."DateTime"
FROM "Comments" AS c
WHERE c."CommentsThreadId" = #__threadId_0
ORDER BY c."DateTime" DESC
LIMIT #__p_1 OFFSET 0
) AS t
INNER JOIN "AspNetUsers" AS a11 ON t."AuthorId" = a11."Id"
ORDER BY t."DateTime" DESC
As you can see, pretty much every single property of RoleDto is being SELECTed separately, instead of being selected just once and having its columns mapped using the AS keyword.
Mappings are done to the following DTOs, property names being true to source entities:
public class UserSimpleDto
{
public string UserName { get; set; }
public string Avatar { get; set; }
public string Title { get; set; }
public RoleDto Role { get; set; }
}
It seems the above and the below are the main culprits. OgmaUser, the source entity for the mapping of the above, contains a list of user roles, of which only one should be present in the target DTO.
OgmaUser.Roles is mapped using EF Core 5's many-to-many setup, but I did try to use an explicit join entity and the result was the exact same.
public class RoleDto
{
public long Id { get; set; }
public string Name { get; set; }
public string? Color { get; set; }
public bool IsStaff { get; set; }
public int Order { get; set; }
}
As you can see, each property of the above is being SELECT ... AS ... separately.
public class CommentDto
{
public long Id { get; set; }
public long CommentsThreadId { get; set; }
public UserSimpleDto Author { get; set; }
public DateTime DateTime { get; set; }
public string Body { get; set; }
}
I doubt this particular DTO has something to do with the result, since the issue occurs with other DTOs that contain UserSimpleDto, but I thought I should include it to present the fullest possible picture.
And the mappings are as follows:
CreateMap<OgmaUser, UserSimpleDto>()
.ForMember(
usd => usd.Role,
opts => opts.MapFrom(u => u.Roles.OrderByDescending(r => r.Order).FirstOrDefault())
);
CreateMap<OgmaRole, RoleDto>();
CreateMap<Comment, CommentDto>()
.ForMember(
cd => cd.Body,
opts => opts.MapFrom(c => Markdown.ToHtml(c.Body, null))
);
The query is generated from the following method:
public async Task<IEnumerable<CommentDto>> GetPaginated(long threadId, int page)
{
return await _context.Comments
.Where(c => c.CommentsThreadId == threadId)
.OrderByDescending(c => c.DateTime)
.ProjectTo<CommentDto>(_mapper.ConfigurationProvider)
.Skip(Math.Max(0, page - 1) * _config.CommentsPerPage)
.Take(_config.CommentsPerPage);
.AsNoTracking()
.ToListAsync();
}
Stack complains that my question is mostly code, and I suppose I do understand that. I've seen myself many people just posting their code, expecting to have it fixed, or even posting their homework and asking for it to be solved.
In this case, however, I believe that the code speaks louder than whatever explanation I could provide, so, true, non-code parts of this question are few and far between. That's the reason why I'm even writing those last paragraph, in fact, as SO will not let me submit my question otherwise.
I believe the best answer is to change UserSimpleDto to
public class UserSimpleDto
{
public string UserName { get; set; }
public string Avatar { get; set; }
public string Title { get; set; }
public IEnumerable<RoleDto> Roles { get; set; }
}
to load all user roles – there aren't all that many – and simply getting only the first one when it's just the first one that's needed. A negligible overhead in the application layer, so to speak, but the resulting query is much simpler which should negate that overhead.
Project after ToListAsync(). Don't pass IQueryable to ProjectTo method.
public async Task<IEnumerable<CommentDto>> GetPaginated(long threadId, int page)
{
return (await _context.Comments
.Where(c => c.CommentsThreadId == threadId)
.OrderByDescending(c => c.DateTime)
.Skip(Math.Max(0, page - 1) * _config.CommentsPerPage)
.Take(_config.CommentsPerPage);
.AsNoTracking()
.ToListAsync())
.ProjectTo<CommentDto>(_mapper.ConfigurationProvider);
}
I have 3 tables: Person, PersonFriend, PersonGroup.
Using LINQ, i want to join the 3 tables, filter using a dynamically generated where clause, and select custom columns with flattened rows (flattened one-to-many relationship table columns).
Pseudo-SQL design:
CREATE TABLE Person (int id, varchar socialclass, date createddate);
CREATE TABLE Person_Friend (int id, id personid references person.id, id friendpersonid references person.id, varchar friendtype);
CREATE TABLE Person_Group (int id, int memberid references person.id, varchar membershiplevel);
Entities:
public class Person
{
public int Id { get; set; }
public string SocialClass { get; set; }
public DateTime? CreatedDate { get; set; }
public ICollection<PersonFriend> Friend { get; set; }
public ICollection<PersonGroup> Group { get; set; }
}
public class PersonFriend
{
public int Id { get; set; }
public int PersonId { get; set; }
public int FriendPersonId { get; set; }
public string FriendType { get; set; }
}
public class PersonGroup
{
public int Id { get; set; }
public int MemberId { get; set; }
public string MembershipLevel { get; set; }
}
query syntax LINQ:
var queryResult = from person in _context.Person
join friend in _context.PersonFriend on person.Id equals friend.FriendPersonId
join group in _context.PersonGroup on person.Id equals group.MemberId
where (friend.PersonId == 1 && friend.FriendType == "type1") || (friend.PersonId == 3 && friend.FriendType == "type2") || ...
select new { person.Id, person.SocialClass, person.CreatedDate, friend.FriendPersonId, friend.FriendType, group.Id, group.MembershipLevel };
Notice the where clause; Given a list of { PersonId, FriendType } object, I want to build the where clause like above.
Since I could not figure building a dynamic where clause for a query syntax LINQ,
I tried converting it to the Method syntax LINQ statement so i can leverage the PredicateBuilder (http://www.albahari.com/nutshell/predicatebuilder.aspx) but I run into the problem during Selecting one-to-many things into a flattened object.
var methodResult = _context.Person
.Include(x => x.Friend)
.Include(x => x.Group)
.Select(person => new { person.Id, person.SocialClass, person.CreatedDate, person.friend.FriendPersonId, person.friend.FriendType, person.group.Id, person.group.MembershipLevel });
notice that the above Select is not possible because friend is a ICollection.
I also tried using the above query syntax LINQ statement without the where clause, making it return a object instead of an annonymous object, and then calling the method .Where() with the predicate builder. But the built expression runs into LINQ => Entity Framework SQL conversion error and executes the where in the application, not in DB.
var queryResultWithoutWhere = from person in _context.Person
join friend in _context.PersonFriend on person.Id equals friend.FriendPersonId
join group in _context.PersonGroup on person.Id equals group.MemberId
select new SelectedObject { PersonId = person.Id, SocialClass = person.SocialClass, CreatedDate = person.CreatedDate, FriendId = friend.FriendPersonId, FriendType = friend.FriendType, GroupId = group.Id, MembershipLevel = group.MembershipLevel };
var predicate = PredicateBuilder.New<SelectedObject>(false);
foreach (var searchObject in searchRequestObjects)
{
predicate.Or(p => p.FriendPersonId == searchObject.FriendPersonId && p.FriendType == searchObject.FriendType);
}
var result = queryResultWithoutWhere.Where(predicate).ToList();
I feel like I tried everything I could, and I cannot seem to generate this SQL. Last resort would be writing a raw SQL string and then executing it, but I really would like to get this working with Entity Framework.
How would I accomplish creating a dynamic where clause, select into a custom flattened object, and have entity framework generate the SQL?
You can use SelectMany to flatten the collections:
var methodResult = Persons
.Include(x => x.Friend)
.Include(x => x.Group)
.SelectMany(person =>
person.Friend.SelectMany(friend =>
person.Group.Select(group =>
new {
person.Id,
person.SocialClass,
person.CreatedDate,
friend.FriendPersonId,
friend.FriendType,
GroupId = group.Id,
group.MembershipLevel
}
)
)
);
I have a Product table that has no relation defined to the translation table. I added a Translation property to the Product POCO as [NotMapped].
**My Product POCO: **
public partial class Product
{
public int ProductID { get; set; }
public double Price { get; set; }
[NotMapped]
public virtual Translation Translation{ get; set; }
/** Other properties **/
}
I also have a Translation table, and like the name says, it contains all the translations.
Now, the right translation can be retrieved from the database by providing three parameters: LanguageID, TranslationOriginID and ValueID.
LanguageID: ID from the language that the user has defined.
TranslationOriginID: Simply said, 'What table contains the entity that I want the translation for?' In other words, this ID points to another table that contains all possible origins. An origin is a table/entity that can have a translation. E.g: The origin in this example is Product.
ValueID: This is the ID of the entity that I want a translation for.
My Translation POCO:
public partial class Translation
{
public int TranslationID { get; set; }
public byte LanguageID { get; set; }
public short TranslationOriginID { get; set; }
public int ValueID { get; set; }
public string TranslationValue { get; set; }
/** Other properties **/
public virtual TranslationOrigin TranslationOrigin { get; set; }
public virtual Language Language { get; set; }
}
When I want to retrieve all products with their Translation, I execute this code:
List<Product> products = context.Products.ToList();
foreach (Product product in products)
{
product.Translation = context.Translations.FirstOrDefault(y => y.LanguageID == 1 && y.TranslationOriginID == 2 && y.ValueID == product.ProductID);
}
Like you can see, I execute for every product in the list another query to get the translation.
My question:
Is it possible to get all the products and their translation in one query? Or even that I automatically retrieve the right translation when I select a product?
I already tried an .Include() and a .Select(). It didn't work, maybe I did something wrong?
I also tried this method, didn't work either.
Btw, I use Entity framework 5 with .NET 4 (so, Entity Framework 4.4).
Thanks in advance.
Greetings
Loetn
Answer
With the example given by Ed Chapel, I came up with a solution.
return (from p in context.Products
join t in context.Translations
on new
{
Id = p.ProductID,
langId = languageID,
tOriginId = translationOriginID
}
equals new
{
Id = d.ValueID,
langId = d.LanguageID,
tOriginId = d.TranslationOriginID
}
into other
from x in other.DefaultIfEmpty()
select new
{
Product = p,
Translation = x
})
.ToList().ConvertAll(x => new Product()
{
Code = x.Product.Code,
Translation = x.Translation,
/** Other properties **/
});
I don't like proper LINQ in most cases. However, join is one scenario where the LINQ is easy than the extensions methods:
from p in context.Products
join t in context.Translations
on t.ValueID equals p.ValueID
&& t.LanguageID == 1
&& t.TranslationOriginID == 2
into joinT
from x in joinT
select new {
Product = p,
Translation = t,
};
You then loop over the result setting x.Product.Translation = x.Translation.
First of all you should realize that your translations table is not structured like a dba would like it You have a non enforced relationship because depending on the OriginId your valueId references a different table.
Because of this you cannot use lazy loading or includes from EF.
My best idea at this point would to manually join the table on an anonymous type(to include your originId). Afterwards you can iterate over the results to set the translation property
The result would look like this :
var data = from p in context.Products
join pt in context.Translations on new{p.Id,2} equals new {pt.ValueId, pt.OriginId} into trans
select new {p, trans};
var result = data.ToList().Select( a =>
{
a.p.Translations = a.trans;
return a.p;
}).ToList();
With the example that Ed Chapel proposed as a solution, I came up with this.
return (from p in context.Products
join t in context.Translations
on new
{
Id = p.ProductID,
langId = languageID,
tOriginId = translationOriginID
}
equals new
{
Id = d.ValueID,
langId = d.LanguageID,
tOriginId = d.TranslationOriginID
}
into other
from x in other.DefaultIfEmpty()
select new
{
Product = p,
Translation = x
})
.ToList().ConvertAll(x => new Product()
{
Code = x.Product.Code,
Translation = x.Translation,
/** Other properties **/
});