We Keep Coding

EF Core SQL Filter Translation

I'm using EF Core 5.0.1 with ASP NET 5.0.1 Web API and I want to build a query with PredicateBuilder using LinqKit.Microsoft.EntityFrameworkCore 5.0.2.1
For the purposes of the question I simplified my model to:
public class User
{
public long IdUser { get; set; }
public string Name { get; set; }
public virtual ICollection<UserDepartment> UserDepartments { get; set; }
}
public class Department
{
public long IdDepartament { get; set; }
public string Name { get; set; }
public virtual ICollection<UserDepartment> UsersDepartment { get; set; }
}
public UserDepartment
{
public long IdUser { get; set; }
public long IdDepartment { get; set; }
public virtual Department { get; set; }
public virtual User { get; set; }
}
One User can have many Departaments and one Departament can have many Users
Three models have its correspondent table in SQL Server and a IEntityTypeConfiguration class with the appropriate relationships set up.
All I want to achieve is to search any User that belongs to any Departament which Department.Name is in a List<String>.
The List<String> contains a list of keywords, not the exact Department Name
Department table has this kind of rows:
IdDepartment
Name
1
Administration
2
HHRR
3
Sales
4
Marketing
And the List<String> can be any keyword like "Admin", "Sal", "Mark" and so on.
First attempt
... was to build a predicate like that:
List<string> kwDepartments = new List<String> {"mark","admin"};
var predicate = PredicateBuilder.New<User>(true);
predicate = predicate.And(x => x.UserDepartments.Where(y => kwDepartments.Any(c => y.Department.Name.Equals(c))).Any());
This produces a SQL with IN operator, like that:
...[t].[Name] IN (N'mark', N'admin'))
Obviously this is not what I want, but if I use .Contains instead of .Equals, an exception is thrown
The LINQ expression could not be translated
I think this is because I'm trying to evaluate a non-primitive value.
Second attempt
... was to iterate over kwDepartments and add an .Or for each string, like that:
foreach (string dep in kwDepartments )
{
predicateDep = predicateDep.Or(x => x.UserDepartments.Where(y=> y.Department.Name.Contains(dep)).Any());
}
predicate = predicate.And(predicateDep);
This returns the match that I expect, but two problems too.
The SQL translation is poor performance as EF Core anidates a INNER JOIN for each keyword. No matter if kwDepartment has two or three elements, but with 100 or 1000 elements will be inadmissible.
Each INNER JOIN on Departments table have a SELECT with all the table fields and i did not need none of them. I've tried to put a .Select(x=> x.Name) statement in the predicate to take only two fields but it make no effect.
Third attempt
... was using Full Text Search using EF.Functions.FreeText but it seems to make no difference.
My goal is to build a predicate that translate in something similar to:
SELECT [c.IdUser]. [c.Name]
FROM [User] AS [c]
WHERE EXISTS (
SELECT 1
FROM [UserDepartment] AS [u]
INNER JOIN (
SELECT [c0].[IdDepartment], [c0].[Name] <--ONLY NEED TWO FIELDS INSTEAD OF ALL FIELDS
FROM [Department] AS [c0]
) AS [t] ON [u].[IdDepartment] = [t].[IdDepartment]
WHERE ([c].[IdUser] = [u].[IdUser]) AND ([t].[Name] like (N'admin%') or [t].[Name] like (N'mark%')))
It is not mandatory to use the LIKE operator, but i put there for better understanding.
Thanks again!

You are on the right track with Or predicate, but instead of multiple Or predicates on user.UserDepatments.Any(single_match) you should create single Or based predicate to be used inside the single user.UserDepatments.Any(multi_or_match).
Something like this:
var departtmentPredicate = kwDepartments
.Select(kw => Linq.Expr((Department d) => EF.Functions.Like(d.Name, "%" + kw + "%")))
.Aggregate(PredicateBuilder.Or);
and then
predicate = predicate.And(u => u.UserDepartments
.Select(ud => ud.Department) // navigate to department
.AsQueryable() // to be able to use departtmentPredicate expression directly
.Any(departtmentPredicate));
With that code and the sample list, DbSet<User>().Where(predicate) is translated to something like this:
DECLARE #__p_1 nvarchar(4000) = N'%mark%';
DECLARE #__p_2 nvarchar(4000) = N'%admin%';
SELECT [u].[IdUser], [u].[Name]
FROM [User] AS [u]
WHERE EXISTS (
SELECT 1
FROM [UserDepartment] AS [u0]
INNER JOIN [Department] AS [d] ON [u0].[IdDepartment] = [d].[IdDepartament]
WHERE ([u].[IdUser] = [u0].[IdUser]) AND (([d].[Name] LIKE #__p_1) OR ([d].[Name] LIKE #__p_2)))

Dynamic OrderBy() using property not in DB throws exception

I have the following models:
public class User
{
public string UserId { get; set; }
public string UserName { get; set; }
public List<Membership> Membership { get; set; }
public bool IsRegistered
{
get { return !String.IsNullOrEmpty(UserName); }
}
}
public class Membership
{
public string MembershipId { get; set; }
public User User { get; set; }
}
In my repository for the Membership entity, I'm attempting to order my collection by doing the following where entries is an IQueryable<Membership> and orderBy is a LamdaExpression equal to { x => x.User.IsRegistered }:
ordered = Queryable.OrderBy(ordered, orderBy);
When I attempt this, I get the following error:
The specified type member 'IsRegistered' is not supported in LINQ to
Entities. Only initializers, entity members, and entity navigation
properties are supported.
How can I order my collection by a property that is not in the database?
Thanks in advance.

Since IsRegistered is a client side function, then you need to do the ordering using LINQ over objects. You can convert your LINQ over SQL to LINQ over objects by causing it to get enumerated. Then once it's a LINQ over objects, you can then order it using LINQ over obect's OrderBy. A common method of doing that is by calling ToList, like:
ordered = Queryable.ToList().OrderBy(ordered, orderBy);
If you want to do the ordering on the database, then you need to convert your clientside code to one that is compatible to SQL, like:
ordered = Queryable.OrderBy(ordered, (x=>x.User.UserName!=null && x.User.UserName!=''));

You can always sort locally. If you want a calculated property to be used for server side ordering you'd need an expression that your LINQ provider (EF) can translate (into SQL in your case).
For your !String.IsNullOrEmpty(UserName) you could use m=>String.Empty!=(m.User.UserName??String.Empty) which works for LINQ2SQL. When checking the generated SQL you should get something like
-- Region Parameters
DECLARE #p0 NVarChar(1000) = ''
DECLARE #p1 NVarChar(1000) = ''
-- EndRegion
SELECT --fields
FROM [Membership] as [t0]
JOIN User AS [t1]
ON -- id
WHERE -- your predicate
ORDER BY
(CASE
WHEN #p0 <> (COALESCE([t1].[UserName],#p1)) THEN 1
WHEN NOT (#p0 <> (COALESCE([t1].[UserName],#p1))) THEN 0
ELSE NULL
END)
I guess EF also translates that likewise.

You can't. Entity framework's order by is mapped to db fields because it's calling "ORDER BY" clause on SQL side.
If you really want to order your records by a non-db field (however, as far as I can see, you can just order your records by UserName in this situation), you should filter your records, call ToList() and OrderBy on that (obviusly this only order your records after the result returned from the database server).

NHibernate - How to return a distinct list of column entries and the count of each entry

Is it possible to replicate the following SQL query in NHibernate?
SELECT Name, COUNT(Name)
FROM Table
GROUP BY Name
Unfortunately I'm not able to get NHibernate to execute this as a raw sql query either as it is not permitted by my current employer.
I've seen examples of returning a count of linked entities but not a count of data in the same table.
I've currently got this working by using two queries. One to get a distinct list of names and one to get the count for each name. I'd like to optimise this to a single database call.
Thanks in advance for any help you can give!

We can do it like this:
// this would be our DTO for result
public class ResultDTO
{
public virtual string Name { get; set; }
public virtual int Count { get; set; }
}
This would be the query
// here we declare the DTO to be used for ALIASing
ResultDTO dto = null;
// here is our query
var result = session.QueryOver<Table>()
.SelectList(l => l
.SelectGroup(x => x.Name).WithAlias(() => dto.Name)
.SelectCount(x => x.Name).WithAlias(() => dto.Count)
)
.TransformUsing(Transformers.AliasToBean<ResultDTO>())
.List<ResultDTO>();

Row_Number() with partition in Nhibernate using CreateCriteria

I am stuck in a problem while creating a query in nhibernate in c#. Actually I have to create a criteria query for following sql statement
select fCompanyID,ROW_NUMBER() over( PARTITION BY fCompanyID order by fPropertyID)
from tARCustomer
But it seems that nhibernate does't support Row_Number() partition by as till now I googled. I need this query functionality in nhibernate but I am not getting how to do it. If anyone did it befor then please let me know / help me to solve out this problem.
Thanks,

Well, NHibernate has solution for anything. This could be the Criteria syntax:
var list = session
.CreateCriteria<Customer>()
.SetProjection
(
Projections.SqlProjection
(
"fCompanyID, ROW_NUMBER() over( PARTITION BY fCompanyID order by fPropertyID) as RowNumber"
, new string[] {"fCompanyID", "RowNumber"}
, new IType[] { NHibernate.NHibernateUtil.Int32, NHibernate.NHibernateUtil.Int32}
)
)
.SetResultTransformer(Transformers.AliasToBean<ResultDTO>())
.List<ResultDTO>()
;
And the ResultDTO like this:
public class ResultDTO
{
public virtual int RowNumber { get; set; }
public virtual int fCompanyID{ get; set; }
}
And now you can work with a list as a set of ResultDTO

SelectMany Count generates explicit sql statement

I have a structure like that:
public class Tag
{
public int TagId { get; set; }
public virtual ICollection<Vacancy> Vacancies { get; set; }
// ...
}
public class Vacancy
{
public int VacancyId { get; set; }
public virtual ICollection<Tag> Tags { get; set; }
// ...
}
These entities are mapped to MS SQL with EF / Code First approach.
After that I fetch somehow from the context (based on user query) a list of tags:
List<Tag> userSelectedTags = ...;
I want to calculate how much vacancies these tags contain:
int count = userSelectedTags.SelectMany(t => t.Vacancies).Count();
It works fine, but it seems this query is not well optimized. When I run into profiler, I get this:
exec sp_executesql N'SELECT
[Extent2].[VacancyId] AS [VacancyId],
[Extent2].[Title] AS [Title],
[Extent2].[Salary] AS [Salary],
...props... etc...
FROM [dbo].[VacancyTagVacancies] AS [Extent1]
INNER JOIN [dbo].[Vacancies] AS [Extent2] ON [Extent1].[Vacancy_VacancyId] = [Extent2].[VacancyId]
WHERE [Extent1].[VacancyTag_VacancyTagId] = #EntityKeyValue1',N'#EntityKeyValue1 int',#EntityKeyValue1=1
In other words, it doesn't use COUNT sql keyword, but just returns the whole collection and calculates it outside SQL Server.
How can I optimize the LINQ query in order to use COUNT here instead of selecting all? Thanks.

Your userSelectedTags isn't a query, which means t.Vacancies becomes a reference to the client-side property, which when enumerated retrieves all the Vacancies for that specific Tag. The Count then runs on the client side.
You need to start from the server side:
int count = (
from tag in context.Tags
where userSelectedTags.Contains(tag)
from v in tag.Vacancies
select v).Count();
You might need to make a userSelectedTagIds instead of userSelectedTags if this doesn't yet work.

One way is to execute the SQL directly like this:
string nativeSQLQuery = "SELECT count(*) FROM <tables join>";
var queryResult = ctx.ExecuteStoreQuery<int>(nativeSQLQuery);