We Keep Coding

EF Core SQL Filter Translation

I'm using EF Core 5.0.1 with ASP NET 5.0.1 Web API and I want to build a query with PredicateBuilder using LinqKit.Microsoft.EntityFrameworkCore 5.0.2.1
For the purposes of the question I simplified my model to:
public class User
{
public long IdUser { get; set; }
public string Name { get; set; }
public virtual ICollection<UserDepartment> UserDepartments { get; set; }
}
public class Department
{
public long IdDepartament { get; set; }
public string Name { get; set; }
public virtual ICollection<UserDepartment> UsersDepartment { get; set; }
}
public UserDepartment
{
public long IdUser { get; set; }
public long IdDepartment { get; set; }
public virtual Department { get; set; }
public virtual User { get; set; }
}
One User can have many Departaments and one Departament can have many Users
Three models have its correspondent table in SQL Server and a IEntityTypeConfiguration class with the appropriate relationships set up.
All I want to achieve is to search any User that belongs to any Departament which Department.Name is in a List<String>.
The List<String> contains a list of keywords, not the exact Department Name
Department table has this kind of rows:
IdDepartment
Name
1
Administration
2
HHRR
3
Sales
4
Marketing
And the List<String> can be any keyword like "Admin", "Sal", "Mark" and so on.
First attempt
... was to build a predicate like that:
List<string> kwDepartments = new List<String> {"mark","admin"};
var predicate = PredicateBuilder.New<User>(true);
predicate = predicate.And(x => x.UserDepartments.Where(y => kwDepartments.Any(c => y.Department.Name.Equals(c))).Any());
This produces a SQL with IN operator, like that:
...[t].[Name] IN (N'mark', N'admin'))
Obviously this is not what I want, but if I use .Contains instead of .Equals, an exception is thrown
The LINQ expression could not be translated
I think this is because I'm trying to evaluate a non-primitive value.
Second attempt
... was to iterate over kwDepartments and add an .Or for each string, like that:
foreach (string dep in kwDepartments )
{
predicateDep = predicateDep.Or(x => x.UserDepartments.Where(y=> y.Department.Name.Contains(dep)).Any());
}
predicate = predicate.And(predicateDep);
This returns the match that I expect, but two problems too.
The SQL translation is poor performance as EF Core anidates a INNER JOIN for each keyword. No matter if kwDepartment has two or three elements, but with 100 or 1000 elements will be inadmissible.
Each INNER JOIN on Departments table have a SELECT with all the table fields and i did not need none of them. I've tried to put a .Select(x=> x.Name) statement in the predicate to take only two fields but it make no effect.
Third attempt
... was using Full Text Search using EF.Functions.FreeText but it seems to make no difference.
My goal is to build a predicate that translate in something similar to:
SELECT [c.IdUser]. [c.Name]
FROM [User] AS [c]
WHERE EXISTS (
SELECT 1
FROM [UserDepartment] AS [u]
INNER JOIN (
SELECT [c0].[IdDepartment], [c0].[Name] <--ONLY NEED TWO FIELDS INSTEAD OF ALL FIELDS
FROM [Department] AS [c0]
) AS [t] ON [u].[IdDepartment] = [t].[IdDepartment]
WHERE ([c].[IdUser] = [u].[IdUser]) AND ([t].[Name] like (N'admin%') or [t].[Name] like (N'mark%')))
It is not mandatory to use the LIKE operator, but i put there for better understanding.
Thanks again!

You are on the right track with Or predicate, but instead of multiple Or predicates on user.UserDepatments.Any(single_match) you should create single Or based predicate to be used inside the single user.UserDepatments.Any(multi_or_match).
Something like this:
var departtmentPredicate = kwDepartments
.Select(kw => Linq.Expr((Department d) => EF.Functions.Like(d.Name, "%" + kw + "%")))
.Aggregate(PredicateBuilder.Or);
and then
predicate = predicate.And(u => u.UserDepartments
.Select(ud => ud.Department) // navigate to department
.AsQueryable() // to be able to use departtmentPredicate expression directly
.Any(departtmentPredicate));
With that code and the sample list, DbSet<User>().Where(predicate) is translated to something like this:
DECLARE #__p_1 nvarchar(4000) = N'%mark%';
DECLARE #__p_2 nvarchar(4000) = N'%admin%';
SELECT [u].[IdUser], [u].[Name]
FROM [User] AS [u]
WHERE EXISTS (
SELECT 1
FROM [UserDepartment] AS [u0]
INNER JOIN [Department] AS [d] ON [u0].[IdDepartment] = [d].[IdDepartament]
WHERE ([u].[IdUser] = [u0].[IdUser]) AND (([d].[Name] LIKE #__p_1) OR ([d].[Name] LIKE #__p_2)))

EF Lambda include navigation properties

I have the following object, called Filter with the following properties:
public int Id { get; set; }
public string Name { get; set; }
public virtual ICollection<Type> Types{ get; set; }
public virtual ICollection<Step> Steps { get; set; }
public virtual ICollection<Flow> Flows { get; set; }
public virtual ICollection<Room> Rooms { get; set; }
When I select a list of Filters from the database, I have no idea how to include the collections (Types, Steps, Flows, Rooms). My code is as follows:
var filters = (
from filter in dbContext.DbSet<Filter>()
let rooms = (
from r in dbContext.DbSet<Room>()
select r
)
let eventTypes = (
from t in dbContext.DbSet<Type>()
select t
)
let processFlows = (
from f in dbContext.DbSet<Flow>()
select f
)
let processFlowSteps = (
from s in dbContext.DbSet<Step>()
select s
)
select filter
).ToList();
My collection of Filter is returned, but the collections inside are empty. Could you please tell me how can I achieve this?
Ps: I do not want to use Include because of performance issues, I don't like how Entity Framework generates the query and I would like to do it this way.

Your method works, you are just doing it slighly wrong.
To include a navigation property, all you have to do is a subselect (using linq), example:
var filters = (from filter in dbContext.DbSet<Filter>()
select new Filter
{
filter.Id,
filter.Name,
Rooms = (from r in dbContext.DbSet<Room>()
where r.FilterId == filter.Id
select r).ToList()
}).ToList();
Keep in mind that EF won't execute the query until you call a return method (ToList, Any, FirstOrDefault, etc). With this, instead of doing those ugly queries you want to avoid by not using Include(), it will simply fire two queries and properly assign the values in the object you want.

You need to use Include extension method:
var filters=dbContext.DbSet<Filter>()
.Include(f=>f.Types)
.Include(f=>f.Steps)
.Include(f=>f.Flows)
.Include(f=>f.Rooms)
.ToList()
Update
#MrSilent, Include extension method was made exactly for the purpose of loading related entities, I think the other option you have is executing a raw sql, but the way you are doing is not the way to go you have four roundtrips to your database and you need to use join instead in order to get the related entities, Include generates those joins for you and it's just one roundtrip.
This is, eg, another way I guess you could do it, but again, it is against the purpose of using EF, the idea of your model is also to represent the relationship between your tables, not just to represent them individually
var query= from f in context.DbSet<Filter>()
from s in f.Steps
from r in f.Rooms
from t in f.Types
from fl in f.Flows
select new {f, s, r, t, fl};

You can use lazy loading , how :
You first need to get the properties that are Include inclined to be virtual and then an empty constructor that is protected access type to do your job well.
public virtual ICollection<Type> Types{ get; set; }
public virtual ICollection<Step> Steps { get; set; }
public virtual ICollection<Flow> Flows { get; set; }
public virtual ICollection<Room> Rooms { get; set; }
And
//FOR EF !
protected Filter() { }
I think this solution will solve your problem.

How to use subquery as a join in NHibernate?

I have the following SQL code:
SELECT err.*,tmp.counted FROM sm_database.error err
LEFT JOIN (
SELECT sme_Hash, COUNT(*) as counted FROM sm_database.error GROUP BY sme_Hash
) tmp
ON tmp.sme_Hash = err.sme_Hash
WHERE sme_Id = 197
Above will get me a extra column counted that should set into my Count property below.
the id 197 will be parameter I will use in a method.
My class name is:
public class Error {
public virtual int Count { get; set; }
public virtual DateTime Date { get; set; }
public virtual int Id { get; set; }
public virtual string Hash { get; set; }
-----------------
//more property's
}
I want to "convert" this into NHibernate with Query(or QueryOver)
Anyone know how to do this properly or could point me in the right direction?
EDIT
I got it with the following:
string query = #"SELECT err.* ,tmp.Count
FROM sm_database.error err
LEFT JOIN (
SELECT sme_Hash as Hash, COUNT(*) as Count FROM sm_database.error GROUP BY sme_Hash
) tmp
ON tmp.Hash = err.sme_Hash
WHERE sme_Id = :id";
var result = session.CreateSQLQuery(query)
.AddEntity(typeof(Error))
.SetParameter("id", id).List<Error>().SingleOrDefault();
It works fine, everthing is pulled except the Count property.
The Count property in my Error class isn't mapped cause we don't have a column in the database because we calculated the count in the query.
Is there a way to get the tmp.Count value in my Error.Count property?
Thanks in advance.

There is no easy answer, no built in solution for this kind of SQL construct.
The options how to go around are:
1) Create special entity and map it to this view: "SELECT sme_Hash, COUNT(*) as counted FROM sm_database.error GROUP BY sme_Hash" It could be either DB view or mapping with subselect (see Mapping native sql to Entity with NHibernate)
2) use native SQL with CreateSQLQuery
Because NHibernate provides querying on top of Entity model. I.e. the FROM is always coming from mapping. No way how to create that as above

NHibernate - How to return a distinct list of column entries and the count of each entry

Is it possible to replicate the following SQL query in NHibernate?
SELECT Name, COUNT(Name)
FROM Table
GROUP BY Name
Unfortunately I'm not able to get NHibernate to execute this as a raw sql query either as it is not permitted by my current employer.
I've seen examples of returning a count of linked entities but not a count of data in the same table.
I've currently got this working by using two queries. One to get a distinct list of names and one to get the count for each name. I'd like to optimise this to a single database call.
Thanks in advance for any help you can give!

We can do it like this:
// this would be our DTO for result
public class ResultDTO
{
public virtual string Name { get; set; }
public virtual int Count { get; set; }
}
This would be the query
// here we declare the DTO to be used for ALIASing
ResultDTO dto = null;
// here is our query
var result = session.QueryOver<Table>()
.SelectList(l => l
.SelectGroup(x => x.Name).WithAlias(() => dto.Name)
.SelectCount(x => x.Name).WithAlias(() => dto.Count)
)
.TransformUsing(Transformers.AliasToBean<ResultDTO>())
.List<ResultDTO>();

SelectMany Count generates explicit sql statement

I have a structure like that:
public class Tag
{
public int TagId { get; set; }
public virtual ICollection<Vacancy> Vacancies { get; set; }
// ...
}
public class Vacancy
{
public int VacancyId { get; set; }
public virtual ICollection<Tag> Tags { get; set; }
// ...
}
These entities are mapped to MS SQL with EF / Code First approach.
After that I fetch somehow from the context (based on user query) a list of tags:
List<Tag> userSelectedTags = ...;
I want to calculate how much vacancies these tags contain:
int count = userSelectedTags.SelectMany(t => t.Vacancies).Count();
It works fine, but it seems this query is not well optimized. When I run into profiler, I get this:
exec sp_executesql N'SELECT
[Extent2].[VacancyId] AS [VacancyId],
[Extent2].[Title] AS [Title],
[Extent2].[Salary] AS [Salary],
...props... etc...
FROM [dbo].[VacancyTagVacancies] AS [Extent1]
INNER JOIN [dbo].[Vacancies] AS [Extent2] ON [Extent1].[Vacancy_VacancyId] = [Extent2].[VacancyId]
WHERE [Extent1].[VacancyTag_VacancyTagId] = #EntityKeyValue1',N'#EntityKeyValue1 int',#EntityKeyValue1=1
In other words, it doesn't use COUNT sql keyword, but just returns the whole collection and calculates it outside SQL Server.
How can I optimize the LINQ query in order to use COUNT here instead of selecting all? Thanks.

Your userSelectedTags isn't a query, which means t.Vacancies becomes a reference to the client-side property, which when enumerated retrieves all the Vacancies for that specific Tag. The Count then runs on the client side.
You need to start from the server side:
int count = (
from tag in context.Tags
where userSelectedTags.Contains(tag)
from v in tag.Vacancies
select v).Count();
You might need to make a userSelectedTagIds instead of userSelectedTags if this doesn't yet work.

One way is to execute the SQL directly like this:
string nativeSQLQuery = "SELECT count(*) FROM <tables join>";
var queryResult = ctx.ExecuteStoreQuery<int>(nativeSQLQuery);