We Keep Coding

EF Core 5 check if all ids from filter exists in related entities

I have two models:
public class Employee
{
public int Id { get; set; }
public IList<Skill> { get; set; }
}
public class Skill
{
public int Id { get; set; }
}
And I have filter with list of skill ids, that employee should contain:
public class Filter
{
public IList<int> SkillIds { get; set; }
}
I want to write query to get all employees, that have all skills from filter.
I tried:
query.Where(e => filter.SkillIds.All(id => e.Skills.Any(skill => skill.Id == id)));
And:
query = query.Where(e => e.Skills
.Select(x => x.Id)
.Intersect(filter.SkillIds)
.Count() == filter.SkillIds.Count);
But as a result I get exception says that query could not be translated.

It is going to be a difficult, if not impossible task, to run a query like this on the sql server side.
This is because to make this work on the SQL side, you would be grouping each set of employee skills into a single row which would need to have a new column for every skill listed in the skills table.
SQL server wasn't really made to handle grouping with an unknown set of columns passed into a query. Although this kind of query is technically possible, it's probably not very easy to do through a model binding framework like ef core.
It would be easier to do this on the .net side using something like:
var employees = _context.Employees.Include(x=>x.Skill).ToList();
var filter = someFilter;
var result = employees.Where(emp => filter.All(skillID=> emp.skills.Any(skill=>skill.ID == skillID))).ToList()

This solution works:
foreach (int skillId in filter.SkillIds)
{
query = query.Where(e => e.Skills.Any(skill => skill.Id == skillId));
}
I am not sure about it's perfomance, but works pretty fast with small amount of data.

I've also encountered this issue several times now, this is the query I've come up with that I found works best and does not result in an exception.
query.Where(e => e.Skills.Where(s => filter.SkillIds.Contains(s.Id)).Count() == filter.SkillIds.Count);

Entity Framework - reusing associated extension methods

I'm trying to reuse an EF6 extension method with an associated entity (one to many relationship). Contrived example:
public class Parent
{
public string State { get; set; }
public ICollection<Child> Children { get; set; }
}
public class Child
{
public string Value { get; set; }
public Parent Parent { get; set; }
}
public static ParentNamedScopes
{
public static IQueryable<Parent> IsReady(this IQueryable<Parent> queryable)
{
return queryable.Where(p => p.State == "Ready" || p.State == "New");
}
}
// ...
var children = db.Children
// my goal, but can't cast Parent to IQueryable<Parent>
// ------------------v
.Where(c => c.Parent.IsReady())
.Where(c => c.Value == "Foobar");
I've seen examples of using AsQueryable() on associated collections in sub queries, but that isn't an options since Parent is a single record. I'm sure I'm missing something obvious and I apologize since my google foo has not turned up the answer today.

One option would be to start your query with the Parents:
var children = db.Parents.IsReady()
.SelectMany(p => p.Children)
.Where(c => c.Value == "Foobar");
The idea that IsReady would convert an IQueryable seems a little off to me, though. If your use case gets more complex, you may need to change that to just give you an Expression<Func<Parent, bool>>, and use something like LINQKit to manipulate your query to make it reusable:
Expression<Func<Parent, bool>> parentIsReady = ParentCriteria.IsReady();
var readyParents = db.Parents.Where(parentIsReady);
var childrenWithReadyParents = db.Children.AsExpandable()
.Where(c => parentIsReady.Invoke(c.Parent))
.Where(c => c.Value == "Foobar");

Orchard CMS ContentQuery with N-to-N relation

In my project I've implemented N-to-N relation between records using this tutorial on OrchardProject web-site. I have 2 parts: MaterialPart & CategoryPart and association record.
Material part
public class MaterialPartRecord : ContentPartRecord {
public MaterialPartRecord() {
Categories = new List<ContentMaterialCategoryRecord>();
}
}
public class MaterialPart : ContentPart<MaterialPartRecord> {
public IEnumerable<CategoryPartRecord> Categories {
get { return Record.Categories.Select(cmcr => cmcr.CategoryPartRecord); }
}
}
CategoryPartRecord
public class CategoryPartRecord : ContentPartRecord {
...
}
public class CategoryPart : ContentPart<CategoryPartRecord> {
...
}
association record:
public class ContentMaterialCategoryRecord {
public virtual int Id { get; set; }
public virtual MaterialPartRecord MaterialPartRecord { get; set; }
public virtual CategoryPartRecord CategoryPartRecord { get; set; }
}
Now I need to select MaterialItems which are linked to certain category. So far I have this method to extract them. It works but I'm not sure that it is correct way to do this.
public IEnumerable<MaterialPart> GetMaterialsByCategory(int catId) {
var cs = new CategoriesService(_oServices);
CategoryPartRecord cat = cs.GetItem(catId).Record;
return _oServices.ContentManager
.Query(VersionOptions.Latest, _contentType)
.Join<CommonPartRecord>()
.OrderByDescending(cpr => cpr.PublishedUtc);
.List()
.Where(ci => ci.IsPublished())
.Select(ci => ci.As<MaterialPart>())
.Where(mp => mp.Categories.Contains(cat)); // < ---- ?
}
So my question is: what is correct way to select materials for required category, which produces optimal SQL query, as we simply need to inner join associated record table with required CategoryPartRecord_Id field value.
thaks!

In case, of M : N with pairing object, we can use QueryOver and subquery. The biggest benefit would be, that we recieve the plain set of material Items, which we can use for paging (Take(), Skip())
var session = ... // get curretn session
CategoryPartRecord category = null;
ContentMaterialCategoryRecord pair = null;
MaterialPartRecord material = null;
var subquery = QueryOver.Of<ContentMaterialCategoryRecord>(() => pair)
// now we will join Categories to be able to filter whatever property
.JoinQueryOver(() => pair.CategoryPartRecord, () => category)
// here is the filter
// there could be IN, >= <= ...
.Where(() => category.ID == 1)
// or
.WhereRestrictionOn(c => c.category.ID).IsIn(new[] {1, 2, 3})
...
// now we will return IDs of the Material we are interested in
.Select(x => pair.MaterialPartRecord.Id);
// finally the clean query over the Materials...
var listOfUsers = session.QueryOver<MaterialPartRecord>(() => material )
.WithSubquery
.WhereProperty(() => material.Id)
.In(subquery)
// paging
.Take(10)
.Skip(10)
.List<MaterialPartRecord>();
So, this will produce the most effective SQL Script, with one subselect, and clean select from material table
NOTE: similar stuff could be done even with LINQ. But QueryOver is NHibernate most native way I'd say. Anyhow, the principe - subquery to filter by category, and main query to load materials will remain the same. Only ONE SQL Select call

Sorting nested collection in projection: Unable to cast object of type 'SortOp' to type 'ProjectOp'

I'm using projection of query results to a custom type, which isn't a part of entity data model:
public sealed class AlgoVersionCacheItem : NotificationObject
{
public int OrderId { get; set; }
public string OrderTitle { get; set; }
public int? CurrentVersion { get; set; }
public int CachedVersion { get; set; }
public IEnumerable<int> AvailableVersions { get; set; }
}
I want AvailableVersions to be sorted in descending order. Hence, I've tried to add sorting for AvailableVersions in projection:
return someQueryable
.Select(version => new AlgoVersionCacheItem
{
OrderId = version.OrderId,
OrderTitle = version.Order.Title,
CurrentVersion = version.Order.CurrentAlgoVersionId,
CachedVersion = version.Id,
AvailableVersions = version
.Order
.AlgoVersions
.Where(v => (allowUncommittedVersions || v.Statuses.Any(s => s.AlgoVersionStatusListItemId == ModelConstants.AlgoVersionCommitted_StatusId)) && v.Id != version.Id)
.OrderByDescending(v => v.Id) // this line will cause exception
.Select(v => v.Id)
})
.Where(item => item.AvailableVersions.Any())
.OrderByDescending(item => item.OrderId)
.ToArray();
With sorting, execution of the query throws an System.Data.EntityCommandCompilationException with System.InvalidCastException as inner exception:
Unable to cast object of type
'System.Data.Entity.Core.Query.InternalTrees.SortOp' to type
'System.Data.Entity.Core.Query.InternalTrees.ProjectOp'
Without .OrderByDescending(v => v.Id) everything works fine.
Is this yet another feature, that isn't supported in Entity Framework, or I've missed something?
P.S. I know, that I can sort items later at client side, but I'm wondering about sorting at the server side.

This is a bug in EF. I was able to repro this on both EF5 and EF6. I think you should be able to workaround the bug by filtering records before creating the results i.e.:
return someQueryable
.Where(version => version.Order.AlgoVersions.Any(v => (allowUncommittedVersions || v.Statuses.Any(s => s.AlgoVersionStatusListItemId == ModelConstants.AlgoVersionCommitted_StatusId)) && v.Id != version.Id))
.Select(version => new AlgoVersionCacheItem
{
OrderId = version.OrderId,
OrderTitle = version.Order.Title,
CurrentVersion = version.Order.CurrentAlgoVersionId,
CachedVersion = version.Id,
AvailableVersions = version
.Order
.AlgoVersions
.Where(v => (allowUncommittedVersions || v.Statuses.Any(s => s.AlgoVersionStatusListItemId == ModelConstants.AlgoVersionCommitted_StatusId)) && v.Id != version.Id)
.OrderByDescending(v => v.Id) // this line will cause exception
.Select(v => v.Id)
})
.OrderByDescending(item => item.OrderId)
.ToArray();
I also have a feeling that this query could be simplified if you go from the other side of relationship (i.e. from Orders) but it may depend on how the someQueryable is created.

In my case the Linq query was selecting a new anonymous construct with one property being set to a collection ToList() extension method. I removed the embedded execution, the error went away, and the system still worked fine.

RavenDB workaround for nested LINQ expression

I have simple set of objects stored in RavenDB:
public class Question
{
public string Id { get; set; }
public DateTime CreatedOn { get; set; }
public ICollection<User> Supporters { get; set; }
public ICollection<Answer> Answers { get; set; }
}
public class Answer
{
public string Id { get; set; }
public bool IsOfficial { get; set; }
}
Now I want to query RavenDB to give me set of questions, ordered firstly by number of supporters, next by condition - if a question has any official answer, and in the and, by question creation date. So I've written a query:
var questions = DocumentSession.Query<Question>().AsQueryable();
questions = questions
.OrderByDescending(x => x.Supporters.Count)
.ThenByDescending(x => x.Answers.Any(a => a.IsOfficial)) //EDIT: source of exception
.ThenByDescending(x => x.CreatedOn)
.Take(15);
var result = questions.ToList();
which throws an exception:
System.InvalidCastException: Unable to cast object of type 'System.Linq.Expressions.MethodCallExpressionN' to type 'System.Linq.Expressions.MemberExpression'
The query is logically correct and works, when I use linq-to-objects, and simply add .ToList() to first line:
var questions = DocumentSession.Query<Question>().Tolist().AsQueryable();
// next lines stay unchanged
I don't want to do it because of performance issues (this change forces all questions to be loaded from database into memory before filtering).
How to make this working without performance impact ? Maybe shell I define an index ? How it should looks like then ?

A custom index for your purposes is basically going to be a recreation of your class with extra fields in it (and some logic to support it). It seems like you don't want to have to add more fields to your current class, are you okay with adding more classes to your project?
Here's an example:
public class Question_WithAnyOfficial: AbstractIndexCreationTask<Question>
{
public class Question_WithAnyOfficial()
{
Map = questions => from question in questions
// New Anonymous Type
select new
{
Id = question.Id,
CreatedOn = question.CreatedOn,
Supporters = question.Supporters,
Answers = question.Answers,
AnyOfficial = question.Answers.Where(a => a.IsOfficial).Any()
};
}
}
Then you can query this:
var questions = DocumentSession.Query<Question_WithAnyOfficial>()
.OrderByDescending(x => x.Supporters.Count)
.ThenByDescending(x => x.AnyOfficial)
.ThenByDescending(x => x.CreatedOn)
.Take(15)
.ToList();
Don't forget that you'll have to register the index when your app starts.

Raven can't support calculations like that inside the LINQ query, so this should work (problem clause removed):
var questions = DocumentSession.Query<Question>()
.OrderByDescending(x => x.Supporters.Count)
//.ThenByDescending(x => x.Answers.Any(a => a.IsOfficial))
.ThenByDescending(x => x.CreatedOn)
.Take(15);
var result = questions.ToList();
If you want to include that logic, you need a field on your class called AreAllAnswersOfficial (or something similar). Then you can put that inside the clause:
var questions = DocumentSession.Query<Question>()
.OrderByDescending(x => x.Supporters.Count)
.ThenByDescending(x => x.AreAllAnswersOfficial)
.ThenByDescending(x => x.CreatedOn)
.Take(15);
var result = questions.ToList();

Based on Bear Alexander response, I've done this like that:
public class QuestionByAnyOfficial : AbstractIndexCreationTask<Question, QuestionByAnyOfficial.Result>
{
public class Result
{
public string Id;
public bool AnyOfficial;
public int SupportersCount;
public DateTime CreatedOn;
}
public QuestionByAnyOfficial()
{
Map = questions => from question in questions
select new
{
Id = question.Id,
AnyOfficial = question.Answers.Any(a => a.IsOfficial),
SupportersCount = question.Supporters.Count,
CreatedOn = question.CreatedOn
};
}
}
var questionIds = DocumentSession.Query<QuestionByAnyOfficial.Result, QuestionByAnyOfficial>()
.OrderByDescending(x => x.SupportersCount)
.ThenByDescending(x => x.AnyOfficial)
.ThenByDescending(x => x.CreatedOn)
.Take(NumberOfQuestions)
.Select(x => x.Id);
var questions = DocumentSession.Load<Question>(questionIds);
var result = questions.ToList();
It works and I believe it is more efficient than my original version. If it can be done in any more elegant way, I'd appreciate any ideas. Regards.