We Keep Coding

EF Core Linq-to-Sql GroupBy SelectMany not working with SQL Server

I am trying the following Linq with LinqPad connecting to SQL Server with EF Core:
MyTable.GroupBy(x => x.SomeField)
.OrderBy(x => x.Key)
.Take(5)
.SelectMany(x => x)
I get this error:
The LINQ expression 'x => x' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explic...
However, this works:
MyTable.AsEnumerable()
.GroupBy(x => x.SomeField)
.OrderBy(x => x.Key)
.Take(5)
.SelectMany(x => x)
I was under the impression that EF Core should be able to translate such an expression.
Am I doing anything wrong?

That exception message is an EF Core message, not EF6.
In EF 6 your expression should work, though with something like a ToList() on the end. I suspect the error you are encountering is that you may be trying to do something more prior to materializing the collection, and that is conflicting with the group by SelectMany evaluation.
For instance, something like this EF might take exception to:
var results = MyTable
.GroupBy(x => x.SomeField)
.OrderBy(x => x.Key)
.Take(5)
.SelectMany(x => x)
.Select(x => new ViewModel { Id = x.Id, Name = x.Name} )
.ToList();
where something like this should work:
var results = MyTable
.GroupBy(x => x.SomeField)
.OrderBy(x => x.Key)
.Take(5)
.SelectMany(x => x.Select(y => new ViewModel { Id = y.Id, Name = y.Name} ))
.ToList();
You don't want to use:
MyTable.AsEnumerable(). ...
As this is materializing your entire table into memory, which might be ok if the table is guaranteed to remain relatively small, but if the production system grows significantly it forms a cascading performance decline over time.
Edit: Did a bit of digging, credit to this post as it does look like another limitation in EF Core's parser. (No idea how something that works in EF6 cannot be successfully integrated into EF Core... Reinventing wheels I guess)
This should work:
var results = MyTable
.GroupBy(x => x.SomeField)
.OrderBy(x => x.Key)
.Take(5)
.Select(x => x.Key)
.SelectMany(x => _context.MyTable.Where(y => y.Key == x))
.ToList();
So for example where I had a Parent and Child table where I wanted to group by ParentId, take the top 5 parents and select all of their children:
var results = context.Children
.GroupBy(x => x.ParentId)
.OrderBy(x => x.Key) // ParentId
.Take(5)
.Select(x => x.Key) // Select the top 5 parent ID
.SelectMany(x => context.Children.Where(c => c.ParentId == x)).ToList();
EF pieces this back together by doing a SelectMany back on the DbSet against the selected group IDs.
Credit to the discussions here: How to select top N rows for each group in a Entity Framework GroupBy with EF 3.1
Edit 2: The more I look at this, the more hacky it feels. Another alternative would be to look at breaking it up into two simpler queries:
var keys = MyTable.OrderBy(x => x.SomeField)
.Select(x => x.SomeField)
.Take(5)
.ToList();
var results = MyTable.Where(x => keys.Contains(x.SomeField))
.ToList();
I think that translates your original example, but the gist is to select the applicable ID/Discriminating keys first, then query for the desired data using those keys. So in the case of my All children from the first 5 parents that have children:
var parentIds = context.Children
.Select(x => x.ParentId)
.OrderBy(x => x)
.Take(5)
.ToList();
var children = context.Children
.Where(x => parentIds.Contains(x.ParentId))
.ToList();

EF Core has limitation for such query, which is fixed in EF Core 6. This is SQL limitation and there is no direct translation to SQL for such GroupBy.
EF Core 6 is creating the following query when translating this GroupBy.
var results = var results = _context.MyTable
.Select(x => new { x.SomeField })
.Distinct()
.OrderBy(x => x.SomeField)
.Take(5)
.SelectMany(x => _context.MyTable.Where(y => y.SomeField == x.SomeField))
.ToList();
It is not most optimal query for such task, because in SQL it can be expressed by Window Function ROW_NUMBER() with PARTITION on SomeField and additional JOIN can be omitted.
Also check this function, which makes such query automatically.
_context.MyTable.TakeDistinct(5, x => x.SomeField);

Linq !Contains() alternative

I'm trying to improve the performance of a query that uses linq and fluent API.
The query looks like this:
var result = DbContext.Set<Parent>()
.Join(DbContext.Set<Child>(),
(parent) => parent.Id,
(child) => child.ParentId,
(parent, child) => cild)
.Select(x => new { x.SomeOtherId, x.AnotherId })
.Where(x => !filters.Contains(x.SomeOtherId))
.Select(x => x.AnotherId )
.Distinct()
.ToListAsync(cancellationToken);
As I understand, .Contains degrades the performance of queries such as this and using a Join is better, e.g
.Join(filters, (c) => c.SomeOtherId, (filterId) => filterId, (c, filterId) => c.AnotherId);
This will give results where there is a match, however how would I find results where there is not a match?
Thanks

NHibernate: Could not resolve property error

I am trying to do a query like:
var a = session.QueryOver<Site>()
.SelectList(
x => x.Select(p => p.SiteName)
.Select(p => p.SiteId).Select(p => p.RegionLocation.City))
.List<object[]>();
but I get the error
could not resolve property: RegionLocation.City of: Entities.Site
The property exists and I can retrieve it using LINQ but QueryOver does not work. What am I doing wrong ?

As far as i remember, with QueryOver, you have to join all entities in the association in order to be able to access it's properties.
This means you ought to do something like:
(notice the .JoinQueryOver)
var a = session.QueryOver<Site>()
.JoinQueryOver(s => s.RegionLocation)
.SelectList(
x => x.Select(p => p.SiteName)
.Select(p => p.SiteId)
.Select(p => p.RegionLocation.City))
.List<object[]>();
Or maybe this will work:
RegionLocation regionLocationAlias = null;
var a = session.QueryOver<Site>()
.JoinAlias(s => s.RegionLocation, () => regionLocationAlias)
.SelectList(
x => x.Select(p => p.SiteName)
.Select(p => p.SiteId)
.Select(() => regionLocationAlias.City))
.List<object[]>();
Also you might want to have a look at https://github.com/davybrion/NHibernateWorkshop/tree/master/NHibernateWorkshop
There's lots of great examples!
Specifically for your problem, have a look at: https://github.com/davybrion/NHibernateWorkshop/blob/master/NHibernateWorkshop/Querying/QueryOver/Projecting.cs

Lambda expressions in C# with If\Else

I built an SQL Query that order table by complex Condition.
here is a example with the same principle: (Table Toto include 3 cols: id ,num1,num2)
select t.id
from Toto as t
group by t.id
order by sum(case when t.id<100 Then t.num1 ELSE t.num2 END)
and my question is, there is option to write this query in Lambda?(using when\else)
thank you!

It would look something like this:
db.Toto
.GroupBy(t => t.id)
.OrderBy(g => g.Sum(t => t.id<100 ? t.num1 : t.num2)
.Select(g => g.Key) // since you're grouping by Id
it's the cleaner equivalent of:
db.Toto
.GroupBy(t => t.id)
.OrderBy(g => g.Sum(t => {
if (t.id<100)
return t.num1;
else
return t.num2;
}
)
.Select(g => g.Key) // since you're grouping by Id

Why can I not use OrderBy() in this lambda expression?

How do I order the following? The orderBy doesnt recognise the x.Name.
var xRefsNames = db.CrossRefs.Where(x => pgNos.Contains(x.PG))
.Select(x => x.Name)
.Distinct()
.OrderBy(x=>x.Name);

Your select is projecting a different object, probably a string based on the name. You want to just order by x.
var xRefsNames = db.CrossRefs.Where(x => pgNos.Contains(x.PG))
.Select(x => x.Name)
.Distinct()
.OrderBy(x=>x);