We Keep Coding

How to write linq query for this sql statement

How would you write a linq query with the following SQL statement. I've tried several methods referenced on stackoverflow but they either don't work with the EF version I'm using (EF core 3.5.1) or the DBMS (SQL Server).
select a.ProductID, a.DateTimeStamp, a.LastPrice
from Products a
where a.DateTimeStamp = (select max(DateTimeStamp) from Products where a.ProductID = ProductID)
For reference, a couple that I've tried (both get run-time errors).
var results = _context.Products
.GroupBy(s => s.ProductID)
.Select(s => s.OrderByDescending(x => x.DateTimeStamp).FirstOrDefault());
var results = _context.Products
.GroupBy(x => new { x.ProductID, x.DateTimeStamp })
.SelectMany(y => y.OrderByDescending(z => z.DateTimeStamp).Take(1))
Thanks!

I understand you would like to have a list of the latest prices of each products?
First of all I prefer to use group by option even over 1st query
select a.ProductID, a.DateTimeStamp, a.LastPrice
from Products a
where a.DateTimeStamp IN (select max(DateTimeStamp) from Products group by ProductID)
Later Linq:
var maxDateTimeStamps = _context.Products
.GroupBy(s => s.ProductID)
.Select(s => s.Max(x => x.DateTimeStamp)).ToArray();
var results = _context.Products.Where(s=>maxDateTimeStamps.Contains(s.DateTimeStamp));
-- all assuming that max datetime stamps are unique

I've managed to do it with the following which replicates the correlated sub query in the original post (other than using TOP and order by instead of the Max aggregate), though I feel like there must be a more elegant way to do this.
var results = from x
in _context.Products
where x.DateTimeStamp == (from y
in _context.Products
where y.ProductID == x.ProductID
orderby y.DateTimeStamp descending
select y.DateTimeStamp
).FirstOrDefault()
select x;

I prefer to break up these queries into IQueryable parts, do you can debug each "step".
Something like this:
IQueryable<ProductOrmEntity> pocoPerParentMaxUpdateDates =
entityDbContext.Products
//.Where(itm => itm.x == 1)/*if you need where */
.GroupBy(i => i.ProductID)
.Select(g => new ProductOrmEntity
{
ProductID = g.Key,
DateTimeStamp = g.Max(row => row.DateTimeStamp)
});
//// next line for debugging..do not leave in for production code
var temppocoPerParentMaxUpdateDates = pocoPerParentMaxUpdateDates.ToListAsync(CancellationToken.None);
IQueryable<ProductOrmEntity> filteredChildren =
from itm
in entityDbContext.Products
join pocoMaxUpdateDatePerParent in pocoPerParentMaxUpdateDates
on new { a = itm.DateTimeStamp, b = itm.ProductID }
equals
new { a = pocoMaxUpdateDatePerParent.DateTimeStamp, b = pocoMaxUpdateDatePerParent.ProductID }
// where
;
IEnumerable<ProductOrmEntity> hereIsWhatIWantItems = filteredChildren.ToListAsync(CancellationToken.None);
That last step, I am putting in an anonymous object. You can put the data in a "new ProductOrmEntity() { ProductID = pocoMaxUpdateDatePerParent.ProductID }...or you can get the FULL ProductOrmEntity object. Your original code, I don't know if getting all columns of the Product object is what you want, or only some of the columns of the object.

How to GroupBy data from many tables in dotnet core Entity Framework

I'm building some marketplace web app, let's say something like e-bay. Typical scenario is:
User makes offer which consists of one or more items and those items are of certain type.After that other users are bidding on that offer.
Here is simplified diagram.
On SQL Fiddle (here) you can see both CREATE TABLE and INSERT INTO statements
Sample data:
There are two offers. On one offer (Id 1) which consists of one item which is type of "watch". There is another offer, (Id 2), which has one item which is of type "headphone".
On both offers there are bids. On watch, there are two bis; one bid with 100 dollars and another with 120. On headphones, there are bids with 50 and 80 dollars.
What I want to achieve is to have average bid per type. In this sample, that means i want to get 110 as average bid for watch and 65 as average bid for headphone. To achieve that using T-SQL, I would write query like this:
SELECT t.name,
avg(amount)
FROM bid b
LEFT JOIN offer o ON b.OfferId = o.id
LEFT JOIN offeritem oi ON o.id = oi.OfferId
LEFT JOIN itemType t ON oi.itemtypeid = t.Id
GROUP BY t.name
So, my question is - how to achieve that in dotnet core 3.0 EntityFramework
Using GroupBy, like this:
_context.Bids
.Include(b => b.Offer)
.ThenInclude(o => o.OfferItems)
.ThenInclude(os => os.ItemType)
.GroupBy(b => b.Offer.OfferItems.First().ItemType.Name);
gives exception:
Client side GroupBy is not supported.
. When I try with projection, like this:
_context.Bids
.Include(b => b.Offer)
.ThenInclude(o => o.OfferItems)
.ThenInclude(os => os.ItemType)
.GroupBy(b => b.Offer.OfferItems.First().ItemType.Name)
.Select(g => new
{
Key = g,
Value = g.Average(b => b.Amount)
});
i get exception again.
Processing of the LINQ .... failed. This may indicate either a bug or
a limitation in EF Core.
EDIT:
This approach
_context.Bids
.Include(b => b.Offer)
.ThenInclude(o => o.OfferItems)
.ThenInclude(os => os.ItemType)
.GroupBy(b => new { b.Offer.OfferItems.First().ItemType.Name}, b => b.Amount)
.Select(g => new
{
Key = g.Key.Code,
Value = g.Average()
});
also threw an exception, but this time:
Cannot use an aggregate or a subquery in an expression used for the
group by list of a GROUP BY clause.
...
So, is there a way to group that data (get simple Average) or should I make another query and iterate throught collection and make calculation myself? That would lower performance for sure (I was hoping I can do server grouping, but as you can see, i got into mentioned issues). Any ideas? Thanks in advance.

In your case it is hard to hide subquery from grouping
You can try it in such way
var joined =
context.Bid
.SelectMany(x =>
x.Offer.OfferItem
.Select(y => new
{
Amount = x.Amount,
Name = y.ItemType.Name
})
.Take(1));
var grouped = from i in joined
group i by i.Name into groups
select new
{
Key = groups.Key,
Amount = groups.Average(x => x.Amount)
};
it gives me a query
SELECT [t].[Name] AS [Key], AVG([t].[Amount]) AS [Amount]
FROM [Bid] AS [b]
INNER JOIN [Offer] AS [o] ON [b].[OfferId] = [o].[Id]
CROSS APPLY (
SELECT TOP(1) [b].[Amount], [i].[Name], [o0].[Id], [i].[Id] AS [Id0], [o0].[OfferId]
FROM [OfferItem] AS [o0]
INNER JOIN [ItemType] AS [i] ON [o0].[ItemTypeId] = [i].[Id]
WHERE [o].[Id] = [o0].[OfferId]
) AS [t]
GROUP BY [t].[Name]

EF Core 2.0 Group By other properties

I have 2 tables:
USERS
UserId
Name
Scores (collection of table Scores)
SCORES
UserId
CategoryId
Points
I need to show all the users and a SUM of their points, but also I need to show the name of the user. It can be filtered by CategoryId or not.
Context.Scores
.Where(p => p.CategoryId == categoryId) * OPTIONAL
.GroupBy(p => p.UserId)
.Select(p => new
{
UserId = p.Key,
Points = p.Sum(s => s.Points),
Name = p.Select(s => s.User.Name).FirstOrDefault()
}).OrderBy(p => p.Points).ToList();
The problem is that when I add the
Name = p.Select(s => s.User.Name).FirstOrDefault()
It takes so long. I don't know how to access the properties that are not inside the GroupBy or are a SUM. This example is very simple becaouse I don't have only the Name, but also other properties from User table.
How can I solve this?

It takes so long because the query is causing client evaluation. See Client evaluation performance issues and how to use Client evaluation logging to identify related issues.
If you are really on EF Core 2.0, there is nothing you can do than upgrading to v2.1 which contains improved LINQ GroupBy translation. Even with it the solution is not straight forward - the query still uses client evaluation. But it could be rewritten by separating the GroupBy part into subquery and joining it to the Users table to get the additional information needed.
Something like this:
var scores = db.Scores.AsQueryable();
// Optional
// scores = scores.Where(p => p.CategoryId == categoryId);
var points = scores
.GroupBy(s => s.UserId)
.Select(g => new
{
UserId = g.Key,
Points = g.Sum(s => s.Points),
});
var result = db.Users
.Join(points, u => u.UserId, p => p.UserId, (u, p) => new
{
u.UserId,
u.Name,
p.Points
})
.OrderBy(p => p.Points)
.ToList();
This still produces a warning
The LINQ expression 'orderby [p].Points asc' could not be translated and will be evaluated locally.
but at least the query is translated and executes as single SQL:
SELECT [t].[UserId], [t].[Points], [u].[UserId] AS [UserId0], [u].[Name]
FROM [Users] AS [u]
INNER JOIN (
SELECT [s].[UserId], SUM([s].[Points]) AS [Points]
FROM [Scores] AS [s]
GROUP BY [s].[UserId]
) AS [t] ON [u].[UserId] = [t].[UserId]

EF Core Mysql performance

I have Mysql database with ~1 500 000 entities. When I try to execute below statement using EF Core 1.1 and Mysql.Data.EntityFrameworkCore 7.0.7-m61 it takes about 40minutes to finish:
var results = db.Posts
.Include(u => u.User)
.GroupBy(g => g.User)
.Select(g => new { Nick = g.Key.Name, Count = g.Count() })
.OrderByDescending(e => e.Count)
.ToList();
On the other hand using local mysql-cli and below statement, takes around 16 seconds to complete.
SELECT user.Name, count(*) c
FROM post
JOIN user ON post.UserId = user.Id
GROUP BY user.Name
ORDER BY c DESC
Am i doing something wrong, or EF Core performance of MySql is so terrible?

Your queries are doing different things. Some issues in your LINQ-to-Entities query:
You call Include(...) which will eagerly load the User for every item in db.Posts.
You call Count() for each record in each group. This could be rewritten to count the records only once per group.
The biggest issue is that you're only using the Name property of the User object. You could select just this field and find the same result. Selecting, grouping, and returning 1.5 million strings should be a fast operation in EF.
Original:
var results =
db.Posts
.Include(u => u.User)
.GroupBy(g => g.User)
.Select(g => new { Nick = g.Key.Name, Count = g.Count() })
.OrderByDescending(e => e.Count)
.ToList();
Suggestion:
var results =
db.Posts
.Select(x => x.User.Name)
.GroupBy(x => x)
.Select(x => new { Name = x.Key, Count = x.Count() })
.OrderByDescending(x => x.Count)
.ToList();
If EF core still has restrictions on the types of grouping statements it allows, you could call ToList after the first Select(...) statement.

EF Linq to Entities Query with groupby sum and min

I am trying to make a Linq to Entities Query working but I can't figure it out.
This is my my SQL query which is working fine:
SELECT SUM(ca.PointValue) as Points, ua.UserFBID, MIN(ua.[Date]) as FirstDate
FROM [LeaderOfNow].[dbo].[QuestionAnswer] ca
inner join [LeaderOfNow].[dbo].[LONUserAnswers] ua
on ca.Id = ua.AnswerId
group by ua.UserFBID
order by Points desc, FirstDate asc
An so far my best attempt is:
var leaders = db.LONUserAnswers
.GroupBy(a => a.UserFBID)
.Select(a =>
new
{
FBID = a.Key,
CurrentPoints = a.Select(v => v.QuestionAnswer.PointValue).Sum(),
FirstAnswered = a.Min(v => v.Date)
})
.OrderByDescending(a => a.CurrentPoints)
.OrderBy(a => a.FirstAnswered)
.Take(10)
.ToList();
However that renders a mess of sql and only respect the first order by and not the second, which I need to work. Any suggestions on what am I doing wrong? Thank you for the help.

When you chain OrderBy-Functions you have to use ThenBy() or ThenDescendingBy() for the latter.