We Keep Coding

How to GroupBy data from many tables in dotnet core Entity Framework

I'm building some marketplace web app, let's say something like e-bay. Typical scenario is:
User makes offer which consists of one or more items and those items are of certain type.After that other users are bidding on that offer.
Here is simplified diagram.
On SQL Fiddle (here) you can see both CREATE TABLE and INSERT INTO statements
Sample data:
There are two offers. On one offer (Id 1) which consists of one item which is type of "watch". There is another offer, (Id 2), which has one item which is of type "headphone".
On both offers there are bids. On watch, there are two bis; one bid with 100 dollars and another with 120. On headphones, there are bids with 50 and 80 dollars.
What I want to achieve is to have average bid per type. In this sample, that means i want to get 110 as average bid for watch and 65 as average bid for headphone. To achieve that using T-SQL, I would write query like this:
SELECT t.name,
avg(amount)
FROM bid b
LEFT JOIN offer o ON b.OfferId = o.id
LEFT JOIN offeritem oi ON o.id = oi.OfferId
LEFT JOIN itemType t ON oi.itemtypeid = t.Id
GROUP BY t.name
So, my question is - how to achieve that in dotnet core 3.0 EntityFramework
Using GroupBy, like this:
_context.Bids
.Include(b => b.Offer)
.ThenInclude(o => o.OfferItems)
.ThenInclude(os => os.ItemType)
.GroupBy(b => b.Offer.OfferItems.First().ItemType.Name);
gives exception:
Client side GroupBy is not supported.
. When I try with projection, like this:
_context.Bids
.Include(b => b.Offer)
.ThenInclude(o => o.OfferItems)
.ThenInclude(os => os.ItemType)
.GroupBy(b => b.Offer.OfferItems.First().ItemType.Name)
.Select(g => new
{
Key = g,
Value = g.Average(b => b.Amount)
});
i get exception again.
Processing of the LINQ .... failed. This may indicate either a bug or
a limitation in EF Core.
EDIT:
This approach
_context.Bids
.Include(b => b.Offer)
.ThenInclude(o => o.OfferItems)
.ThenInclude(os => os.ItemType)
.GroupBy(b => new { b.Offer.OfferItems.First().ItemType.Name}, b => b.Amount)
.Select(g => new
{
Key = g.Key.Code,
Value = g.Average()
});
also threw an exception, but this time:
Cannot use an aggregate or a subquery in an expression used for the
group by list of a GROUP BY clause.
...
So, is there a way to group that data (get simple Average) or should I make another query and iterate throught collection and make calculation myself? That would lower performance for sure (I was hoping I can do server grouping, but as you can see, i got into mentioned issues). Any ideas? Thanks in advance.

In your case it is hard to hide subquery from grouping
You can try it in such way
var joined =
context.Bid
.SelectMany(x =>
x.Offer.OfferItem
.Select(y => new
{
Amount = x.Amount,
Name = y.ItemType.Name
})
.Take(1));
var grouped = from i in joined
group i by i.Name into groups
select new
{
Key = groups.Key,
Amount = groups.Average(x => x.Amount)
};
it gives me a query
SELECT [t].[Name] AS [Key], AVG([t].[Amount]) AS [Amount]
FROM [Bid] AS [b]
INNER JOIN [Offer] AS [o] ON [b].[OfferId] = [o].[Id]
CROSS APPLY (
SELECT TOP(1) [b].[Amount], [i].[Name], [o0].[Id], [i].[Id] AS [Id0], [o0].[OfferId]
FROM [OfferItem] AS [o0]
INNER JOIN [ItemType] AS [i] ON [o0].[ItemTypeId] = [i].[Id]
WHERE [o].[Id] = [o0].[OfferId]
) AS [t]
GROUP BY [t].[Name]

C# : How to get anonymous type from LINQ result

I need to get NewsImage field and list of categories Ids that associated with the news in Many to Many relationship ... but it gives me error:
The type of one of the expressions in the join clause is incorrect.Type inference failed in the call to 'Join'.
My code looks like this
var Result1 = (from c in db.News
join d in db.Categories
on c.NewsId equals d.News.Select(l => l.NewsId)
where c.NewsId == 1
select new { c.NewsImagePath, d.CategoryId }).ToList();

Assuming you have a navigation property defining the n-n relation I would write:
var result = db.News
.Where(x => x.NewsId == 1)
.SelectMany(x => x.Categories,
(news, category) => new { news.NewsImagePath, category.CategoryId })
.ToList();

The problem is inside the on statement.
on c.NewsId equals d.News.Select( l => l.NewsId )
The Select on the right-hand side will return a IEnumerable of news, which is not what you want.
Something like this would technically work:
on c.NewsId equals d.News.Select( l => l.NewsId ).FirstOrDefault()
But it does not make sense logically.
I suspect the whole query should be built differently. I think you want to join when the category list of news contains the news item. In that case, you can't use the join statement, it would look somewhat like this:
from n in db.News
from c in db.Categories
where c.News.Select( ne => ne.NewsId ).Contains( n.NewsId )
select new { n.NewsImagePath, c.CategoryId }

Convert SQL query with multiple GroupBy columns to LINQ

SELECT
[TimeStampDate]
,[User]
,count(*) as [Usage]
FROM [EFDP_Dev].[Admin].[AuditLog]
WHERE [target] = '995fc819-954a-49af-b056-387e11a8875d'
GROUP BY [Target], [User] ,[TimeStampDate]
ORDER BY [Target]
My database table has the columns User, TimeStampDate, and Target (which is a GUID).
I want to retrieve all items for each date for each user and display count of entries.
The above SQL query works. How can I convert it into LINQ to SQL? Am using EF 6.1 and my entity class in C# has all the above columns.
Create Filter basically returns an IQueryable of the entire AuditLogSet :
using (var filter = auditLogRepository.CreateFilter())
{
var query = filter.All
.Where(it => it.Target == '995fc819-954a-49af-b056-387e11a8875d')
.GroupBy(i => i.Target, i => i.User, i => i.TimeStamp);
audits = query.ToList();
}
Am not being allowed to group by on 3 columns in LINQ and I am also not sure how to select like the above SQL query with count. Fairly new to LINQ.

You need to specify the group by columns in an anonymous type like this:-
var query = filter.All
.Where(it => it.Target == '995fc819-954a-49af-b056-387e11a8875d')
.GroupBy(x => new { x.User, x.TimeStampDate })
.Select(x => new
{
TimeStampDate= x.Key.TimeStampDate,
User = x.Key.User,
Usage = x.Count()
}).ToList();

Many people find query syntax simpler and easier to read (this might not be the case, I don't know), here's the query syntax version anyway.
var res=(from it in filter.All
where it.Target=="995fc819-954a-49af-b056-387e11a8875d"
group it by new {it.Target, it.User, it.TimeStampDate} into g
orderby g.Key.Target
select new
{
TimeStampDate= g.Key.TimeStampDate,
User=g.Key.User,
Usage=g.Count()
});
EDIT: By the way you don't need to group by Target neither OrderBy, since is already filtered, I'm leaving the exact translation of the query though.

To use GroupBy you need to create an anonymous object like this:
filter.All
.Where(it => it.Target == '995fc819-954a-49af-b056-387e11a8875d')
.GroupBy(i => new { i.Target, i.User, i.TimeStamp });

It is unnecessary to group by target in your original SQL.
filter.All.Where( d => d.Target == "995fc819-954a-49af-b056-387e11a8875d")
.GroupBy(d => new {d.User ,d.TimeStampDate} )
.Select(d => new {
User = d.Key.User,
TimeStampDate = d.Key.TimeStampDate,
Usage = d.Count()
} );

Eliminate loop and use Linq instead

Say I have a List as below:
List<R> lstR = GetR();
Now I want a Linq statement to get menus assigned to R, I achieved this by using a loop and then using Linq to get the menus as below:
List<int> ids = new List<int>();
foreach (R r in lstR)
{
ids.Add(r.Id);
}
menu = (from s in db.Menu
where ids.Contains(s.R.Id)
select s.MenuText).Distinct();
Now as far as I know the above is two loop(Linq is using internal loop). would I be able to combine these two statements i.e. not do the first loop to get the ids?

In both lstR and db.Menu are either in-memory data sets (Linq-to-Objects) or IQueryable collections from your database, you can do this:
menu =
(from s in db.Menu
where lstR.Select(r => r.Id)
.Contains(s.R.Id)
select s.MenuText)
.Distinct();
Or this:
menu =
(from s in db.Menu
join r in lstR on s.R.Id equals r.Id
select s.MenuText)
.Distinct();
However, since List<R> exists in memory and db.Menu is an IQueryable, you're options are limited. You could materialize db.Menu into an IEnumerable, so you can process it in memory:
List<R> lstR = GetR();
menu =
(from s in db.Menu.AsEnumerable()
join r in lstR on s.R.Id equals r.Id
select s.MenuText)
.Distinct();
But, this can be costly if there are a lot of records. It's better to do something like this, which admittedly doesn't look much different from what you already have:
List<R> lstR = GetR();
var ids = lstR.Select(r => r.Id).ToList(); // or .ToArray();
menu =
(from s in db.Menu
where ids.Contains(s.R.Id)
select s.MenuText)
.Distinct();
But in truth, the best option is to see if you can refactor GetR so that it returns an IQueryable<R> from your database. That way you can use both of the first two options without needing to materialize any sets into memory first. And by the way, once you've done that and set up navigation properties, you can probably do something like this:
IQueryable<R> lstR = GetR();
menu =
(from r in lstR
from s in r.Menus
select s.MenuText)
.Distinct();

It can be done like.
menu = (from s in db.Menu
where lstR.Select(item => item.Id).Contains(s.R.Id)
select s.MenuText).Distinct();
But i wouldnt combine those two statements, because if you use a HashSet it will speed up:
var ids = new HashSet<int>(lstR);
menu = (from s in db.Menu
where ids.Contains(s.R.Id)
select s.MenuText).Distinct();
This will be faster i guess. The problem with the first one is, every s in db.Menu The list is iterated for creating a list of id's select().

You coud use the linq projection method Select():
ids = lstR.Select(p => p.Id);

menu = db.Menu.Where(s => GetR().Select(r => r.Id).Contains(s.R.Id))
.Select(s => s.MenuText)
.Distinct();
but it will be to complex. It will be better if you'l write like this
var ids = GetR().Select(r => r.Id);
menu = db.Menu.Where(s => ids.Contains(s.R.Id))
.Select(s => s.MenuText)
.Distinct();

Use Join
var result = (from s in db.Menu
join r in lstR on s.Id equals r.ID
select s.MenuText).Distinct();

How to group using LINQ, then get value of largest group

Here's what I have so far:
var bestReason =
from p in successfulReasons
group p by p.Reason into g
select new { Excuse = g.Key, ExcuseCount = g.Count() };
What I need to do now is return one reason that is the best reason, determined by which were successful in the past.
Sample data:
ID,Reason
---------
0,Weather
1,Traffic
2,Illness
3,Weather
4,Traffic
5,Traffic
6,Pirates
should return "Traffic"
Would like to do it all in one LINQ statement, if possible.
Thanks.
EDIT: If there are 7 Pirate Attacks, and 7 Traffic Accidents, I'm ok with returning either one (the first alphabetically would be fine).

var bestReason = successfulReasons
.GroupBy(r => r.Reason)
.OrderByDescending(grp => grp.Count())
.First().Key;

If I understand your question correctly, you can do:
string bestReason =
(from p in successfulReasons
orderby p.Reason
group p by p.Reason into g
orderby g.Count() descending
select g.Key).FirstOrDefault();

var group = excuses.GroupBy(m => m.Reason)
.OrderByDescending(m => m.Count())
.Select(m => m.Key)
.FirstOrDefault();
Which produces the following sql statement:
SELECT TOP (1) [t1].[Reason]
FROM (
SELECT COUNT(*) AS [value], [t0].[Reason]
From [dbo].[Excuses] As [t0]
GROUP BY [t0].[Reason]
) As [t1]
ORDER BY [t1].[value] DESC
Since this is a moderately complicated IQueryable expression, you might consider compiling it to speed up the response time:
Func<ExcusesDataContext, string> commonResult = CompiledQuery.Compile(
(ExcusesDataContext c) => c.Excuses.GroupBy(m => m.Reason).OrderByDescending(m => m.Count()).Select(m => m.Key).FirstOrDefault()
);
Console.WriteLine(commonResult(new ExcusesDataContext()));
Console.ReadLine();
You could also just call the stored procedure via a repository and snag the particular value that you're looking for. This would be the fastest path to happiness, but the least fun to maintain:
string excuse = this.repo.Excuses.MostCommonFor(ProblemList.BeingLate);