I want to get items from the sqlite database (There are about 6,000 items in the database)
Some items are duplicates Therefore, I want to remove this duplicate information when I get items
I used the following code
public async static Task<List<myModel>> GetAllItems()
{
using var db = new dbContext();
var query =
from item in db.myTable.GroupBy(x => x.Id).Select(x => x.First())
select new myModel
{
Id = item.Id,
Name = item.Name,
...
};
return await query.ToListAsync();
}
But I get the following error
System.InvalidOperationException: 'The LINQ expression 'GroupByShaperExpression:
KeySelector: m.Id,
ElementSelector:EntityShaperExpression:
EntityType: myTable
ValueBufferExpression:
ProjectionBindingExpression: EmptyProjectionMember
IsNullable: False
.First()' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.'
Update (EF Core 6.0):
EF Core 6.0 did add support for some additional operators to GroupBy result set, including the one in question, so now the original LINQ query should work intact.
Original:
Since currently (hopefully v6.0 would add some) EF Core does not support GroupBy result operators other than key/aggregates projections, it cannot be used to implement the top N items per group function, which is what you basically are trying to do (for N == 1).
So as a workaround (without 3rd party extensions), you have to do that manually by (1) using subquery for selecting the unique keys, and then (2) using it as filter for correlated limiting subquery. e.g. something like
var query = db.myTable.Select(x => new { x.Id }).Distinct() // (1)
.SelectMany(key => db.myTable.Where(x => x.Id == key.Id).Take(1)) // (2)
// The rest is the same as the original
.Select(item => new myModel
{
Id = item.Id,
Name = item.Name,
...
});
First, change your var to IEnumerable<HWGPackageModel>
Second, add AsEnumerable() before GroupBy
public static IEnumerable<myModel> GetAllItems()
{
var db = new dbContext();
IEnumerable<myModel> query =
from item in db.myTable.AsEnumerable().GroupBy(x => x.Id).Select(x => x.FirstOrDefault())
select new myModel
{
Id = item.Id,
...
};
return query;
}
Related
I have a database that contains duplicate records, but only their ID field is the same and the rest of their information is different for example:
Id: test, version: 1.0.0
Id: test, version: 2.1.3
Id: something, version: 4.0.0
Id: something, version: 5.0.0
...
I can get all items without duplication with the following codes
using var db = new dbContext();
var query =
from item in db.my.Select(x => new { x.Id }).Distinct()
.SelectMany(key => db.myTable.Where(x => x.Id == key.Id).Take(1))
select new myModel
{
Id = item.Id,
Versions = <Here we need all related versions>
};
return await query.ToListAsync();
Now I want to get all versions of an ID and put it in the Versions in select new myModel{}
But I do not know how to do this
UPDATE:
i added this line to query
from versions in db.myTable.AsEnumerable().GroupBy(x => x.Id)
and
select new myModel
{
Id = item.Id,
Versions = versions.Select(x => x.Version).ToList()
};
but i got an error
System.InvalidOperationException: 'Processing of the LINQ expression 'GroupByShaperExpression:
This requires different approach than the one for returning single item per group.
The required operation here makes no sense to be implemented server side, hence should be done client side, with the only potential optimization of not retrieving unneeded data from the database.
First, use server side query to select all data needed. Then materialize that query in memory and perform GroupBy and final projection there.
e.g.
var dbQuery = db.my.Select(x => new
{
x.Id,
x.Version,
});
var data = await dbQuery.ToListAsync();
var result = data
.GroupBy(x => x.Id)
.Select(g => new myModel
{
Id = g.Key,
Versions = g.Select(x => x.Version).ToList(),
});
I'm having trouble with a LINQ query.
var cRecords = context.ClassificationRecords
.GroupBy(p => p.ImageRecordId)
.Select(g => g.OrderByDescending(c => c.Created).FirstOrDefault())
.ToList();
What I'm trying to do is get a list of the last dated classification record for each user. I'll include the user and some other stuff, but that's the stripped down version.
The problem is that the above is giving me an error:
System.InvalidOperationException: 'The LINQ expression '(GroupByShaperExpression:
KeySelector: (c.ImageRecordId),
ElementSelector:(EntityShaperExpression:
EntityType: ClassificationRecord
ValueBufferExpression:
(ProjectionBindingExpression: EmptyProjectionMember)
IsNullable: False
)
)
.OrderByDescending(c => c.Created)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.'
I've seen this query used in several places on here and other websites, so it should be working..?
I'm using .Net Core 3.1 with EntityFrameworkCore 3.1.11
The query is being ran inside a hangfire worker program, so not a Blazor server - not sure if it makes a difference.
One possible work around is to modify your query into something EF 3.1 can translate - how efficient this will be depends on your database engine. I think this should work but you need the primary key to match the records:
var cRecords = context.ClassificationRecords
.Where(c => c.PrimaryKey == context.ClassificationRecords.Where(c2 => c2.ImageRecordId == c.ImageRecordId)
.OrderByDescending(c2 => c2.Created)
.First().PrimaryKey)
.ToList();
This query is not possible with EFC. After GroupBy you can select only grouping key or aggregation result. Faster query here is Window Functions usage which are not supported by EF.
So there is workaround:
var grouped =
from c in ontext.ClassificationRecords
group c by c.ImageRecordId into g
select new
{
ImageRecordId = g.Key,
Created = g.Max(x => x.Created)
};
var query =
from g in grouped
join c in context.ClassificationRecords
on new { g.Created, g.ImageRecordId } equals { c.Created, c.ImageRecordId }
select c;
var result = query.ToList();
The goal is to get the first DateTime and Last DateTime from a collection on an Entity (Foreign Key). My Entity is an organization and my collection are Invoices. I'm grouping results since Organizations unfortunately are not Unique. I'm dealing with duplicate data and cannot assume my organizations are unique so I'm grouping by a Number field on my Entity.
I'm using .NET Core 2.1.2 with Entity Framework.
I'm trying to get the following query generated from LINQ:
SELECT MIN([organization].[Id]) AS Id, MIN([organization].[Name]) AS Name,
MIN([organization].[Number]) AS Number, MIN([invoice].[Date])
AS First, MAX([invoice].[Date]) AS Last
FROM [organization]
INNER JOIN [invoice] ON [invoice].[OrganizationId] = [organization].[Id]
GROUP BY [organization].[Number], [organization].[Name]
ORDER BY [organization].[Name]
However I have no idea how to get to write the LINQ query to get it to generate this result.
I got as far as:
await _context
.Organization
.Where(z => z.Invoices.Any())
.GroupBy(organization => new
{
organization.Number,
organization.Name
})
.Select(grouping => new
{
Id = grouping.Min(organization => organization.Id),
Name = grouping.Min(organization => organization.Name),
Number= grouping.Min(organization => organization.Number),
//First = ?,
//Last = ?
})
.OrderBy(z => z.Name)
.ToListAsync();
I have no clue how to write the LINQ query in such a way that it generates the above.
I have a couple questions still:
Are the Min statements for Id, Name and Number correct ways of getting the
first element in the grouping?
Do I need a join statement or is "WHERE EXISTS" better (this got generated before I changed the code)?
Does anyone know how to finish writing the LINQ statement? Because I have to get the first and last Date from the Invoices Collection on my Organization Entity:
organization.Invoices.Min(invoice => invoice.Date)
organization.Invoices.Max(invoice => invoice.Date)
Here is the trick.
To make inner join by using collection navigation property simple use SelectMany and project all primitive properties that you need later (this is important for the current EF Core query translator). Then perform the GroupBy and project the key properties / aggregates. Finally do the ordering.
So
var query = _context
.Organization
.SelectMany(organization => organization.Invoices, (organization, invoice) => new
{
organization.Id,
organization.Number,
organization.Name,
invoice.Date
})
.GroupBy(e => new
{
e.Number,
e.Name
})
.Select(g => new
{
Id = g.Min(e => e.Id),
Name = g.Key.Name,
Number = g.Key.Number,
First = g.Min(e => e.Date),
Last = g.Max(e => e.Date),
})
.OrderBy(e => e.Name);
is translated to
SELECT MIN([organization].[Id]) AS [Id], [organization].[Name], [organization].[Number],
MIN([organization.Invoice].[Date]) AS [First], MAX([organization.Invoice].[Date]) AS [Last]
FROM [Organization] AS [organization]
INNER JOIN [Invoice] AS [organization.Invoice] ON [organization].[Id] = [organization.Invoice].[OrganizationId]
GROUP BY [organization].[Number], [organization].[Name]
ORDER BY [organization].[Name]
I am using entity framework and doing a group by over a table. My query is a follows:-
var brokerPaymentLists = dbContext.BrokerPayments
.Include("PaymentDetail")
.Where(bp => bp.IdPaymentStatus == (long)EntityModel.Additions.Variables.PaymentStatus.ALLOTED)
.GroupBy(bp => bp.IdBroker,
(key, g) => new
{
IdBroker = key.Value,
BrokerPayments = g.ToList()
}).ToList();
I have included PaymentDetail but after grouping by i can see that the paymentdetail for each item in the BrokerPayments i null. Any suggestion why this is the case, also how can i do the group by such that I can my my paymentDetail insisde each of the BrokerPayments;
The eagerly loading by using Include requires the shape of the data to do not be changed since the Include is applied. In your case this means the query must return IQueryable<BrokerPayments>. But the GroupBy operator changes the shape because it returns IQueryable<IGrouping<TKey, TSource>>. Same will happen with projections and custom joins.
As a workaround you can execute grouping in LINQ to Objects like:
var brokerPaymentLists = dbContext.BrokerPayments
.Include("PaymentDetail")
.Where(bp => bp.IdPaymentStatus == (long)EntityModel.Additions.Variables.PaymentStatus.ALLOTED)
.AsEnumerable()
.GroupBy(bp => bp.IdBroker,
(key, g) => new
{
IdBroker = key.Value,
BrokerPayments = g
});
NOTE: pay attention that the query exectuion will not be defferd
I have a PostInfo table and LikeInfo Table,PostInfo have list of LikeInfo (for post likes).PostInfo table have a NotMapped property named of "LikeCount".
I want select list of PostInfoes and join to LikeInfo table and calcualte count of LikeInfoes of post and fill count of this to LikeCount property.
Here is my sample:
var query = context.PostInfoes.Where(x => x.UserId == userId).Include(x => x.LikeInfoes);
foreach (var item in query)
{
item.LikeCount = item.LikeInfoes.Count;
}
return query.ToList();
That is not good way for me because I do foreach on query and set LikeCount manualy however I dont want/need include full peoperties of LikeInfo table in this case.
I'm finding the best and easy way to fill this property.
Since EF6 does not allow projecting to entity type, you need to use LINQ to Entities query with intermediate anonymous type projection containing all necessary data, then switch to LINQ to Objects (via AsEnumerable()) and do the final projection, using a delegate block to perform the necessary unmapped property fixups:
var result = context.PostInfoes
.Where(x => x.UserId == userId)
.Select(x => new { Info = x, LikeCount = x.LikeInfoes.Count() })
.AsEnumerable()
.Select(x =>
{
x.Info.LikeCount = x.LikeCount;
return x.Info;
})
.ToList();