We Keep Coding

How to write linq query for this sql statement

How would you write a linq query with the following SQL statement. I've tried several methods referenced on stackoverflow but they either don't work with the EF version I'm using (EF core 3.5.1) or the DBMS (SQL Server).
select a.ProductID, a.DateTimeStamp, a.LastPrice
from Products a
where a.DateTimeStamp = (select max(DateTimeStamp) from Products where a.ProductID = ProductID)
For reference, a couple that I've tried (both get run-time errors).
var results = _context.Products
.GroupBy(s => s.ProductID)
.Select(s => s.OrderByDescending(x => x.DateTimeStamp).FirstOrDefault());
var results = _context.Products
.GroupBy(x => new { x.ProductID, x.DateTimeStamp })
.SelectMany(y => y.OrderByDescending(z => z.DateTimeStamp).Take(1))
Thanks!

I understand you would like to have a list of the latest prices of each products?
First of all I prefer to use group by option even over 1st query
select a.ProductID, a.DateTimeStamp, a.LastPrice
from Products a
where a.DateTimeStamp IN (select max(DateTimeStamp) from Products group by ProductID)
Later Linq:
var maxDateTimeStamps = _context.Products
.GroupBy(s => s.ProductID)
.Select(s => s.Max(x => x.DateTimeStamp)).ToArray();
var results = _context.Products.Where(s=>maxDateTimeStamps.Contains(s.DateTimeStamp));
-- all assuming that max datetime stamps are unique

I've managed to do it with the following which replicates the correlated sub query in the original post (other than using TOP and order by instead of the Max aggregate), though I feel like there must be a more elegant way to do this.
var results = from x
in _context.Products
where x.DateTimeStamp == (from y
in _context.Products
where y.ProductID == x.ProductID
orderby y.DateTimeStamp descending
select y.DateTimeStamp
).FirstOrDefault()
select x;

I prefer to break up these queries into IQueryable parts, do you can debug each "step".
Something like this:
IQueryable<ProductOrmEntity> pocoPerParentMaxUpdateDates =
entityDbContext.Products
//.Where(itm => itm.x == 1)/*if you need where */
.GroupBy(i => i.ProductID)
.Select(g => new ProductOrmEntity
{
ProductID = g.Key,
DateTimeStamp = g.Max(row => row.DateTimeStamp)
});
//// next line for debugging..do not leave in for production code
var temppocoPerParentMaxUpdateDates = pocoPerParentMaxUpdateDates.ToListAsync(CancellationToken.None);
IQueryable<ProductOrmEntity> filteredChildren =
from itm
in entityDbContext.Products
join pocoMaxUpdateDatePerParent in pocoPerParentMaxUpdateDates
on new { a = itm.DateTimeStamp, b = itm.ProductID }
equals
new { a = pocoMaxUpdateDatePerParent.DateTimeStamp, b = pocoMaxUpdateDatePerParent.ProductID }
// where
;
IEnumerable<ProductOrmEntity> hereIsWhatIWantItems = filteredChildren.ToListAsync(CancellationToken.None);
That last step, I am putting in an anonymous object. You can put the data in a "new ProductOrmEntity() { ProductID = pocoMaxUpdateDatePerParent.ProductID }...or you can get the FULL ProductOrmEntity object. Your original code, I don't know if getting all columns of the Product object is what you want, or only some of the columns of the object.

linq - find item in list within multiple lists

I have a highly nested class, and trying to find a single item buried deep within. The following gives me an error "Can't convert type match to bool', although I don't see why it thinks I'm trying to return a boolean.
var match = community.TeamLeagues
.Where(x => x.Seasons
.Where(y => y.Divisions
.Where(z => z.Matches
.Where(a => a.Id == "1234").FirstOrDefault())));

Where by itself returns a (deferred) enumerable of items and cannot as such be used as a condition by the outer Where. What you probably want to do is to use Contains(), Any() or All() inside the outer Wheres that will return the result you're looking for.
Something like this might be what you're after:
var match = community.TeamLeagues.Where(t =>
t.Seasons.Any(
s => s.Divisions.Any(
d => d.Matches.Any(
m => m.Id == "1234")
)));

The Where method needs to evaluate an expression that returns a bool. Your nested Wheres are not doing that - the only Where that is, is the last one a => a.Id == "1234", all the other expressions are returning an IEnumerable.

z.Matches.Where(a => a.Id == "1234").FirstOrDefault() returns a object of type Match(your collection item type of the IEnumerable Matches) (or null), no boolean value. I guess you need to check if there are entires in matches that have a Id 1234. Use Any to evaluate a condition:
var match = community.TeamLeagues.Where(x =>
x.Seasons.Any(y =>
y.Divisions.Any(z =>
z.Matches.Any(a => a.Id == "1234")
)));
[items.Where(x => x.Id == 4).Any() is the same as items.Any(x => x.Id == 4)]
This returns you all TeamLeagues which contain a Season which contain a Division which contain a Match which has a element with the id 1234.

To make it simple you can also use the Matches table directly and using a ViewModel you can represent your view.
like:
var MyViewModel = (from l in Mathes
where l.Id == "1234"
select new MyViewModel
{
Id = l.Id,
MatchName = l.Name,
}).ToList();

Couldn't get it working with linq, but works with query syntax.
var leagueMatch = (from teamLeague in community.TeamLeagues
from season in teamLeague.Seasons
from division in season.Divisions
from match in division.Matches.Where(x => x.Id == "1234")
select match).FirstOrDefault();

how to use linq to return a value from 4 tables

I am reasonably new to Linq and it seems quite easy to iuse but I am having an issue when trying to extract a value from a table that is linked/constrained by 3 other tables.
I have this in my SQL DB:
I am using Asp.Net 4 and Entity Framework 6.
I have as a parameter the 'DatabaseName'.
I ultimately want to get the SubscriptionRef that is assigned to this name.
I could do this step-by-step (ie using multiple linqs) but I thought it would look 'clean' using just 1 linq statment.
I have got as far as this:
var names = o.RegisteredNames.Where(d => d.DatabaseName == DBName).Where(d => d.ClientNames.Where(f => f.ClientId == f.Client.ClientId).FirstOrDefault();
But I get the error:
Cannot implicitly convert type 'Services.ClientName' to 'bool'

You have a Problem here:
d => d.ClientNames.Where(f => f.ClientId == f.Client.ClientId)
f => ... returns a single ClientName or null, which causes your error, because there should be a boolean.
If you want this first value or null, you should replace
.Where(d => d.ClientNames ...
//with
.Select(d => d.ClientNames ...

Try this:
o.RegisteredNames.First(d => d.DatabaseName == DBName).ClientNames.Select(x=>x.Client.Subscription.SubscriptionRef)
It should give you list go SubscriptionRef.

You can try with one LINQ query like...
var names = o.RegisteredNames.Where(d => d.DatabaseName == DBName ).FirstOrDefault();

You might wanna try sql style:
var client = from c in db.Clients
join cn in db.ClientNames on c.ClientId equals cn.ClientId
join rn in db.RegisteredNames on cn.RegisteredNamesId equals rn.RegisteredNameId
where rn.DatabaseName == "YourDBName"
select c;
But it also depends on how your objects were built.

Try using join:
var names = (
from names in o.RegisteredNames.Where(d => d.DatabaseName == DBName)
join cnames in o.ClientNames on names.RegisteredNamesId equals cnames.RegisteredNamesId
select cnames.ClientId
).FirstOrDefault();
Add as many joins as you want.

Try this
It works in List,
var option1= o.RegisteredNames
.Where(g => g.DbName == "YourDbName")
.Where(h => h.ClientNames.Any(f => f == 5))
.FirstOrDefault();
var option2= o.RegisteredNames
.FirstOrDefault(h => h.DbName == "Name" && h.ClientNames.Any(j => j == 1));

LINQ Query descending tree

I have this entity structure:
Categories->Has many "Groups"->Has many "Types"->Has many "Items"
I need to get the descending tree (with includes) filtering by Item.Color == "blue", that is:
var tree = from c in db.Categories
.Include(ct =>
ct.Groups
.Select(gr =>
gr.Types
.Select(pr => pr.Products)
)
)
join g in db.Groups on c.CategoryId equals g.CategoryId
join t in db.Types on g.GroupId equals t.GroupId
join i in db.Items on t.TypeId equals i.TypeId
where i.Color == "blue" // example filter
select c;
With this query I have the descending tree, however I get Items with other colors.
I need only descending tree for blue items.
Thanks!

I am assuming you are using EntityFramework.
All you need to do is an include all the way down, and then use the .Any predicate on your entities. E.g.
var tree = db.Categories.Include(ct => ct.Groups.Select(gr =>
gr.Types.Select(pr =>
pr.Products)))
.Where(c => c.Groups.Any(g =>
g.Types.Any(t =>
t.Items.Any(i =>
i.Color == "blue"))))
.ToList()
A word of warning, if a type has multiple items, and only 1 is blue, you will get all of its Items, not just the blue one. But if a type has no blue items, you wont get it.
If you want to filter at each level, you will need to do projections at each level.
EDIT
So if you want to filter you have 2 options:
Projection at each level
Start by getting all blue items, and then retrieve theirs types, groups and categories.
Examples
1. Projections at each level
var tree = db.Categories.Include(ct => ct.Groups.Select(gr =>
gr.Types.Select(pr =>
pr.Products)))
.Where(c => c.Groups.Any(g =>
g.Types.Any(t =>
t.Items.Any(i =>
i.Color == "blue"))))
.Select(c => new { Groups = c.Groups.Where(g =>
g.Types.Any(t =>
t.Items.Any(i =>
i.Color == "blue"))
.Select( g => new { Types = g.Types.Where(t =>
t.Items.Any(i =>
i.Color == "blue"))
.Select(t => new { Items = t.Items.Where(i =>
i.Color == "blue")
.ToList() // Return list of anonymous types (
You cannot project back an entity type, you have to retrieve the annonymous types first, and then use Linq2Objects to project into the Entity type as EF will not allow you to do this. Often what I do is directly project into a view model ready for display.

SQL Azure vs. On-Premises Timeout Issue - EF

I'm working on a report right now that runs great with our on-premises DB (just refreshed from PROD). However, when I deploy the site to Azure, I get a SQL Timeout during its execution. If I point my development instance at the SQL Azure instance, I get a timeout as well.
Goal: To output a list of customers that have had an activity created during the search range, and when that customer is found, get some other information about that customer regarding policies, etc. I've removed some of the properties below for brevity (as best I can)...
UPDATE
After lots of trial and error, I can get the entire query to run fairly consistently within 1000MS so long as this block of code is not executed.
CurrentStatus = a.Activities
.Where(b => b.ActivityType.IsReportable)
.OrderByDescending(b => b.DueDateTime)
.Select(b => b.Status.Name)
.FirstOrDefault(),
With this code in place, things begin to go haywire. I think this Where clause is a big part of it: .Where(b => b.ActivityType.IsReportable). What is the best way to grab the status name?
EXISTING CODE
Any thoughts as to why SQL Azure would timeout whereas on-premises would turn this around in less than 100MS?
return db.Customers
.Where(a => a.Activities.Where(
b => b.CreatedDateTime >= search.BeginDateCreated
&& b.CreatedDateTime <= search.EndDateCreated).Count() > 0)
.Where(a => a.CustomerGroup.Any(d => d.GroupId== search.GroupId))
.Select(a => new CustomCustomerReport
{
CustomerId = a.Id,
Manager = a.Manager.Name,
Customer = a.FirstName + " " + a.LastName,
ContactSource= a.ContactSource!= null ? a.ContactSource.Name : "Unknown",
ContactDate = a.DateCreated,
NewSale = a.Sales
.Where(p => p.Employee.IsActive)
.OrderByDescending(p => p.DateCreated)
.Select(p => new PolicyViewModel
{
//MISC PROPERTIES
}).FirstOrDefault(),
ExistingSale = a.Sales
.Where(p => p.CancellationDate == null || p.CancellationDate <= myDate)
.Where(p => p.SaleDate < myDate)
.OrderByDescending(p => p.DateCreated)
.Select(p => new SalesViewModel
{
//MISC PROPERTIES
}).FirstOrDefault(),
CurrentStatus = a.Activities
.Where(b => b.ActivityType.IsReportable)
.OrderByDescending(b => b.DueDateTime)
.Select(b => b.Disposition.Name)
.FirstOrDefault(),
CustomerGroup = a.CustomerGroup
.Where(cd => cd.GroupId == search.GroupId)
.Select(cd => new GroupViewModel
{
//MISC PROPERTIES
}).FirstOrDefault()
}).ToList();

I cannot give you a definite answer but I would recommend approaching the problem by:
Run SQL profiler locally when this code is executed and see what SQL is generated and run. Look at the query execution plan for each query and look for table scans and other slow operations. Add indexes as needed.
Check your lambdas for things that cannot be easily translated into SQL. You might be pulling the contents of a table into memory and running lambdas on the results, which will be very slow. Change your lambdas or consider writing raw SQL.
Is the Azure database the same as your local database? If not, pull the data locally so your local system is indicative.
Remove sections (i.e. CustomerGroup then CurrentDisposition then ExistingSale then NewSale) and see if there is a significant performance improvement after removing the last section. Focus on the last removed section.
Looking at the line itself:
You use ".Count() > 0" on line 4. Use ".Any()" instead, since the former goes through every row in the database to get you an accurate count when you just want to know if at least one row satisfies the requirements.
Ensure fields referenced in where clauses have indexes, such as IsReportable.

Short answer: use memory.
Long answer:
Because of either bad maintenance plans or limited hardware, running this query in one big lump is what's causing it to fail on Azure. Even if that weren't the case, because of all the navigation properties you're using, this query would generate a staggering number of joins. The answer here is to break it down in smaller pieces that Azure can run. I'm going to try to rewrite your query into multiple smaller, easier to digest queries that use the memory of your .NET application. Please bear with me as I make (more or less) educated guesses about your business logic/db schema and rewrite the query accordingly. Sorry for using the query form of LINQ but I find things such as join and group by are more readable in that form.
var activityFilterCustomerIds = db.Activities
.Where(a =>
a.CreatedDateTime >= search.BeginDateCreated &&
a.CreatedDateTime <= search.EndDateCreated)
.Select(a => a.CustomerId)
.Distinct()
.ToList();
var groupFilterCustomerIds = db.CustomerGroup
.Where(g => g.GroupId = search.GroupId)
.Select(g => g.CustomerId)
.Distinct()
.ToList();
var customers = db.Customers
.AsNoTracking()
.Where(c =>
activityFilterCustomerIds.Contains(c.Id) &&
groupFilterCustomerIds.Contains(c.Id))
.ToList();
var customerIds = customers.Select(x => x.Id).ToList();
var newSales =
(from s in db.Sales
where customerIds.Contains(s.CustomerId)
&& s.Employee.IsActive
group s by s.CustomerId into grouped
select new
{
CustomerId = grouped.Key,
Sale = grouped
.OrderByDescending(x => x.DateCreated)
.Select(new PolicyViewModel
{
// properties
})
.FirstOrDefault()
}).ToList();
var existingSales =
(from s in db.Sales
where customerIds.Contains(s.CustomerId)
&& (s.CancellationDate == null || s.CancellationDate <= myDate)
&& s.SaleDate < myDate
group s by s.CustomerId into grouped
select new
{
CustomerId = grouped.Key,
Sale = grouped
.OrderByDescending(x => x.DateCreated)
.Select(new SalesViewModel
{
// properties
})
.FirstOrDefault()
}).ToList();
var currentStatuses =
(from a in db.Activities.AsNoTracking()
where customerIds.Contains(a.CustomerId)
&& a.ActivityType.IsReportable
group a by a.CustomerId into grouped
select new
{
CustomerId = grouped.Key,
Status = grouped
.OrderByDescending(x => x.DueDateTime)
.Select(x => x.Disposition.Name)
.FirstOrDefault()
}).ToList();
var customerGroups =
(from cg in db.CustomerGroups
where cg.GroupId == search.GroupId
group cg by cg.CustomerId into grouped
select new
{
CustomerId = grouped.Key,
Group = grouped
.Select(x =>
new GroupViewModel
{
// ...
})
.FirstOrDefault()
}).ToList();
return customers
.Select(c =>
new CustomCustomerReport
{
// ... simple props
// ...
// ...
NewSale = newSales
.Where(s => s.CustomerId == c.Id)
.Select(x => x.Sale)
.FirstOrDefault(),
ExistingSale = existingSales
.Where(s => s.CustomerId == c.Id)
.Select(x => x.Sale)
.FirstOrDefault(),
CurrentStatus = currentStatuses
.Where(s => s.CustomerId == c.Id)
.Select(x => x.Status)
.FirstOrDefault(),
CustomerGroup = customerGroups
.Where(s => s.CustomerId == c.Id)
.Select(x => x.Group)
.FirstOrDefault(),
})
.ToList();

Hard to suggest anything without seeing actual table definitions, espectially the indexes and foreign keys on Activities entity.
As far I understand Activity (CustomerId, ActivityTypeId, DueDateTime, DispositionId). If this is standard warehousing table (DateTime, ClientId, Activity), I'd suggest the following:
If number of Activities is reasonably small, then force the use of CONTAINS by
var activities = db.Activities.Where( x => x.IsReportable ).ToList();
...
.Where( b => activities.Contains(b.Activity) )
You can even help the optimiser by specifying that you want ActivityId.
Indexes on Activitiy entity should be up to date. For this particular query I suggest (CustomerId, ActivityId, DueDateTime DESC)
precache Disposition table, my crystal ball tells me that it's dictionary table.
For similar task to avoid constantly hitting Activity table I made another small table (CustomerId, LastActivity, LastVAlue) and updated it as the status changed.