We Keep Coding

LINQ: How to Write INNER JOIN and WHERE Condition

I have two database tables--
ReconFinalCost (Id, LeaseDocNumber, SnapshotId)
ReconSnapshot (Id, FiscalYear, FiscalQuarter)
ReconFinalCost.SnapshotId is a foreign-key to ReconSnapshot.Id.
In my C# code, I have a List<ReconFinalCost> leaselist. I want to write a LINQ command that inner joins the ReconFinalCost and ReconSnapshot tables on (ReconFinalCost.SnapshotId == ReconSnapshot.Id) and KEEPS all ReconFinalCost items in List<ReconFinalCost> leaselist for which there DOES NOT EXIST another ReconFinalCost item with an identical LeaseDocNumber but for which there is a SnapshotId associated with a ReconSnapshot with identical FiscalYear, but larger FiscalQuarter.
Here's that query, which I'm trying to write in LINQ, in SQL. I want to KEEP all items in List<ReconFinalCost> leaselist that are returned by the SQL query.
select * from "ReconFinalCost"
inner join "ReconSnapshot" on "ReconFinalCost"."SnapshotId" = "ReconSnapshot"."Id"
where NOT EXISTS (
SELECT 1 FROM "ReconFinalCost" B
inner join "ReconSnapshot" RS2 on RS2."Id" = B."SnapshotId"
where "ReconFinalCost"."LeaseDocNumber" = B."LeaseDocNumber" and
( RS2."FiscalYear" = "ReconSnapshot"."FiscalYear" and
RS2."FiscalQuarter" > "ReconSnapshot"."FiscalQuarter" )
)
So far I have:
List<ReconFinalCost> leaselist;
List<ReconFinalCost> originalLeaseList = leaselist;
leaselist = leaselist
.Where(x => !originalLeaseList.Any(
y => y.LeaseDocNumber == x.LeaseDocNumber &&
getIdsOfFutureQuarterSnapshots(x.SnapshotId).Any(
futureSnapshotId => futureSnapshotId == y.SnapshotId) ) )
.OrderBy(x => x.LeaseDocNumber).ToList();
...
private List<long> getIdsOfFutureQuarterSnapshots(long? snapshotId)
{
// this function returns a list of Ids of ReconSnapshots with the same (FiscalYear) as that of snapshotId, except in a FUTURE quarter
ReconSnapshot givenSnapshot = repository.GetReconSnapshotByID( (int)snapshotId );
return _context.ReconSnapshots.Where(r => r.FiscalYear == givenSnapshot.FiscalYear &&
r.FiscalQuarter > givenSnapshot.FiscalQuarter ).Select(r => r.Id).ToList();
}
What is the LINQ syntax for inner joining the ReconFinalCost and ReconSnapshot tables on (ReconFinalCost.SnapshotId == ReconSnapshot.Id), and then specifying the where clause for identical FiscalYear but larger FiscalQuarter, instead of having to call a separate function like I've done above?

Improving LINQ query for many-to-many relation

I have a database with the following schema:
Now, I'm trying to pull all landingpages for a domain and sort those by the first UrlFilter's FilterType that matches a certain group. This is the LINQ I've come up with so far:
var baseQuery = DbSet.AsNoTracking()
.Where(e => EF.Functions.Contains(EF.Property<string>(e, "Url"), $"\"{searchTerm}*\""))
.Where(e => e.DomainLandingPages.Select(lp => lp.DomainId).Contains(domainId));
var count = baseQuery.Count();
var page = baseQuery
.Select(e => new
{
LandingPage = e,
UrlFilter = e.LandingPageUrlFilters.FirstOrDefault(f => f.UrlFilter.GroupId == groupId)
})
.Select(e => new
{
e.LandingPage,
FilterType = e.UrlFilter == null ? UrlFilterType.NotCovered : e.UrlFilter.UrlFilter.UrlFilterType
})
.OrderBy(e => e.FilterType)
.Skip(10).Take(75).ToList();
Now, while this technically works, it's quite slow with execution times ranging from 10-30 seconds, which is not good enough for the use case. The LINQ is translated to the following SQL:
SELECT [l1].[Id], [l1].[LastUpdated], [l1].[Url], CASE
WHEN (
SELECT TOP(1) [l].[LandingPageId]
FROM [LandingPageUrlFilters] AS [l]
INNER JOIN [UrlFilters] AS [u] ON [l].[UrlFilterId] = [u].[Id]
WHERE ([l1].[Id] = [l].[LandingPageId]) AND ([u].[GroupId] = #__groupId_3)) IS NULL THEN 4
ELSE (
SELECT TOP(1) [u0].[UrlFilterType]
FROM [LandingPageUrlFilters] AS [l0]
INNER JOIN [UrlFilters] AS [u0] ON [l0].[UrlFilterId] = [u0].[Id]
WHERE ([l1].[Id] = [l0].[LandingPageId]) AND ([u0].[GroupId] = #__groupId_3))
END AS [FilterType]
FROM [LandingPages] AS [l1]
WHERE CONTAINS([l1].[Url], #__Format_1) AND #__domainId_2 IN (
SELECT [d].[DomainId]
FROM [DomainLandingPages] AS [d]
WHERE [l1].[Id] = [d].[LandingPageId]
)
ORDER BY CASE
WHEN (
SELECT TOP(1) [l2].[LandingPageId]
FROM [LandingPageUrlFilters] AS [l2]
INNER JOIN [UrlFilters] AS [u1] ON [l2].[UrlFilterId] = [u1].[Id]
WHERE ([l1].[Id] = [l2].[LandingPageId]) AND ([u1].[GroupId] = #__groupId_3)) IS NULL THEN 4
ELSE (
SELECT TOP(1) [u2].[UrlFilterType]
FROM [LandingPageUrlFilters] AS [l3]
INNER JOIN [UrlFilters] AS [u2] ON [l3].[UrlFilterId] = [u2].[Id]
WHERE ([l1].[Id] = [l3].[LandingPageId]) AND ([u2].[GroupId] = #__groupId_3))
END
OFFSET #__p_4 ROWS FETCH NEXT #__p_5 ROWS ONLY
Now my question is, how can I improve the execution time of this? Either by SQL or LINQ
EDIT: So I've been tinkering with some raw SQL and this is what I've come up with:
with matched_urls as (
select l.id, min(f.urlfiltertype) as Filter
from landingpages l
join landingpageurlfilters lpf on lpf.landingpageid = l.id
join urlfilters f on lpf.urlfilterid = f.id
where f.groupid = #groupId
and contains(Url, '"barz*"')
group by l.id
) select l.id, 5 as Filter
from landingpages l
where #domainId in (
select domainid
from domainlandingpages dlp
where l.id = dlp.landingpageid
) and l.id not in (select id from matched_urls ) and contains(Url, '"barz*"')
union select * from matched_urls
order by Filter
offset 10 rows fetch next 30 rows only
This performs somewhat okay, cutting the execution time down to ~5 seconds. As this is to be used for a table search I would however like to get it down even further. Is there any way to improve this SQL?

You're right to have a look at the generated SQL. In general, I would advise to learn SQL, write a performing SQL query and work your way back (either use a stored procedure or raw SQL, or design your LINQ query with that same philosophy.
I suspect this will be better (not tested):
var page = (
from e in baseQuery
let urlFilter = e.LandingPageUrlFilters.OrderBy(f => f.UrlFilterType).FirstOrDefault(f => f.UrlFilter.GroupId == groupId)
let filterType = urlFilter == null ? UrlFilterType.NotCovered : e.UrlFilter.UrlFilter.UrlFilterType
select new
{
LandingPage = e,
FilterType = filterType
}
).Skip(10).Take(75).ToList();

one of the way to improve the execution time is see execution plan in SSMS (SQL Server Management Studio).
After look on the execution plan you can design some indexes, or if you have no experiences with this, you can see if SSMS recommends some indexes.
Next try to create the indexes and execute the query again and see if execution time was improved.
Note: this is only one of many possible ways to improve execution time...

Convert SQL to EF Linq

I have the following query:
SELECT COUNT(1)
FROM Warehouse.WorkItems wi
WHERE wi.TaskId = (SELECT TaskId
FROM Warehouse.WorkItems
WHERE WorkItemId = #WorkItemId)
AND wi.IsComplete = 0;
And since we are using EF, I'd like to be able to use the Linq functionality to generate this query. (I know that I can give it a string query like this, but I would like to use EF+Linq to generate the query for me, for refactoring reasons.)
I really don't need to know the results of the query. I just need to know if there are any results. (The use of an Any() would be perfect, but I can't get the write code for it.)
So... Basically, how do I write that SQL query as a LINQ query?
Edit: Table Structure
WorkItemId - int - Primary Key
TaskId - int - Foreign Key on Warehouse.Tasks
IsComplete - bool
JobId - int
UserName - string
ReportName - string
ReportCriteria - string
ReportId - int - Foreign Key on Warehouse.Reports
CreatedTime - DateTime

The direct translation could be something like this
var result = db.WorkItems.Any(wi =>
!wi.IsComplete && wi.TaskId == db.WorkItems
.Where(x => x.WorkItemId == workItemId)
.Select(x => x.TaskId)
.FirstOrDefault()));
Taking into account the fact that SQL =(subquery), IN (subquery) and EXISTS(subquery) in nowadays modern databases are handled identically, you can try this instead
var result = db.WorkItems.Any(wi =>
!wi.IsComplete && db.WorkItems.Any(x => x.WorkItemId == workItemId
&& x.TaskId == wi.TaskId));

Turns out that I just needed to approach the problem from a different angle.
I came up with about three solutions with varying Linq syntaxes:
Full method chain:
var q1 = Warehouse.WorkItems
.Where(workItem => workItem.TaskId == (from wis in Warehouse.WorkItems
where wis.WorkItemId == workItemId
select wis.TaskId).First())
.Any(workItem => !workItem.IsComplete);
Mixed query + method chain:
var q2 = Warehouse.WorkItems
.Where(workItem => workItem.TaskId == Warehouse.WorkItems
.Where(wis => wis.WorkItemId == workItemId)
.Select(wis => wis.TaskId)
.First())
.Any(workItem => !workItem.IsComplete);
Full query:
var q3 = (from wi in Warehouse.WorkItems
where wi.TaskId == (from swi in Warehouse.WorkItems
where swi.WorkItemId == workItemId
select swi.TaskId).First()
where !wi.IsComplete
select 1).Any();
The only problems with this is that it comes up with some really jacked up SQL:
SELECT
(CASE
WHEN EXISTS(
SELECT NULL AS [EMPTY]
FROM [Warehouse].[WorkItems] AS [t0]
WHERE (NOT ([t0].[IsComplete] = 1)) AND ([t0].[TaskId] = ((
SELECT TOP (1) [t1].[TaskId]
FROM [Warehouse].[WorkItems] AS [t1]
WHERE [t1].[WorkItemId] = #p0
)))
) THEN 1
ELSE 0
END) AS [value]

You can use the Any() function like so:
var result = Warehouse.WorkItems.Any(x => x.WorkItemId != null);
In short, you pass in your condition, which in this case is checking whether or not any of the items in your collection have an ID
The variable result will tell you whether or not all items in your collection have ID's.
Here's a helpful webpage to help you get started with LINQ: http://www.dotnetperls.com/linq

Subquery in the original SQL was a useless one, thus not a good sample for Any() usage. It is simply:
SELECT COUNT(*)
FROM Warehouse.WorkItems wi
WHERE WorkItemId = #WorkItemId
AND wi.IsComplete = 0;
It looks like, since the result would be 0 or 1 only, guessing the purpose and based on seeking how to write Any(), it may be written as:
SELECT CASE WHEN EXISTS ( SELECT *
FROM Warehouse.WorkItems wi
WHERE WorkItemId = #WorkItemId AND
wi.IsComplete = 0 ) THEN 1
ELSE 0
END;
Then it makes sense to use Any():
bool exists = db.WorkItems.Any( wi => wi.WorkItemId == workItemId & !wi.IsComplete );
EDIT: I misread the original query in a hurry, sorry. Here is an update on the Linq usage:
bool exists = db.WorkItems.Any( wi =>
db.WorkItems
.SingleOrDefault(wi.WorkItemId == workItemId).TaskId == wi.TaskId
&& !wi.IsComplete );
If the count was needed as in the original SQL:
var count = db.WorkItems.Count( wi =>
db.WorkItems
.SingleOrDefault(wi.WorkItemId == workItemId).TaskId == wi.TaskId
&& !wi.IsComplete );
Sorry again for the confusion.

Linq query to SQL query

I have a Linq Query that works well but I need to write the SQL Query
Can Anybody help me write it?
this query will search the database foreach a.h and a.HV in the view with the filters of time and model and in the end it checks the option Filter.M that if it is selected it will search for all the data selected in this DropDownCheckBoxes`
How can i write the this where and select part in SQL command?
ret1 = (from a in View
where
a.LastRefreshTime>=Filter.From && a.LastRefreshTime<=Filter.To && a.ModelCode == mdlCode &&
Filter.PN.Select(epn => epn.Substring(0, 11)).Contains(a.H) &&
Filter.PN.Select(epn => epn.Substring(14, 2)).Contains(a.HV)
select new RData
{
v = a.v,
Date = a.LastRefreshTime,
UserId = a.UserId,
M = a.Name,
}).Distinct().AsQueryable();
ret = ret1.Where(nr =>
Filter.M == null || !Filter.M.Any() || Filter.M.Contains(nr.M)
).ToList();

Here's a start for you
select a.v v,
a.LastRefreshTime "Date",
a.UserId,
a.Name
from a
where a.LastRefreshTime>= arg_filter_from
and a.LastRefreshTime<= arg_filter_to
and a.ModelCode = arg_mdlCode
.
.
.
In this query you'll need to replace 'arg_...' with the appropriate values or arguments you want.
Contains is roughly equivalent to "IN" in SQL. For example:
where a.Name in ('jim', 'bob', 'joe')
In can also be used with a subselect which is roughly what I think Filter.PN.Select is doing though I'm not a linq expert. Example:
where a.H in (Select foo from PN_Table)
Or simpler example continuing on the my previous name example:
where a.Name in (select first_name from table)

If we supposed that the Filter.PN list represent a table FilterPN in your sql database, that will be your converted code for the first linq query
select distinct a.v, a.LastRefreshTime, a.UserId, a.Name
from [view] a
where a.LastRefreshTime>= 'Filter.From' and
a.LastRefreshTime<='Filter.To' and a.ModelCode = 'mdlCode' and
exists(select top 1 * from FilterPN where Substring(epn, 1, 11) = a.H) and
exists(select top 1 * from FilterPN where Substring(eenter code herepn, 15, 2) = a.HV)
think to replace the enquoted variables with ur real values 'Filter.To'...

How can I convert this SQL Query into LINQ (OVER (PARTITION BY Date))

Here's the query I'm trying to convert into Linq:
SELECT R.Code,
R.FlightNumber,
S.[Date],
S.Station,
R.Liters,
SUM(R.Liters) OVER (PARTITION BY Year([Date]), Month([Date]), Day([Date])) AS Total_Liters
FROM S INNER JOIN
R ON S.ID = R.SID
WHERE (R.Code = 'AC')
AND FlightNumber = '124'
GROUP BY Station, Code, FlightNumber, [Date], Liter
ORDER BY R.FlightNumber, [Date]
Thanks for any help.
UPDATE: Here is the Linq code I'm trying it on; I cannot make the OVER PARTITION by Date.
var test =
(from record in ent.Records join ship in ent.Ship on record.ShipID equals ship.ID
orderby ship.Station
where ship.Date > model.StartView && ship.Date < model.EndView && ship.Station == model.Station && record.FlightNumber == model.FlightNumber
group record by new {ship.Station, record.Code, record.FlightNumber, ship.Date, record.AmountType1} into g
select new { g.Key.Station, g.Key.Code, g.Key.FlightNumber, g.Key.Date, AmmountType1Sum = g.Sum(record => record.AmountType1) });

Execute query first without aggregation:
var test =
(from record in ent.Records join ship in ent.Ship on record.ShipID equals ship.ID
orderby ship.Station
where ship.Date > model.StartView && ship.Date < model.EndView && ship.Station == model.Station && record.FlightNumber == model.FlightNumber
select new {ship.Station, record.Code, record.FlightNumber, ship.Date, record.AmountType1};
Then calculate sum
var result =
from row in test
select new {row.Station, row.Code, row.FlightNumber, row.Date, row.AmountType1,
AmountType1Sum = test.Where(r => r.Date == row.Date).Sum(r => r.AmountType1) };
This should produce the same effect as database query. Code above may contain errors, because I wrote it only here.

I've answered a similar thread on: LINQ to SQL and a running total on ordered results
On that thread it was like this:
var withRuningTotals = from i in itemList
select i.Date, i.Amount,
Runningtotal = itemList.Where( x=> x.Date == i.Date).
GroupBy(x=> x.Date).
Select(DateGroup=> DateGroup.Sum(x=> x.Amount)).Single();
In you situation, you might have to join the two tables together first while grouping, then run the same concept above on the joined table result.