We Keep Coding

how to write group by clause with row_number-1 order by time desc in linq

i want to join 2 tables overview and transformerlocations table. i want to find the latest record of each deviceimei and then join with locations table.
[HttpGet]
[Route("overview_locations")]
public HttpResponseMessage GetoverviewLocations()
{
var overviewDetails = _dbContext.Overviews.ToList();
if (overviewDetails.Count != 0)
{
return Request.CreateResponse(HttpStatusCode.OK, overviewDetails, Configuration.Formatters.JsonFormatter);
}
else
{
}
}
sample data
2020-11-23 01:03:07.000 8673220311024 0 0 23.842163 91.280693
2020-11-23 01:01:06.000 8673220311024 0 0 23.842163 91.280693
2020-11-23 01:00:00.000 8645020301067 0 0 23.841940 91.280306
Exp op
2020-11-23 01:03:07.000 8673220311024 0 0 23.842163 91.280693
2020-11-23 01:00:00.000 8645020301067 0 0 23.841940 91.280306
sql
select *
from
(select
o.devicetimestamp,
o.deviceimei,
o.oti_a,
oti_t,
ts.latitude,
ts.longitude,
row_number() over(partition by o.deviceimei order by o.devicetimestamp desc) rn
from overview
inner join transformerlocations ts
on o.deviceimei = ts.imei
where ts.latitude is not null) t
where rn = 1

To translate filtering by the first ROW_NUMBER() you need to group by the PARTITION and then order each group as desired and select the first item in each group.
var ans = from o in overview
join ts in transformerlocations on o.deviceimei equals ts.imei
where ts.latitude != null
group new { o, ts } by o.deviceimei into otsg
select (
from ots in otsg
orderby ots.o.devicetimestamp descending
select new {
ots.o.devicetimestamp,
ots.o.deviceimei,
ots.o.oti_a,
ots.ts.latitude,
ots.ts.longitude
}
).FirstOrDefault();

Linq query in EF Core 2 when using joins and pagination order by column alias doesn't work

We are currently trying to write out sorting into our server-side pagination using Linq and EF Core 2. We are running into an issue where the column alias being produced by Linq does not work while using pagination. However if we do not paginate it works as intended.
All of the columns within the outputted queries are aliases as we have different property names in the model and database column names are different, but this shouldn't make a difference to our knowledge.
This is the Linq query without the pagination:
var source = from p in _ppmRepository.GetAll()
join jt in _jobTypeRepository.GetAll() on p.PpmFkeyInSeq equals jt.Id into jtdata
from jt in jtdata.DefaultIfEmpty()
join a in _assetRepository.GetAll() on p.PpmFkeyArSeq equals a.Id into aData
from a in aData.DefaultIfEmpty()
where p.PpmFkeyBgSeq == bldId
orderby p.PpmFreq
select new BuildingPpmListViewModel
{
PpmId = p.Id,
PpmFreq = p.PpmFreq,
PpmNextService = p.PpmNextService,
TotalCost = p.TotalCost,
PpmPeriodUnits = p.PpmPeriodUnits,
PpmFkeyPriDesc = p.PpmFkeyPriDesc,
JtTitle = jt.JtTitle,
AssetId = p.PpmFkeyArSeq,
AssetDescription = a.AssetDescription,
IsDeleted = p.IsDeleted
};
source = source.Where(i => i.JtTitle.Contains("audit") && i.AssetDescription.Contains("df"));
This is the outputted query produced by ef core which works:
SELECT [p].[PPM_SEQ] AS [PpmId], [p].[PPM_FREQ] AS [PpmFreq], [p].[PPM_NEXT_SERVICE] AS [PpmNextService],
CAST([p].[TotalCost] AS float) AS [TotalCost], [p].[PPM_PERIOD_UNITS] AS [PpmPeriodUnits], [p].[PPM_FKEY_PRI_DESC] AS [PpmFkeyPriDesc],
[t].[jt_title] AS [JtTitle], [p].[PPM_FKEY_AR_SEQ] AS [AssetId], [t0].[AR_DESCRIPTION] AS [AssetDescription], [p].[Deleted] AS [IsDeleted]
FROM [PPMs] AS [p]
LEFT JOIN (
SELECT [j].*
FROM [JobTypes] AS [j]
) AS [t] ON [p].[PPM_FKEY_IN_SEQ] = [t].[jt_seq]
LEFT JOIN (
SELECT [a].*
FROM [Assets] AS [a]
) AS [t0] ON [p].[PPM_FKEY_AR_SEQ] = [t0].[ar_seq]
WHERE ([p].[PPM_FKEY_BG_SEQ] = 172) AND ((CHARINDEX(N'audit', [t].[jt_title]) > 0) AND (CHARINDEX(N'df', [t0].[AR_DESCRIPTION]) > 0))
ORDER BY [PpmFreq]
This is the Linq query with the pagination:
var source = from p in _ppmRepository.GetAll()
join jt in _jobTypeRepository.GetAll() on p.PpmFkeyInSeq equals jt.Id into jtdata
from jt in jtdata.DefaultIfEmpty()
join a in _assetRepository.GetAll() on p.PpmFkeyArSeq equals a.Id into aData
from a in aData.DefaultIfEmpty()
where p.PpmFkeyBgSeq == bldId
orderby p.PpmFreq
select new BuildingPpmListViewModel
{
PpmId = p.Id,
PpmFreq = p.PpmFreq,
PpmNextService = p.PpmNextService,
TotalCost = p.TotalCost,
PpmPeriodUnits = p.PpmPeriodUnits,
PpmFkeyPriDesc = p.PpmFkeyPriDesc,
JtTitle = jt.JtTitle,
AssetId = p.PpmFkeyArSeq,
AssetDescription = a.AssetDescription,
IsDeleted = p.IsDeleted
};
source = source.Where(i => i.JtTitle.Contains("audit") && i.AssetDescription.Contains("df")).Skip(0).Take(50);
This is the output of the pagination where in the over function order by PpmFreq is the alias of [p].[PPM_FREQ] that SQL can not find:
SELECT [t1].[PpmId], [t1].[PpmFreq], [t1].[PpmNextService], [t1].[TotalCost], [t1].[PpmPeriodUnits],
[t1].[PpmFkeyPriDesc], [t1].[JtTitle], [t1].[AssetId], [t1].[AssetDescription], [t1].[IsDeleted]
FROM (
SELECT [p].[PPM_SEQ] AS [PpmId], [p].[PPM_FREQ] AS [PpmFreq], [p].[PPM_NEXT_SERVICE] AS [PpmNextService],
CAST([p].[TotalCost] AS float) AS [TotalCost], [p].[PPM_PERIOD_UNITS] AS [PpmPeriodUnits], [p].[PPM_FKEY_PRI_DESC] AS
[PpmFkeyPriDesc], [t].[jt_title] AS [JtTitle], [p].[PPM_FKEY_AR_SEQ] AS [AssetId], [t0].[AR_DESCRIPTION] AS [AssetDescription],
[p].[Deleted] AS [IsDeleted], ROW_NUMBER() OVER(ORDER BY [PpmFreq]) AS [__RowNumber__]
FROM [PPMs] AS [p]
LEFT JOIN (
SELECT [j].*
FROM [JobTypes] AS [j]
) AS [t] ON [p].[PPM_FKEY_IN_SEQ] = [t].[jt_seq]
LEFT JOIN (
SELECT [a].*
FROM [Assets] AS [a]
) AS [t0] ON [p].[PPM_FKEY_AR_SEQ] = [t0].[ar_seq]
WHERE (([p].[PPM_FKEY_BG_SEQ] = 172)) AND ((CHARINDEX(N'audit', [t].[jt_title]) > 0)
AND (CHARINDEX(N'df', [t0].[AR_DESCRIPTION]) > 0))
) AS [t1]
WHERE ([t1].[__RowNumber__] > 0) AND ([t1].[__RowNumber__] <= (50))
This looks to be where our issues are coming from as we can slightly modify it to get a correct result from the database:
ROW_NUMBER() OVER(ORDER BY [PpmFreq]) AS [__RowNumber__]
If we were to modify the above statement to also include the table alias as [p].[PPM_FREQ], like so: ROW_NUMBER() OVER(ORDER BY [p].[PPM_FREQ]) AS [__RowNumber__] then our issues are resolved, but that doesnt seem possible with our current linq query.

See if following works better :
var source = (from p in _ppmRepository.GetAll()
join jt in _jobTypeRepository.GetAll() on p.PpmFkeyInSeq equals jt.Id into jtdata
from jt in jtdata.DefaultIfEmpty()
join a in _assetRepository.GetAll() on p.PpmFkeyArSeq equals a.Id into aData
from a in aData.DefaultIfEmpty()
select new BuildingPpmListViewModel
{
PpmId = p.Id,
PpBgSeq = p.PpmFkeyBgSeq,
PpmFreq = p.PpmFreq,
PpmNextService = p.PpmNextService,
TotalCost = p.TotalCost,
PpmPeriodUnits = p.PpmPeriodUnits,
PpmFkeyPriDesc = p.PpmFkeyPriDesc,
JtTitle = jt.JtTitle,
AssetId = p.PpmFkeyArSeq,
AssetDescription = a.AssetDescription,
IsDeleted = p.IsDeleted
})
.Where(x => x.PpBgSeq == bldId)
.OrderBy(x => x.PpmFreq)
.ToList();

This is a known issue of that we have later filed with the ef core team directly.
This is a known issue which has been fixed for upcoming release of 2.1
You can see more details and possible work-around here
github.com/aspnet/EntityFrameworkCore/issues/9535`
Smit Patel
If you run a nightly build you can fix the above issue.

time out expired issue in lambda method for contains

I have this code:
string[] splits = keyword.Split(' ');
var fproducts = (from products in db.tbl_Product
where splits.Any(item => products.Prod_Name_Fa.Contains(item) ||
products.shortDesc.Contains(item) || products.Prod_Code.Contains(item))
select products).Distinct();
if splits have 2 or 3 words it works but when I put 5 words in that I get time out expired error,
Is there any other solution to search an array in a field ?
Update:
this is the sql output :
{SELECT
[Extent1].[ID] AS [ID],
[Extent1].[Register_Date] AS [Register_Date],
[Extent1].[tbl_VATID] AS [tbl_VATID],
[Extent1].[Prod_Name_Fa] AS [Prod_Name_Fa],
[Extent1].[Prod_Code] AS [Prod_Code],
[Extent1].[Prod_Description_Fa] AS [Prod_Description_Fa],
[Extent1].[Prod_Description_En] AS [Prod_Description_En],
[Extent1].[Prod_How_To_Use] AS [Prod_How_To_Use],
[Extent1].[Prod_Keywords] AS [Prod_Keywords],
[Extent1].[Prod_Model] AS [Prod_Model],
[Extent1].[User_Rating] AS [User_Rating],
[Extent1].[Admin_Rating] AS [Admin_Rating],
[Extent1].[Seen_Counter] AS [Seen_Counter],
[Extent1].[Is_Ashantyvn_Salable] AS [Is_Ashantyvn_Salable],
[Extent1].[shortDesc] AS [shortDesc],
[Extent1].[basePrice] AS [basePrice],
[Extent1].[Discount] AS [Discount],
[Extent1].[discountDateFrom] AS [discountDateFrom],
[Extent1].[discountDateTo] AS [discountDateTo],
[Extent1].[Sex] AS [Sex],
[Extent1].[Publish] AS [Publish],
[Extent1].[basePic] AS [basePic],
[Extent1].[suggestDate] AS [suggestDate],
[Extent1].[isSuggest] AS [isSuggest],
[Extent1].[homesuggestDate] AS [homesuggestDate],
[Extent1].[homeisSuggest] AS [homeisSuggest],
[Extent1].[suggestNum] AS [suggestNum],
[Extent1].[tagOrder] AS [tagOrder]
FROM [dbo].[tbl_Product] AS [Extent1]
WHERE EXISTS (SELECT
1 AS [C1]
FROM (SELECT
N'رژ' AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
UNION ALL
SELECT
N'لب' AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable2]
UNION ALL
SELECT
N'دو' AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable3]
UNION ALL
SELECT
N'رنگ' AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable4]
UNION ALL
SELECT
N'لوویو' AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable5]) AS [UnionAll4]
WHERE (( CAST(CHARINDEX([UnionAll4].[C1], [Extent1].[Prod_Name_Fa]) AS int)) > 0) OR (( CAST(CHARINDEX([UnionAll4].[C1], [Extent1].[shortDesc]) AS int)) > 0) OR (( CAST(CHARINDEX([UnionAll4].[C1], [Extent1].[Prod_Code]) AS int)) > 0)
)}

I'm not sure what mechanism Entity Framework is using to translate your query, but it's likely that you're getting something unnecessarily complicated. Try composing your own intersection of queries returning results for each keyword, like so:
string[] splits = keyword.Split(' ');
if(!splits.Any())
return db.tbl_Product; // or something
var queries = splits
.Select(item => db.tbl_Product
.Where(product => product.Prod_Name_Fa.Contains(item) ||
product.shortDesc.Contains(item) ||
product.Prod_Code.Contains(item))));
return items.Aggregate((q1, q2) => q1.Union(q2));

For best performance, I have to select just ID of the model not all,
it should be like this :
string[] splits = keyword.Split(' ');
var fproducts = (from products in db.tbl_Product
where splits.Any(item => products.Prod_Name_Fa.Contains(item) ||
products.shortDesc.Contains(item) || products.Prod_Code.Contains(item))
select products.ID).Distinct();
products.ID
and then select the records when equals these IDs.

LINQ Count multiple values from a collection

In SQL what I'm trying to accomplish is
SELECT
SUM(CASE WHEN Kendo=1 THEN 1 ELSE 0 END) as KendoCount,
SUM(CASE WHEN Icenium=1 THEN 1 ELSE 0 END) as IceniumCount
FROM
Contacts
I'd like to do this in a C# program using LINQ.
Contacts is a List where Contact has many Booleans such as Kendo and Icenium and I need to know how many are true for each of the Booleans.

At least with LINQ to SQL, the downside of the count functions is that it requires separate SQL requests for each .count method. I suspect Jessie is trying to run a single scan over the table rather than multiple scans for each predicate. Depending on the logic and number of columns you are creating, this may not perform as well. Closer to the original request, try using sum with a ternary if clause as such (from Northwind):
from e in Employees
group e by "" into g
select new {
isUs = g.Sum (x => x.Country == "USA" ? 1 : 0),
NotUs = g.Sum (x => x.Country != "USA" ? 0 : 1)
}
LINQ to SQL generates the following (YMMV with other ORM's):
SELECT SUM(
(CASE
WHEN [t1].[Country] = #p1 THEN #p2
ELSE #p3
END)) AS [isUs], SUM(
(CASE
WHEN [t1].[Country] <> #p4 THEN #p5
ELSE #p6
END)) AS [NotUs]
FROM (
SELECT #p0 AS [value], [t0].[Country]
FROM [Employees] AS [t0]
) AS [t1]
GROUP BY [t1].[value]

var KendoCount = db.Contacts.Where(x => x.Kendo).Count();
var IceniumCount = db.Contacts.Where(x => x.Icenium).Count();

I would do this as two separate queries:
int kendoCount = db.Contacts.Count(c => c.Kendo);
int iceniumCount = db.Contacts.Count(c => c.Icenium);
Given that these queries will automatically translate into optimized SQL, this will likely be similar in speed or even potentially faster than any query option, and is far simpler to understand.
Note that, if this is for Entity Framework, you'll need to write this as:
int kendoCount = db.Contacts.Where(c => c.Kendo).Count();
int iceniumCount = db.Contacts.Where(c => c.Icenium).Count();

var result = Contacts
.GroupBy(c => new
{
ID = "",
})
.Select(c => new
{
KendoCount = c.Sum(k => k.Kendo ? 1 : 0),
IceniumCount = c.Sum(k => k.Icenium ? 1: 0),
})
.ToArray()

LINQ to SQL using GROUP BY and COUNT(DISTINCT)

I have to perform the following SQL query:
select answer_nbr, count(distinct user_nbr)
from tpoll_answer
where poll_nbr = 16
group by answer_nbr
The LINQ to SQL query
from a in tpoll_answer
where a.poll_nbr = 16 select a.answer_nbr, a.user_nbr distinct
maps to the following SQL query:
select distinct answer_nbr, distinct user_nbr
from tpoll_answer
where poll_nbr = 16
So far, so good. However the problem raises when trying to GROUP the results, as I'm not being able to find a LINQ to SQL query that maps to the first query I wrote here (thank you LINQPad for making this process a lot easier). The following is the only one that I've found that gives me the desired result:
from answer in tpoll_answer where answer.poll_nbr = 16 _
group by a_id = answer.answer_nbr into votes = count(answer.user_nbr)
Which in turns produces the follwing ugly and non-optimized at all SQL query:
SELECT [t1].[answer_nbr] AS [a_id], (
SELECT COUNT(*)
FROM (
SELECT CONVERT(Bit,[t2].[user_nbr]) AS [value], [t2].[answer_nbr], [t2].[poll_nbr]
FROM [TPOLL_ANSWER] AS [t2]
) AS [t3]
WHERE ([t3].[value] = 1) AND ([t1].[answer_nbr] = [t3].[answer_nbr]) AND ([t3].[poll_nbr] = #p0)
) AS [votes]
FROM (
SELECT [t0].[answer_nbr]
FROM [TPOLL_ANSWER] AS [t0]
WHERE [t0].[poll_nbr] = #p0
GROUP BY [t0].[answer_nbr]
) AS [t1]
-- #p0: Input Int (Size = 0; Prec = 0; Scale = 0) [16]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 3.5.30729.1
Any help will be more than appreciated.

There isn't direct support for COUNT(DISTINCT {x})), but you can simulate it from an IGrouping<,> (i.e. what group by returns); I'm afraid I only "do" C#, so you'll have to translate to VB...
select new
{
Foo= grp.Key,
Bar= grp.Select(x => x.SomeField).Distinct().Count()
};
Here's a Northwind example:
using(var ctx = new DataClasses1DataContext())
{
ctx.Log = Console.Out; // log TSQL to console
var qry = from cust in ctx.Customers
where cust.CustomerID != ""
group cust by cust.Country
into grp
select new
{
Country = grp.Key,
Count = grp.Select(x => x.City).Distinct().Count()
};
foreach(var row in qry.OrderBy(x=>x.Country))
{
Console.WriteLine("{0}: {1}", row.Country, row.Count);
}
}
The TSQL isn't quite what we'd like, but it does the job:
SELECT [t1].[Country], (
SELECT COUNT(*)
FROM (
SELECT DISTINCT [t2].[City]
FROM [dbo].[Customers] AS [t2]
WHERE ((([t1].[Country] IS NULL) AND ([t2].[Country] IS NULL)) OR (([t1]
.[Country] IS NOT NULL) AND ([t2].[Country] IS NOT NULL) AND ([t1].[Country] = [
t2].[Country]))) AND ([t2].[CustomerID] <> #p0)
) AS [t3]
) AS [Count]
FROM (
SELECT [t0].[Country]
FROM [dbo].[Customers] AS [t0]
WHERE [t0].[CustomerID] <> #p0
GROUP BY [t0].[Country]
) AS [t1]
-- #p0: Input NVarChar (Size = 0; Prec = 0; Scale = 0) []
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 3.5.30729.1
The results, however, are correct- verifyable by running it manually:
const string sql = #"
SELECT c.Country, COUNT(DISTINCT c.City) AS [Count]
FROM Customers c
WHERE c.CustomerID != ''
GROUP BY c.Country
ORDER BY c.Country";
var qry2 = ctx.ExecuteQuery<QueryResult>(sql);
foreach(var row in qry2)
{
Console.WriteLine("{0}: {1}", row.Country, row.Count);
}
With definition:
class QueryResult
{
public string Country { get; set; }
public int Count { get; set; }
}

The Northwind example cited by Marc Gravell can be rewritten with the City column selected directly by the group statement:
from cust in ctx.Customers
where cust.CustomerID != ""
group cust.City /*here*/ by cust.Country
into grp
select new
{
Country = grp.Key,
Count = grp.Distinct().Count()
};

Linq to sql has no support for Count(Distinct ...). You therefore have to map a .NET method in code onto a Sql server function (thus Count(distinct.. )) and use that.
btw, it doesn't help if you post pseudo code copied from a toolkit in a format that's neither VB.NET nor C#.

This is how you do a distinct count query. Note that you have to filter out the nulls.
var useranswercount = (from a in tpoll_answer
where user_nbr != null && answer_nbr != null
select user_nbr).Distinct().Count();
If you combine this with into your current grouping code, I think you'll have your solution.

simple and clean example of how group by works in LINQ
http://www.a2zmenu.com/LINQ/LINQ-to-SQL-Group-By-Operator.aspx

I wouldn't bother doing it in Linq2SQL. Create a stored Procedure for the query you want and understand and then create the object to the stored procedure in the framework or just connect direct to it.