We Keep Coding

LINQ Query Multiple Group and count of latest record - Oracle DB

I tried to divided Linq queries into 3 (total, success, fail) but so far "Total" Linq query is working fine. Please help me to get "Success", "Fail" columns (it has mulitple statuses and we have to check the last column of each transaction and destination)
Note: you need to group by ProcessTime, TransactionId, Destination and check last column whether it is success or Fail then apply count (we are using oracle as backend)
LINQ for Total count
var query = (from filetrans in context.FILE_TRANSACTION
join route in context.FILE_ROUTE on filetrans.FILE_TRANID equals route.FILE_TRANID
where
filetrans.PROCESS_STRT_TIME >= fromDateFilter && filetrans.PROCESS_STRT_TIME <= toDateFilter
select new { PROCESS_STRT_TIME = DbFunctions.TruncateTime((DateTime)filetrans.PROCESS_STRT_TIME), filetrans.FILE_TRANID, route.DESTINATION }).
GroupBy(p => new { p.PROCESS_STRT_TIME, p.FILE_TRANID, p.DESTINATION });
var result = query.GroupBy(x => x.Key.PROCESS_STRT_TIME).Select(x => new { x.Key, Count = x.Count() }).ToDictionary(a => a.Key, a => a.Count);

Check this solution. If it gives wrong result, then I need more details.
var fileTransQuery =
from filetrans in context.AFRS_FILE_TRANSACTION
where accountIds.Contains(filetrans.ACNT_ID) &&
filetrans.PROCESS_STRT_TIME >= fromDateFilter && filetrans.PROCESS_STRT_TIME <= toDateFilter
select filetrans;
var routesQuery =
from filetrans in fileTransQuery
join route in context.AFRS_FILE_ROUTE on filetrans.FILE_TRANID equals route.FILE_TRANID
select route;
var lastRouteQuery =
from d in routesQuery.GroupBy(route => new { route.FILE_TRANID, route.DESTINATION })
.Select(g => new
{
g.Key.FILE_TRANID,
g.Key.DESTINATION,
ROUTE_ID = g.Max(x => x.ROUTE_ID)
})
from route in routesQuery
.Where(route => d.FILE_TRANID == route.FILE_TRANID && d.DESTINATION == route.DESTINATION && d.ROUTE_ID == route.ROUTE_ID)
select route;
var recordsQuery =
from filetrans in fileTransQuery
join route in lastRouteQuery on filetrans.FILE_TRANID equals route.FILE_TRANID
select new { filetrans.PROCESS_STRT_TIME, route.CRNT_ROUTE_FILE_STATUS_ID };
var result = recordsQuery
.GroupBy(p => DbFunctions.TruncateTime((DateTime)p.PROCESS_STRT_TIME))
.Select(g => new TrendData
{
TotalCount = g.Sum(x => x.CRNT_ROUTE_FILE_STATUS_ID != 7 && x.CRNT_ROUTE_FILE_STATUS_ID != 8 ? 1 : 0)
SucccessCount = g.Sum(x => x.CRNT_ROUTE_FILE_STATUS_ID == 7 ? 1 : 0),
FailCount = g.Sum(x => failureStatus.Contains(x.CRNT_ROUTE_FILE_STATUS_ID) ? 1 : 0),
Date = g.Min(x => x.PROCESS_STRT_TIME)
})
.OrderBy(x => x.Date)
.ToList();

LINQ query to retrieve pivoted data taking too long

I am working on a LINQ query which includes some pivot data as below
var q = data.GroupBy(x => new
{
x.Med.Name,
x.Med.GenericName,
}).ToList().Select(g =>
new SummaryDto
{
Name= g.Key.Name,
GenericName = g.Key.GenericName,
Data2012 = g.Where(z => z.ProcessDate.Year == 2012).Count(),
Data2013 = g.Where(z => z.ProcessDate.Year == 2013).Count(),
Data2014 = g.Where(z => z.ProcessDate.Year == 2014).Count(),
Data2015 = g.Where(z => z.ProcessDate.Year == 2015).Count(),
Data2016 = g.Where(z => z.ProcessDate.Year == 2016).Count(),
Data2017 = g.Where(z => z.ProcessDate.Year == 2017).Count(),
TotalCount = g.Count(),
}).AsQueryable();
return q;
The above LINQ takes too long as it queries grp q.Count()*6 times. If there are 10000 records, then it queries 60000 times
Is there a better way to make this faster?

Add year to the group key, then group again, and harvest per-group counts:
return data.GroupBy(x => new {
x.Med.Name
, x.Med.GenericName
, x.ProcessDate.Year
}).Select(g => new {
g.Key.Name
, g.Key.GenericName
, g.Key.Year
, Count = g.Count()
}).GroupBy(g => new {
g.Name
, g.GenericName
}).Select(g => new SummaryDto {
Name = g.Key.Name
, GenericName = g.Key.GenericName
, Data2012 = g.SingleOrDefault(x => x.Year == 2012)?.Count ?? 0
, Data2013 = g.SingleOrDefault(x => x.Year == 2013)?.Count ?? 0
, Data2014 = g.SingleOrDefault(x => x.Year == 2014)?.Count ?? 0
, Data2015 = g.SingleOrDefault(x => x.Year == 2015)?.Count ?? 0
, Data2016 = g.SingleOrDefault(x => x.Year == 2016)?.Count ?? 0
, Data2017 = g.SingleOrDefault(x => x.Year == 2017)?.Count ?? 0
, TotalCount = g.Sum(x => x.Count)
}).AsQueryable();
Note: This approach is problematic, because year is hard-coded in the SummaryDto class. You would be better off passing your DTO constructor an IDictionary<int,int> with counts for each year. If you make this change, the final Select(...) would look like this:
.Select(g => new SummaryDto {
Name = g.Key.Name
, GenericName = g.Key.GenericName
, TotalCount = g.Sum(x => x.Count)
, DataByYear = g.ToDictionary(i => i.Year, i => i.Count)
}).AsQueryable();

I suggest grouping inside the group by year and then converting to a dictionary to access the counts. Whether it is faster to group with year first and then count in-memory depends on the distribution of the initial grouping, but with the database it may depend on how efficiently it can group by year, so I would test to determine which seems fastest.
In any case grouping by year after the initial grouping is about 33% faster than your query in-memory, but again it is vastly dependent on the distribution. As the number of initial groups increase, the grouping by Year queries slow down to match the original query. Note that the original query without any year counts is about 1/3 the time.
Here is grouping after the database grouping:
var q = data.GroupBy(x => new {
x.Med.Name,
x.Med.GenericName,
}).ToList().Select(g => {
var gg = g.GroupBy(d => d.ProcessDate.Year).ToDictionary(d => d.Key, d => d.Count());
return new SummaryDto {
Name = g.Key.Name,
GenericName = g.Key.GenericName,
Data2012 = gg.GetValueOrDefault(2012),
Data2013 = gg.GetValueOrDefault(2013),
Data2014 = gg.GetValueOrDefault(2014),
Data2015 = gg.GetValueOrDefault(2015),
Data2016 = gg.GetValueOrDefault(2016),
Data2017 = gg.GetValueOrDefault(2017),
TotalCount = g.Count(),
};
}).AsQueryable();

Add missing dates to list

I have written a solution which basically adds missing date and sets the sales property for that date in my collection to 0 where it's missing like this:
int range = Convert.ToInt32(drange);
var groupedByDate = tr.Union(Enumerable.Range(1, Convert.ToInt32(range))
.Select(offset => new MyClassObject
{
Date = DateTime.Now.AddDays(-(range)).AddDays(offset),
Sales = 0
})).GroupBy(x => x.Date)
.Select(item => new MyClassObject
{
Sales = item.Sum(x => x.Sales),
Date = item.Key
})
.OrderBy(x => x.Date)
.ToList();
The first solution where the dates from DB were grouped by and they were missing looked like this:
var groupedByDate = tr
.GroupBy(x => x.TransactionDate.Date)
.Select(item => new MyClassObject
{
Sales = item.Sum(x => x.QuantityPurchased),
Date = item.Key.ToString("yyyy-MM-dd")
})
.OrderBy(x => x.Date)
.ToList();
I don't really like the way I did it in first solution, the code looks very messy and I honestly believe it can be written in a better manner..
Can someone help me out with this?
P.S. The first solution above that I've shown works just fine, but I would like to write something better which is more prettier to the eyes, and it looks quite messy (the first solution I wrote)...

How about generate the date range and then left join that with the result from your original query. And than set Sales to 0 when there is no match.
int range = 2;
var startDate = DateTime.Now;
var dates = Enumerable.Range(1, range)
.Select(offset => startDate.AddDays(-offset).Date);
var groupedByDate = from date in dates
join tmp in groupedByDate on date equals tmp.Date into g
from gr in g.DefaultIfEmpty()
select new MyClassObject
{
Sales = gr == null ? 0 : gr.Sales,
Date = date
};

Here is the easy way to do this:
var lookup = tr.ToLookup(x => x.TransactionDate.Date, x => x.QuantityPurchased);
var quantity = lookup[new DateTime(2017, 6, 29)].Sum();
If you want a range of dates then it's just this:
var startDate = new DateTime(2017, 6, 1)
var query =
from n in Enumerable.Range(0, 30)
let TransactionDate = startDate.AddDays(n)
select new
{
TransactionDate,
QuantityPurchases = lookup[TransactionDate].Sum(),
};
Simple.

Linq Min in Select new & Multiple GroupBy columns

I want to write following query in Linq
INSERT INTO INOUTNEW (CODE,INDATE,TIME_DATE1,INOUTFLAG,TIME_FLD1,TIME_FLD2,TIME_FLD3)
SELECT CODE,MIN(INDATE),TIME_DATE1,'I',TIME_FLD1,TIME_FLD2,'31/05/2015' FROM INOUT
WHERE TIME_FLD1='T0003' AND INDATE >= '31/05/2015' AND INDATE <= '31/05/2015'
AND TIME_DATE1='31/05/2015'
GROUP BY CODE,TIME_DATE1,TIME_FLD1,TIME_FLD2
SO I am trying this :-
var data = ctx.tblInOut.Where(m => m.CompanyId == companyId && m.Time_Field1 == item.ShiftCode && m.InDate == StrInStart && m.InDate <= StrInEnd && m.Time_Date1 == InputDate).Select(m =>
new
{
EmployeeId = m.EmployeeId,
InDate = Min(m.InDate),
Time_Date1 = m.Time_Date1,
InOutFlag = m.InOutFlag
}).ToList();
I am stuck in Min Part. How to get Min in Select? And How to add multiple GroupBy in Linq?

Try something like this:
var data = ctx.tblInOut
.Where(m =>
m.CompanyId == companyId &&
m.Time_Field1 == item.ShiftCode &&
m.InDate == StrInStart &&
m.InDate <= StrInEnd &&
m.Time_Date1 == InputDate
)
.GroupBy(m =>
new {
m.Code,
m.Time_Date1,
m.Time_FLD1,
m.Time_FLD2
})
.Select(g =>
new
{
m.Key.Code,
InDate = m.Min(gg => gg.InDate),
m.Key.Time_Date1,
Something = "I",
m.Key.Time_FLD1,
m.Key.Time_FLD2,
SomeDate = "31/05/2015"
}).ToList();
To get Min, you must group first - otherwise it's trying to call Min on a single element.
.Key simply references the key of the group (in this case, a tuple of Code, Date1, Time_FLD1, Time_FLD2)

Linq C# Sum in Group By

I'm trying to transform the SQL that is here http://sqlfiddle.com/#!6/a1c8d/2 in linq below. The expected result is what is in sqlfiddle, but my LINQ returns more rows.
PS: In sqlfiddle the fields are reduced to not increase pollution and stay focused on my problem.
resultado.Dados =
(
from a in db.AgendaHorario
join b in db.Agenda on a.AgendaID equals b.AgendaID
select new
{
a.AgendaID,
Horario = a.Horario,
Controle = a.Controle,
Cor = b.Cor,
Agenda = b.Sigla
}).AsEnumerable()
.GroupBy(g => new
{
g.AgendaID,
Horario = g.Horario.ToString("dd/MM/yyyy"),
Data = g.Horario.ToString("yyyy-MM-dd"),
g.Controle,
g.Agenda,
g.Cor
})
.Select(s => new
{
id = s.Key.AgendaID,
title = s.Key.Agenda,
start = s.Key.Data,
color = String.IsNullOrEmpty(s.Key.Cor) ? "3a87ad" : s.Key.Cor,
className = "",
someKey = 1,
allDay = false,
Resultado0 = s.Sum(m => m.Controle == "L" ? 1 : 0).ToString(),
Resultado1 = s.Sum(m => m.Controle == "B" ? 1 : 0).ToString()
});

As per the comments, this addresses the question of how to repeat your SqlFiddle in Linq. Note that the projection to a String Date cannot be converted to Sql directly, so I've had to early materialize with AsEnumerable() (obviously, in your real query, apply any filters prior to materializing!). You could probably do the grouping on just the date part using SqlFunctions, e.g. 3 x applications of SqlFunctions.DatePart will allow you to group by dd, MM and YYYY
var dados = db.AgendaHorarios1
.AsEnumerable()
.GroupBy(ah => ah.Horario.ToString("dd/MM/yyyy"))
.Select(g => new {Horario = g.Key,
Livre = g.Count(x => x.Controle == "L"),
Bloq = g.Count(x => x.Controle == "B"),
Aged = g.Count(x => x.Controle == "A")});