We Keep Coding

Slow LINQ query because of Where+ToDictionary

I have this query that I need to run:
IEnumerable<MerchantWithParameters> merchants =
from i in (
from m in d.GetTable<Merchant>()
join mtp in d.GetTable<MerchantToParameters>() on m.Id equals mtp.MerchantId into mtps
from mtp in mtps.DefaultIfEmpty()
join cp in d.GetTable<ContextParameters>() on mtp.ContextParametersId equals cp.Id into cps
from cp in cps.DefaultIfEmpty()
select new {Merchant = m, ContextParameter = cp}
)
group i by new { i.Merchant.Id } into ig
select new MerchantWithParameters()
{
Id = ig.Key.Id,
Parameters = ig.Where(g => g.ContextParameter != null).ToDictionary(g => g.ContextParameter.Key, g => g.ContextParameter.Text)
};
For some reason it takes really long time for this query to be completed.
I believe that it has something to do with
Parameters = ig.Where(g => g.ContextParameter != null).ToDictionary(g => g.ContextParameter.Key, g => g.ContextParameter.Text)
Because when I remove this line, query starts to execute really fast.
Could you please show me what am I doing wrong?
UPDATE:
I am using ToList() to extract data from the database.

It is known SQL limitation. You cannot get grouped items, only grouping key or aggregation result. Since you need all records, we can do grouping on the client side, but previously maximally limit retrieved data.
var query =
from m in d.GetTable<Merchant>()
from mtp in d.GetTable<MerchantToParameters>().LeftJoin(mtp => m.Id == mtp.MerchantId)
from cp in d.GetTable<ContextParameters>().LeftJoin(cp => mtp.ContextParametersId == cp.Id)
select new
{
MerchantId = m.Id,
ContextParameterKey = (int?)cp.Key,
ContextParameterText = cp.Text
};
var result =
from q in query.AsEnumerable()
group q by q.MerchantId into g
select new MerchantWithParameters
{
Id = g.Key,
Parameters = g.Where(g => g.ContextParameterKey != null)
.ToDictionary(g => g.ContextParameterKey.Value, g => g.ContextParameterText)
};
var merchants = result.ToList();

Simplify many LINQ Join

I have a rather complex LINQ query but that is the point of the question
var result = await _context.TblBreakpoints
.GroupBy(b => new { b.BpgroupId, b.ResultType, b.DrugId, b.Susceptible, b.LowIntermediate, b.Intermediate, b.Resistant})
.Join(_context.TblBreakpointgroups,
bg => bg.Key.BpgroupId,
g => g.BpgroupId,
(bg, g) => new
{
GroupId = bg.Key.BpgroupId,
DrugId = bg.Key.DrugId,
Susceptible = bg.Key.Susceptible,
LowIntermediate = bg.Key.LowIntermediate,
Intermediate = bg.Key.Intermediate,
Method = bg.Key.ResultType,
Resistant = bg.Key.Resistant,
StandardId = g.BpstandardId,
GroupName = g.BpgroupName,
Count = bg.Count(),
})
.Join(_context.TblBreakpointStandards,
i => i.StandardId,
j => j.BpstandardId,
(i, j) => new
{
Standard = j.Bpstandard,
GroupId = i.GroupId,
GroupName = i.GroupName,
Count = i.Count,
Method = i.Method,
DrugId = i.DrugId,
Susceptible = i.Susceptible,
LowIntermediate = i.LowIntermediate,
Intermediate = i.Intermediate,
Resistant = i.Resistant
})
.Join(_context.TblDrugs,
i => i.DrugId,
j => j.DrugId,
(i, j) => new
{
DrugName = j.DrugName,
Standard = i.Standard,
GroupId = i.GroupId,
GroupName = i.GroupName,
Count = i.Count,
Method = i.Method,
Susceptible = i.Susceptible,
LowIntermediate = i.LowIntermediate,
Intermediate = i.Intermediate,
Resistant = i.Resistant
})
.Join(_context.TblBreakpointgroupmembers,
i => i.GroupId,
j => j.BpgroupId,
(i, j) => new
{
OrganismId = j.OrganismId,
Standard = i.Standard,
GroupId = i.GroupId,
GroupName = i.GroupName,
Count = i.Count,
Method = i.Method,
DrugName = i.DrugName,
Susceptible = i.Susceptible,
LowIntermediate = i.LowIntermediate,
Intermediate = i.Intermediate,
Resistant = i.Resistant
})
.Join(_context.TblOrganismNames,
i => i.OrganismId,
j => j.OrganismId,
(i, j) => new BreakpointSummary
{
OrganismName = j.OrganismName,
Standard = i.Standard,
GroupName = i.GroupName,
Count = i.Count,
Method = i.Method,
DrugName = i.DrugName,
Susceptible = i.Susceptible,
LowIntermediate = i.LowIntermediate,
Intermediate = i.Intermediate,
Resistant = i.Resistant
})
.ToListAsync().ConfigureAwait(false);
From the query with each Join I keep passing the previous values and add the value(s) that come from the join. It is already tedious with just 5 joins, it would get even more so with more joins. Is there a better way that I am missing?
I think the equivalent SQL is
WITH bpg (BPGroupId, ResultType, DrugId, Susceptible, LowIntermediate, Intermediate, Resistant, Count)
AS (
SELECT BPGroupId, ResultType, DrugId, Susceptible, LowIntermediate, Intermediate, Resistant, COUNT(BPGroupId)
FROM dbo.tbl_Breakpoint a
GROUP BY BPGroupId,
ResultType,
DrugId,
Susceptible,
LowIntermediate,
Intermediate,
Resistant
)
SELECT a.BpgroupName, b.BPStandard, c.DrugName, e.OrganismName, CTE.ResultType, CTE.Susceptible, CTE.LowIntermediate, CTE.Intermediate, CTE.Resistant, CTE.Count
FROM dbo.tbl_breakpointgroup a
INNER JOIN bpg CTE ON a.BPGroupId = CTE.BPGroupId
INNER JOIN tbl_BreakpointStandard b ON b.BPStandardId = a.BPStandardId
INNER JOIN tbl_Drug c ON c.DrugID = CTE.DrugId
INNER JOIN tbl_breakpointgroupmember d ON d.BPGroupId = CTE.BPGroupId
INNER JOIN tbl_OrganismName e ON e.OrganismId = d.OrganismId
WHERE a.BPGroupId = CTE.BPGroupId

In general when using manual joins in LINQ, it's better to use the query syntax since it provides range variables (which correspond to table/query aliases in SQL) transparency. e.g. (in pseudo code)
from a in queryA
join b in queryB on ... // can use any field from a
join c in queryC on ... // can use any field from a and b
join d in queryD on ... // can use any field from a, b and c
...
select new
{
// can use any field for a, b, c, d etc.
}
The same with method syntax is a bit more complicated, but the principle is to wrap the previous "variables" in simple tuple like anonymous types until you get to the final projection, e.g. (in pseudo code)
queryA
.Join(queryB, a => {a fields}, b => {b fields), (a, b) => new { a, b }) // first result
.Join(queryC, r => {r.a, r.b fields), c => { c fields }, (r, c) => new { r.a, r.b, c } // second result
.Join(queryD, r => {r.a, r.b, r.c fields), d => { d fields }, (r, d) => new { r.a, r.b, r.c, d } // third result
...
.Select(r => new { r.a, r.b, r.c, r.d... fields });
Applying it to your example, the corresponding query syntax could be like (note that sub queries inside can use whatever syntax is appropriate):
var query =
from cte in _context.TblBreakpoints
.GroupBy(b => new { b.BpgroupId, b.ResultType, b.DrugId, b.Susceptible, b.LowIntermediate, b.Intermediate, b.Resistant})
.Select(g => new
{
g.Key.BpgroupId, g.Key.ResultType, g.Key.DrugId, g.Key.Susceptible, h.Key.LowIntermediate, g.Key.Intermediate, g.Key.Resistant,
Count = g.Count(),
})
join a in _context.TblBreakpointgroups on cte.BpgroupId equals a.BpgroupId
join b in _context.TblBreakpointStandards on a.BpstandardId equals b.BpstandardId
join c in _context.TblDrugs on cte.DrugId equals c.DrugId
join d in _context.TblBreakpointgroupmembers on cte.BpgroupId equals d.BpgroupId
join e in _context.TblOrganismNames on d.OrganismId equals e.OrganismId
select new BreakpointSummary
{
OrganismName = e.OrganismName,
Standard = b.Bpstandard,
GroupName = a.BpgroupName,
Count = cte.Count,
Method = cte.ResultType,
DrugName = d.DrugName,
Susceptible = cte.Susceptible,
LowIntermediate = cte.LowIntermediate,
Intermediate = cte.Intermediate,
Resistant = cte.Resistant,
};
You can convert it to method syntax using the aforementioned rules, but for me it doesn't worth the effort.

Linq Dynamic Query With Group By

I need extra where clause for my Linq query. For example if customer choose a date filter so i need to date filter to my query etc... When i try to myQuery.Where predicate there is visible just group by's field.
How can i append new where condition to my query.
//for example i need dynamically append o.OrderDate==Datetime.Now or another where clause
var myQuery =(from o in _db.Orders
join l in _db.OrderLines.Where(x => x.ParaBirimi == model.ParaBirimi) on o.orderId equals
l.OrderId
where o.OrderDate.Value.Year == year1
group o by new {o.OrderDate.Value.Month}
into g
select
new
{
Month = g.Key.Month,
Total = g.Select(t => t.OrderLines.Sum(s => s.OrderTotal)).FirstOrDefault()
});

You are too late at the end of the query to add new Where. You have already grouped the data, and projected it, removing nearly all the fields.
Try:
var baseQuery = from o in _db.Orders
join l in _db.OrderLines.Where(x => x.ParaBirimi == model.ParaBirimi) on o.orderId equals l.OrderId
where o.OrderDate.Value.Year == year1
select new { Order = o, OrderLine = l };
if (something)
{
baseQuery = baseQuery.Where(x => x.Order.Foo == "Bar");
}
var myQuery = (from o in baseQuery
group o by new { o.Order.OrderDate.Value.Month }
into g
select
new
{
Month = g.Key.Month,
Total = g.Sum(t => t.OrderLine.OrderTotal)
});
Clearly you can have multiple if. Each .Where() is in && (AND) with the other conditions.
Note how the result of the join is projected in an anonymous class that has two properties: Order and OrderLine

Linq to select data from one table not in other table

Hi i have the following code to select data from one table not in other table
var result1 = (from e in db.Users
select e).ToList();
var result2 = (from e in db.Fi
select e).ToList();
List<string> listString = (from e in result1
where !(from m in result2
select m.UserID).Contains(e.UserID)
select e.UserName).ToList();
ViewBag.ddlUserId = listString;
Am getting value inside listString .But got error while adding listString to viewbag.
Unable to cast object of type 'System.Collections.Generic.List`1[System.String]' to type 'System.Collections.Generic.IEnumerable`1[Main.Models.Admin.User]'.

First, could you update your question with the entire method so that we can see what might be going on with the ViewBag? Because your code should work just fine, assigning whatever value to the ViewBag is no problem normally:
ViewBag.property1 = 0;
ViewBag.property1 = "zero";
works just fine. ViewBag is dynamic. Now, you could get that error if you would later try to assing ViewBag.ddlUserId to something that actually is the wrong type.
I would like you to rewrite your statement as well, let me explain why. Assume for a moment that you have a lot ( > 100.000) of User records in your db.Users and we assume the same for Fi as well. In your code, result1 and result2 are now two lists, one containing >100.000 User objects and the other >100.000 Fi objects. Then these two lists are compared to each other to produce a list of strings. Now imagine the resource required for your web server to process this. Under the assumption that your actually using/accessing a separate SQL server to retrieve your data from, it would be a lot better and faster to let that server do the work, i.e. producing the list of UserID's.
For that you'd either use Kirill Bestemyanov's answer or the following:
var list = (from user in db.Users
where !db.Fi.Any(f => f.UserID == user.UserID)
select user.UserName).ToList()
This will produce just one query for the SQL server to execute:
SELECT
[Extent1].[UserName] AS [UserName]
FROM [dbo].[Users] AS [Extent1]
WHERE NOT EXISTS (SELECT
1 AS [C1]
FROM [dbo].[Fi] AS [Extent2]
WHERE [Extent2].[UserID] = [Extent1].[UserID]
)}
which in the end is what you want...
Just to clarify more:
var list = (from user in db.Users
where !db.Fi.Any(f => f.UserID == user.UserID)
select user.UserName).ToList()
can be written as the following lambda expression as well:
var list = db.Users.Where(user => !db.Fi.Any(f => f.UserID == user.UserID))
.Select(user => user.UserName).ToList()
which from the looks of it is slightly different from Kirill Bestemyanov's answer (which I slightly modified, just to make it look more similar):
var list = db.Users.Where(user => !db.Fi.Select(f => f.UserID)
.Contains(user.UserID))
.Select(user => user.UserName).ToList();
But, they will in fact produce the same SQL Statement, thus the same list.

I will rewrite it to linq extension methods:
List<string> listString = db.Users.Where(e=>!db.Fi.Select(m=>m.UserID)
.Contains(e.UserID))
.Select(e=>e.UserName).ToList();
try it, it should work.

Try this it is very simple.
var result=(from e in db.Users
select e.UserID).Except(from m in db.Fi
select m.UserID).ToList();

var res = db.tbl_Ware.where(a => a.tbl_Buy.Where(c => c.tbl_Ware.Title.Contains(mtrTxtWareTitle.Text)).Select(b => b.Ware_ID).Contains(a.ID));
This mean in T-SQL is:
SELECT * FROM tbl_Ware WHERE id IN (SELECT ware_ID, tbl_Buy WHErE tbl_Ware.title LIKE '% mtrTxtwareTitle.Text %')

getdata = (from obj in db.TblManageBranches
join objcountr in db.TblManageCountries on obj.Country equals objcountr.iCountryId.ToString() into objcount
from objcountry in objcount.DefaultIfEmpty()
where obj.IsActive == true
select new BranchDetails
{
iBranchId = obj.iBranchId,
vBranchName = obj.vBranchName,
Addressline1 = obj.Addressline1,
Adminemailid = obj.Adminemailid,
BranchType = obj.BranchType,
Country = objcountry.vCountryName,
CreatedBy = obj.CreatedBy,
CreatedDate = obj.CreatedDate,
iArea = obj.iArea,
iCity = obj.iCity,
Infoemailid = obj.Infoemailid,
Landlineno = obj.Landlineno,
Mobileno = obj.Mobileno,
iState = obj.iState,
Pincode = obj.Pincode,
Processemailid = obj.Processemailid,
objbranchbankdetails = (from objb in db.TblBranchesBankDetails.Where(x => x.IsActive == true && x.iBranchId == obj.iBranchId)
select new ManageBranchBankDetails
{
iBranchId = objb.iBranchId,
iAccountName = objb.iAccountName,
iAccountNo = objb.iAccountNo,
iBankName = objb.iBankName,
iAccountType = objb.iAccountType,
IFSCCode = objb.IFSCCode,
SWIFTCode = objb.SWIFTCode,
CreatedDate = objb.CreatedDate,
Id = objb.Id
}).FirstOrDefault(),
objbranchcontactperson = (from objc in db.tblbranchcontactpersons.Where(x => x.Isactive == true && x.branchid == obj.iBranchId)
select new ManageBranchContactPerson
{
branchid = objc.branchid,
createdate = objc.createdate,
Id = objc.Id,
iemailid = objc.iemailid,
ifirstname = objc.ifirstname,
ilandlineno = objc.ilandlineno,
ilastname = objc.ilastname,
imobileno = objc.imobileno,
title = objc.title,
updateddate=objc.updateddate,
}).ToList(),
}).OrderByDescending(x => x.iBranchId).ToList();

getdata = (from obj in db.TblManageBranches join objcountr in db.TblManageCountries on obj.Country equals objcountr.iCountryId.ToString() into objcount from objcountry in objcount.DefaultIfEmpty() where obj.IsActive == true
select new BranchDetails
{
iBranchId = obj.iBranchId,
vBranchName = obj.vBranchName,
objbranchbankdetails = (from objb in db.TblBranchesBankDetails.Where(x => x.IsActive == true && x.iBranchId == obj.iBranchId)
select new ManageBranchBankDetails
{
iBranchId = objb.iBranchId,
iAccountName = objb.iAccountName,
}).FirstOrDefault(),
objbranchcontactperson = (from objc in db.tblbranchcontactpersons.Where(x => x.Isactive == true && x.branchid == obj.iBranchId)
select new ManageBranchContactPerson
{
branchid = objc.branchid,
createdate = objc.createdate,
Id = objc.Id,
iemailid = objc.iemailid,
}).ToList(),
}).OrderByDescending(x => x.iBranchId).ToList();

Translate SQL to lambda LINQ with GroupBy and Average

I spend a few hours trying to translate simple SQL to lambda LINQ
SELECT ID, AVG(Score) FROM myTable
GROUP BY ID
Any idea?

from t in myTable
group t by new {
t.ID
} into g
select new {
Average = g.Average(p => p.Score),
g.Key.ID
}
or Lambda
myTable.GroupBy(t => new {ID = t.ID})
.Select (g => new {
Average = g.Average (p => p.Score),
ID = g.Key.ID
})

The equivalent in Linq-to-Objects would be something like the below.
var results = from row in myTable
group row by row.Id into rows
select new
{
Id = rows.Key,
AverageScore = rows.Average(row => row.Score)
};
It's only slightly different for an ORM like entity framework. Namely, you would need to go through the data context or an appropriate DbSet/ObjectSet.

var _result = from a in myTable
group a by a.ID into g
select new
{
ID = g.Key.ID,
AverageResult = g.Average(x => x.Score)
}