We Keep Coding

Entity Framework: How to optimize this below linq query?

I need some suggestions that how to improve this below query.
from o in this.DbContext.Set<School>().AsNoTracking()
from s in o.Teachers.DefaultIfEmpty()
where SchoolCodes.Contains(o.Code)
select new TabularItem
{
SchoolId = o.Id,
SchoolCode = o.Code,
SchoolPurchaseOrderReference = o.PurchaseOrderReference,
SchoolDescription = o.OrderDescription,
SchoolActivityStatus = o.ActivityStatusesInternal.FirstOrDefault(os => os.ActivityName == orderLoggingActivity),
Type = o.TypesAsString,
CustomerCode = o.CustomerCode,
TeacherId = s == null ? (Guid?)null : s.Id,
TeacherCode = s == null ? null : s.Code,
TeacherCustomerReference = s == null ? null : s.CustomerReference,
TeacherIsImported = s == null ? (bool?)null : s.IsImported,
TeacherIsRegisteredUnderModification = s == null ? (bool?)null : s.IsRegisteredUnderModification,
TeacherStatus = s == null ? null : s.StatusAsString,
TeacherStatusChangeDate = s == null ? (DateTimeOffset?)null : s.StatusChangeDate,
IsReportInProgress = s == null ? false : s.IsReportInProgress,
TeacherActivityStatus = s == null ? null : s.ActivityStatusesInternal.FirstOrDefault(ss => ss.ActivityName == orderLoggingActivity),
TeacherHasUnresolvedIssue = s.TeacherIssuesInternal.Any(si => unresolvedIssueStatuses.Contains(si.StatusAsString)),
TeacherHasAdvancePaymentInProgressInvoiceableItem = s.FractionsInternal.SelectMany(x => x.TestPRepetitionsInternal).Any(x => x.InvoiceableItem.IsAdvancePaymentInProgress),
TeacherHasInvoicingInProgressInvoiceableItem = s.FractionsInternal.SelectMany(x => x.TestPRepetitionsInternal).Any(x => x.InvoiceableItem.IsInvoicingInProgress && x.InvoiceableItem.InvoicingStatusAsString != doNotInvoiceStatus),
HasSchoolBasedInvoiceableItems = s.School.InvoiceableItemsInternal.Any(item => item.InvoicingStatusAsString != orderBasedInvoiceableItemStatus),
SchoolHasInvoicingInProgressInvoiceableItem = s.School.InvoiceableItemsInternal.Any(x => x.IsInvoicingInProgress && x.InvoicingStatusAsString != doNotInvoiceStatus)
};
Here School--> Teacher --> Fraction --> TestPRepetition --> InvoiceableItem relation between tables.
please suggest me where i can improve performance. This will hit only once, so i cant use compiled query. there is no use.

Simple. DO not load all the data.
Teacher -> Fraction -> TestRPepetition multiplies the amount odf data you pull.
Ef is meant to pull the data you need, now load a lot of related data into memory IN CASE YOU MAY NEED IT ONE DAY.
Pull the minimum amount of data you need in this moment, go back to the database when you need more. Optimize from there when you run into problems by adding preloads, but always keep to the minimum you need.
Right now you load all data related to all tachers within a specific code. This is likely a ridiculous amount of data that mostly is noise and not properly used in further processing.

There are a couple options, but few are in EF
Clean up the null values in the database
Build the implied entity relations like TeacherHasInvoicingInProgressInvoiceableItem into the database (via foreign key or mapping table)
Prefetch repeat statements like o.ActivityStatusesInternal.FirstOrDefault

Null value in the result of a left outer join linq causes error

I have linq query, that left outer join two tables. I found if a value of a field returns null,, then I will get an error message:
"The cast to value type 'System.Int32' failed because the materialized value is null. Either the result type's generic parameter or the query must use a nullable type."
I copied my linq below:
var SrvRef = from s in db.SrvHeads
join r in db.Referrants on s.svhReferrer equals r.refID into r_join
from r in r_join.DefaultIfEmpty()
where s.svhAccID == accId &&
s.svhHeadCnt == HeadId
select new
{
s.svhBalance,
r.refID
};
bool FBeenPaid = SrvRef.FirstOrDefault().svhBalance == 0M; //this causes error
How can I fix this problem?

I'm slightly surprised at the kind of error you're getting, but there are two places you need to take account of the possibility of the result being null:
Within the query, where r can be null. (If you don't want to match when there are no elements in r_join matching s, you shouldn't be using a left outer join)
In the result itself: you're using FirstOrDefault() which will return null if SrvRef is empty.
So at first glance it should probably be something like:
var query = from s in db.SrvHeads
join r in db.Referrants on s.svhReferrer equals r.refID into r_join
from r in r_join.DefaultIfEmpty()
where s.svhAccID == accId && s.svhHeadCnt == HeadId
select new
{
s.svhBalance,
refId = r == null ? 0 : r.refID // Adjust to the appropriate type of refID
};
var result = query.FirstOrDefault();
bool beenPaid = result != null && result.svhBalance == 0m;
With C# 6, you can change the bottom two lines to:
bool beenPaid = query.FirstOrDefault()?.svhBalance == 0m ?? false;
Having said that:
You're not currently using refId in the result anyway - why are you including it in the result?
Are you sure you want a left outer join at all?
Are you sure that taking the first result is really what you want? What if there are multiple results in the join?
Is there any reason you're not doing the whole thing in a query? Something like:
var paid = db.SrvHeads
.Where(s => s.svhAccID == accId && s.svhHeadCnt == HeadId)
.Any(s => db.Refererrants.Any(r => s.svhReferrer == r.refID
&& s.svhBalance == 0m);
.. but just for the precise semantics you want.

I had a similar issue.
Cause: You are using from "r" in r_join.DefaultIfEmpty(). You cannot use same alias name for left outer join.
Solution: Use different alias name if DefaultIfEmpty() cases. Eg: rEmpty
I modified the below query and its working.
var SrvRef = from s in db.SrvHeads
join r in db.Referrants on s.svhReferrer equals r.refID into r_join
from rEmpty in r_join.DefaultIfEmpty()
where s.svhAccID == accId &&
s.svhHeadCnt == HeadId
select new
{
s.svhBalance,
refID = rEmpty == null ? 0 : rEmpty.refID
};

what i think is causing an error is that svhBalance is an int32 value type and you are accessing a null value returned by SrvRef.FirstOrDefault().
please try the following line and let me know if it helped you.
if svhBalance is an int value type
var SrvRefObj= SrvRef.FirstOrDefault(); bool FBeenPaid = (((SrvRefObj!=null)&&(SrvRefObj.svhBalance
!=null))?(SrvRefObj.svhBalance == 0):(false))
else if it's a decimal value type
var SrvRefObj= SrvRef.FirstOrDefault(); bool FBeenPaid = (((SrvRefObj!=null)&&(SrvRefObj.svhBalance
!=null))?(SrvRefObj.svhBalance == 0M):(false))

No property or field 'p' exists in type 'productByCategory'

I am trying to create a dynamic query for filtering my data results. I am building a linqToSql query containing subqueries and filtering the result after that using
System.Linq.Dynamic. But my Approach is ending up in errors and i cant find a good amount of documentations over the library samples.
here is what i am trying to do. i am using a list to filter data by given filter ids, after i fetch the result with added sub query:
List<long?> filter_list = new List<long?>();
string query = "";
if(mapped_filters!=null)
if (mapped_filters.Length > 0)
{
int i=0;
foreach (string item in mapped_filters)
{
long filter = 0;
long.TryParse(item, out filter);
filter_list.Add(filter);
query += " p.prod_attrs.Contains(#"+i+") ||";
i++;
}
}
query = query.TrimEnd(new char []{ '|','|'});
and following is the code for linq fetch on my products:
var productByCategory = db.products.Where(p => p.is_active == true && p.mainCat_id == cat_id).
Select(p => new productByCategory
{
id = p.id ,
prod_name = p.product_name,
prod_price = db.product_pricings.Where(pr => pr.prod_id == p.id && pr.currency_id == 2).FirstOrDefault(),
//filters below
prod_attrs = db.prod_attr_maps.Where(pa => pa.prod_id == p.id && pa.is_active == true
&& pa.currency_id == 2)
.Select(pa => pa.attr_id).ToList()
}).Where(p =>
(p.prod_price.our_price + p.prod_price.shipping_cost) >= min &&
(p.prod_price.our_price + p.prod_price.shipping_cost) <= max)
.Where("p=>"+query,filter_list)
.OrderBy(p => p.newItem).ToList();
but this ends up in error :
No property or field 'p' exists in type 'productByCategory'
..... Parse Exception
So far i have tried replacing
.Where("p=>"+query,filter_list)
To
.Where(""+query,filter_list)
and query part to
query += " prod_attrs.Contains(#"+i+") ||";
but still same error p replaced by prod_attrs exception.
EDIT
i did some direct string placement in the query and it seems like a problem with dynamic linq dll, when i do query like this :
.Where("id==1 && prod_name.Contains(\"a\")") it works fine
but with
.Where("id==1 && prod_name.Contains(1)")
it ends up in error :
An exception of type 'System.InvalidOperationException' occurred in
System.Core.dll but was not handled in user code
Additional information: No generic method 'Contains' on type
'System.Linq.Enumerable' is compatible with the supplied type
arguments and arguments. No type arguments should be provided if the
method is non-generic.
So it clearly seems to be a functionality issue with dynamic linq library.

Entity Framework causing Timeout Error

I am working with the following Entity Framework query. I know there's a lot going on here but am hoping it's clear enough that someone might be able to spot the issue.
var lineItems = from li in Repository.Query<CostingLineItem>()
let cid = (li.ParentCostingPackage != null) ?
li.ParentCostingPackage.ParentCostingEvent.ProposalSection.Proposal.Costing.Id :
li.ParentCostingEvent.ProposalSection.Proposal.Costing.Id
where cid == costingId &&
li.OriginalProductId.HasValue &&
(li.Quantity.HasValue && li.Quantity.Value > 0) && // li.QuantityUnitMultiplier
Classifications.Contains(li.OriginalProduct.ClassificationEnumIndex)
let selectedChoiceId = li.OriginalPackageOptionId.HasValue ?
(from c in li.OriginalPackageOption.CostingLineItems
orderby (c.IsIncluded ?? false) ? -2 : (c.IsDefaultItem ?? false) ? -1 : c.Id
select (int)c.OriginalPackageOptionChoiceId).FirstOrDefault() :
0
where selectedChoiceId == 0 || (li.OriginalPackageOptionChoiceId.HasValue && li.OriginalPackageOptionId.Value == selectedChoiceId)
let hasProviderAvailable = li.OriginalProductItem.ProductItemVendors.Any(
piv => piv.ProductPricings.Any(pp => pp.ProductItemVendor.CompanyId != null || pp.ProductItemVendor.HotelId != null))
select new
{
LineItem = li,
ProductItem = li.OriginalProductItem,
Product = li.OriginalProduct,
Vendors = li.CostingLineItemVendors,
HasProviderAvailable = hasProviderAvailable
};
As is, this query generates the following run-time error:
The wait operation timed out
If I change the section that declares selectedChoiceId to the following, the error goes away:
let selectedChoiceId = 0
Can anyone see how that code is consistently causing a time-out error?
(Note: This code is part of a large application that has been running for several years. So I really don't think this has anything to do with the connection string or anything like that. If I make the change above, it works consistently.)

The query can be simplified in a number of ways, which should make it easier to optimize by the database engine.
Firstly, you can remove a number of null checks (HasValue), because they're not relevant in SQL, but they do bloat the generated SQL.
Secondly, I think this check involving selectedChoiceId can be greatly simplified. This is what I think the statement could look like:
from li in Repository.Query<CostingLineItem>()
let cid = (li.ParentCostingPackage != null) ?
li.ParentCostingPackage.ParentCostingEvent.ProposalSection.Proposal.Costing.Id :
li.ParentCostingEvent.ProposalSection.Proposal.Costing.Id
where cid == costingId &&
li.OriginalProductId.HasValue &&
li.Quantity > 0 && // no null check
Classifications.Contains(li.OriginalProduct.ClassificationEnumIndex)
let selectedChoiceId = (from c in li.OriginalPackageOption.CostingLineItems
orderby c.IsIncluded ? -2 : c.IsDefaultItem ? -1 : c.Id // no null checks
select (int)c.OriginalPackageOptionChoiceId).FirstOrDefault()
where !li.OriginalPackageOptionId.HasValue || li.OriginalPackageOptionId == selectedChoiceId
let hasProviderAvailable = li.OriginalProductItem.ProductItemVendors.Any(
piv => piv.ProductPricings.Any(pp => pp.ProductItemVendor.CompanyId != null || pp.ProductItemVendor.HotelId != null))
select new
{
LineItem = li,
ProductItem = li.OriginalProductItem,
Product = li.OriginalProduct,
Vendors = li.CostingLineItemVendors,
HasProviderAvailable = hasProviderAvailable
}
For the rest, of course there are the usual suspects. Better indexes may become more important as the database volume increases. Checking for (and fixing) database fragmentation can also have a significant impact.

I think this will give you a better performance but not sure if it'll fix the problem :
let selectedChoiceId = li.OriginalPackageOptionId.HasValue
? (from c in li.OriginalPackageOption.CostingLineItems
let cOrder = (c.IsIncluded ?? false) ? -2 : (c.IsDefaultItem ?? false) ? -1 : c.Id
orderby cOrder
select (int) c.OriginalPackageOptionChoiceId).FirstOrDefault()
: 0

Linq if-elseif-else using linq query in asp.net

In my asp.net application i am using linq. I need a help what is the syntax for if-elseif-else using linq in single line.
genericReportList =
(from CD in list
select new GENERICREPORT
{
CITATIONNO = CD.CITATIONNO,
DATE = CD.DATE,
LOCATION = CD.LOCATION,
//STATUS = CD.STATUS,
PLATENO = Utilities.DecryptData(CD.PLATENO),
PSOURCE = CD.PSOURCE,
MAKE = CD.MAKE,
ID = Utilities.DecryptData(CD.ID),
NATIONALITY = CD.NATIONALITY,
SOURCE = CD.SOURCE,
NAME = Utilities.DecryptData(CD.NAME),
VIOLATION = CD.VIOLATION,
STATUS = CD.STATUS == short.Parse("1") ? "Complete" : "Incomplete"
}).ToList();
If STATUS = CD.STATUS == short.Parse("1") ? "Complete" : and 2 for "Incomplete" and 3 for "Void"

I don't understand why you are doing short.Parse("1"). This will always be 1. If you want multiple if-else in a one-liner, combine ternary operators:
STATUS = CD.STATUS == 1 ? "Complete" : CD.STATUS == 2 ? "Incomplete" : "Void"
If this is going to be used in the context of Entity Framework (or other ORM with IQueryable support), it will translate to a CASE WHEN SQL statement.

What I understand from your question if I'm not wrong, is that you might be asking about where clause.
if yes then you can always use multiple where in your query.
example :
Collection.Where(x => x.Age == 10)
.Where(x => x.Name == "Fido")
.Where(x => x.Fat == true)
more about LINQ query
http://msdn.microsoft.com/en-us/library/gg509017.aspx

You can just continue with what you already have.
STATUS = CD.STATUS == 1 ? "Complete" : (CD.STATUS == 2 ? "Incomplete" : "InProgress")

There nothing wrong with writing your own method and using it in a LINQ query. It will be far more understandable and readable than a long conditional operator.
Consider using something like:
private String GetStatus(int value)
{
if (value == 1)
return "Complete";
if (value == 2)
return "Incomplete";
if (value == 3)
return "Void";
}
Then you would call it like:
STATUS = GetStatus(CD.STATUS)