We Keep Coding

Is there a better way to fix a LINQ expression that could not be translated than my own solution?

For my LINQ query I have left joined a documents information table onto itself to grab only distinct documents with a top page count and certain social security numbers. I got an error on my first attempt. But I was able to fix it by toList() all the steps. However, I only want to ToList() the last part but I don't know how. I don't want each step to be doing a SQL call... I want to be building an IQUERYABLE in parts. Then once built do the SQL call once....
Here was the first attempt:
// get documents of a type doc or eDoc that contains the social security number thats in a list of property ids
var documents = db.DocListFullPathWithTagsLimitedVws.AsNoTracking()
.Where(s => listOfSSNs.Contains(s.PropId)
&& s.StrVal == SSNparam && (s.DocumentType == "Doc" ||
s.DocumentType == "eDoc"));
// get top page count of each document
var queryCount = documents.GroupBy(x => new { x.Tocid, x.PropId })
.Select(x => new
{
Tocid = x.Key.Tocid,
PropId = x.Key.PropId,
PageNum = x.Count()
});
// distinct as the documents can be repeated from property ids
var queryDistinct = queryCount.DistinctBy(x => x.Tocid);
// left join on orginal documents to get
var queryCombine =
(from document in documents
join qd in queryDistinct on document.Tocid equals qd.Tocid into gj
from subset in gj.DefaultIfEmpty()
select new DocListFullPathWithTagsLimitedVw
{
DocumentName = document.DocumentName,
Tocid = document.Tocid,
EdocStoreid = document.EdocStoreid,
Storeid = document.Storeid,
EdocExt = document.EdocExt,
PropId = document.PropId,
StrVal = document.StrVal,
FullPathAndFilename = document.FullPathAndFilename,
DocumentType = document.DocumentType,
VolName = document.VolName,
Modified = document.Modified,
Created = document.Created,
PageNum = subset.PageNum
}).ToList();
return queryCombine;
Gets this error Message:
[LaserficheDocFinder] : System.InvalidOperationException: The LINQ expression 'DbSet<DocListFullPathWithTagsLimitedVw>()
.Where(s => __listOfSSNs_0
.Contains(s.PropId) && s.StrVal == __SSNparam_1 && s.DocumentType == "Doc" || s.DocumentType == "eDoc")
.GroupBy(x => new {
Tocid = x.Tocid,
PropId = x.PropId
})
.Select(x => new {
Tocid = x.Key.Tocid,
PropId = x.Key.PropId,
PageNum = x
.AsQueryable()
.Count()
})
.DistinctBy(x => x.Tocid)' could not be translated. Either rewrite the query in a form that can be translated,
or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable',
'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
Here is my solution. (but I want it all to be IQueryable untill the end...)
var documents = db.DocListFullPathWithTagsLimitedVws.AsNoTracking()
.Where(s => listOfSSNs.Contains(s.PropId)
&& s.StrVal == SSNparam && (s.DocumentType == "Doc" ||
s.DocumentType == "eDoc" )).ToList();
// get top page count of each document
var queryCount = documents.GroupBy(x => new { x.Tocid, x.PropId })
.Select(x => new
{
Tocid = x.Key.Tocid,
PropId = x.Key.PropId,
PageNum = x.Count()
}).ToList();
// distinct as the documents can be repeated from props
var queryDistinct = queryCount.DistinctBy(x => x.Tocid);
var queryDistinctList = queryDistinct.AsEnumerable().ToList();
var queryCombine =
(from document in documents
join qd in queryDistinctList on document.Tocid equals qd.Tocid into gj
from subset in gj.DefaultIfEmpty()
where (document.DocumentType == "Doc" && document.PageNum == subset.PageNum && document.PropId == subset.PropId)
|| (document.DocumentType == "eDoc" && document.PageNum == null && subset.PageNum == 1 && document.PropId == subset.PropId )
|| (document.DocumentType == "eDoc" && document.PageNum != null && document.PageNum == subset.PageNum && document.PropId == subset.PropId)
|| (document.FullPathAndFilename == null && document.PageNum == null && subset.PageNum == 1 && document.PropId == subset.PropId)
select new DocListFullPathWithTagsLimitedVw
{
DocumentName = document.DocumentName,
Tocid = document.Tocid,
EdocStoreid = document.EdocStoreid,
Storeid = document.Storeid,
EdocExt = document.EdocExt,
PropId = document.PropId,
StrVal = document.StrVal,
FullPathAndFilename = document.FullPathAndFilename,
DocumentType = document.DocumentType,
VolName = document.VolName,
Modified = document.Modified,
Created = document.Created,
PageNum = subset.PageNum
}).ToList();
return queryCombine;

You can remove DistinctBy and use other technique how to get first element. It is possible to add OrderBy if it is omitted. Also Left join is not needed, outer items should exist.
// get documents of a type doc or eDoc that contains the social security number thats in a list of property ids
var documents = db.DocListFullPathWithTagsLimitedVws.AsNoTracking()
.Where(s => listOfSSNs.Contains(s.PropId)
&& s.StrVal == SSNparam && (s.DocumentType == "Doc" ||
s.DocumentType == "eDoc"));
// get top page count of each document
var queryCount = documents.GroupBy(x => new { x.Tocid, x.PropId })
.Select(x => new
{
Tocid = x.Key.Tocid,
PropId = x.Key.PropId,
PageNum = x.Count()
});
var queryCombine =
(from document in documents
from subset in queryCount.Where(x => x.TocId == document.Tocid).Take(1)
select new DocListFullPathWithTagsLimitedVw
{
DocumentName = document.DocumentName,
Tocid = document.Tocid,
EdocStoreid = document.EdocStoreid,
Storeid = document.Storeid,
EdocExt = document.EdocExt,
PropId = document.PropId,
StrVal = document.StrVal,
FullPathAndFilename = document.FullPathAndFilename,
DocumentType = document.DocumentType,
VolName = document.VolName,
Modified = document.Modified,
Created = document.Created,
PageNum = subset.PageNum
}).ToList();
return queryCombine;

Include Newest Note and Comment in List

I have to include both the newest Note and Comment with view model that I am passing to my list view but I cannot figure out how to include it in the view model.
I tried to do include but after I type p it will not give me a list of my properties for MinimumProductInfo which ProductNotes is a property for it.
Here is the controller code I am trying:
public ActionResult MinimumProductInfoList()
{
var Model = _minimumProductInfo.GetAll();
var Notes = db.ProductNotes.Where(p => p.NoteTypeFlag == "p").OrderByDescending(p => p.NoteDate).First();
var Comments = db.ProductNotes.Where(p => p.NoteTypeFlag == "c").OrderByDescending(p => p.NoteDate).First();
var Model = db.MinimumProductInfo.Include(p => p)
var ViewModel = Model.Select(x => new ProductInfoWithNoteList { MinimumProductInfoID = x.MinimumProductInfoID, ItemCode = x.ItemCode, MinimumOnHandQuantity = x.MinimumOnHandQuantity, MaximumOHandQuantity = x.MaximumOHandQuantity, MinimumOrderQuantity = x.MinimumOrderQuantity, LeadTimeInWeeks = x.LeadTimeInWeeks });
return View(ViewModel);
}
Everything else is working except now I need to include the latest note and latest comment in to my viewmodel
This is what I have now with WithMetta's help:
public ActionResult MinimumProductInfoList()
{
var productInfoViewModelCollection =
from x in db.MinimumProductInfo
let pnote =
(from inner_pnote in db.ProductNotes
where x.MinimumProductInfoID == inner_pnote.MinimumProductInfoID
&& inner_pnote.NoteTypeFlag == "p"
orderby inner_pnote.NoteDate
select inner_pnote).FirstOrDefault()
let cnote =
(from inner_cnote in db.ProductNotes
where x.MinimumProductInfoID == inner_cnote.MinimumProductInfoID
&& inner_cnote.NoteTypeFlag == "c"
orderby inner_cnote.NoteDate
select inner_cnote).FirstOrDefault()
select new ProductInfoWithNoteList
{
MinimumProductInfoID = x.MinimumProductInfoID,
ItemCode = x.ItemCode,
MinimumOnHandQuantity = x.MinimumOnHandQuantity,
MaximumOHandQuantity = x.MaximumOHandQuantity,
MinimumOrderQuantity = x.MinimumOrderQuantity,
LeadTimeInWeeks = x.LeadTimeInWeeks,
Comment = cnote.ToString(),
PermanentNote = pnote.ToString()
};
return View(productInfoViewModelCollection);
}

Maybe something like this using LINQ.
var productInfoViewModelCollection =
from x in db.MinimumProductInfo
where x != null
let pnote =
(from inner_pnote in db.ProductNotes
where inner_pnote != null
&& x.MinimumProductInfoID == inner_pnote.MinimumProductInfoID
&& inner_pnote.NoteTypeFlag == "p"
orderby inner_pnote.NoteDate descending
select inner_pnote).FirstOrDefault()
let cnote =
(from inner_cnote db.ProductNotes
where inner_cnote != null
&& x.MinimumProductInfoID == inner_cnote.MinimumProductInfoID
&& inner_cnote.NoteTypeFlag == "c"
orderby inner_cnote.NoteDate descending
select inner_cnote).FirstOrDefault()
select new ProductInfoWithNoteList {
MinimumProductInfoID = x.MinimumProductInfoID,
ItemCode = x.ItemCode,
MinimumOnHandQuantity = x.MinimumOnHandQuantity,
MaximumOHandQuantity = x.MaximumOHandQuantity,
MinimumOrderQuantity = x.MinimumOrderQuantity,
LeadTimeInWeeks = x.LeadTimeInWeeks,
Comment = cnote,
PermanentNote = pnote
};

Get all if variable is null else get matching

Can't figure out the logic for Get all records if variable is null else get where officername = officer.
var res = (from h in db.BalanceHistories
where temp.Contains(h.LoanType ?? 0)
&& ((officer != null && h.OfficerName.ToLower() == officer.ToLower()) || ("Get all records"))
group h by new { h.Date.Value.Month, h.Date.Value.Year } into p
select new
{
Month = p.Key.Month,
Year = p.Key.Year,
Count = p.Count(),
Balance = p.Sum(x => x.Balance),
Delinquent = p.Sum(x => x.Delinquent)
}).ToList();

While you can make a compound if statement, this will (usually -- currently always) pass this compound statement on to the database. In many cases, this can cause index misses which if done correctly wouldn't be missed. In non-Microsoft SQL servers, these types of queries are also known to just plain not work (MySQL, DB2). It's better to just write the query correctly in the first place:
var query = db.BalanceHistories
.Where(h=>temp.Contains(h.LoanType ?? 0));
if (officer!=null)
{
// Depending on your database, the ToLower()s here may not be needed.
query=query.Where(h=>h.OfficerName.ToLower() == officer.ToLower()))
}
var res=(from h in query
group h by new { h.Date.Value.Month, h.Date.Value.Year } into p
select new
{
Month = p.Key.Month,
Year = p.Key.Year,
Count = p.Count(),
Balance = p.Sum(x => x.Balance),
Delinquent = p.Sum(x => x.Delinquent)
});

Just swap ("Get all records") with (officer == null), Although i would suggest you to put this condition at first.
Explanation:
If officer == null than respect to short circut law we will return true and the rest of our condition won't matter.
Else means officer != null and thus we want to check if it matches our h.OfficerName data.
End result:
res = (from h in db.BalanceHistories
where temp.Contains(h.LoanType ?? 0)
&& (
officer == null
|| h.OfficerName.ToLower() == officer.ToLower())
)
group h by new { h.Date.Value.Month, h.Date.Value.Year } into p
select new
{
Month = p.Key.Month,
Year = p.Key.Year,
Count = p.Count(),
Balance = p.Sum(x => x.Balance),
Delinquent = p.Sum(x => x.Delinquent)
}).ToList();

Have you tried:
var res = (from h in db.BalanceHistories
where temp.Contains(h.LoanType ?? 0)
&& ((officer == null) || (h.OfficerName.ToLower() == officer.ToLower()))
group h by new { h.Date.Value.Month, h.Date.Value.Year } into p
select new
{
Month = p.Key.Month,
Year = p.Key.Year,
Count = p.Count(),
Balance = p.Sum(x => x.Balance),
Delinquent = p.Sum(x => x.Delinquent)
}).ToList();

LINQ pivot top 5 rows to columns

The desired output is a set of rows containing details about a physician and the top 5 procedures they performed (by volume) that are not in our benchmarking tables - with a minimum of one per month. Even with my limited LINQ experience, I have managed to get this working, but the performance in abysmal. In the spirit of becoming better at LINQ, I thought I'd see if there was a better way to approach this (besides coding in SQL).
Here is the entire method, although I believe it is the final statement (var report =) that deserves the most attention.
//Get the Providers for the selected AccessLevel & Report Period
var providerRiskLevels = GetProviderRiskLevelForAccessLevel(userAccessLevelId, reportPeriodId, providerNamePiece);
short benchmarkYear = (
from rp in db.ReportPeriods
where rp.ReportPeriodId == reportPeriodId
select rp.BenchmarkDataYear
).FirstOrDefault();
byte Duration = (
from rp in db.ReportPeriods
where rp.ReportPeriodId == reportPeriodId
select rp.Duration
).FirstOrDefault();
//Summarize counts by procedure code (ignoring modifiers)
var codeCounts =
from pp in db.ProviderProductions
//332 - restrict to valid codes
join pc in db.ProcedureCodes on pp.ProcedureCode equals pc.ProcedureCode1
where pp.ReportPeriodId == reportPeriodId
//show deleted codes as potential issue
&& pc.TerminatedDate == null
&& (codeFilter == null || pp.ProcedureCode == codeFilter)
join pc in db.PspsProcedures
on new { col1 = pp.ProcedureCode, col2 = benchmarkYear } equals new { col1 = pc.ProcedureCode, col2 = pc.Year } into left
from l_d in left.DefaultIfEmpty()
where l_d == null
group pp by
new { pp.ProviderId, pp.ProcedureCode }
into g
orderby g.Key.ProviderId, g.Sum(x => x.Volume) descending
where g.Sum(x => x.Volume) > Duration
select new NewCodesEntity
{
ProviderId = g.Key.ProviderId,
ProcedureCode = g.Key.ProcedureCode,
Volume = g.Sum(x => x.Volume)
};
//Get top 5 procedures performed by provider
var newCodes =
from cc in codeCounts
group cc by
new { cc.ProviderId }
into g
select new ProviderNewCodesEntity
{
ProviderId = g.Key.ProviderId,
Codes = g.OrderByDescending(x => x.Volume).Take(5).ToList()
};
//Build the report
var report =
from pr in providerRiskLevels
join nc in newCodes on pr.Provider.ProviderId equals nc.ProviderId
where pr.RiskCategoryId == RiskCategoryIds.VisibleRisk
&& (filterRiskLevelNums.Contains(pr.RiskLevelNum))
&& (filterSpecialtyId == 0 || pr.Provider.Specialty.SpecialtyId == filterSpecialtyId)
select new ProbeAuditEntity
{
ProviderId = pr.Provider.ProviderId,
ProviderName = pr.Provider.Name,
ProviderCode = pr.Provider.ProviderCode,
SpecialtyName = pr.Provider.Specialty.Name,
SpecialtyCode = pr.Provider.Specialty.SpecialtyCode,
VisibleRisk = pr.RiskScore,
NewCode1 = nc.Codes.OrderByDescending(c => c.Volume).FirstOrDefault().ProcedureCode,
NewCode2 = nc.Codes.OrderByDescending(c => c.Volume).Skip(1).FirstOrDefault().ProcedureCode,
NewCode3 = nc.Codes.OrderByDescending(c => c.Volume).Skip(2).FirstOrDefault().ProcedureCode,
NewCode4 = nc.Codes.OrderByDescending(c => c.Volume).Skip(3).FirstOrDefault().ProcedureCode,
NewCode5 = nc.Codes.OrderByDescending(c => c.Volume).Skip(4).FirstOrDefault().ProcedureCode
};
return report;
I appreciate your suggestions.

Give this a try for the report:
//Build the report
var report =
(from pr in providerRiskLevels
join nc in newCodes on pr.Provider.ProviderId equals nc.ProviderId
where pr.RiskCategoryId == RiskCategoryIds.VisibleRisk
&& filterRiskLevelNums.Contains(pr.RiskLevelNum)
&& (filterSpecialtyId == 0
|| pr.Provider.Specialty.SpecialtyId == filterSpecialtyId)
select new
{
RiskLevel = pr,
NewCodes = nc.Codes.OrderByDescending(c => c.Volumn).Select(c => c.ProcedureCode).Take(5)
})
.AsEnumerable()
.Select(x => new ProbeAuditEntity
{
ProviderId = x.RiskLevel.Provider.ProviderId,
ProviderName = x.RiskLevel.Provider.Name,
ProviderCode = x.RiskLevel.Provider.ProviderCode,
SpecialtyName = x.RiskLevel.Provider.Specialty.Name,
SpecialtyCode = x.RiskLevel.Provider.Specialty.SpecialtyCode,
VisibleRisk = x.RiskLevel.RiskScore,
NewCodes = x.NewCodes.Concat(Enumerable.Repeat(<DefaultProcedureCodeHere>, 5)).Take(5)
});
Or, if ProbeAuditEntity is a class you could do this:
//Build the report
var report =
(from pr in providerRiskLevels
join nc in newCodes on pr.Provider.ProviderId equals nc.ProviderId
where pr.RiskCategoryId == RiskCategoryIds.VisibleRisk
&& filterRiskLevelNums.Contains(pr.RiskLevelNum)
&& (filterSpecialtyId == 0
|| pr.Provider.Specialty.SpecialtyId == filterSpecialtyId)
select new ProbeAuditEntity
{
ProviderId = x.RiskLevel.Provider.ProviderId,
ProviderName = x.RiskLevel.Provider.Name,
ProviderCode = x.RiskLevel.Provider.ProviderCode,
SpecialtyName = x.RiskLevel.Provider.Specialty.Name,
SpecialtyCode = x.RiskLevel.Provider.Specialty.SpecialtyCode,
VisibleRisk = x.RiskLevel.RiskScore,
NewCodes = nc.Codes.OrderByDescending(c => c.Volumn).Select(c => c.ProcedureCode).Take(5)
})
.ToList();
report
.ForEach(x => x.NewCodes = x.NewCodes.Concat(Enumerable.Repeat(<DefaultProcedureCodeHere>, 5)).Take(5);

The entity or complex type cannot be constructed in a LINQ to Entities query

On our online billing application, we give a billing summary of what bills the customer received and the payments they made.
In order for this to work, I have to first pull the payments then match them to the bills. So I have do something like:
foreach (BillPaymentSummary payment in billPayments)
{
DateTime dt = payment.DueDate;
// Debug errors on this next line
var summary = (from a in db.BillHistories
where a.CustomerId == customerNumber && a.DueDate == dt && a.Type == "BILL"
select new BillSummary
{
Id = a.Id,
CustomerId = a.CustomerId,
DueDate = a.DueDate,
PreviousBalance = a.PreviousBalance.Value,
TotalBill = a.TotalBill.Value,
Type = a.Type,
IsFinalBill = a.IsFinalBill
}).SingleOrDefault();
if (summary != null)
{
summary.PayDate = payment.PaidDate;
summary.AmountPaid = payment.AmountPaid;
returnSummaries.Add(summary);
}
else
{
summary = (from a in db.BillHistories
where a.CustomerId == customerNumber && a.DueDate == payment.DueDate && a.Type == "ADJ "
select new BillSummary
{
Id = a.Id,
CustomerId = a.CustomerId,
DueDate = a.DueDate,
PreviousBalance = a.PreviousBalance.Value,
TotalBill = a.TotalBill.Value,
Type = a.Type,
IsFinalBill = a.IsFinalBill
}).SingleOrDefault();
if (summary != null)
{
summary.PayDate = payment.PaidDate;
summary.AmountPaid = payment.AmountPaid;
returnSummaries.Add(summary);
}
}
}
I have been playing with this, but no matter what I do, I get the following error message:
The entity or complex type 'UtilityBill.Domain.Concrete.BillSummary' cannot be constructed in a LINQ to Entities query.
Is it because I am running queries within queries? How can I get around this error?
I have tried searching Google for an answer and see many answers, but none of them seem to explain my problem.

You cannot project onto a mapped entity. You would have to call ToList() before doing your mapping.
Or better yet, change to the following (calling FirstOrDefault will execute the query and allow you to populate your object):
var summary = db.BillHistories.FirstOrDefault(a => a.CustomerId == customerNumber && a.DueDate == dt && a.Type == "BILL").Select(x => new BillSummary
{
Id = a.Id,
CustomerId = a.CustomerId,
DueDate = a.DueDate,
PreviousBalance = a.PreviousBalance.Value,
TotalBill = a.TotalBill.Value,
Type = a.Type,
IsFinalBill = a.IsFinalBill
});
To decouple yourself from the Entity Framework you may want to also consider using a different model class to return instead of the Entity Framework model.

What I ended up doing was:
foreach (BillPaymentSummary payment in billPayments)
{
var data = db.BillHistories.Where(b => b.CustomerId == customerNumber && b.DueDate == payment.DueDate && b.Type == "B").FirstOrDefault();
if (data != null) // There is a bill history
{
returnSummaries.Add(new BillSummary
{
Id = data.Id,
CustomerId = data.CustomerId,
DueDate = data.DueDate,
PreviousBalance = data.PreviousBalance,
TotalBill = data.TotalBill,
Type = (data.Type.Trim() == "B" ? "BILL" : (data.Type == "A" ? "ADJ" : "")),
IsFinalBill = data.IsFinalBill,
PayDate = payment.PaidDate,
AmountPaid = payment.AmountPaid
});
}
else // No bill history record, look for an adjustment
{
data = db.BillHistories.FirstOrDefault(b => b.CustomerId == customerNumber && b.DueDate == payment.DueDate && b.Type == "A");
if (data != null)
{
returnSummaries.Add(new BillSummary
{
Id = data.Id,
CustomerId = data.CustomerId,
DueDate = data.DueDate,
PreviousBalance = data.PreviousBalance,
TotalBill = data.TotalBill,
Type = (data.Type.Trim() == "B" ? "BILL" : (data.Type == "A" ? "ADJ" : "")),
IsFinalBill = data.IsFinalBill,
PayDate = payment.PaidDate,
AmountPaid = payment.AmountPaid
});
}
}
db.SaveChanges();
}