We Keep Coding

Why Same where clause need to write multiple times in a Linq Query for following SQL

What will be the proper linq syntax of below SQL Query ?
select a.id, a.AppointmentStatusID, ad.ID as DetailID
from [dbo].[Appointment] a, [dbo].[AppointmentDetail] ad
where a.[ID] = ad.[AppointmentID]
and a.CompanyID = 'a3dea87a-804e-4115-98cf-472988cf1678'
and a.LocationID = '3165caca-2a48-46f0-bbed-578cff29167t'
and ad.AppDateFrom <= {ts '2017-11-14 23:59:31'}
and ad.AppDateTo >= {ts '2017-11-14 00:00:00'}
and ad.[ApprovalStatusID] = 2
Problem I faced:
I required to filter Where Condition two times 1st at the time within the join & 2nd time during the object.Select expression, please check bellow
var results = (from a in appointments
join ad in _appointmentDetailRepository.GetAll() on a.ID equals ad.AppointmentID
where ad.ApprovalStatusID == 2
&& DbFunctions.TruncateTime(ad.AppDateFrom) <= DbFunctions.TruncateTime(viewmodel.AppointmentDate)
&& DbFunctions.TruncateTime(ad.AppDateTo) >= DbFunctions.TruncateTime(viewmodel.AppointmentDate)
orderby a.ID
select new Appointment
{
ID = a.ID,
CompanyID = a.CompanyID,
LocationID = a.LocationID,
AppointmentDetail = a.AppointmentDetail.Select(ad => new AppointmentDetail
{
ID = ad.ID,
AppDateFrom = ad.AppDateFrom,
AppDateTo = ad.AppDateTo,
AppointmentStatusID = ad.AppointmentStatusID,
}).Where(ad=> ad.ApprovalStatusID == 2
&& DbFunctions.TruncateTime(ad.AppDateFrom) <= DbFunctions.TruncateTime(viewmodel.AppointmentDate)
&& DbFunctions.TruncateTime(ad.AppDateTo) >= DbFunctions.TruncateTime(viewmodel.AppointmentDate)).ToList()
}).GroupBy(x => x.ID).Select(x => x.DefaultIfEmpty().FirstOrDefault());
Query : Why I required to write Where clause 2 times ?
Required Result
An Appointment Object --> Containing ICollection<AppoinmentDetails> if Details.Where Condition == True

From what I see (without knowing the model you have), it looks like you should use the already joined and filtered Details from ad instead of looking it up again from the Property a.AppointmentDetail...
Untested:
select new Appointment
{
ID = a.ID,
CompanyID = a.CompanyID,
LocationID = a.LocationID,
AppointmentDetail = ad.ToList(), // <-- don't you think?
...
}

For the given SQL query
select a.id, a.AppointmentStatusID, ad.ID as DetailID
from [dbo].[Appointment] a, [dbo].[AppointmentDetail] ad
where a.[ID] = ad.[AppointmentID]
and a.CompanyID = 'a3dea87a-804e-4115-98cf-472988cf1678'
and a.LocationID = '3165caca-2a48-46f0-bbed-578cff29167t'
and ad.AppDateFrom <= {ts '2017-11-14 23:59:31'}
and ad.AppDateTo >= {ts '2017-11-14 00:00:00'}
and ad.[ApprovalStatusID] = 2
LINQ query can be written as
var results = (from a in appointments
join ad in appointmentDetails on a.ID equals ad.AppointmentID
where ad.ApprovalStatusID == 2
&& a.CompanyID == "a3dea87a-804e-4115-98cf-472988cf1678"
&& a.LocationID == "3165caca-2a48-46f0-bbed-578cff29167t"
&& ad.AppDateFrom.Date <= viewmodel.AppointmentDate.Date
&& ad.AppDateTo.Date >= viewmodel.AppointmentDate.Date
select new
{
ID = a.ID,
AppointmentStatusID = a.AppointmentStatusID,
DetailID = ad.ID
}).ToList();
You can also write it like
var results = appointmentDetails
.Where(ad => ad.AppDateFrom.Date <= viewmodel.AppointmentDate.Date
&& ad.AppDateTo.Date >= viewmodel.AppointmentDate.Date
&& ad.ApprovalStatusID == 2
&& ad.Appointment.CompanyID == "a3dea87a-804e-4115-98cf-472988cf1678"
&& ad.Appointment.LocationID == "3165caca-2a48-46f0-bbed-578cff29167t")
.Select(ad =>
new
{
ID = ad.Appointment.ID,
AppointmentStatusID = ad.Appointment.AppointmentStatusID,
DetailID = ad.ID
})
.ToList()
As per the updated question, to get the the Appointment object with a collection of AppointmentDetails, please try this query
var results = appointmentDetails
.Where(ad => ad.AppDateFrom.Date <= viewmodel.AppointmentDate.Date
&& ad.AppDateTo.Date >= viewmodel.AppointmentDate.Date
&& ad.ApprovalStatusID == 2
&& ad.Appointment.CompanyID == "a3dea87a-804e-4115-98cf-472988cf1678"
&& ad.Appointment.LocationID == "3165caca-2a48-46f0-bbed-578cff29167t")
.Select(ad =>
new
{
ID = ad.Appointment.ID,
AppointmentStatusID = ad.Appointment.AppointmentStatusID,
Detail = ad
})
.AsEnumerable()
.GroupBy(a => new { a.ID, a.AppointmentStatusID })
.Select(a => new Appointment
{
ID = a.Key.ID,
AppointmentStatusID = a.Key.AppointmentStatusID,
AppointmentDetails = a.Select(d => d.Detail).ToList()
})
.ToList();

Linq to SQL grouping with embedded lists causes too many queries

I am developing a query to grab and join some SQL tables in C# and am having some trouble with grouping and enumerables within the dataset. My query is below. This gives me the data in the format I'm looking for, but it takes way too long when I try to add the enumerated list as indicated below. When I look under the hood I can see it is executing way too many SQL queries. I'd like to get it to just one. Using LinqPad:
void Main()
{
var nightlyRuns = (from a in LoadTestSummaries
join b in LoadTestTestSummaryData
on a.LoadTestRunId equals b.LoadTestRunId
where a.TargetStack == "LoadEnv" &&
a.TestGuid != null &&
a.StartTime != null &&
a.LoadTestRunId != null
orderby a.StartTime
group new {a, b} by new
{
a.TestGuid,
a.Name,
a.Description,
a.StartTime,
a.Duration,
a.NumAgents,
a.NumHosts,
a.PassFail,
a.ResultsFilePath,
a.Splunk
}
into g
let scenarioStart = g.Min(s => s.a.StartTime) ?? g.Min(s => s.a.DateCreated)
let testCases = g.Select(s => s.b)
orderby scenarioStart
select new
{
TestGuid = g.Key.TestGuid,
ScenarioRun = new
{
Name = g.Key.Name,
Description = g.Key.Description,
StartTime = scenarioStart,
Duration = g.Key.Duration,
NumAgents = g.Key.NumAgents,
NumHosts = g.Key.NumHosts,
Result = g.Key.PassFail,
ResultsFilePath = g.Key.ResultsFilePath,
SplunkLink = g.Key.Splunk,
// PROBLEM: Causes too many queries:
TestRuns = from t in testCases select t.TestCaseId
}
}).ToLookup(g => g.TestGuid, g => g.ScenarioRun);
nightlyRuns["ba593f66-695f-4fd1-99c3-71253a2e4981"].Dump();
}
The "TestRuns" line is causing the excessive queries. Any idea what I am doing wrong here?
Thanks for any insight.

Tough answer to test but I think we can avoid the grouping and multiple queries with something like this: (https://msdn.microsoft.com/en-us/library/bb311040.aspx)
var nightlyRuns = (from a in LoadTestSummaries
join b in LoadTestTestSummaryData
on a.LoadTestRunId equals b.LoadTestRunId
where a.TargetStack == "LoadEnv" &&
a.TestGuid != null &&
a.StartTime != null &&
a.LoadTestRunId != null
into testGroup
select new
{
TestGuid = a.TestGuid,
ScenarioRun = new
{
Name = a.TestGuid,
Description = a.Description,
StartTime = a.StartTime ?? a.DateCreated,
Duration = a.Duration,
NumAgents = g.Key.NumAgents,
NumHosts = a.NumHosts,
Result = a.PassFail,
ResultsFilePath = a.ResultsFilePath,
SplunkLink = a.Splunk,
// PROBLEM: Causes too many queries:
TestRuns =testGroup
}
}).OrderBy(x=>x.StartTime).ToLookup(x => x.TestGuid, x => x.ScenarioRun);
nightlyRuns["ba593f66-695f-4fd1-99c3-71253a2e4981"].Dump();

Get all if variable is null else get matching

Can't figure out the logic for Get all records if variable is null else get where officername = officer.
var res = (from h in db.BalanceHistories
where temp.Contains(h.LoanType ?? 0)
&& ((officer != null && h.OfficerName.ToLower() == officer.ToLower()) || ("Get all records"))
group h by new { h.Date.Value.Month, h.Date.Value.Year } into p
select new
{
Month = p.Key.Month,
Year = p.Key.Year,
Count = p.Count(),
Balance = p.Sum(x => x.Balance),
Delinquent = p.Sum(x => x.Delinquent)
}).ToList();

While you can make a compound if statement, this will (usually -- currently always) pass this compound statement on to the database. In many cases, this can cause index misses which if done correctly wouldn't be missed. In non-Microsoft SQL servers, these types of queries are also known to just plain not work (MySQL, DB2). It's better to just write the query correctly in the first place:
var query = db.BalanceHistories
.Where(h=>temp.Contains(h.LoanType ?? 0));
if (officer!=null)
{
// Depending on your database, the ToLower()s here may not be needed.
query=query.Where(h=>h.OfficerName.ToLower() == officer.ToLower()))
}
var res=(from h in query
group h by new { h.Date.Value.Month, h.Date.Value.Year } into p
select new
{
Month = p.Key.Month,
Year = p.Key.Year,
Count = p.Count(),
Balance = p.Sum(x => x.Balance),
Delinquent = p.Sum(x => x.Delinquent)
});

Just swap ("Get all records") with (officer == null), Although i would suggest you to put this condition at first.
Explanation:
If officer == null than respect to short circut law we will return true and the rest of our condition won't matter.
Else means officer != null and thus we want to check if it matches our h.OfficerName data.
End result:
res = (from h in db.BalanceHistories
where temp.Contains(h.LoanType ?? 0)
&& (
officer == null
|| h.OfficerName.ToLower() == officer.ToLower())
)
group h by new { h.Date.Value.Month, h.Date.Value.Year } into p
select new
{
Month = p.Key.Month,
Year = p.Key.Year,
Count = p.Count(),
Balance = p.Sum(x => x.Balance),
Delinquent = p.Sum(x => x.Delinquent)
}).ToList();

Have you tried:
var res = (from h in db.BalanceHistories
where temp.Contains(h.LoanType ?? 0)
&& ((officer == null) || (h.OfficerName.ToLower() == officer.ToLower()))
group h by new { h.Date.Value.Month, h.Date.Value.Year } into p
select new
{
Month = p.Key.Month,
Year = p.Key.Year,
Count = p.Count(),
Balance = p.Sum(x => x.Balance),
Delinquent = p.Sum(x => x.Delinquent)
}).ToList();

LINQ pivot top 5 rows to columns

The desired output is a set of rows containing details about a physician and the top 5 procedures they performed (by volume) that are not in our benchmarking tables - with a minimum of one per month. Even with my limited LINQ experience, I have managed to get this working, but the performance in abysmal. In the spirit of becoming better at LINQ, I thought I'd see if there was a better way to approach this (besides coding in SQL).
Here is the entire method, although I believe it is the final statement (var report =) that deserves the most attention.
//Get the Providers for the selected AccessLevel & Report Period
var providerRiskLevels = GetProviderRiskLevelForAccessLevel(userAccessLevelId, reportPeriodId, providerNamePiece);
short benchmarkYear = (
from rp in db.ReportPeriods
where rp.ReportPeriodId == reportPeriodId
select rp.BenchmarkDataYear
).FirstOrDefault();
byte Duration = (
from rp in db.ReportPeriods
where rp.ReportPeriodId == reportPeriodId
select rp.Duration
).FirstOrDefault();
//Summarize counts by procedure code (ignoring modifiers)
var codeCounts =
from pp in db.ProviderProductions
//332 - restrict to valid codes
join pc in db.ProcedureCodes on pp.ProcedureCode equals pc.ProcedureCode1
where pp.ReportPeriodId == reportPeriodId
//show deleted codes as potential issue
&& pc.TerminatedDate == null
&& (codeFilter == null || pp.ProcedureCode == codeFilter)
join pc in db.PspsProcedures
on new { col1 = pp.ProcedureCode, col2 = benchmarkYear } equals new { col1 = pc.ProcedureCode, col2 = pc.Year } into left
from l_d in left.DefaultIfEmpty()
where l_d == null
group pp by
new { pp.ProviderId, pp.ProcedureCode }
into g
orderby g.Key.ProviderId, g.Sum(x => x.Volume) descending
where g.Sum(x => x.Volume) > Duration
select new NewCodesEntity
{
ProviderId = g.Key.ProviderId,
ProcedureCode = g.Key.ProcedureCode,
Volume = g.Sum(x => x.Volume)
};
//Get top 5 procedures performed by provider
var newCodes =
from cc in codeCounts
group cc by
new { cc.ProviderId }
into g
select new ProviderNewCodesEntity
{
ProviderId = g.Key.ProviderId,
Codes = g.OrderByDescending(x => x.Volume).Take(5).ToList()
};
//Build the report
var report =
from pr in providerRiskLevels
join nc in newCodes on pr.Provider.ProviderId equals nc.ProviderId
where pr.RiskCategoryId == RiskCategoryIds.VisibleRisk
&& (filterRiskLevelNums.Contains(pr.RiskLevelNum))
&& (filterSpecialtyId == 0 || pr.Provider.Specialty.SpecialtyId == filterSpecialtyId)
select new ProbeAuditEntity
{
ProviderId = pr.Provider.ProviderId,
ProviderName = pr.Provider.Name,
ProviderCode = pr.Provider.ProviderCode,
SpecialtyName = pr.Provider.Specialty.Name,
SpecialtyCode = pr.Provider.Specialty.SpecialtyCode,
VisibleRisk = pr.RiskScore,
NewCode1 = nc.Codes.OrderByDescending(c => c.Volume).FirstOrDefault().ProcedureCode,
NewCode2 = nc.Codes.OrderByDescending(c => c.Volume).Skip(1).FirstOrDefault().ProcedureCode,
NewCode3 = nc.Codes.OrderByDescending(c => c.Volume).Skip(2).FirstOrDefault().ProcedureCode,
NewCode4 = nc.Codes.OrderByDescending(c => c.Volume).Skip(3).FirstOrDefault().ProcedureCode,
NewCode5 = nc.Codes.OrderByDescending(c => c.Volume).Skip(4).FirstOrDefault().ProcedureCode
};
return report;
I appreciate your suggestions.

Give this a try for the report:
//Build the report
var report =
(from pr in providerRiskLevels
join nc in newCodes on pr.Provider.ProviderId equals nc.ProviderId
where pr.RiskCategoryId == RiskCategoryIds.VisibleRisk
&& filterRiskLevelNums.Contains(pr.RiskLevelNum)
&& (filterSpecialtyId == 0
|| pr.Provider.Specialty.SpecialtyId == filterSpecialtyId)
select new
{
RiskLevel = pr,
NewCodes = nc.Codes.OrderByDescending(c => c.Volumn).Select(c => c.ProcedureCode).Take(5)
})
.AsEnumerable()
.Select(x => new ProbeAuditEntity
{
ProviderId = x.RiskLevel.Provider.ProviderId,
ProviderName = x.RiskLevel.Provider.Name,
ProviderCode = x.RiskLevel.Provider.ProviderCode,
SpecialtyName = x.RiskLevel.Provider.Specialty.Name,
SpecialtyCode = x.RiskLevel.Provider.Specialty.SpecialtyCode,
VisibleRisk = x.RiskLevel.RiskScore,
NewCodes = x.NewCodes.Concat(Enumerable.Repeat(<DefaultProcedureCodeHere>, 5)).Take(5)
});
Or, if ProbeAuditEntity is a class you could do this:
//Build the report
var report =
(from pr in providerRiskLevels
join nc in newCodes on pr.Provider.ProviderId equals nc.ProviderId
where pr.RiskCategoryId == RiskCategoryIds.VisibleRisk
&& filterRiskLevelNums.Contains(pr.RiskLevelNum)
&& (filterSpecialtyId == 0
|| pr.Provider.Specialty.SpecialtyId == filterSpecialtyId)
select new ProbeAuditEntity
{
ProviderId = x.RiskLevel.Provider.ProviderId,
ProviderName = x.RiskLevel.Provider.Name,
ProviderCode = x.RiskLevel.Provider.ProviderCode,
SpecialtyName = x.RiskLevel.Provider.Specialty.Name,
SpecialtyCode = x.RiskLevel.Provider.Specialty.SpecialtyCode,
VisibleRisk = x.RiskLevel.RiskScore,
NewCodes = nc.Codes.OrderByDescending(c => c.Volumn).Select(c => c.ProcedureCode).Take(5)
})
.ToList();
report
.ForEach(x => x.NewCodes = x.NewCodes.Concat(Enumerable.Repeat(<DefaultProcedureCodeHere>, 5)).Take(5);

This LINQ-to-Entities join seems overly complex, am I doing something wrong?

I have two queries that I need to combine in LINQ that both actually come from the same table. The reason for this is that one of the queries needs to get the max of a field for each day and then sum the days together where the second query can just sum everything right off the bat. Here is the first query:
var queryDownload = from p in
(from p in almdbContext.cl_contact_event
where p.time_of_contact >= startDate && p.time_of_contact < endDate && p.input_file_name.Contains(inputFileName) && listName.Contains(listName)
group p by new
{
date = EntityFunctions.CreateDateTime(p.time_of_contact.Value.Year, p.time_of_contact.Value.Month, p.time_of_contact.Value.Day, 0, 0, 0),
listName = p.contact_list_name
} into g
select new
{
date = g.Key.date,
listName = g.Key.listName,
download = g.Max(a => a.total_number_of_records)
})
group p by p.listName into g
select new
{
listName = g.Key,
totalDownload = g.Sum(a => a.download),
};
This is the second:
var queryPenData = from p in almdbContext.cl_contact_event
where p.time_of_contact >= startDate && p.time_of_contact < endDate && p.input_file_name.Contains(inputFileName) && listName.Contains(listName)
group p by p.contact_list_name into g
select new
{
listName = g.Key,
dials = g.Sum(a => a.ov_dial_start_time != null ? 1 : 0),
agentConnects = g.Sum(a => a.agent_login_name != null ? 1 : 0),
abandons = g.Sum(a => a.response_status == "DAC" || a.response_status == "DAD" ? 1 : 0),
rightPartyContacts = g.Sum(a => a.response_status == "PTP" || a.response_status == "RPC" ? 1 : 0),
promiseToPays = g.Sum(a => a.response_status == "PTP" ? 1 : 0),
talkTime = g.Sum(a => EntityFunctions.DiffSeconds(a.ov_call_connected_time, a.ov_trunk_released_time)) ?? 0,
wrapTime = g.Sum(a => EntityFunctions.DiffSeconds(a.ov_trunk_released_time, a.record_released_time)) ?? 0
};
And this is the query joining them together.
var queryJoin = from qd in queryDownload
join qp in queryPenData
on qd.listName equals qp.listName
select new
{
listName = qp.listName,
download = qd.totalDownload,
dials = qp.dials,
agentConnects = qp.agentConnects,
abandons = qp.abandons,
rightPartyContacts = qp.rightPartyContacts,
promiseToPays = qp.promiseToPays,
talkTime = qp.talkTime,
wrapTime = qp.wrapTime
};
This seems extremely verbose/roundabout to me. Is there a better way I can write/approach this to shrink/simplify the code?

For your last query could you not just do something like this?
var queryJoin = from qd in queryDownload join qpd in queryPenData on qd.listname equals qpd....
Should be able to do this:
var queryJoin = from qd in queryDownload
join qp in queryPenData
on qd.listName equals qp.listName
select new
{
qp, qd
};
From what I can tell you've already formed your data in queryPenData so there is no reason to re-assign it to new variables in the final join. Just select the object which will allow you to traverse into the anonymous type in queryPenData. I think...