LINQ to Entity query execution time - c#

I have this LINQ to Entity query which fetches around 5000 Plus records. I have been noticed is that the higher the records returned the more time it takes to execute the query and return the results.
lstUserAssignments = context.UserAssessments.Where(a=>a.UserID == UserID && a.CourseID == ClassID && a.ProductID == a.ProductID && a.AppID == ApplicationID && a.IsDeleted == false && a.Score_Percentage >= 0).Select(a=> new UserAssessmentEntity()
{
ApplicationID = a.AppID,
AttemptNo = a.AttemptNo,
CourseID = a.CourseID,
Score_Percentage = (float)(a.Score_Percentage != null ? a.Score_Percentage : 0),
Status = a.Status,
UserAssessmentID = a.UserAssessmentID,
UserID = a.UserID,
UserScore = a.User_Score,
CreatedDateTime = a.CreatedDateTime,
ModifiedDateTime = a.ModifiedDateTime,
TimeSpent=a.UserAssessmentDetails.Sum(i=>i.TimeSpent??0),
InstructorFeedbackText = a.UserAssessmentDetails.FirstOrDefault()!=null?a.UserAssessmentDetails.FirstOrDefault().InstructorFeedbackText:string.Empty
}).ToList();
To make the query simple. There is UserAssessment and UserAssessmentDetails table. The first one is the parent and the latter the child. There is some performance issue in the query and i feel its in calculating TimeSpent and InstructorFeedback.
Can anyone point my finger on the problem.

Take and Skip:
context.UserAssessments.Where(a=>a.UserID == UserID && a.CourseID == ClassID && a.ProductID == a.ProductID && a.AppID == ApplicationID && a.IsDeleted == false && a.Score_Percentage >= 0).Select(a=> new UserAssessmentEntity()
{
ApplicationID = a.AppID,
AttemptNo = a.AttemptNo,
CourseID = a.CourseID,
Score_Percentage = (float)(a.Score_Percentage != null ? a.Score_Percentage : 0),
Status = a.Status,
UserAssessmentID = a.UserAssessmentID,
UserID = a.UserID,
UserScore = a.User_Score,
CreatedDateTime = a.CreatedDateTime,
ModifiedDateTime = a.ModifiedDateTime,
TimeSpent=a.UserAssessmentDetails.Sum(i=>i.TimeSpent??0),
InstructorFeedbackText = a.UserAssessmentDetails.FirstOrDefault()!=null?a.UserAssessmentDetails.FirstOrDefault().InstructorFeedbackText:string.Empty
}).Take(10).Skip(10).ToList();

Related

42P18: could not determine data type of parameter $12

I'm trying to run this query, but it tells me this error:
42P18: could not determine data type of parameter $12
bds.DataSource = (from vendas in conexao.Vendas
join nota in conexao.NotaFiscal on vendas.IdNotaFIscal equals nota.Id
join clientes in conexao.Cliente on vendas.IdCliente equals clientes.Id
where (nota.DataEmissao >= periodoInicial || periodoInicial == null) &&
(nota.DataEmissao <= periodoFinal || periodoFinal == null) &&
(clientes.Id == idCliente || idCliente == null) &&
(nota.NumeroNotaFiscal >= notaInicial || notaInicial == null) &&
(nota.NumeroNotaFiscal <= notaFinal || notaFinal == null) &&
(nota.Modelo == mod || mod == null)
orderby nota.Id descending
select new
{
Id = nota.Id,
ClienteId = clientes.NomeRazao,
Chave = nota.Chave,
DataEmissao = nota.DataEmissao,
Status = nota.Status,
NumeroNota = nota.NumeroNotaFiscal,
Modelo = nota.Modelo,
}).ToList();
The error occurs in this line below, if I withdraw, works normal, I have tried to change, however I need this filter.I can't understand what I'm doing wrong.
(nota.Modelo == mod || mod == null)
And here's how I fill out the variable
string mod = String.Empty;
if(modelo != "TODOS") mod = modelo == "NF-E" ? "55" : modelo== "NFC-e" ? "65" : null;
Any idea how I can fix it? Thank you.
Try to separate filtering. Such parameters is bad for LINQ Translation and performance. I hope it will solve your problem.
var query =
from vendas in conexao.Vendas
join nota in conexao.NotaFiscal on vendas.IdNotaFIscal equals nota.Id
join clientes in conexao.Cliente on vendas.IdCliente equals clientes.Id
select new { vendas, nota, clientes };
if (periodoInicial != null)
query = query.Where(x => x.nota.DataEmissao >= periodoInicial);
if (periodoFinal != null
query = query.Where(x => x.nota.DataEmissao <= periodoFinal);
if (idCliente != null)
query = query.Where(x => x.clientes.Id == idCliente);
if (notaInicial != null)
query = query.Where(x => x.nota.NumeroNotaFiscal >= notaInicial);
if (notaFinal != null)
query = query.Where(x => x.nota.NumeroNotaFiscal <= notaFinal);
if (mod != null)
query = query.Where(x => x.nota.Modelo == mod);
var result =
from q in query
orderby q.nota.Id descending
select new
{
Id = q.nota.Id,
ClienteId = q.clientes.NomeRazao,
Chave = q.nota.Chave,
DataEmissao = q.nota.DataEmissao,
Status = q.nota.Status,
NumeroNota = q.nota.NumeroNotaFiscal,
Modelo = q.nota.Modelo,
};
bds.DataSource = result.ToList();

Nullable object must have a value in join tables query

I have three tables TableTeam, TableUser and TableLink. I want to join them to get a list.
var teams = _db.TableTeam;
var users = _db.TableUser;
var managerIds = teams.Select(x=>x.ManagerId).ToList();
var list = (from user in _db.TableUser
where user.EndDate == null && user.ManagerId != null && managerIds.Contains(user.ManagerId.Value)
let r = _db.TableLink.FirstOrDefault(l => l.ProjectId == projectId && l.UserId == user.UserId)
let t = _db.TableTeam.FirstOrDefault(m => m.ManagerId == user.ManagerId)
select new MyDto
{
ProjectId = projectId,
UserId = user.UserId,
IsAssigned = r.AssignedId > 0 ?true : false,
AssignedId = r.AssignedId,
Percent = r.Percent.HasValue ? (r.Percent.Value).ToString() + "%": ""
}).GroupBy(d => d.UserId).ToList();
For the table TableLink, the corresponding dataSet class in entity framework is:
public LinkDto
{
public int AssignedId {get;set;}
public short ProjectId {get;set;}
public short UserId {get;set;}
public decimal? Percent {get;set;}
// ...
}
However I got the error:
Nullable object must have a value
I think that it may be caused by decimal? Percent. But not sure how to fix it. Also I can use the store d procedure as well, which means if you can solve it by SQL query, that is also great.
It turns out I have to move let codes before where clause then add the condition to where.
Such as
var list = (from user in _db.TableUser
let r = _db.TableLink.FirstOrDefault(l => l.ProjectId == projectId && l.UserId == user.UserId)
let t = _db.TableTeam.FirstOrDefault(m => m.ManagerId == user.ManagerId)
where user.EndDate == null && user.ManagerId != null && managerIds.Contains(user.ManagerId.Value)
&& r != null && t != null
select new MyDto
{
ProjectId = projectId,
UserId = user.UserId,
IsAssigned = r.AssignedId > 0 ?true : false,
AssignedId = r.AssignedId,
Percent = r.Percent.HasValue ? (r.Percent.Value).ToString() + "%": ""
}).GroupBy(d => d.UserId).ToList();

Slow LINQ Query on Join

I'm using EF 6 with .NET Core. I have written the following query to select some data into a custom object. At the moment the main query returns over 25000 records. Then the filters are applied. This query is running very slow. I'm just trying to find out if I'm doing any mistakes here. Why it could be so slow and is the filtering happening on the memory rather than on the database side?
public IQueryable<TicketViewModel> GetTickets(int companyId, string locale, TicketFilterViewModel filters, Guid customerRef, int? take, int currentUser)
{
IQueryable<TicketViewModel> tickets = (from to in _context.Ticket
join co in _context.Company on to.CompanyId equals co.CompanyId
join con in _context.Contact on to.ContactId equals con.ContactId
join site in _context.Site on to.SiteId equals site.SiteId
join country in _context.Country on site.CountryId equals country.CountryId
join cus in _context.Customer on con.CustomerId equals cus.CustomerId
join tic_type in _context.TicketType on to.TicketTypeId equals tic_type.TicketTypeId
join ts in _context.TicketStatus on to.TicketStatusId equals ts.TicketStatusId
join sb in _context.ServiceBoard on to.ServiceBoardId equals sb.ServiceBoardId into ob1
from a in ob1.DefaultIfEmpty()
join u in _context.User on to.TechnitianId equals u.Id into ob2
from b in ob2.DefaultIfEmpty()
join pr in _context.Priority on to.PriorityId equals pr.PriorityId into ob3
from c in ob3.DefaultIfEmpty()
where to.CompanyId == companyId && (customerRef == Guid.Empty || cus.RefNo == customerRef)
&& to.MergedIntoTicketId == null
select new TicketViewModel
{
CreatedOn = Helpers.Custom.UtcToStandardTime(locale, to.AddedOnUtc).ToString("dd/MM/yyyy hh:mm tt"),
CustomerName = cus.CustomerName,
TicketNumber = to.TicketNumber,
TicketTitle = to.TicketTitle,
RefNo = to.RefNo,
CompanyName = co.CompanyName,
ContactName = string.Concat(con.FirstName, " ", con.LastName),
SiteAddress = String.Concat(site.AddressLine1, ", ", site.AddressLine2, ", ", site.Suburb, ", ", site.State, ", ", site.PostCode, ", ", country.Name),
ServiceBoardName = a.BoardName,
TechnicianName = string.Concat(b.FirstName, " ", b.LastName),
PriorityName = c.PriorityName,
TicketStatus = ts.StatusName, //d.StatusName,
ServiceBoardId = to.ServiceBoardId,
TicketStatusId = to.TicketStatusId,
CustomerId = cus.CustomerId,
PriorityId = to.PriorityId,
ContractId = site.ContractId,
TechnitianId = to.TechnitianId,
TicketId = to.TicketId,
StatusCategoryId = ts.CategoryId,//,//d.CategoryId,
DueOnUtc = to.DueOnUtc,
DefaultStatusId = ts.DefaultId,//d.DefaultId,
ClosedOnUtc = to.ClosedOnUtc,
ResolvedOnUtc = to.ResolvedOnUtc,
InitialResponseMade = to.InitialResponseMade,
TicketTypeId = to.TicketTypeId,
TicketTypeName = tic_type.TicketTypeName
}).OrderByDescending(o => o.TicketNumber);
bool isFiltersHit = false;
if (filters != null)
{
if (tickets != null && tickets.Count() > 0 && filters.serviceboard_selectedItems != null && filters.serviceboard_selectedItems.Count > 0)
{
isFiltersHit = true;
tickets = tickets.Where(x => x.ServiceBoardId != null && filters.serviceboard_selectedItems.Select(o => o.serviceBoardId).Contains(x.ServiceBoardId.Value));
}
if (tickets != null && tickets.Count() > 0 && filters.status_selectedItems != null && filters.status_selectedItems.Count > 0)
{
isFiltersHit = true;
tickets = tickets.Where(x => x.TicketStatusId != null && filters.status_selectedItems.Select(o => o.ticketStatusId).Contains(x.TicketStatusId.Value));
}
if (tickets != null && tickets.Count() > 0 && filters.type_selectedItems != null && filters.type_selectedItems.Count > 0)
{
isFiltersHit = true;
tickets = tickets.Where(x => filters.type_selectedItems.Select(o => o.ticketTypeId).Contains(x.TicketTypeId));
}
if (tickets != null && tickets.Count() > 0 && filters.technician_selectedItems != null && filters.technician_selectedItems.Count > 0)
{
isFiltersHit = true;
tickets = tickets.Where(x => x.TechnitianId != null && filters.technician_selectedItems.Select(o => o.id).Contains(x.TechnitianId.Value));
}
if (tickets != null && tickets.Count() > 0 && filters.customer_selectedItems != null && filters.customer_selectedItems.Count > 0)
{
isFiltersHit = true;
tickets = tickets.Where(x => x.CustomerId != 0 && filters.customer_selectedItems.Select(o => o.customerId).Contains(x.CustomerId));
}
if (tickets != null && tickets.Count() > 0 && filters.priority_selectedItems != null && filters.priority_selectedItems.Count > 0)
{
isFiltersHit = true;
tickets = tickets.Where(x => x.PriorityId != null && filters.priority_selectedItems.Select(o => o.priorityId).Contains(x.PriorityId.Value));
}
if (tickets != null && tickets.Count() > 0 && filters.contract_selectedItems != null && filters.contract_selectedItems.Count > 0)
{
isFiltersHit = true;
tickets = tickets.Where(x => x.ContractId != null && filters.contract_selectedItems.Select(o => o.contractId).Contains(x.ContractId.Value));
}
if (tickets != null && tickets.Count() > 0 && filters.source_selectedItems != null && filters.source_selectedItems.Count > 0)
{
isFiltersHit = true;
tickets = tickets.Where(x => x.SourceId != 0 && filters.source_selectedItems.Select(o => o.item_id).Contains(x.SourceId));
}
}
if (take.HasValue)
return tickets.Take(take.Value);
return tickets;
}
If you compose the query based on your conditions from the filters, you will see a big improvement. Meaning, create a base query ( base and minimum conditions ) and then add the conditions one by one, depending on what your filters are. Execute the query after all filters are checked.
P.S: you don't really need to call tickets.Count() on every filter. Do it once, in the beginning.

Why Same where clause need to write multiple times in a Linq Query for following SQL

What will be the proper linq syntax of below SQL Query ?
select a.id, a.AppointmentStatusID, ad.ID as DetailID
from [dbo].[Appointment] a, [dbo].[AppointmentDetail] ad
where a.[ID] = ad.[AppointmentID]
and a.CompanyID = 'a3dea87a-804e-4115-98cf-472988cf1678'
and a.LocationID = '3165caca-2a48-46f0-bbed-578cff29167t'
and ad.AppDateFrom <= {ts '2017-11-14 23:59:31'}
and ad.AppDateTo >= {ts '2017-11-14 00:00:00'}
and ad.[ApprovalStatusID] = 2
Problem I faced:
I required to filter Where Condition two times 1st at the time within the join & 2nd time during the object.Select expression, please check bellow
var results = (from a in appointments
join ad in _appointmentDetailRepository.GetAll() on a.ID equals ad.AppointmentID
where ad.ApprovalStatusID == 2
&& DbFunctions.TruncateTime(ad.AppDateFrom) <= DbFunctions.TruncateTime(viewmodel.AppointmentDate)
&& DbFunctions.TruncateTime(ad.AppDateTo) >= DbFunctions.TruncateTime(viewmodel.AppointmentDate)
orderby a.ID
select new Appointment
{
ID = a.ID,
CompanyID = a.CompanyID,
LocationID = a.LocationID,
AppointmentDetail = a.AppointmentDetail.Select(ad => new AppointmentDetail
{
ID = ad.ID,
AppDateFrom = ad.AppDateFrom,
AppDateTo = ad.AppDateTo,
AppointmentStatusID = ad.AppointmentStatusID,
}).Where(ad=> ad.ApprovalStatusID == 2
&& DbFunctions.TruncateTime(ad.AppDateFrom) <= DbFunctions.TruncateTime(viewmodel.AppointmentDate)
&& DbFunctions.TruncateTime(ad.AppDateTo) >= DbFunctions.TruncateTime(viewmodel.AppointmentDate)).ToList()
}).GroupBy(x => x.ID).Select(x => x.DefaultIfEmpty().FirstOrDefault());
Query : Why I required to write Where clause 2 times ?
Required Result
An Appointment Object --> Containing ICollection<AppoinmentDetails> if Details.Where Condition == True
From what I see (without knowing the model you have), it looks like you should use the already joined and filtered Details from ad instead of looking it up again from the Property a.AppointmentDetail...
Untested:
select new Appointment
{
ID = a.ID,
CompanyID = a.CompanyID,
LocationID = a.LocationID,
AppointmentDetail = ad.ToList(), // <-- don't you think?
...
}
For the given SQL query
select a.id, a.AppointmentStatusID, ad.ID as DetailID
from [dbo].[Appointment] a, [dbo].[AppointmentDetail] ad
where a.[ID] = ad.[AppointmentID]
and a.CompanyID = 'a3dea87a-804e-4115-98cf-472988cf1678'
and a.LocationID = '3165caca-2a48-46f0-bbed-578cff29167t'
and ad.AppDateFrom <= {ts '2017-11-14 23:59:31'}
and ad.AppDateTo >= {ts '2017-11-14 00:00:00'}
and ad.[ApprovalStatusID] = 2
LINQ query can be written as
var results = (from a in appointments
join ad in appointmentDetails on a.ID equals ad.AppointmentID
where ad.ApprovalStatusID == 2
&& a.CompanyID == "a3dea87a-804e-4115-98cf-472988cf1678"
&& a.LocationID == "3165caca-2a48-46f0-bbed-578cff29167t"
&& ad.AppDateFrom.Date <= viewmodel.AppointmentDate.Date
&& ad.AppDateTo.Date >= viewmodel.AppointmentDate.Date
select new
{
ID = a.ID,
AppointmentStatusID = a.AppointmentStatusID,
DetailID = ad.ID
}).ToList();
You can also write it like
var results = appointmentDetails
.Where(ad => ad.AppDateFrom.Date <= viewmodel.AppointmentDate.Date
&& ad.AppDateTo.Date >= viewmodel.AppointmentDate.Date
&& ad.ApprovalStatusID == 2
&& ad.Appointment.CompanyID == "a3dea87a-804e-4115-98cf-472988cf1678"
&& ad.Appointment.LocationID == "3165caca-2a48-46f0-bbed-578cff29167t")
.Select(ad =>
new
{
ID = ad.Appointment.ID,
AppointmentStatusID = ad.Appointment.AppointmentStatusID,
DetailID = ad.ID
})
.ToList()
As per the updated question, to get the the Appointment object with a collection of AppointmentDetails, please try this query
var results = appointmentDetails
.Where(ad => ad.AppDateFrom.Date <= viewmodel.AppointmentDate.Date
&& ad.AppDateTo.Date >= viewmodel.AppointmentDate.Date
&& ad.ApprovalStatusID == 2
&& ad.Appointment.CompanyID == "a3dea87a-804e-4115-98cf-472988cf1678"
&& ad.Appointment.LocationID == "3165caca-2a48-46f0-bbed-578cff29167t")
.Select(ad =>
new
{
ID = ad.Appointment.ID,
AppointmentStatusID = ad.Appointment.AppointmentStatusID,
Detail = ad
})
.AsEnumerable()
.GroupBy(a => new { a.ID, a.AppointmentStatusID })
.Select(a => new Appointment
{
ID = a.Key.ID,
AppointmentStatusID = a.Key.AppointmentStatusID,
AppointmentDetails = a.Select(d => d.Detail).ToList()
})
.ToList();

The entity or complex type cannot be constructed in a LINQ to Entities query

On our online billing application, we give a billing summary of what bills the customer received and the payments they made.
In order for this to work, I have to first pull the payments then match them to the bills. So I have do something like:
foreach (BillPaymentSummary payment in billPayments)
{
DateTime dt = payment.DueDate;
// Debug errors on this next line
var summary = (from a in db.BillHistories
where a.CustomerId == customerNumber && a.DueDate == dt && a.Type == "BILL"
select new BillSummary
{
Id = a.Id,
CustomerId = a.CustomerId,
DueDate = a.DueDate,
PreviousBalance = a.PreviousBalance.Value,
TotalBill = a.TotalBill.Value,
Type = a.Type,
IsFinalBill = a.IsFinalBill
}).SingleOrDefault();
if (summary != null)
{
summary.PayDate = payment.PaidDate;
summary.AmountPaid = payment.AmountPaid;
returnSummaries.Add(summary);
}
else
{
summary = (from a in db.BillHistories
where a.CustomerId == customerNumber && a.DueDate == payment.DueDate && a.Type == "ADJ "
select new BillSummary
{
Id = a.Id,
CustomerId = a.CustomerId,
DueDate = a.DueDate,
PreviousBalance = a.PreviousBalance.Value,
TotalBill = a.TotalBill.Value,
Type = a.Type,
IsFinalBill = a.IsFinalBill
}).SingleOrDefault();
if (summary != null)
{
summary.PayDate = payment.PaidDate;
summary.AmountPaid = payment.AmountPaid;
returnSummaries.Add(summary);
}
}
}
I have been playing with this, but no matter what I do, I get the following error message:
The entity or complex type 'UtilityBill.Domain.Concrete.BillSummary' cannot be constructed in a LINQ to Entities query.
Is it because I am running queries within queries? How can I get around this error?
I have tried searching Google for an answer and see many answers, but none of them seem to explain my problem.
You cannot project onto a mapped entity. You would have to call ToList() before doing your mapping.
Or better yet, change to the following (calling FirstOrDefault will execute the query and allow you to populate your object):
var summary = db.BillHistories.FirstOrDefault(a => a.CustomerId == customerNumber && a.DueDate == dt && a.Type == "BILL").Select(x => new BillSummary
{
Id = a.Id,
CustomerId = a.CustomerId,
DueDate = a.DueDate,
PreviousBalance = a.PreviousBalance.Value,
TotalBill = a.TotalBill.Value,
Type = a.Type,
IsFinalBill = a.IsFinalBill
});
To decouple yourself from the Entity Framework you may want to also consider using a different model class to return instead of the Entity Framework model.
What I ended up doing was:
foreach (BillPaymentSummary payment in billPayments)
{
var data = db.BillHistories.Where(b => b.CustomerId == customerNumber && b.DueDate == payment.DueDate && b.Type == "B").FirstOrDefault();
if (data != null) // There is a bill history
{
returnSummaries.Add(new BillSummary
{
Id = data.Id,
CustomerId = data.CustomerId,
DueDate = data.DueDate,
PreviousBalance = data.PreviousBalance,
TotalBill = data.TotalBill,
Type = (data.Type.Trim() == "B" ? "BILL" : (data.Type == "A" ? "ADJ" : "")),
IsFinalBill = data.IsFinalBill,
PayDate = payment.PaidDate,
AmountPaid = payment.AmountPaid
});
}
else // No bill history record, look for an adjustment
{
data = db.BillHistories.FirstOrDefault(b => b.CustomerId == customerNumber && b.DueDate == payment.DueDate && b.Type == "A");
if (data != null)
{
returnSummaries.Add(new BillSummary
{
Id = data.Id,
CustomerId = data.CustomerId,
DueDate = data.DueDate,
PreviousBalance = data.PreviousBalance,
TotalBill = data.TotalBill,
Type = (data.Type.Trim() == "B" ? "BILL" : (data.Type == "A" ? "ADJ" : "")),
IsFinalBill = data.IsFinalBill,
PayDate = payment.PaidDate,
AmountPaid = payment.AmountPaid
});
}
}
db.SaveChanges();
}

Categories