I have a function that is called to retrieve transactions by a specified start and end date. I am now updating this function to also return ALL transactions(start/end date are null) for certain cases.
When start and end date are null (which are properties of the TransactionsCriteria class) I need the last code block in the snippet below executed. I can't wrap that snippet of code in an if statement since I need it executed for other cases as well, when start and end date are NOT null.
How do I accomplish implementing this logic with the below code block?
Thank you.
public async Task<(int total, List<Transaction> records)> GetAllTransactionsAsync(int fundId, TransactionsCriteria criteria)
{
IQueryable<Transaction> transactions = _db.Transactions.Include(t => t.DepositSource).Include(t => t.WithdrawalSource).Include(t => t.ReconciliationReport).Where(t => t.FundId == fundId);
if (criteria.UseClearedDate)
{
transactions = transactions.Where(t => t.ReconciliationReport != null && (t.ReconciliationReport.DateCreated.Date >= criteria.StartDate.Date
&& t.ReconciliationReport.DateCreated.Date <= criteria.EndDate.Date));
}
if (criteria.Reconcile)
{
transactions = transactions.Where(t =>
t.Date.Date <= criteria.EndDate.Date);
}
else
{
transactions = transactions.Where(t =>
t.Date.Date >= criteria.StartDate.Date
&& t.Date.Date <= criteria.EndDate.Date);
}
transactions = transactions.Where(t => (criteria.Statuses.Contains(t.ReconciliationId == null ? t.IsVoided ? TransactionStatus.Void : TransactionStatus.Outstanding : TransactionStatus.Cleared))
&& criteria.Sources.Contains(t.TransactionSource) && criteria.Types.Contains(t.TransactionType));
By virtue of it being an if...else statement, matching the condition criteria.StartDate == null && criteria.EndDate == null causes it to enter the if block and skip the else block.
Related
I have a complex where clause in my EF linq statement which repeats a subquery expression, on _db.OPESRRecoveryElements, but with different parameters, one of which is depending on records from the main entity, OPCases/OPCaseDto.
The query as it is works, but its hard for people to read. Ideally I'd like to be able to create an expression which could be re-used at the 3 necessary points and would still allow it to execute as a single, server-side SQL statement.
Is there a way to create an Expression / IQueryable definition which can be used for a subquery like this?
List<OPCaseDto> opCases = await _db.OPCases
.ProjectTo<OPCaseDto>(_autoMapperConfig, null, requestedExpands)
.Where(c =>
c.OPStatusId == OPStatusIds.AwaitingRecoveryElement
&& (
(c.OPCategoryLetter == "B"
// Only need a gross pensionable element if there is an outstanding gross pensionable figure
&& (c.GrossOverpaidPensionable - c.GrossRecoveredPensionable == 0
|| _db.OPESRRecoveryElements.Any(e => !e.NonPensionable && e.OPRecoveryMethod.OPTypeLetter == "G"
&& !e.OPRecoveryPlans.Any(rp
=> (rp.RecoveryStatus == OPRecoveryStatuses.NotStarted || rp.RecoveryStatus == OPRecoveryStatuses.InRecovery)
&& rp.AssignmentNo == c.RecoveryAssignmentNo)))
// Only need a gross non-pensionable element if there is an outstanding gross non-pensionable figure
&& (c.GrossOverpaidNonPensionable - c.GrossRecoveredNonPensionable == 0
|| _db.OPESRRecoveryElements.Any(e => e.NonPensionable && e.OPRecoveryMethod.OPTypeLetter == "G"
&& !e.OPRecoveryPlans.Any(rp
=> (rp.RecoveryStatus == OPRecoveryStatuses.NotStarted || rp.RecoveryStatus == OPRecoveryStatuses.InRecovery)
&& rp.AssignmentNo == c.RecoveryAssignmentNo))))
|| (c.OPCategoryLetter == "D"
// Don't need to check for an outstanding net figure - if the case is net and isn't complete, there will be one!
&& _db.OPESRRecoveryElements.Any(e => e.OPRecoveryMethod.OPTypeLetter == "N"
&& !e.OPRecoveryPlans.Any(rp
=> (rp.RecoveryStatus == OPRecoveryStatuses.NotStarted || rp.RecoveryStatus == OPRecoveryStatuses.InRecovery)
&& rp.AssignmentNo == c.RecoveryAssignmentNo)))
)
)
.AsNoTracking()
.ToListAsync();
If it wasn't for the c.RecoveryAssignmentNo part, I could easily create an expression like:
public Expression<Func<OPESRRecoveryElement, bool>> NoActiveRecoveryPlans(string opType, bool nonPen)
{
return e => e.OPRecoveryMethod.OPTypeLetter == opType
&& e.NonPensionable == nonPen
&& !e.OPRecoveryPlans.Any(rp
=> (rp.RecoveryStatus == OPRecoveryStatuses.NotStarted || rp.RecoveryStatus == OPRecoveryStatuses.InRecovery));
}
and use it like:
(c.OPCategoryLetter == "B"
// Only need a gross pensionable element if there is an outstanding gross pensionable figure
&& (c.GrossOverpaidPensionable - c.GrossRecoveredPensionable == 0
|| _db.OPESRRecoveryElements.Any(NoActiveRecoveryPlans("G", false)))
and it would get executed before the query to get the OPCases.
I could also fetch all the OPCaseDto records and OPESRRecoveryElements as separate queries and filter in memory, but I don't want to do that.
If I add a parameter to the function, string assignmentNo, I (unsurprisingly) get an error - "Unable to cast object of type 'System.Linq.Expressions.InstanceMethodCallExpression3' to type 'System.Linq.Expressions.LambdaExpression'"
I am developing a hospitality domain application (.Net Core 2.2) in which i am developing a reporting module. From a dashboard few filters are available to fetch records from Data Base.
Below is the DTO i am using to contains filters
public class SearchDto
{
public DateTime DateForm {get;set;}
public DateTime DateTo {get;set;}
public DateSearchType SearchType {get;set;}
public string RegionId {get;set;}
public OrdersStatus status {get;set;}
public string PaymentModeTypes {get;set;}
public string channel {get;set;}
}
Here DateSearchType is a enum with value
Start // service Start Date
End // service End Date
Creation // Order Creation Date
Also OrdersStatus (an enum) with values like All , Confirmed , Cancelled , PaymnetFailed etc
PaymentModeTypes can be a single string or comma seprated string for ex : "NetBanking, CreditCard, DebitCard, Cash"
RegionId is also a single string or comma seprated string as "101, 102, 102"
Same for Channel either "Web" or "Web, Mobile"
Curently ef core expressssion i am using is as follow
var v = Database.Orders.List(
x => ((SearchType == DateSearchType.Start) ? x.Services.Any(y => (y.MinStartTime >= DateForm.Date && y.MinStartTime <= DateTo.Date)) || DateForm.Date.Equals(DateTime.MinValue) : true)
&& ((SearchType == DateSearchType.End) ? x.Services.Any(y => y.MaxEndTime >= DateForm.Date && y.MaxEndTime <= DateTo.Date) : true)
&& ((SearchType == DateSearchType.Creation) ? x.BookingDate.Date >= DateForm.Date && x.BookingDate.Date <= DateTo.Date : true)
&& (RegionId.Length == 0 || RegionId.Contains(x.RegionId))
&& (status == OrdersStatus.All || x.Status == status)
&& (string.IsNullOrEmpty(PaymentModeTypes) || PaymentTypes.Contains(x.PaymentType))
&& (string.IsNullOrEmpty(channel) || channels.Contains(x.ChannelName)), //Where
x => x.Guests,
x => x.Services
);
Here Guests and Services are two another tables set as navigation property in Orders Model
This expression is working fine but taking too much time to execute , any good approach to optimize it or right way to rewrite this code ?
Also what is the best practice to exclude any filter if its value is not provided.
Few Filters are not mandatory, they can be supplied or not, so query execution has to be of dynamic nature. current implementation if a filter value is not supplied
&& (string.IsNullOrEmpty(PaymentModeTypes) || PaymentTypes.Contains(x.PaymentType))
Can anybody suggest some good material or any piece of code regarding this so i can optimize it.
Please ignore Typo as i am not habitual to use dark theme
Edit 1:Client Evaluation set as off
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder
.ConfigureWarnings(warnings => warnings.Throw(RelationalEventId.QueryClientEvaluationWarning));
}
There is some inefficiency in the generated SQL which might be causing the performance issue.
Before EF Core one is expected to conditionally chain multiple Where calls or use some predicate builder utility to build conditionally Where predicate having only the necessary conditions.
This normally is not needed in EF Core, because it tries to automatically eliminate such conditions. It does that for logical expressions (|| , &&), but failing to do so for the conditional expressions (? :). So the solution is to replace the later with the equivalent logical expressions.
You are doing that for most of your conditions, but not for the first 3. So replace
x => ((SearchType == DateSearchType.Start) ? x.Services.Any(y => (y.MinStartTime >= DateForm.Date && y.MinStartTime <= DateTo.Date)) || DateForm.Date.Equals(DateTime.MinValue) : true)
&& ((SearchType == DateSearchType.End) ? x.Services.Any(y => y.MaxEndTime >= DateForm.Date && y.MaxEndTime <= DateTo.Date) : true)
&& ((SearchType == DateSearchType.Creation) ? x.BookingDate.Date >= DateForm.Date && x.BookingDate.Date <= DateTo.Date : true)
with
x => (SearchType != DateSearchType.Start || x.Services.Any(y => y.MinStartTime >= DateForm.Date && y.MinStartTime <= DateTo.Date) || DateForm.Date.Equals(DateTime.MinValue))
&& (SearchType != DateSearchType.End || x.Services.Any(y => y.MaxEndTime >= DateForm.Date && y.MaxEndTime <= DateTo.Date))
&& (SearchType != DateSearchType.Creation || x.BookingDate.Date >= DateForm.Date && x.BookingDate.Date <= DateTo.Date)
and see if that helps. For sure the generated SQL will be optimal.
Noticed that you are using different variable names for comma separated strings and Contains filters. So I'm assuming you have something like this
public IEnumerable<string> PaymentTypes => (PaymentModeTypes ?? "").Split(", ");
public IEnumerable<string> channels => (channel ?? "").Split(", ");
which is good and will generate SQL IN (...) conditions when necessary.
You might consider doing the same for regions, e.g. add
public IEnumerable<string> RegionIds => (RegionId ?? "").Split(", ");
and replace
&& (RegionId.Length == 0 || RegionId.Contains(x.RegionId))
with
&& (string.IsNullOrEmpty(RegionId) || RegionIds.Contains(x.RegionId))
I have a List property that I am setting like so:
testCard.LstSummaries =
db.Summaries.Where(
x =>
(x.AID == aId || x.AInformation.RegNumber == aRegNumber) && DbFunctions.TruncateTime(x.Day) == DateTime.Today.Date &&
x.deleted == false).ToList();
Then I have a conditional statement:
if (testCard.LstSummaries.Count > 0)
{
if (
testCard.LstSummaries.All(
x =>
(x.AID == aId || // ERROR HAPPENS ON THIS LINE
x.AInformation.RegNumber == aRegNumber) &&
DbFunctions.TruncateTime(x.Day) == DateTime.Today.Date && x.deleted == false))
{
// .... do something
}
I get an error:
This function can only be invoked from LINQ to Entities.
I want to avoid to make multiple calls to the database.. furthermore testCard.LstSummaries already has the values I am looking for.. but if I do this:
if (testCard.LstSummaries.Count > 0)
{
if (
db.Summaries.All(
x =>
(x.AID == aId || // NO ERROR
x.AInformation.RegNumber == aRegNumber) &&
DbFunctions.TruncateTime(x.Day) == DateTime.Today.Date && x.deleted == false))
{
// .... do something
}
I feel like making this call to the database is pointless because I would be retrieving the same results that are already stored in testCard.LstSummaries, but I can't invoke .All() because it's not LINQ to Entities.
Is there a workaround for this?
Problem is with DbFunctions.TruncateTime(x.Day), because it is converted to sql on runtime. Try to check without it.
I have faced a strange problem. When user comes to any page of my web app
I do check if user has permissions to access it, and provide trial period if its first time to come.
Here is my piece of code:
List<string> temp_workers_id = new List<string>();
...
if (temp_workers_id.Count > 6)
{
System.Data.SqlTypes.SqlDateTime sqlDate = new System.Data.SqlTypes.SqlDateTime(DateTime.Now.Date);
var rusers = dbctx.tblMappings.Where(tm => temp_workers_id.Any(c => c == tm.ModelID));
var permissions = dbctx.UserPermissions
.Where(p => rusers
.Any(ap => ap.UserID == p.UserID)
&& p.DateStart != null
&& p.DateEnd != null
&& p.DateStart <= sqlDate.Value
&& p.DateEnd >= sqlDate.Value);
if (permissions.Count() < 1)
{
permissions = dbctx.UserPermissions
.Where(p => rusers
.Any(ap => ap.UserID == p.UserID)
&& p.DateStart == null
&& p.DateEnd == null);
var used = dbctx.UserPermissions
.Where(p => rusers
.Any(ap => ap.UserID == p.UserID)
&& p.DateStart != null
&& p.DateEnd != null);
if (permissions.Count() > 0 && used.Count() < 1)
{
var p = permissions.First();
using (Models.TTTDbContext tdbctx = new Models.TTTDbContext())
{
var tp = tdbctx.UserPermissions.SingleOrDefault(tup => tup.UserID == p.UserID);
tp.DateStart = DateTime.Now.Date;
tp.DateEnd = DateTime.Now.Date.AddDays(60);
tdbctx.SaveChanges();
}
here the First() method throws exception:
Sequence contains no elements
how that even could be?
EDIT:
I dont think that user opens two browsers and navigate here at the same time, but could be the concurrency issue?
You claim you only found this in the server logs and didn't encounter it during debugging. That means that between these lines:
if (permissions.Count() > 0)
{
var p = permissions.First();
Some other process or thread changed your database, so that the query didn't match any documents anymore.
This is caused by permissions holding a lazily evaluated resource, meaning that the query is only executed when you iterate it (which Count() and First()) do.
So in the Count(), the query is executed:
SELECT COUNT(*) ... WHERE ...
Which returns, at that moment, one row. Then the data is modified externally, causing the next query (at First()):
SELECT n1, n2, ... WHERE ...
To return zero rows, causing First() to throw.
Now for how to solve that, is up to you, and depends entirely on how you want to model this scenario. It means the second query was actually correct: at that moment, there were no more rows that fulfilled the query criteria. You could materialize the query once:
permissions = query.Where(...).ToList()
But that would mean your logic operates on stale data. The same would happen if you'd use FirstOrDefault():
var permissionToApply = permissions.FirstOrDefault();
if (permissionToApply != null)
{
// rest of your logic
}
So it's basically a lose-lose scenario. There's always the chance that you're operating on stale data, which means that the next code:
tdbctx.UserPermissions.SingleOrDefault(tup => tup.UserID == p.UserID);
Would throw as well. So every time you query the database, you'll have to write the code in such a way that it can handle the records not being present anymore.
I am getting this error when I execute this line, through break points I detected this error.
Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
int? dressSerialNo;
var lstDress = (
from yy in currContext.OrderDressings
where yy.OrderID == this.OrderID
&& yy.OrderItemID == this.orderItemID
&& yy.ProductID == this.ProductID
select yy
).ToList();
if (lstDress.Count > 0)
{
dressSerialNo = (
from yy in lstDress
where yy.OrderID == this.OrderID
&& yy.OrderItemID == this.orderItemID
&& yy.ProductID == this.ProductID
select (int?)yy.SrNo
).Max();
dressSerialNo += dressSerialNo + 1;
}
else dressSerialNo = 1;
solution:- In my project I was mainlining transaction in two for some module old method with ado.net and for newly developed modules I was using the entity framework so it was creating problem in transaction. So it went aberrant.
You are using Linq-To-Entities. There is an issue with the connection to your database server. Common causes for this are:
The query is taking longer than the timeout specified in the context.
There are network related issues causing a delay.
You can optionally change the command timeout (see this question about how to do this).
You don't want to materialize your all your entities using .ToList().
You could write a single query that returns only what you're interested in:
// Get the entity that contains the max value, using Ordering
var myMaxIfAny = currContext.OrderDressings
.Where(yy => yy.OrderID == this.OrderID && yy.OrderItemID == this.orderItemID && yy.ProductID == this.ProductID)
.OrderByDescending(z => z.SrNo)
.FirstOrDefault();
if (myMaxIfAny != null)
{
// Then you got a value, retrieve the Max using myMaxIfAny.SrNo
// ...
}
else
{
// ...
}
I've formatted and commented your code:
int? dressSerialNo;
// Get all OrderDressings with matching OrderID, orderItemID and ProductID as a List<OrderDressing>
var lstDress = (from yy in currContext.OrderDressings
where yy.OrderID == this.OrderID
&& yy.OrderItemID == this.orderItemID
&& yy.ProductID == this.ProductID
select yy)
.ToList();
// If any were found,
if (lstDress.Count > 0)
{
// Execute the Where again (what else will the list contain?) and select all yy.SrNo
dressSerialNo = (from yy in lstDress
where yy.OrderID == this.OrderID
&& yy.OrderItemID == this.orderItemID
&& yy.ProductID == this.ProductID
select (int?)yy.SrNo)
.Max(); // And take the Max() of that
// Add dressSerialNo + 1 to dressSerialNo.
dressSerialNo += dressSerialNo + 1;
}
else dressSerialNo = 1;
Which seems it can be corrected and reduced to:
int? serialNumber = (from yy in currContext.OrderDressings
where yy.OrderID == this.OrderID
&& yy.OrderItemID == this.orderItemID
&& yy.ProductID == this.ProductID
select yy.SrNo)
.DefaultIfEmpty() // Might not be necessary
.Max();
if (!serialNumber.HasValue)
{
serialNumber = 1;
}
else
{
serialNumber++;
}
Please note this can cause concurrency issues if two people execute this at the same time.
The query against the database is taking too long. There are many reasons as to why this could be happening.
Try running the sql statement generated from the linq directly to the database to see if it takes as long.
Check and see if any of your columns have massive data. (like a string column filled with large volume of data)
Meanwhile try adding this to end of your connection string
Connection Timeout=30000;
I have found the issue it was creating time out problem,
in my application transaction was maintained in 2 style,
one with old ado.net style, and another with EF style,
so it created a chaos. I am to make it uniform all to change in entity-framework.