I have an operation that takes a serializable QueryModel and converts it to an Expression to be passed to Entity Framework. My query against the database looks like:
public IEnumerable<PhotoVerifySessionOverview> FindSessions(Expression<Func<vwPhotoVerifySession, bool>> predicate, PaginationModel model)
{
var sessions = Context.vwPhotoVerifySessions
.AsQueryable()
.Where(predicate)
.OrderBy(string.Format("{0} {1}", model.OrderByColumn, model.OrderByDirection))
.Skip(model.Offset)
.Take(model.PageSize);
return Mapper.Map<IEnumerable<PhotoVerifySessionOverview>>(sessions);
}
and my predicate builder looks like:
var predicate = PredicateBuilder.True<vwPhotoVerifySession>();
//Add the tenant to the where clause
if (model.TenantId.HasValue)
predicate.And(p => p.TenantId == model.TenantId.Value);
else
predicate.And(p => p.TenantReferenceId == model.TenantReferenceId);
//Add a date range if one is present
if (model.CreatedOnRange != default(DateRange))
{
var endDate = model.CreatedOnRange.End == default(DateTime) ? DateTime.Now : model.CreatedOnRange.End;
predicate.And(p => p.CreatedOn >= model.CreatedOnRange.Start && p.CreatedOn <= endDate);
}
//Include status filtering if any filters are present
if (model.StatusFilter != null && model.StatusFilter.Any())
{
//use Id and name to search for status
predicate.And(p => model.StatusFilter.Any(f => f.StatusId == p.StatusId || p.Status == f.Name));
}
var pagination = model as PaginationModel;
var sessions = Manager.FindSessions(predicate, pagination);
return sessions;
The problem is, my Where clause is not being evaluated and all results are being returned. Is there something else I should be doing to get the Where statement to work correctly?
You need to assign predicate back to itself.
predicate = predicate.And(p => p.TenantId == model.TenantId.Value);
Related
I have following method which applies AND condition. But I would like to have OR condition based on parameter in filter.
public async Task<IList<PerformanceReportUser>> GetUsersForPerformanceReport(PerformanceReportFilter filter)
{
var query = _context.Set<EntityUser>()
.AsNoTracking()
.AsQueryable();
if(filter != null)
{
if (filter.Modified.HasValue)
{
query = query.Where(q => q.Modified >= filter.Modified);
}
// How can I have dynamically the OR condition based on filter parameter?
if (filter.Created.HasValue)
{
query = query.Where(q => q.Created >= filter.Created);
}
}
var result = query.Select(user => ToDomain(user));
return await result.ToListAsync();
}
Basically in SQL, I am expecting something like below.
If parameter is 'Or':
Select *
From EntityUser
Where Created >= '2021-01-01' Or Modified >= '2022-02-02'
If parameter is 'And'
Select *
From EntityUser
Where Created >= '2021-01-01' And Modified >= '2022-02-02'
If you want OR inside this types of querys, you have to do in same line, something similar to this:
var query = _context.Set<EntityUser>()
.AsNoTracking()
.AsQueryable();
if(filter != null)
{
query = query.Where(q =>
(filter.Modified.HasValue && q.Modified >= filter.Modified)
||
(filter.Created.HasValue && q.Created >= filter.Created)
);
}
EDITED
If you want dinamicaly condition based on filter use something similar to this...
var query = _context.Set<EntityUser>()
.AsNoTracking()
.AsQueryable();
if(filter != null)
{
if(filter.condition == "OR"){
query = query.Where(q =>
(filter.Modified.HasValue && q.Modified >= filter.Modified)
||
(filter.Created.HasValue && q.Created >= filter.Created)
);
}else{
query = query.Where(q =>
(filter.Modified.HasValue && q.Modified >= filter.Modified)
&&
(filter.Created.HasValue && q.Created >= filter.Created)
);
}
}
How can I merge this below query into a single query? I think multiple if's will add unnecessarily to code complexity.
var query = scenario.Where(x => x.ScenarioNumber == ScenarioNumber).Select(x => x).Distinct().ToList();
// Match exception number else return wild card entries.
if(query.Any(x=>x.ExcepNumber == ExcepNumber))
{
query = query.Where(x => x.ExcepNumber == ExcepNumber).ToList();
}
else
{
query = query.Where(x => x.ExcepNumber == "**").ToList();
}
// Match regime else return wild card entries.
if (query.Any(x => x.Regime == Regime))
{
query = query.Where(x => x.Regime == Regime).ToList();
}
else
{
query = query.Where(x => x.Regime == "**").ToList();
}
finalResponse = query.Select(x => x.TestColumn).Distinct().ToList();
I have a suggestion. You could extract it into a generic method, in which you would pass a selector function which specifies the property to be used for the filtering and the 2 choices that you would like to be compared (and the collection of course).
private List<T> EitherFilterOrWildCard<T>(Func<T, string> selector, string compare, string elseCompare, List<T> query)
{
var temp = query.Where(x => selector(x) == compare);
return temp.Any() ? temp.ToList() : query.Where(x => selector(x) == elseCompare).ToList();
}
Your if statements would then shrink to these 2 lines:
query = EitherFilterOrWildCard<MyClass>(x => x.ExcepNumber, ExcepNumber, "**", query);
query = EitherFilterOrWildCard<MyClass>(x => x.Regime, Regime, "**", query);
I have some filter parameters in the Controller of an ASP.NET MVC project and I need to create Where clause dynamically according to these parameters. If isActive parameter is true, it will get the records having the StatusId = 1. There are also userName and labId parameters in the method that should be matched in the Where clause.
public ActionResult GetStudents(int labId, string userName, bool isAll)
{
var allRecords = repository.Students;
//If isAll, get all the records having StatusId = 1
var result = allRecords.Where(m => (isAll) || m.StatusId == 1);
//???
}
I use the filter above, but I have no idea what is the most suitable way (conventions) for multiple parameters in order to fetch the result fast. Any idea?
Note: I want to filter for all of three parameters and Where clause should contain all of the combinations according to the parameter's values (also is null or empty).
var predicate = PredicateBuilder.False<Record>();
if(isAll)
predicate = predicate.AND(d => d.StatusId ==1);
predicate = predicate.AND(d => d.labID == labid && d.username = username);
return allRecords.Where(predicate);`
You can use a predicate builder
You can concatenate linq-methods as they all return an IEnumerable<T> and are combined using something like an SQL-And (dependinng on what LINQ-provider you use):
IEnumerable<Student> result = allRecords;
if(labelId.HasValue)
result = result.Where(x => x.LabelId == labelId);
else
result = result.Where(x => x.LabelId == 0); // or whatever your default-behaviour is
if(isAll)
result = result.Where(x => x.StatusId == 1);
else
result = result.Where(x => x.StatusId == 0); // or whateever your default-behaviour is when isAll is false
if(!String.IsNullOrEmpty(userName))
result = result.Where(x => x.Name == userName);
else
result = result.Where(x => x.Name == "Claus"); // or whatever the default-behaviour is when the param isn´t set
Do like this
public ActionResult GetStudents(int labId, string userName, bool isAll)
{
var allRecords = repository.Students;
//If isAll, get all the records having StatusId = 1
if (isAll)
{
var result = allRecords.Where(m => m.StatusId == 1 && m.UserName == userName && m.LabId == labId);
}
else
{
// do else things
}
}
you need something like below
public ActionResult GetStudents(int labId, string userName, bool isAll)
{
var allRecords = repository.Students;
//If isAll, get all the records having StatusId = 1
if (isAll)
{
var result = allRecords.Where(m => m.StatusId == 1
&& m.LabId == labId
&& m.UserName == username);
//or
var result = from record in allRecords
where record != null &&
record.StatusId == 1
&& !string.IsNullOrWhiteSpace(record.UserName)
&& record.UserName.Equals(username)
&& record.Labid = labId
select record;
}
else
{
// do else things
}
}
I have created a simple method using EF 6 that will query with grouping based on some input information and some possible Type and SubType values, as the following
public int GetOriginal(DateTime startDate, DateTime endDate, List<int> userIds)
{
DateTime dt = DateTime.UtcNow;
var ret = DbContext.ContactFeedback
.Where(c => c.FeedbackDate >= startDate &&
c.FeedbackDate <= endDate && userIds.Contains(c.UserId) &&
(c.Type == FeedbackType.A || c.Type == FeedbackType.B || c.Type == FeedbackType.C))
.GroupBy(x => new {TruncateTime = DbFunctions.TruncateTime(x.FeedbackDate), x.LeadId, x.UserId})
.Count();
Console.WriteLine(string.Format("{0}",DateTime.UtcNow - dt));
return ret;
}
It works as expected, however if I try to create a new auxiliar method that receives the "query" (Func type object) as input to be run, I see a very big difference in performance which I'm not able to explain, because they should run exactly the same.
Here is my rewritten methods
public int GetRewritten(DateTime startDate, DateTime endDate, List<int> userIds)
{
DateTime dt = DateTime.UtcNow;
var query = new Func<ContactFeedback, bool>(c => c.FeedbackDate >= startDate && c.FeedbackDate <= endDate && userIds.Contains(c.UserId) &&
(c.Type == FeedbackType.A || c.Type == FeedbackType.B ||
c.Type == FeedbackType.C));
var ret = GetTotalLeadsByFeedback(query);
Console.WriteLine(string.Format("{0}",DateTime.UtcNow - dt));
return ret;
}
private int GetTotalLeadsByFeedback(Func<ContactFeedback, bool> query)
{
return DbContext.ContactFeedback
.Where(query)
.GroupBy(x => new { TruncateTime = DbFunctions.TruncateTime(x.FeedbackDate), x.LeadId, x.UserId })
.Count();
}
Here are the running times in seconds
GetOriginal with 1 userId:0.0156318 - With ~100 usersIds: 0.1455635
GetRewritten with 1 userId:0.4742711 - With ~100 usersIds: 7.2555701
As you can see the difference is huge, anyone can share a light on why this occurs?
I'm running everything on Azure with a SQL Server DB if it helps
I see a very big difference in performance which I'm not able to explain, because they should run exactly the same.
They're considerably different in approach. The first part of your initial method's query:
DbContext.ContactFeedback
.Where(c => c.FeedbackDate >= startDate &&
c.FeedbackDate <= endDate && userIds.Contains(c.UserId) &&
(c.Type == FeedbackType.A || c.Type == FeedbackType.B || c.Type == FeedbackType.C))
Is equivalent to:
DbContext.ContactFeedback
.Where(new Expression<Func<ContactFeedback, bool>>(new Func<ContactFeedback, bool>(c => c.FeedbackDate >= startDate && c.FeedbackDate <= endDate && userIds.Contains(c.UserId) &&
(c.Type == FeedbackType.A || c.Type == FeedbackType.B ||
c.Type == FeedbackType.C)))
When you call .Where on an IQueryable<T> it will (barring a case where the type implementing IQueryable<T> has its own appliable .Where which would be strange) call into:
public static IQueryable<TSource> Where<TSource>(
this IQueryable<TSource> source,
Expression<Func<TSource, bool>> predicate
)
Bearing in mind that lambdas in source code can be turned into either a Func<…> or an Expression<Func<…>> as applicable.
Entity Framework then combines this query with the GroupBy and finally upon Count() turns the entire query into the appropriate SELECT COUNT … query, which the database performs (just how quickly depending on table contents and what indices are set, but which should be reasonably quick) and then a single value is sent back from the database for EF to obtain.
Your version though has explicitly assigned the lambda to a Func<ContactFeedback, bool>. As such using it with Where it has to call into:
public static IEnumerable<TSource> Where<TSource>(
this IEnumerable<TSource> source,
Func<TSource, bool> predicate
)
So to do the Where EF has to do retrieve every column of every row from the database, and then filter out those rows for which that Func returns true, then group them in memory (which requires storing the partially-constructed groups) before doing a Count by a mechanism like:
public int Count<T>(this IEnumerable<T> source)
{
/* some attempts at optimising that don't apply to this case and so in fact just waste a tiny amount omitted */
int tally = 0;
using(var en = source.GetEnumerator())
while(en.MoveNext())
++tally;
return tally;
}
This is a lot more work with a lot more traffic between the EF and database, and so a lot slower.
A rewrite of the sort you attempted would be better approximated by:
public int GetRewritten(DateTime startDate, DateTime endDate, List<int> userIds)
{
DateTime dt = DateTime.UtcNow;
var query = new Expression<Func<ContactFeedback, bool>>(c => c.FeedbackDate >= startDate && c.FeedbackDate <= endDate && userIds.Contains(c.UserId) &&
(c.Type == FeedbackType.A || c.Type == FeedbackType.B ||
c.Type == FeedbackType.C));
var ret = GetTotalLeadsByFeedback(query);
Console.WriteLine(string.Format("{0}",DateTime.UtcNow - dt));
return ret;
}
private int GetTotalLeadsByFeedback(Expression<Func<ContactFeedback, bool>> predicate)
{
return DbContext.ContactFeedback
.Where(predicate)
.GroupBy(x => new { TruncateTime = DbFunctions.TruncateTime(x.FeedbackDate), x.LeadId, x.UserId })
.Count();
}
(Note also that I changed the name of the predicate to predicate, as predicate is more commonly used for predicates, query for a source along with zero or more methods acting upon it; so DbContext.ContactFeedback, DbContext.ContactFeedback.Where(predicate) and DbContext.ContactFeedback.Where(predicate).GroupBy(x => new { TruncateTime = DbFunctions.TruncateTime(x.FeedbackDate), x.LeadId, x.UserId }) would all be queries if enumerated, and DbContext.ContactFeedback.Where(predicate).GroupBy(x => new { TruncateTime = DbFunctions.TruncateTime(x.FeedbackDate), x.LeadId, x.UserId }).Count() is a query that immediately executes and returns a single value).
Conversely, the form you ended up with could be written back into the style of GetOriginal as:
public int GetNotOriginal(DateTime startDate, DateTime endDate, List<int> userIds)
{
DateTime dt = DateTime.UtcNow;
var ret = DbContext.ContactFeedback
.AsEnumerable()
.Where(c => c.FeedbackDate >= startDate &&
c.FeedbackDate <= endDate && userIds.Contains(c.UserId) &&
(c.Type == FeedbackType.A || c.Type == FeedbackType.B || c.Type == FeedbackType.C))
.GroupBy(x => new {TruncateTime = DbFunctions.TruncateTime(x.FeedbackDate), x.LeadId, x.UserId})
.Count();
Console.WriteLine(string.Format("{0}",DateTime.UtcNow - dt));
return ret;
}
Note the AsEnumerable forcing the Where and everything that follows to be executed in the .NET application, rather than on the database.
I am trying to write a LINQ statement with some optional where clauses. This is for a search. The user can select a specific site to search or search against all sites:
var query =
_db.STEWARDSHIP
.OrderBy(r => r.SITE.SITE_NAME)
.Where(r => r.SITE_ID == SiteId)
.Where(r => r.VISIT_TYPE_VAL.VISIT_TYPE_ID == VisitTypeId)
.Select(r => new
{
id = r.STEWARDSHIP_ID,
name = r.SITE.SITE_NAME,
visit_type = r.VISIT_TYPE_VAL.VISIT_TYPE_DESC,
visit_date = r.VISIT_DATE
});
return query;
So when the method gets SiteId = 14, for instance, no problem. However, when it gets SiteId = null, then that where clause should not be considered.
Thanks
Eric
That's easy:
var query = _db.STEWARDSHIP.OrderBy(r => r.SITE.SITE_NAME);
if (SiteId != null)
{
query = query.Where(r => r.SITE_ID == SiteId);
}
query = query.Where(r => r.SITE.SITE_TYPE_VAL.SITE_TYPE_ID == SiteTypeId)
.Select(r => new
{
id = r.STEWARDSHIP_ID,
name = r.SITE.SITE_NAME,
visit_type = r.VISIT_TYPE_VAL.VISIT_TYPE_DESC,
visit_date = r.VISIT_DATE
});
return query;
This works because queries compose nicely - and they really only represent queries; it's only when you try to fetch data from them that the query is actually executed.
Can't you just edit the where clause to something like
.Where(r=>SiteId == null || r.SiteId == SiteId)
you can use where clause in one statement ..like this ..
.Where(r => SiteID == null || r.SITE_ID == SiteID)
I'm stealing a trick from TSQL. Just check for the null value as well.
...
.Where(r => SiteID == null || r.SITE_ID == SiteID)
...
The SQL example is this:
WHERE (SITE_ID = #given OR #given IS NULL) --return matches or all
Though if that value is mutable and you want the value at the time the query was built, try this instead:
var localSiteID = SiteID;
...
.Where(r => localSiteID == null || r.SITE_ID == SiteID)
...