We Keep Coding

Get filter criteria from function in c# LINQ to SQL

I have the following LINQ query. Is there a way to outsource the central part into a method?
IQueryable<vwResCache> clientDataset = db.vwResCaches.Where(r =>
clientResources.Contains(r.ID_Resource) &&
r.StartDate < prospectDate &&
(
r.StartDate >= retrospectDate ||
r.StartDate < retrospectDate && (
(clientConfig.FinishReq_FinishedPA && clientConfig.FinishReq_DeliveryNotesPrinted && (!r.FinishedPA.Value || !r.DeliveryNotesPrinted.Value)) ||
(clientConfig.FinishReq_FinishedPA && !clientConfig.FinishReq_DeliveryNotesPrinted && !r.FinishedPA.Value) ||
(!clientConfig.FinishReq_FinishedPA && clientConfig.FinishReq_DeliveryNotesPrinted && !r.DeliveryNotesPrinted.Value)
)
));
As you can see, the must indented part depends on the local object clientConfig. I want to exclude that into s separate method like so:
IQueryable<vwResCache> clientDataset = db.vwResCaches.Where(r =>
clientResources.Contains(r.ID_Resource) &&
r.StartDate < prospectDate &&
(
r.StartDate >= retrospectDate ||
r.StartDate < retrospectDate && GetFilterCriteria(clientConfig, r)
));
This method should look something like:
FilterExpression GetFilterExpression(ClientConfig clientConfig, c)
{
if (clientConfig.FinishReq_FinishedPA && clientConfig.FinishReq_DeliveryNotesPrinted)
return new FilterExpression(c, r => !r.FinishedPA.Value || !r.DeliveryNotesPrinted.Value);
if (clientConfig.FinishReq_FinishedPA)
return new FilterExpression(c, r => !r.FinishedPA.Value);
if (clientConfig.FinishReq_DeliveryNotesPrinted)
return new FilterExpression(c, r => !r.DeliveryNotesPrinted.Value);
}
Is such a thing possible? Or would the effort not justify the benefit?

Calling .Where(...) multiple times has the same effect as combining the individual filters with &&.
Unfortunately, there's no simple equivalent for combining filters with ||, so your filter function would need to include the StartDate logic for each branch.
Something like this should work:
static Expression<Func<vwResCache, bool>> BuildClientConfigFilter(
DateTime retrospectDate,
ClientConfig clientConfig)
{
if (clientConfig.FinishReq_FinishedPA
&& clientConfig.FinishReq_DeliveryNotesPrinted)
{
return r => r.StartDate >= retrospectiveDate
|| (r.StartDate < retrospectiveDate
&& (!r.FinishedPA.Value || !r.DeliveryNotesPrinted.Value));
}
if (clientConfig.FinishReq_FinishedPA
&& !clientConfig.FinishReq_DeliveryNotesPrinted)
{
return r => r.StartDate >= retrospectiveDate
|| (r.StartDate < retrospectiveDate
&& !r.FinishedPA.Value);
}
if (!clientConfig.FinishReq_FinishedPA
&& clientConfig.FinishReq_DeliveryNotesPrinted)
{
return r => r.StartDate >= retrospectiveDate
|| (r.StartDate < retrospectiveDate
&& !r.DeliveryNotesPrinted.Value;
}
return r => r.StartDate >= retrospectiveDate;
}
Usage:
IQueryable<vwResCache> clientDataset = db.vwResCaches
.Where(r => clientResources.Contains(r.ID_Resource))
.Where(r => r.StartDate < prospectDate)
.Where(BuildClientConfigFilter(retrospectDate, clientConfig));

Prevent Entity Framework query from being evaluated locally

I have this query that is being partially evaluated locally and I am wondering how to prevent it.
It seems to be the conditional select that is causing the problem.
var fetchQuery = (
from udg in _entities.UberDirectGroups.AsNoTracking()
join uber in _entities.Ubers.AsNoTracking()
on udg.Id equals uber.Id
let memberCount = (
from t in _entities.UberDirects.AsNoTracking()
join u in _entities.Ubers.AsNoTracking()
on t.UberToId equals u.Id
where t.UberFromId == udg.Id && !u.Deleted
select u.UberTypeId == (byte)UberType.User ? 1 :
u.UberTypeId == (byte)UberType.Department ? u.Department.Users.Where(user => user.Deleted == false).Count() :
u.UberTypeId == (byte)UberType.Office ? u.tblOffice.tblDepartment.SelectMany(d => d.Users).Where(user => user.Deleted == false).Count() :
u.UberTypeId == (byte)UberType.ProjectGroup ? u.Group.GroupMembers.Select(pm => pm.User).Where(user => user.Deleted == false).Count() :
u.UberTypeId == (byte)UberType.Role ? _entities.Roles.Where(r => r.RoleDataId == u.tblRoleData.Id).Select(r => r.tblUser).Where(user => user.Deleted == false).Count() : 0
).Sum()
where
udg != null &&
uber != null &&
uber.InstanceId == instanceId &&
(!isSearch || udg.Name.Contains(searchText))
select new TargetGroupProjection
{
id = udg.Id,
name = udg.Name,
created = uber.Date,
toCount = memberCount
}
);

EntityFremework: could a select before a where optimize this?

I'm trying to gain performance on this query, and I'd like to know if calling a Select() before the Where() I could have some improvement:
public async Task<List<PostValues>> GetValuesToTheDashboard(DataFilter filter, CancellationToken cancellationToken) {
long startTimestanp = Helpers.UnixTimeNow(filter.StartDate);
long endTimestanp = Helpers.UnixTimeNow(filter.EndDate);
return await
_context.CatchDetails.Where(
x => x.Monitoring.Client.Id == filter.CustomerId && x.Data.published >= startTimestanp
&& x.Data.published <= endTimestanp
&& ((filter.Sentiment == Sentiments.ALL) || x.Sentiment_enum == filter.Sentiment)
&& (filter.MonitoringId == 0 || x.Monitoring.id == filter.MonitoringId)
&& (filter.KeywordId == 0 || x.Keyword.Id == filter.KeywordId)
&& (filter.MotiveId == 0 || x.Motive.Id == filter.MotiveId)
&& (filter.SocialNetwork.Count == 0 || filter.SocialNetwork.Any(s => x.Data.social_media == s))
&& (filter.Busca == "" || x.Data.content_snippet.Contains(filter.Busca))
&& (filter.Gender.Count == 0 || filter.Gender.Any(g => x.Data.extra_author_attributes.gender_enum == g)))
.Select(s => new PostValues() {
CatchDetailsId=s.Id,
Monitoring=s.Monitoring.name,
Keyword=s.Keyword.Text,
Motive=s.Motive.Name,
Sentiment=s.Sentiment_enum,
Gender=s.Data.extra_author_attributes.gender_enum,
SocialMedia=s.Data.social_media,
Country=s.Data.extra_author_attributes.world_data.country_code,
State=s.Data.extra_author_attributes.world_data.region,
Published=s.Data.published
}).ToListAsync(cancellationToken);
}

There might be a way how to improve performance, but it won't be with switching Select and Where (as Chetan mentioned in the comment).
You could build the query in a sequence and based on the filter get a simpler query in the end. This would go like this:
var query = _context.CatchDetails.Where(
x => x.Monitoring.Client.Id == filter.CustomerId && x.Data.published >= startTimestanp
&& x.Data.published <= endTimestanp);
if (filter.Sentiment != Sentiments.ALL)
{
query = query.Where(x => x.Sentiment_enum == filter.Sentiment);
}
if (filter.MonitoringId != 0)
{
query = query.Where(x => x.Monitoring.id == filter.MonitoringId);
}
[...]
return await
query.Select(s => new PostValues() {
[...]
}).ToListAsync(cancellationToken);

Do not forget, the variable query is already on memory of the application when the SQL returns data. If there is many results it could throw memory exception.
I suggest that you limit the range of date on that search.

Filtering an Iqueryable object by a group of rules

I've been filtetring an Iqueryable object by some rules, using successive Where clauses to do && filtering, how can i do the same for || filtering?
Anyone has a suggestion?
Using another words, i want to apply for all rules || filtering.
I'm going to show a portion of the code to exemplify what im asking.
Where i get the pages object:
public IQueryable<gbd_Pages> getAllPagesByOrderAndDir(int template_id, string name = "", int sortOrder = 3, string sortDirection = "asc")
{
IQueryable<gbd_Pages> Listpages;
if (sortDirection == "asc")
{
Listpages = from p in _db.gbd_Pages
from pc in (from c in p.gbd_Content
where c.IsActive == true && c.IsDeleted == false && c.Template_Id == template_id
&& !string.IsNullOrEmpty(name) ? c.Content.ToLower().Contains(name.ToLower()) : true
&& c.gbd_Template_Fields.SortOrder == sortOrder
select c).Take(1)
orderby pc.Content ascending
where p.IsActive == true && p.IsDeleted == false
select p;
}
else
{
Listpages = from p in _db.gbd_Pages
from pc in (from c in p.gbd_Content
where c.IsActive == true && c.IsDeleted == false && c.Template_Id == template_id
&& !string.IsNullOrEmpty(name) ? c.Content.ToLower().Contains(name.ToLower()) : true
&& c.gbd_Template_Fields.SortOrder == sortOrder
select c).Take(1)
orderby pc.Content descending
where p.IsActive == true && p.IsDeleted == false
select p;
}
return Listpages;
}
The function where i do the filtering. Actually it's only doing && filtering for all rules:
public IEnumerable<gbd_Pages> PagesFiltered(List<Rule> newRules, string search, int sortColumnId, string sortColumnDir, out int count)
{
GBD.FrontOffice.Controllers.SegmentsController segmentCtrl = new GBD.FrontOffice.Controllers.SegmentsController();
var Pages = controller.getAllPagesByOrderAndDir(1, search, sortColumnId, sortColumnDir);
if (newRules.Count > 0)//FILTRA A LISTAGEM DE ACORDO COM AS REGRAS
{
foreach (var rule in newRules)
{
if (rule.Value!=null)
{
gbd_Template_Fields templateField = controller.getTemplateFieldById(1, rule.Template_Field_Id);
gbd_Segment_Operators segOperator = segmentCtrl.getOperatorById(rule.Operator_Id);
if (segOperator.Value == "==")//IGUALDADE
if (rule.Value.Trim() != "")
{
if (templateField != null && templateField.Field_Id == 3) //tipo data
{
try
{
DateTime test_datetime = DateTime.Parse(rule.Value); // testa se a data introduzida é valida
Pages = Pages.Where(p => p.gbd_Content.Any(c => c.Templates_Fields_Id == rule.Template_Field_Id && c.IsActive == true && c.IsDeleted == false && (!string.IsNullOrEmpty(c.Content) && c.Content.Trim() == rule.Value.Trim())));
}
catch (Exception ex)
{
}
}
else if (templateField != null && templateField.Field_Id == 12 && !string.IsNullOrEmpty(rule.Value))//Lista múltipla de valores
{
rule.Value = rule.Value.Replace(',', '|');
Pages = Pages.Where(p => p.gbd_Content.Any(c => c.Templates_Fields_Id == rule.Template_Field_Id && c.IsActive == true && c.IsDeleted == false && (!string.IsNullOrEmpty(c.Content) && c.Content == rule.Value)));
}
else
Pages = Pages.Where(p => p.gbd_Content.Any(c => c.Templates_Fields_Id == rule.Template_Field_Id && c.IsActive == true && c.IsDeleted == false && (c.Content != null && c.Content.ToLower().Trim() == rule.Value.ToLower().Trim())));
}
else
{
Pages = Pages.Where(p => !p.gbd_Content.Any(c => c.Templates_Fields_Id == rule.Template_Field_Id) || p.gbd_Content.Any(c => c.Templates_Fields_Id == rule.Template_Field_Id && c.IsActive == true && c.IsDeleted == false && (c.Content == null || c.Content.ToLower().Trim() == rule.Value.ToLower().Trim())));
}
else if (segOperator.Value == ">")//MAIOR QUE
{
if (templateField != null && templateField.Field_Id == 3) //tipo data
{
try
{
DateTime test_datetime = DateTime.Parse(rule.Value); // testa se a data introduzida é valida
Pages = Pages.Where(p => p.gbd_Content.Any(c => c.Templates_Fields_Id == rule.Template_Field_Id && c.IsActive == true && c.IsDeleted == false && (!string.IsNullOrEmpty(c.Content) && (c.Content.Substring(6, 4).CompareTo(rule.Value.Trim().Substring(6, 4)) > 0 || c.Content.Substring(6, 4).CompareTo(rule.Value.Trim().Substring(6, 4)) == 0 && c.Content.Substring(3, 2).CompareTo(rule.Value.Trim().Substring(3, 2)) > 0 || c.Content.Substring(6, 4).CompareTo(rule.Value.Trim().Substring(6, 4)) == 0 && c.Content.Substring(3, 2).CompareTo(rule.Value.Trim().Substring(3, 2)) == 0 && c.Content.Substring(0, 2).CompareTo(rule.Value.Trim().Substring(0, 2)) > 0))));
}
catch (Exception ex)
{
}
}
else if (templateField != null && templateField.Field_Id == 5)//tipo número
Pages = Pages.Where(p => p.gbd_Content.Any(c => c.Templates_Fields_Id == rule.Template_Field_Id && c.IsActive == true && c.IsDeleted == false && (c.Content != null && c.Content != "" && (c.Content.Trim().CompareTo(rule.Value.Trim()) > 0 || (rule.Value.Trim().StartsWith("-") && c.Content.Trim().CompareTo(rule.Value.Trim()) > 0 || (c.Content.Trim().CompareTo("0") > 0))))));//procurar outra solução https://stackoverflow.com/questions/7740693/big-issue-in-converting-string-to-datetime-using-linq-to-entities
else if (templateField != null && templateField.Field_Id == 6) //tipo telefone //ESTA CONDIÇÃO NÃO FOI ADICIONADA AINDA A BD
{
try
{
Pages = Pages.AsEnumerable().Where(p => p.gbd_Content.Any(c => c.Templates_Fields_Id == rule.Template_Field_Id && c.IsActive == true && c.IsDeleted == false && (c.Content != null && c.Content != "" && (Convert.ToInt32(c.Content.Split(' ')[1]) > Convert.ToInt32(rule.Value))))).AsQueryable();//TODO: procurar outra solução
}
catch (Exception ex) { }
}
}
many more rules...
else if (segOperator.Value == "StartsWith")//COMEÇA COM
{
if (rule.Value.Trim() != "")
Pages = Pages.Where(p => p.gbd_Content.Any(c => c.Templates_Fields_Id == rule.Template_Field_Id && c.IsActive == true && c.IsDeleted == false && (c.Content != null && c.Content.ToLower().Trim().StartsWith(rule.Value.ToLower().Trim()))));
}
else if (segOperator.Value == "EndsWith")//ACABA COM
{
if (rule.Value.Trim() != "")
Pages = Pages.Where(p => p.gbd_Content.Any(c => c.Templates_Fields_Id == rule.Template_Field_Id && c.IsActive == true && c.IsDeleted == false && (c.Content != null && c.Content.ToLower().Trim().EndsWith(rule.Value.ToLower().Trim()))));
}
}
}
}
count = Pages.Select(d => d.Id).Count();
return Pages;
}

The Where statement is implicitly using AND in all cases. To get an OR statement equivalent, you'll need to use the double pipe operator (||) inside of a Where statement with something like:
.Where(p => p.A == 'b' || p.C == 'd')
Unfortunately there is no simple way to add successive optional statements the .NET framework. It would be great to have something like .Where(p => p.A == 'b').Or(p => p.C == 'd') but generating a query from this seems impractical - if there was an another Where statement, then how would Linq know how to group it relative to the other two? (A || B && C) versus ((A || B) && C).
That said, there is an external library (LINQKit) that can help fill the gap. See Dynamic where clause (OR) in Linq to Entities for reference.

You can work around the limitations of linq by just adding all the results into a List<>
List<gbd_Pages> allPages = new List<gdb_Pages>();
foreach (var rule in newRules)
{
IEnumerable<gbd_Pages> rulePages;
// current logic (except you need to not overwrite the Pages variable)
// instead of the pattern "Pages = Pages.Where( ..." use "rulePages = Pages.Where( ..."
allPages.AddRange(rulePages);
}
return allPages;

LINQ to Entities: nullable datetime in where clause

I have a where clause that looks up child objects on an entity:
var Lookup = row.Offenses.Where(x => x.Desc == co.Desc && x.Action == co.Action && x.AppealYN == co.AppealYN && x.OffDate == co.OffDate).ToList();
Sometimes the co.OffDate can be null, which will cause an exception. Right now, the only way I can think of to get around that, is to use an if statement:
if (co.OffDate.HasValue)
{
var Lookup = row.Offenses.Where(x => x.Desc == co.Desc && x.Action == co.Action && x.AppealYN == co.AppealYN && x.OffDate == co.OffDate).ToList();
}
else
{
var Lookup = row.Offenses.Where(x => x.Desc == co.Desc && x.Action == co.Action && x.AppealYN == co.AppealYN).ToList();
}
Is there anyway I can re-write the linq query to accomplish what the if statement does? I still want to do a lookup, even if the co.OffDate is null.

You could insert a ternary into your Where filter:
var Lookup = row.Offenses
.Where(x =>
x.Desc == co.Desc
&& x.Action == co.Action
&& x.AppealYN == co.AppealYN
&& (co.OffDate.HasValue ? x.OffDate == co.OffDate : true)
).ToList();

I would rewrite it to be more readable (in my opinion):
var query = row.Offenses.Where(x => x.Desc == co.Desc
&& x.Action == co.Action
&& x.AppealYN == co.AppealYN)
if (co.OffenseDate.HasValue)
{
query = query.Where(x.OffDate == co.OffenseDate);
}
var Lookup = query.ToList();