We Keep Coding

How can I pull a repeated where clause expression from linq into a function?

I have a large project where I have dozens of linq statements where I am looking for a matching record by checking several fields to see if they match or both field and compared field are null.
var testRecord = new { firstField = "bob", secondField = (string)null, thirdField = "ross" };
var matchRecord = dataContext.RecordsTable.FirstOrDefault(vi =>
(vi.first == testRecord.firstField || ((vi.first == null || vi.first == string.Empty) && testRecord.firstField == null))
&& (vi.second == testRecord.secondField || ((vi.second == null || vi.second == string.Empty) && testRecord.secondField == null))
&& (vi.third == testRecord.thirdField || ((vi.third == null || vi.third == string.Empty) && testRecord.thirdField == null)));
//do stuff with matchRecord
Ideally I would replace all that duplicated code (used around 50 times across the system I'm working on) with something like the following
Expression<Func<string, string, bool>> MatchesOrBothNull = (infoItem, matchItem) => (
infoItem == matchItem || ((infoItem == null || infoItem == string.Empty) && matchItem == null));
var matchRecord = dataContext.RecordsTable.FirstOrDefault(vi =>
MatchesOrBothNull(vi.first, testRecord.firstField)
&& MatchesOrBothNull(vi.second, testRecord.secondField)
&& MatchesOrBothNull(vi.third, testRecord.thirdField));
//do stuff with matchRecord
My question is two-fold: First, is there a matched or both null function already available? (I've looked without luck).
Second, the code block above compiles, but throws a "no supported translation to sql" error, is there a way to have a function in the where clause? I know that there is a translation because it works if I don't pull it into the function. How can I get that translated?

First of all you can check whether string is null or empty with single code which is called : String.IsNullOrEmpty(vi.first). You need a method like this one :
public bool MatchesOrBothNull(string first,string second){
if(first==second||String.IsNullOrEmpty(first)||String.IsNullOrEmpty(second))
return true;
else return false;
}
You can use it in where clause
var matchRecord = dataContext.RecordsTable.Where(vi =>
MatchesOrBothNull(vi.first, testRecord.firstField)
&& MatchesOrBothNull(vi.second, testRecord.secondField)
&& MatchesOrBothNull(vi.third, testRecord.thirdField)
).FirstOrDefault();

Entity Framework: Count() very slow on large DbSet and complex WHERE clause

I need to perform a count operation on this Entity Framework (EF6) data set using a relatively complex expression as a WHERE clause and expecting it to return about 100k records.
The count operation is obviously where records are materialized and therefore the slowest of operations to take place. The count operation is taking about 10 seconds in our production environment which is unacceptable.
Note that the operation is performed on the DbSet directly (db being the Context class), so no lazy loading should be taking place.
How can I further optimize this query in order to speed up the process?
The main use case is displaying an index page with multiple filter criteria but the the function is also used for writing generic queries to the ParcelOrderstable as required for other operations in the service classes which might be a bad idea resulting in very complex queries resulting from laziness and might potentially be a future problem.
The count is later used for pagination, and a much smaller number of records (e.g. 500) is actually displayed. This is a database-first project using SQL Server.
ParcelOrderSearchModel is a C#-class that serves to encapsualte query parameters and is used exclusively by service classes in order to call the GetMatchingOrdersfunction.
Note that on the majority of calls, the majority of the parameters of ParcelOrderSearchModel will be null.
public List<ParcelOrderDto> GetMatchingOrders(ParcelOrderSearchModel searchModel)
{
// cryptic id known --> allow public access without login
if (String.IsNullOrEmpty(searchModel.KeyApplicationUserId) && searchModel.ExactKey_CrypticID == null)
throw new UnableToCheckPrivilegesException();
Func<ParcelOrder, bool> userPrivilegeValidation = (x => false);
if (searchModel.ExactKey_CrypticID != null)
{
userPrivilegeValidation = (x => true);
}
else if (searchModel.KeyApplicationUserId != null)
userPrivilegeValidation = privilegeService.UserPrivilegeValdationExpression(searchModel.KeyApplicationUserId);
var criteriaMatchValidation = CriteriaMatchValidationExpression(searchModel);
var parcelOrdersWithNoteHistoryPoints = db.HistoryPoint.Where(hp => hp.Type == (int)HistoryPointType.Note)
.Select(hp => hp.ParcelOrderID)
.Distinct();
Func<ParcelOrder, bool> completeExpression = order => userPrivilegeValidation(order) && criteriaMatchValidation(order);
searchModel.PaginationTotalCount = db.ParcelOrder.Count(completeExpression);
// todo: use this count for pagination
}
public Func<ParcelOrder, bool> CriteriaMatchValidationExpression(ParcelOrderSearchModel searchModel)
{
Func<ParcelOrder, bool> expression =
po => po.ID == 1;
expression =
po =>
(searchModel.KeyUploadID == null || po.UploadID == searchModel.KeyUploadID)
&& (searchModel.KeyCustomerID == null || po.CustomerID == searchModel.KeyCustomerID)
&& (searchModel.KeyContainingVendorProvidedId == null || (po.VendorProvidedID != null && searchModel.KeyContainingVendorProvidedId.Contains(po.VendorProvidedID)))
&& (searchModel.ExactKeyReferenceNumber == null || (po.CustomerID + "-" + po.ReferenceNumber) == searchModel.ExactKeyReferenceNumber)
&& (searchModel.ExactKey_CrypticID == null || po.CrypticID == searchModel.ExactKey_CrypticID)
&& (searchModel.ContainsKey_ReferenceNumber == null || (po.CustomerID + "-" + po.ReferenceNumber).Contains(searchModel.ContainsKey_ReferenceNumber))
&& (searchModel.OrKey_Referencenumber_ConsignmentID == null ||
((po.CustomerID + "-" + po.ReferenceNumber).Contains(searchModel.OrKey_Referencenumber_ConsignmentID)
|| (po.VendorProvidedID != null && po.VendorProvidedID.Contains(searchModel.OrKey_Referencenumber_ConsignmentID))))
&& (searchModel.KeyClientName == null || po.Parcel.Name.ToUpper().Contains(searchModel.KeyClientName.ToUpper()))
&& (searchModel.KeyCountries == null || searchModel.KeyCountries.Contains(po.Parcel.City.Country))
&& (searchModel.KeyOrderStates == null || searchModel.KeyOrderStates.Contains(po.State.Value))
&& (searchModel.KeyFromDateRegisteredToOTS == null || po.DateRegisteredToOTS > searchModel.KeyFromDateRegisteredToOTS)
&& (searchModel.KeyToDateRegisteredToOTS == null || po.DateRegisteredToOTS < searchModel.KeyToDateRegisteredToOTS)
&& (searchModel.KeyFromDateDeliveredToVendor == null || po.DateRegisteredToVendor > searchModel.KeyFromDateDeliveredToVendor)
&& (searchModel.KeyToDateDeliveredToVendor == null || po.DateRegisteredToVendor < searchModel.KeyToDateDeliveredToVendor);
return expression;
}
public Func<ParcelOrder, bool> UserPrivilegeValdationExpression(string userId)
{
var roles = GetRolesForUser(userId);
Func<ParcelOrder, bool> expression =
po => po.ID == 1;
if (roles != null)
{
if (roles.Contains("ParcelAdministrator"))
expression =
po => true;
else if (roles.Contains("RegionalAdministrator"))
{
var user = db.AspNetUsers.First(u => u.Id == userId);
if (user.RegionalAdministrator != null)
{
expression =
po => po.HubID == user.RegionalAdministrator.HubID;
}
}
else if (roles.Contains("Customer"))
{
var customerID = db.AspNetUsers.First(u => u.Id == userId).CustomerID;
expression =
po => po.CustomerID == customerID;
}
else
{
expression =
po => false;
}
}
return expression;
}

If you can possibly avoid it, don't count for pagination. Just return the first page. It's always expensive to count and adds little to the user experience.
And in any case you're building the dynamic search wrong.
You're calling IEnumerable.Count(Func<ParcelOrder,bool>), which will force client-side evaluation where you should be calling IQueryable.Count(Expression<Func<ParcelOrder,bool>>). Here:
Func<ParcelOrder, bool> completeExpression = order => userPrivilegeValidation(order) && criteriaMatchValidation(order);
searchModel.PaginationTotalCount = db.ParcelOrder.Count(completeExpression);
But there's a simpler, better pattern for this in EF: just conditionally add criteria to your IQueryable.
eg put a method on your DbContext like this:
public IQueryable<ParcelOrder> SearchParcels(ParcelOrderSearchModel searchModel)
{
var q = this.ParcelOrders();
if (searchModel.KeyUploadID != null)
{
q = q.Where( po => po.UploadID == searchModel.KeyUploadID );
}
if (searchModel.KeyCustomerID != null)
{
q = q.Where( po.CustomerID == searchModel.KeyCustomerID );
}
//. . .
return q;
}

Store math operator in variable

I am creating an application in C# MVVM. I have a simple question. Is there any possibility to store math operator in variable? I have a code like that:
public ICollectionView FilteredCollection
{
get
{
return filteredCollection;
}
set
{
filteredCollection = value;
OnPropertyChanged("FilteredCollection");
}
}
FilteredCollection.Filter = x => (
(string.IsNullOrEmpty(DynamicSearchEmployeeName) || ((Employee)x).EmployeeName.Contains(DynamicSearchEmployeeName))
&& (DynamicSearchEmployeeID == null || ((Employee)x).EmployeeID == DynamicSearchEmployeeID)
&& (string.IsNullOrEmpty(DynamicSearchEmployeeSalary) || ((Employee)x).EmployeeSalary == Convert.ToInt32(DynamicSearchEmployeeSalary))
&& (string.IsNullOrEmpty(DynamicSearchEmployeeDesigner) || ((Employee)x).EmployeeName.Contains(DynamicSearchEmployeeDesigner))
&& (string.IsNullOrEmpty(DynamicSearchEmployeeEmailID) || ((Employee)x).EmployeeName.Contains(DynamicSearchEmployeeEmailID))
);
What I want to achieve:
In fourth line (DynamicSearchEmployeeSalary) math operator should be depended on following conditions:
if (IsPressedEqual == true)
VARIABLE = "=="
if (IsPressedLess == true)
VARIABLE = "<"
if (IsPressedGreater == true)
VARIABLE = ">"
if (IsPressedLess == true && IsPressedEqual == true)
VARIABLE = "<="
if (IsPressedGreater == true && IsPressedEqual == true)
VARIABLE = ">="
Scenario:
For example I put a value like 10000 in textbox, then click on button with "=" operator. As a result I want to receive Employees with Salary equals than 10000.
Then I click on ">". And I have Employees with Salary greater and equals 10000.
FilteredCollection.Filter = x => (
(string.IsNullOrEmpty(DynamicSearchEmployeeName) || ((Employee)x).EmployeeName.Contains(DynamicSearchEmployeeName))
&& (DynamicSearchEmployeeID == null || ((Employee)x).EmployeeID == DynamicSearchEmployeeID)
&& (string.IsNullOrEmpty(DynamicSearchEmployeeSalary) || ((Employee)x).EmployeeSalary VARIABLE Convert.ToInt32(DynamicSearchEmployeeSalary))
&& (string.IsNullOrEmpty(DynamicSearchEmployeeDesigner) || ((Employee)x).EmployeeName.Contains(DynamicSearchEmployeeDesigner))
&& (string.IsNullOrEmpty(DynamicSearchEmployeeEmailID) || ((Employee)x).EmployeeName.Contains(DynamicSearchEmployeeEmailID))

I've made a simple Rule Based Engine ... I think it will help you in your issue ...
please find it as a nuget package here: https://www.nuget.org/packages/IbnSherien.RuleBasedEngine/
you can create a rule like this:
var rule = RuleEngine.CreateRule<Employee>()
.If<Employee>(e => e.EmployeeSalary).GreaterThan(DynamicSearchEmployeeSalary)
.Validate();
FilteredCollection.Filter = x => (
(string.IsNullOrEmpty(DynamicSearchEmployeeName) || ((Employee)x).EmployeeName.Contains(DynamicSearchEmployeeName))
&& (DynamicSearchEmployeeID == null || ((Employee)x).EmployeeID == DynamicSearchEmployeeID)
&& (string.IsNullOrEmpty(DynamicSearchEmployeeSalary) || rule.Match((Employee)x).IsMatch)
&& (string.IsNullOrEmpty(DynamicSearchEmployeeDesigner) || ((Employee)x).EmployeeName.Contains(DynamicSearchEmployeeDesigner))
&& (string.IsNullOrEmpty(DynamicSearchEmployeeEmailID) || ((Employee)x).EmployeeName.Contains(DynamicSearchEmployeeEmailID))
feel free to add any comments or contribute to the package

Adding More than One Criteria to a Predicate List

I have Companies that I am attempting to filter down using criteria. Each Company has a CurrentStatus, and the method to filter is called everytime the user checks a CheckBox to define a filter. I currently have this working almost exactly how I want it, aside from one thing. This is what I have;
private void FilterCompanyType(object sender, RoutedEventArgs e)
{
criteria.Clear();
if (currentCheckBox.IsChecked == true)
{
criteria.Add(new Predicate<CompanyModel>(x => x.CurrentStatus == 1));
}
if (nonCurrentCheckBox.IsChecked == true)
{
criteria.Add(new Predicate<CompanyModel>(x => x.CurrentStatus == 0));
}
foreach (CheckBox checkBox in companyFilters.Children)
{
if (!CheckCheckBoxes())
{
dataGrid.ItemsSource = null;
compDetailsLabel.Content = string.Empty;
}
else
{
dataGrid.ItemsSource = CompanyICollectionView;
CompanyICollectionView.Filter = dynamic_Filter;
SetSelectedCompany(selectedIndex);
dataGrid.SelectedIndex = 0;
}
}
}
As I said this is working OK, it works for when the user wants to see a list of Companies where CurrentStaus == 1 OR a list of Companies where CurrentStatus == 0. However, currently the user cannot see a list of Companies, if both CheckBoxes are checked where CurrentStatus == 0 AND CurrentStatus == 1.
I have tried adding this but it does not work with both CheckBoxes checked;
if (nonCurrentCheckBox.IsChecked == true && currentCheckBox.IsChecked == true)
{
criteria.Add(new Predicate<CompanyModel>(x => x.CurrentStatus == 0));
criteria.Add(new Predicate<CompanyModel>(x => x.CurrentStatus == 1));
}
This just returns an empty DataGrid. How can I change the Predicate to allow for both?

If I clearly understand, if statements should look like this.
if ( currentCheckBox.IsChecked == true && nonCurrentCheckBox.IsChecked == false )
{
criteria.Add( new Predicate<CompanyModel>( x => x.CurrentStatus == 1 ) );
}
else if ( nonCurrentCheckBox.IsChecked == true && currentCheckBox.IsChecked == false )
{
criteria.Add( new Predicate<CompanyModel>( x => x.CurrentStatus == 0 ) );
}
else if ( nonCurrentCheckBox.IsChecked == true && currentCheckBox.IsChecked == true )
{
criteria.Add( new Predicate<CompanyModel>( x => ( x.CurrentStatus == 0 || x.CurrentStatus == 1 ) ) );
}

Linq To Entities: Checking Multiple Columns Against against a Set of Values

I have a MySQL DB that I'm trying to query with the Entity Framework.
I have a Match table that looks like this:
match_id, winning_hero1, winning_hero2, winning_hero3, winning_hero4, winning_hero5
with a winning_hero being identified by an integer.
given a list of hero ids, I want to return all matches where these heroes were all on the winning team together.
something like this:
public List<MatchEntity> GetAllMatchesWithWinningHeroes(List<int> heroList)
{
List<MatchEntity> matchList = null;
using (var context = new dotaEntities())
{
switch (heroList.Count)
{
case 1:
matchList = context.MatchEntities.Where(m => m.winning_hero1 == heroList[0]
|| m.winning_hero2 == heroList[0]
|| m.winning_hero3 == heroList[0]
|| m.winning_hero4 == heroList[0]
|| m.winning_hero5 == heroList[0]).ToList();
break;
case 2:
matchList = context.MatchEntities.Where(m => (m.winning_hero1 == heroList[0]
|| m.winning_hero2 == heroList[0]
|| m.winning_hero3 == heroList[0]
|| m.winning_hero4 == heroList[0]
|| m.winning_hero5 == heroList[0])
&& (m.winning_hero1 == heroList[1]
|| m.winning_hero2 == heroList[1]
|| m.winning_hero3 == heroList[1]
|| m.winning_hero4 == heroList[1]
|| m.winning_hero5 == heroList[1])).ToList();
break;
case 3:
//etc..
}
return matchList;
}
}
Is there a "nicer" way to go about selecting these matches?
Thank you.

What you need is this:
using (var context = new dotaEntities())
{
var matchList = context.MatchEntities.Where(m =>
heroList.Contains(m.winning_hero1) &&
heroList.Contains(m.winning_hero2) &&
heroList.Contains(m.winning_hero3) &&
heroList.Contains(m.winning_hero4) &&
heroList.Contains(m.winning_hero5)).ToList();
}
But, your data design is wrong. In your Match table, you should only have a MatchID (PK), and probably MatchName or MatchDate, and in your Hero table, you should have HeroID (PK), HeroName,... Then, you should have a third table (MatchHero) just to create a many-to-many relationship between the two tables. In your MatchHero table, you'll have MatchID and HeroID.

I think this is what you want:
using (var context = new dotaEntities())
{
matchList = context.MatchEntities.Where(m=>
( heroList.Contains(m.winning_hero1)?1:0 +
heroList.Contains(m.winning_hero2)?1:0 +
heroList.Contains(m.winning_hero3)?1:0 +
heroList.Contains(m.winning_hero4)?1:0 +
heroList.Contains(m.winning_hero5)?1:0
) == heroList.Count
).ToList();
}
This assumes that all winning_heros are different.