I was originally using a foreach loop and then for each element in the loop, I perform a LINQ query like so:
foreach (MyObject identifier in identifiers.Where(i => i.IsMarkedForDeletion == false))
{
if (this.MyEntities.Identifiers.Where(pi => identifier.Field1 == pi.Field1 && identifier.Field2 == pi.Field2 && identifier.Field3 == pi.Field3).Any())
{
return false;
}
}
return true;
Then I modified it like so:
if (identifiers.Any(i => !i.IsMarkedForDeletion && this.MyEntities.Identifiers.Where(pi => i.Field1 == pi.Field1 && i.Field2 == pi.Field2 && i.Field3 == pi.Field3).Any()))
{
return false;
}
return true;
My question is this still the wrong way to use LINQ? Basically, I want to eliminate the need for the foreach loop (which seems like I should be able to get rid of it) and also make the DB query faster by not performing separate DB queries for each element of a list. Instead, I want to perform one query for all elements. Thanks!
You can change your code in this way, and it will be converted to SQL statement as expected.
To prevent runtime errors during transformation, it will be better to save DBSet to the IQueryable variable; identifiers should be IQueryable too, so you should change your code into something like this (to be honest, Resharper converted your foreach in this short labda):
IQueryable<MyObject2> identifiers = MyEntities.Identifiers.Where(i => i.IsMarkedForDeletion == false);
IQueryable<MyObject2> ids = MyEntities.Identifiers.AsQueryable();
return identifiers.All(identifier => !ids.Any(pi => identifier.Field1 == pi.Field1 && identifier.Field2 == pi.Field2 && identifier.Field3 == pi.Field3));
If identifiers is in memory collection you can change code in this way (hope that fields are string):
IQueryable<MyObject2> ids = MyEntities.Identifiers.AsQueryable();
string[] values = identifiers.Where(i => i.IsMarkedForDeletion == false).Select(i => String.Concat(i.Field1, i.Field2, i.Field3)).ToArray();
return !ids.Any(i => values.Contains(i.Field1 + i.Field2 + i.Field3));
Unfortunately your modified version will be executed exactly the same way (i.e. multiple database queries) as in the original foreach approach because EF does not support database query with joins to in memory collection (except for primitive and enumeration type collections), so if you try the most logical way
bool result = this.MyEntities.Identifiers.Any(pi => identifiers.Any(i =>
!i.IsMarkedForDeletion &&
i.Field1 == pi.Field1 && i.Field2 == pi.Field2 && i.Field3 == pi.Field3));
you'll get
NotSupportedException: Unable to create a constant value of type 'YourType'. Only primitive types or enumeration types are supported in this context.
The only way to let EF execute a single database query is to manually build a LINQ query with Concat per each item from in memory collection, like this
IQueryable<Identifier> query = null;
foreach (var item in identifiers.Where(i => !i.IsMarkedForDeletion))
{
var i = item;
var subquery = this.MyEntities.Identifiers.Where(pi =>
pi.Field1 == i.Field1 && pi.Field2 == i.Field2 && pi.Field3 == i.Field3);
query = query != null ? query.Concat(subquery) : subquery;
}
bool result = query != null && query.Any();
See Logging and Intercepting Database Operations of how to monitor the EF actions.
I would use it as follows:
if (identifiers.Where(i => !i.IsMarkedForDeletion &&
this.MyEntities.Identifiers.Field1 == i.Field1 &&
this.MyEntities.Identifiers.Field2 == i.Field2 &&
this.MyEntities.Identifiers.Field3 == i.Field3).Any()))
{
return false;
}
return true;
I hope this helps. Even though it is more to type out, it is more understandable and readable then using multiple 'where' statements.
The situation
I have a method that takes in a POCO. This POCO is like the following
private class SearchCriteria
{
public string name get;set;
public string age get;set;
public string talent get;set;
..
....
}
The method basically has a query to the db , that uses the above criteria.
public void query(SearchCriteria crit)
{
if(crit.name!=null && crit.age!=null && crit.talent !=null)
{
dbContext.Students.Where(c=>c.name ==crit.name && c.age==crit.age...)
}
else if(crit.name !=null && crit.age!=null)
{
}
else if(....
{
}
As you can see there is a definite problem above , where in, in case of large number of criteria, I will have to write a lot of if-elses to drop out specific arguments from the where clause .
The possible solution ?
I am actually new to the lambda expressions world but I believe we must be having a facility which would allow us to do something like below.
dbContext.Students.Where(processCriteria(searchCriteriaPOCO)).
Can you folks lead me to the proper direction ?. Thanks
Get a queryable and then keep adding where clauses to it. That way you only need to test each possible criteria the once and also only generate the number of where clauses that are absolutely needed.
IQueryable<Student> q = dbContext.Students.AsQueryable();
if (crit.name != null)
q = q.Where(c => c.name == crit.name);
if (crit.age != null)
q = q.Where(c => c.age== crit.age);
Let me start by saying that this answer uses the same basic idea as #PhilWright's answer. It just wraps it up in an extension method that applies this pattern for you, and allows you to have a syntax that reads nice.
public static class SearchExtensions
{
public static IQueryable<Student> Query(this SearchCriteria criteria, IQueryable<Student> studentQuery)
{
return studentQuery
.Match(criteria.name, (student) => student.name == criteria.name)
.Match(criteria.age, (student) => student.age == criteria.age)
.Match(criteria.talent, (student) => student.talent == criteria.talent);
// add expressions for other fields if needed.
}
private static IQueryable<Student> Match<T>(
this IQueryable<Student> studentQuery,
T criterionValue,
Expression<Func<Student, bool>> whereClause) where T : class
{
// only use the expression if the criterion value is non-null.
return criterionValue == null ? studentQuery : studentQuery.Where(whereClause);
}
}
You can then use it in your code like this:
var criteria = new SearchCriteria() {
name = "Alex",
talent = "Nosepicking"
};
var results = criteria.Query(dbContext.Students);
Maybe I'm missing something, as the code example is not the clearest I've seen, but for your specific example, I would think the following should be fine:
dbContext.Students.Where(c => (crit.name == null || crit.name == c.name) &&
(crit.age == null || crit.age == c.age) &&
(crit.talent == null || crit.talent == c.talent));
No need to chain a bunch of if statements.
For more complicated scenarios, you might prefer something like PredicateBuilder
You can use a pattern like this:
dbContext.Students.Where(c=>(crit.name == null || c.name ==crit.name) && ...)
A search criterion which is null will give a subexpression which is always true.
I've got a method which can accept an optional int? value as a number of items to Take from a collection. I want to return all items if a null value is passed. Right now I have to duplicate my query to accomplish this
if(take == null)
{
x = db.WalkingDeadEps.Where(x => x.BicyclesCouldHaveSavedLives == true).ToList()
}
else
{
x = db.WalkingDeadEps.Where(x => x.BicyclesCouldHaveSavedLives == true).Take(take).ToList()
}
Is there a simpler way? Something like this?
.Take(take != null ? take : "all")
with Linq you have the option to store your query in variables. it will not be executed until you call ToList or equivalent methods on it.
var query = db.WalkingDeadEps.Where(x => x.BicyclesCouldHaveSavedLives == true);
x = take.HasValue ? query.Take(take.Value).ToList() : query.ToList();
I have this function:
public static IQueryable<Article> WhereArticleIsLive(this IQueryable<Article> q)
{
return q.Where(x =>
x != null
&& DateTime.UtcNow >= x.PublishTime
&& x.IsPublished
&& !x.IsDeleted);
}
And it works just fine in this query:
from a in Articles.WhereArticleIsLive()
where a.Id == 5
select new { a.Title }
But it doesn't work in this only slightly more complex query:
from s in Series
from a in Articles.WhereArticleIsLive()
where s.Id == a.SeriesId
select new { s, a }
I get this error message:
NotSupportedException: LINQ to Entities does not recognize the method 'System.Linq.IQueryable1[TheFraser.Data.Articles.Article] WhereArticleIsLive(System.Linq.IQueryable1[TheFraser.Data.Articles.Article])' method, and this method cannot be translated into a store expression.
Any idea why? Is there another way to consolidate query parameters like this?
Thanks in advance.
EDIT: corrections by Craig.
I'm leaving this here, because I think it's a valuable tool: Use linqkit! But not for solving this question though :-)
Instead of returning IQueryable, use Expression to factor out predicates. E.g. you could define the following static method on Article:
public static Expression<Func<Article,bool>> IsLive()
{
return x =>
x != null
&& DateTime.UtcNow >= x.PublishTime
&& x.IsPublished
&& !x.IsDeleted
}
Then, ensure to store a reference to this expression when building your query, something along the lines of (not tested):
var isLive = Article.IsLive();
from s in Series
from a in Articles.Where(isLive)
where s.Id == a.SeriesId
select new { s, a }
I've got a lot of ugly code that looks like this:
if (!string.IsNullOrEmpty(ddlFileName.SelectedItem.Text))
results = results.Where(x => x.FileName.Contains(ddlFileName.SelectedValue));
if (chkFileName.Checked)
results = results.Where(x => x.FileName == null);
if (!string.IsNullOrEmpty(ddlIPAddress.SelectedItem.Text))
results = results.Where(x => x.IpAddress.Contains(ddlIPAddress.SelectedValue));
if (chkIPAddress.Checked)
results = results.Where(x => x.IpAddress == null);
...etc.
results is an IQueryable<MyObject>.
The idea is that for each of these innumerable dropdowns and checkboxes, if the dropdown has something selected, the user wants to match that item. If the checkbox is checked, the user wants specifically those records where that field is null or an empty string. (The UI doesn't let both be selected at the same time.) This all adds to the LINQ Expression which gets executed at the end, after we've added all the conditions.
It seems like there ought to be some way to pull out an Expression<Func<MyObject, bool>> or two so that I can put the repeated parts in a method and just pass in what changes. I've done this in other places, but this set of code has me stymied. (Also, I'd like to avoid "Dynamic LINQ", because I want to keep things type-safe if possible.) Any ideas?
I'd convert it into a single Linq statement:
var results =
//get your inital results
from x in GetInitialResults()
//either we don't need to check, or the check passes
where string.IsNullOrEmpty(ddlFileName.SelectedItem.Text) ||
x.FileName.Contains(ddlFileName.SelectedValue)
where !chkFileName.Checked ||
string.IsNullOrEmpty(x.FileName)
where string.IsNullOrEmpty(ddlIPAddress.SelectedItem.Text) ||
x.FileName.Contains(ddlIPAddress.SelectedValue)
where !chkIPAddress.Checked ||
string.IsNullOrEmpty(x. IpAddress)
select x;
It's no shorter, but I find this logic clearer.
In that case:
//list of predicate functions to check
var conditions = new List<Predicate<MyClass>>
{
x => string.IsNullOrEmpty(ddlFileName.SelectedItem.Text) ||
x.FileName.Contains(ddlFileName.SelectedValue),
x => !chkFileName.Checked ||
string.IsNullOrEmpty(x.FileName),
x => string.IsNullOrEmpty(ddlIPAddress.SelectedItem.Text) ||
x.IpAddress.Contains(ddlIPAddress.SelectedValue),
x => !chkIPAddress.Checked ||
string.IsNullOrEmpty(x.IpAddress)
}
//now get results
var results =
from x in GetInitialResults()
//all the condition functions need checking against x
where conditions.All( cond => cond(x) )
select x;
I've just explicitly declared the predicate list, but these could be generated, something like:
ListBoxControl lbc;
CheckBoxControl cbc;
foreach( Control c in this.Controls)
if( (lbc = c as ListBoxControl ) != null )
conditions.Add( ... );
else if ( (cbc = c as CheckBoxControl ) != null )
conditions.Add( ... );
You would need some way to check the property of MyClass that you needed to check, and for that you'd have to use reflection.
Have you seen the LINQKit? The AsExpandable sounds like what you're after (though you may want to read the post Calling functions in LINQ queries at TomasP.NET for more depth).
Don't use LINQ if it's impacting readability. Factor out the individual tests into boolean methods which can be used as your where expression.
IQueryable<MyObject> results = ...;
results = results
.Where(TestFileNameText)
.Where(TestFileNameChecked)
.Where(TestIPAddressText)
.Where(TestIPAddressChecked);
So the the individual tests are simple methods on the class. They're even individually unit testable.
bool TestFileNameText(MyObject x)
{
return string.IsNullOrEmpty(ddlFileName.SelectedItem.Text) ||
x.FileName.Contains(ddlFileName.SelectedValue);
}
bool TestIPAddressChecked(MyObject x)
{
return !chkIPAddress.Checked ||
x.IpAddress == null;
}
results = results.Where(x =>
(string.IsNullOrEmpty(ddlFileName.SelectedItem.Text) || x.FileName.Contains(ddlFileName.SelectedValue))
&& (!chkFileName.Checked || string.IsNullOrEmpty(x.FileName))
&& ...);
Neither of these answers so far is quite what I'm looking for. To give an example of what I'm aiming at (I don't regard this as a complete answer either), I took the above code and created a couple of extension methods:
static public IQueryable<Activity> AddCondition(
this IQueryable<Activity> results,
DropDownList ddl,
Expression<Func<Activity, bool>> containsCondition)
{
if (!string.IsNullOrEmpty(ddl.SelectedItem.Text))
results = results.Where(containsCondition);
return results;
}
static public IQueryable<Activity> AddCondition(
this IQueryable<Activity> results,
CheckBox chk,
Expression<Func<Activity, bool>> emptyCondition)
{
if (chk.Checked)
results = results.Where(emptyCondition);
return results;
}
This allowed me to refactor the code above into this:
results = results.AddCondition(ddlFileName, x => x.FileName.Contains(ddlFileName.SelectedValue));
results = results.AddCondition(chkFileName, x => x.FileName == null || x.FileName.Equals(string.Empty));
results = results.AddCondition(ddlIPAddress, x => x.IpAddress.Contains(ddlIPAddress.SelectedValue));
results = results.AddCondition(chkIPAddress, x => x.IpAddress == null || x.IpAddress.Equals(string.Empty));
This isn't quite as ugly, but it's still longer than I'd prefer. The pairs of lambda expressions in each set are obviously very similar, but I can't figure out a way to condense them further...at least not without resorting to dynamic LINQ, which makes me sacrifice type safety.
Any other ideas?
#Kyralessa,
You can create extension method AddCondition for predicates that accepts parameter of type Control plus lambda expression and returns combined expression. Then you can combine conditions using fluent interface and reuse your predicates. To see example of how it can be implemented see my answer on this question:
How do I compose existing Linq Expressions
I'd be wary of the solutions of the form:
// from Keith
from x in GetInitialResults()
//either we don't need to check, or the check passes
where string.IsNullOrEmpty(ddlFileName.SelectedItem.Text) ||
x.FileName.Contains(ddlFileName.SelectedValue)
My reasoning is variable capture. If you're immediately execute just the once you probably won't notice a difference. However, in linq, evaluation isn't immediate but happens each time iterated occurs. Delegates can capture variables and use them outside the scope you intended.
It feels like you're querying too close to the UI. Querying is a layer down, and linq isn't the way for the UI to communicate down.
You may be better off doing the following. Decouple the searching logic from the presentation - it's more flexible and reusable - fundamentals of OO.
// my search parameters encapsulate all valid ways of searching.
public class MySearchParameter
{
public string FileName { get; private set; }
public bool FindNullFileNames { get; private set; }
public void ConditionallySearchFileName(bool getNullFileNames, string fileName)
{
FindNullFileNames = getNullFileNames;
FileName = null;
// enforce either/or and disallow empty string
if(!getNullFileNames && !string.IsNullOrEmpty(fileName) )
{
FileName = fileName;
}
}
// ...
}
// search method in a business logic layer.
public IQueryable<MyClass> Search(MySearchParameter searchParameter)
{
IQueryable<MyClass> result = ...; // something to get the initial list.
// search on Filename.
if (searchParameter.FindNullFileNames)
{
result = result.Where(o => o.FileName == null);
}
else if( searchParameter.FileName != null )
{ // intermixing a different style, just to show an alternative.
result = from o in result
where o.FileName.Contains(searchParameter.FileName)
select o;
}
// search on other stuff...
return result;
}
// code in the UI ...
MySearchParameter searchParameter = new MySearchParameter();
searchParameter.ConditionallySearchFileName(chkFileNames.Checked, drpFileNames.SelectedItem.Text);
searchParameter.ConditionallySearchIPAddress(chkIPAddress.Checked, drpIPAddress.SelectedItem.Text);
IQueryable<MyClass> result = Search(searchParameter);
// inform control to display results.
searchResults.Display( result );
Yes it's more typing, but you read code around 10x more than you write it. Your UI is clearer, the search parameters class takes care of itself and ensures mutually exclusive options don't collide, and the search code is abstracted away from any UI and doesn't even care if you use Linq at all.
Since you are wanting to repeatedly reduce the original results query with innumerable filters, you can use Aggregate(), (which corresponds to reduce() in functional languages).
The filters are of predictable form, consisting of two values for every member of MyObject - according to the information I gleaned from your post. If every member to be compared is a string, which may be null, then I recommend using an extension method, which allows for null references to be associated to an extension method of its intended type.
public static class MyObjectExtensions
{
public static bool IsMatchFor(this string property, string ddlText, bool chkValue)
{
if(ddlText!=null && ddlText!="")
{
return property!=null && property.Contains(ddlText);
}
else if(chkValue==true)
{
return property==null || property=="";
}
// no filtering selected
return true;
}
}
We now need to arrange the property filters in a collection, to allow for iterating over many. They are represented as Expressions for compatibility with IQueryable.
var filters = new List<Expression<Func<MyObject,bool>>>
{
x=>x.Filename.IsMatchFor(ddlFileName.SelectedItem.Text,chkFileName.Checked),
x=>x.IPAddress.IsMatchFor(ddlIPAddress.SelectedItem.Text,chkIPAddress.Checked),
x=>x.Other.IsMatchFor(ddlOther.SelectedItem.Text,chkOther.Checked),
// ... innumerable associations
};
Now we aggregate the innumerable filters onto the initial results query:
var filteredResults = filters.Aggregate(results, (r,f) => r.Where(f));
I ran this in a console app with simulated test values, and it worked as expected. I think this at least demonstrates the principle.
One thing you might consider is simplifying your UI by eliminating the checkboxes and using an "<empty>" or "<null>" item in your drop down list instead. This would reduce the number of controls taking up space on your window, remove the need for complex "enable X only if Y is not checked" logic, and would enable a nice one-control-per-query-field.
Moving on to your result query logic, I would start by creating a simple object to represent a filter on your domain object:
interface IDomainObjectFilter {
bool ShouldInclude( DomainObject o, string target );
}
You can associate an appropriate instance of the filter with each of your UI controls, and then retrieve that when the user initiates a query:
sealed class FileNameFilter : IDomainObjectFilter {
public bool ShouldInclude( DomainObject o, string target ) {
return string.IsNullOrEmpty( target )
|| o.FileName.Contains( target );
}
}
...
ddlFileName.Tag = new FileNameFilter( );
You can then generalize your result filtering by simply enumerating your controls and executing the associated filter (thanks to hurst for the Aggregate idea):
var finalResults = ddlControls.Aggregate( initialResults, ( c, r ) => {
var filter = c.Tag as IDomainObjectFilter;
var target = c.SelectedValue;
return r.Where( o => filter.ShouldInclude( o, target ) );
} );
Since your queries are so regular, you might be able to simplify the implementation even further by using a single filter class taking a member selector:
sealed class DomainObjectFilter {
private readonly Func<DomainObject,string> memberSelector_;
public DomainObjectFilter( Func<DomainObject,string> memberSelector ) {
this.memberSelector_ = memberSelector;
}
public bool ShouldInclude( DomainObject o, string target ) {
string member = this.memberSelector_( o );
return string.IsNullOrEmpty( target )
|| member.Contains( target );
}
}
...
ddlFileName.Tag = new DomainObjectFilter( o => o.FileName );