We Keep Coding

C# - How to use Expression<Func> with parametrs in LINQ where

Task: I need to give some expression with parameters into LINQ's where to get some data from database, but have an error above
This example of working expression:
var shopExp = GetPersonForShop(PersonTypeIds.Director, new Guid("adda423f-8c38-40e0-9f39-6deceb787bc0")); // id
Where(shopExp)
But i need assign id dynamically, but got error above :
_repository.Persons
.Where(GetPersonForShop(PersonTypeIds.Director, person.PersonId)
And got error:
{"Unable to cast object of type 'System.Linq.Expressions.InstanceMethodCallExpression2' to type 'System.Linq.Expressions.LambdaExpression'."}
How does function for where(linq) look:
private Expression<Func<Person, bool>> GetPersonForShop(PersonTypeIds personTypeId, Guid personId)
{
return person => person .PeronTypeId== (int) personTypeId && person .PersonId == personId;
}
This is approximate look like out production, just change names of parametrs code
How can I add expression with parameters to Where clause??

Lambda expressions use => notation. Try something like this:
var idToFind = new Guid("adda423f-8c38-40e0-9f39-6deceb787bc0");
var result = _repository.Persons
.Where(p => p.TypeId == PersonTypeIds.Director && p.PersonId == idToFind);
In this expression, p represents each Person record in the Persons table, compared one-by-one using the boolean expression that follows it.
Depending on your datasource, the comparison for each p will either be done by .NET in memory, or it will happen inside your database using a SQL WHERE clause which is constructed from the boolean expression. The last would be optimal because it would mean that not the entire Persons table has to be transferred into .NET memory before comparison can take place.
Update - To apply the same condition multiple times without repeating it in your code, while still keeping the advantages of LINQ to SQL translation intact, you can put the condition in an Expression<Func<Person, bool>> object and then use that multiple times:
Expression<Func<Person, bool>> expression =
p => p.TypeId == PersonTypeIds.Director && p.PersonId == idToFind;
var result1 = datasource1.Where(expression);
var result2 = datasource2.Where(expression);
var result3 = datasource3.Where(expression);
Or through a method that produces the Expression object:
var result1 = datasource1.Where(GetExpression(idToFind));
var result2 = datasource2.Where(GetExpression(idToFind));
var result3 = datasource3.Where(GetExpression(idToFind));
public Expression<Func<Person, bool>> GetExpression(Guid idToFind)
{
return p => p.TypeId == PersonTypeIds.Director && p.PersonId == idToFind;
}
Or alternatively you can use a helper method:
var result1 = FilterByTypeAndId(datasource1, idToFind);
var result2 = FilterByTypeAndId(datasource2, idToFind);
var result3 = FilterByTypeAndId(datasource3, idToFind);
public IQueryable<Person> FilterByTypeAndId(IQueryable<Person> datasource, Guid idToFind)
{
return datasource.Where(p => p.TypeId == PersonTypeIds.Director && p.PersonId == idToFind);
}

based on the previous response, I am going to give you a few alternatives and suggestions.
var idToFind = new Guid("adda423f-8c38-40e0-9f39-6deceb787bc0");
var result = _repository
.Persons
.Where(p => p.TypeId == PersonTypeIds.Director)
.Where(p => p.PersonId == idToFind)
.ToList();
First is doing the where clause in 2 steps and then, adding the ToList(), with the ToList(), you will deal with collections and LINQ that is pretty useful. And by doing the where clause in 2 steps, is more for readable purposes.

Linq query takes too long when using Func & OrderByDescending

Consider this code:
public List<Clients> GetFilteredClients(DateTime? FromDate = null,
DateTime? ToDate = null,
int? fromLocationType = null,
int? toLocationType = null)
{
Func<Clients, bool> fromDateFilter = f => true;
if (FromDate.HasValue)
{
fromDateFilter = z => z.Insert_Date.Value.Date >= FromDate.Value.Date;
}
Func<Clients, bool> toDateFilter = f => true;
if (ToDate.HasValue)
{
toDateFilter = z => z.Insert_Date.Value.Date <= ToDate.Value.Date;
}
Func<Clients, bool> fromLocationTypeFilter = f => true;
if (fromLocationType.HasValue)
{
fromOrgFilter = z => z.LocationTypeId >= fromLocationType.Value;
}
Func<Clients, bool> toLocationTypeFilter = f => true;
if (toLocationType.HasValue)
{
toLocationTypeFilter = z => z.LocationTypeId <= toLocationType.Value;
}
var filtered = DB_Context.Clients
.Where(fromDateFilter)
.Where(toDateFilter)
.Where(fromLocationTypeFilter)
.Where(toLocationTypeFilter)
.OrderByDescending(k => k.Id)
.Take(1000)
.ToList();
return filtered;
}
I have something like 100K records in the DB, I need only the top 1000 that answer to the requirements of:
.Where(fromDateFilter)
.Where(toDateFilter)
.Where(fromLocationTypeFilter)
.Where(toLocationTypeFilter)
However the execution time still takes something like 10 seconds.
Any idea why?

You must use Expression<Func<...>> rather than Func<...>. When you use Func, only the enumerable methods can be used on the queryable, which in this case means you first download everything to memory, and then do the filtering. If you switch over to Expression<...>, the O/RM will do the filtering on the DB server, rather than in your application.
Also, there's better ways to do what you're doing. For example, you can build the conditions like so:
var query = DB_Context.Clients.AsQueryable();
if (FromDate.HasValue) query = query.Where(...);
if (ToDate.HasValue) query = query.Where(...);
...
return query.OrderByDescending(k => k.Id).Take(1000).ToList();
Of course, this means that whatever DB provider you're using must be able to support the kind of filtering you're trying to do - you'll need to consult the documentation.

You are using delegates instead LINQ expressions. That leads to processing a data by your application and not by SQL Server.
LINQ expressions look like lambda expressions thanks for the syntax, but they are not same thing. The compiler takes a decision what to create (delegates or LINQ expressions) depending on the situation.
If an object implements the IQueriable interface, then the compiler uses the Queryable class and generates LINQ expression trees, which later can be translated into a SQL query or other form by the specific IQueryProvider.
Otherwise, the compiler uses extensions from the Enumerable class, which create iterators over source collection (all records from the table in your case).
As an example. The code bellow will be compilled into LINQ expressions.
// Source code
IQueryable<Clients> source = null;
IQueryable<Clients> result = source.Where(c => c.LocationTypeId >= 1);
// Compiller generated code
IQueryable<Clients> source = null;
Expression parameterC = Expression.Parameter(typeof(Clients), "c");
IQueryable<Clients> result = Queryable.Where<Clients>(
source,
Expression.Lambda<Func<Clients, bool>>(
Expression.LessThanOrEqual(
Expression.Property(
parameterC ,
typeof(Clients).GetProperty("LocationTypeId").GetGetMethod()
),
Expression.Constant(1, typeof(int))
),
new ParameterExpression[]
{
parameterC
}
);
And this code uses delegates:
// Source code
IQueryable<Clients> source = null;
Func<Clients, bool> filter = c => c.LocationTypeId >= 1;
IEnumerable<Clients> result = source.Where(filter );
// Compiller generated code
IQueryable<Clients> source = null;
Func<Clients, bool> filter = c => c.LocationTypeId >= 1;
IEnumerable<Clients> result = Enumerable.Where(source, filter);
So, to solve you problem use Expression<Func<Clients, bool>> instead of Func<Clients, bool>:
IQueryable<Clients> result = DB_Context.Clients;
if (someFilter.HasValue)
result = result.Where(c => c.SomeProperty == someFilter.Value);
// other filters
return query
.OrderByDescending(k => k.Id)
.Take(1000)
.ToList();

Is there a neat way of doing a ToList within a LINQ query using query syntax?

Consider the code below:
StockcheckJobs =
(from job in (from stockcheckItem in MDC.StockcheckItems
where distinctJobs.Contains(stockcheckItem.JobId)
group stockcheckItem by new { stockcheckItem.JobId, stockcheckItem.JobData.EngineerId } into jobs
select jobs).ToList()
let date = MJM.GetOrCreateJobData(job.Key.JobId).CompletedJob.Value
orderby date descending
select new StockcheckJobsModel.StockcheckJob()
{
JobId = job.Key.JobId,
Date = date,
Engineer = (EngineerModel)job.Key.EngineerId,
MatchingLines = job.Count(sti => sti.Quantity == sti.ExpectedQuantity),
DifferingLines = job.Count(sti => sti.Quantity != sti.ExpectedQuantity)
}).ToList()
There is a ToList() in the middle because the GetOrCreateJobData method can't be translated into sql.
As a result I've had to surround the first part of my query in brackets to do this, then I've used an outer query to finish up.
I know I could split this into two variables, but I don't want to do that (this is within an object initialiser too).
Is there some other syntax I can use to increase readability, preferably removing the need for an outer an inner query, when I have to do a ToList (or otherwise get to linq-to-objects) in the middle of a linq query?
In an ideal world I'd like something like this (as close as is possible anyway):
StockcheckJobs =
from stockcheckItem in MDC.StockcheckItems
where distinctJobs.Contains(stockcheckItem.JobId)
group stockcheckItem by new { stockcheckItem.JobId, stockcheckItem.JobData.EngineerId } into jobs
MAGIC_DO_BELOW_AS_LINQ-TO-OBJECTS_KEYWORD_OR_SYNTAX
let date = MJM.GetOrCreateJobData(jobs.Key.JobId).CompletedJob.Value
orderby date descending
select new StockcheckJobsModel.StockcheckJob()
{
JobId = jobs.Key.JobId,
Date = date,
Engineer = new ThreeSixtyScheduling.Models.EngineerModel() { Number = jobs.Key.EngineerId },
MatchingLines = jobs.Count(sti => sti.Quantity == sti.ExpectedQuantity),
DifferingLines = jobs.Count(sti => sti.Quantity != sti.ExpectedQuantity)
};

You can fix the issue of GetOrCreateJobData not being translatable to SQL.
By implementing a custom query translator for the specified method call expression, you can gain control over how LINQ-to-SQL interprets the method. There is a good article explaining this procedure and linking to relevant resources available at: http://www.codeproject.com/Articles/32968/QueryMap-Custom-translation-of-LINQ-expressions
Alternatively, you could refactor the GetOrCreateJobData method to an extension method which builds the same logic with expressions, so that LINQ-to-SQL can interpret it naturally. Depending on the complexity of the method, this may be more or less feasible than my first suggestion.

I would raise two points with the question:
I really don't think there's any readability issue with introducing an extra variable here. In fact, I think it makes it more readable as it separates the "locally executing" code from the code executing on the database.
To simply switch to LINQ-To-Objects, AsEnumerable is preferable to ToList.
That said, here's how you can stay in query-land all the way without an intermediate AsEnumerable() / ToList() on the entire query-expression : by tricking the C# compiler into using your custom extension methods rather than the BCL. This is possible since C# uses a "pattern-based" approach (rather than being coupled with the BCL) to turn query-expressions into method-calls and lambdas.
Declare evil classes like these:
public static class To
{
public sealed class ToList { }
public static readonly ToList List;
// C# should target this method when you use "select To.List"
// inside a query expression.
public static List<T> Select<T>
(this IEnumerable<T> source, Func<T, ToList> projector)
{
return source.ToList();
}
}
public static class As
{
public sealed class AsEnumerable { }
public static readonly AsEnumerable Enumerable;
// C# should target this method when you use "select As.Enumerable"
// inside a query expression.
public static IEnumerable<T> Select<T>
(this IEnumerable<T> source, Func<T, AsEnumerable> projector)
{
return source;
}
}
And then you can write queries like this:
List<int> list = from num in new[] { 41 }.AsQueryable()
select num + 1 into result
select To.List;
IEnumerable<int> seq = from num in new[] { 41 }.AsQueryable()
select num + 1 into result
select As.Enumerable into seqItem
select seqItem + 1; // Subsequent processing
In your case, your query would become:
StockcheckJobs =
from stockcheckItem in MDC.StockcheckItems
where distinctJobs.Contains(stockcheckItem.JobId)
group stockcheckItem by new { stockcheckItem.JobId, stockcheckItem.JobData.EngineerId } into jobs
select As.Enumerable into localJobs // MAGIC!
let date = MJM.GetOrCreateJobData(localJobs.Key.JobId).CompletedJob.Value
orderby date descending
select new StockcheckJobsModel.StockcheckJob()
{
JobId = localJobs.Key.JobId,
Date = date,
Engineer = new ThreeSixtyScheduling.Models.EngineerModel() { Number = localJobs.Key.EngineerId },
MatchingLines = localJobs.Count(sti => sti.Quantity == sti.ExpectedQuantity),
DifferingLines = localJobs.Count(sti => sti.Quantity != sti.ExpectedQuantity)
};
I really don't see this as any sort of improvement, though. Rather, it's pretty heavy abuse of a language feature.

I find that using method syntax makes things clearer, but that's just personal preference. It certainly makes the top half of the query better, but using a let, while possible in method syntax, is a bit more work.
var result = stockcheckItem in MDC.StockcheckItems
.Where(item => distinctJobs.Contains(item.JobId))
.GroupBy(item => new { item.JobId, item.JobData.EngineerId })
.AsEnumerable() //switch from Linq-to-sql to Linq-to-objects
.Select(job => new StockcheckJobsModel.StockcheckJob()
{
JobId = job.Key.JobId,
Date = MJM.GetOrCreateJobData(job.Key.JobId).CompletedJob.Value,
Engineer = (EngineerModel)job.Key.EngineerId,
MatchingLines = job.Count(sti => sti.Quantity == sti.ExpectedQuantity),
DifferingLines = job.Count(sti => sti.Quantity != sti.ExpectedQuantity)
})
.Orderby(item => item.Date)
.ToList()

One option is to do all the SQL compatible work up front into an anonymous type,
var jobs =
(from job in (from stockcheckItem in MDC.StockcheckItems
where distinctJobs.Contains(stockcheckItem.JobId)
group stockcheckItem by new
{ stockcheckItem.JobId, stockcheckItem.JobData.EngineerId }
into jobs
select new
{
JobId = job.Key.JobId,
Engineer = (EngineerModel)job.Key.EngineerId,
MatchingLines =
job.Count(sti => sti.Quantity == sti.ExpectedQuantity),
DifferingLines =
job.Count(sti => sti.Quantity != sti.ExpectedQuantity)
}
).AsEnumerable()
StockcheckJobs = jobs.Select(j => new StockcheckJobsModel.StockcheckJob
{
JobId = j.JobId,
Date = MJM.GetOrCreateJobData(j.JobId).CompletedJob.Value,
Engineer = j.EngineerId,
MatchingLines = j.MatchingLines,
DifferingLines = j.DifferingLines
}).OrderBy(j => j.Date).ToList();
Obviously not tested, but you get the idea.

Adding a where/order by clause to an IQueryable

I have ths function to query a set of records from the DB:
public IQueryable<PointTransactionViewModel> GetPointTransactions(int UserID)
{
return
(
from PointTransaction p in entities.PointTransaction
join ActivityLog a in entities.ActivityLog
on p.TransactionID equals a.TransactionID
where p.UserID == UserID
select new PointTransactionViewModel
{
ID = p.TransactionID,
Balance = p.Balance,
Points = p.Amount,
RelatedActivityID = a.ID,
When = p.When,
Sender = p.SenderUserInfo.CompleteName
}
);
}
I wish to add an additional cause, like this
var entries = GetPointTransaction(1);
return entries.OrderbyDescending.Where( x => x.When >= start && w.When <= end).
( x => x.When);
However, I seem to need to create a new query from the existing one for this to work. But, I have seem this work before without creating a new query, in the code snippet before:
public PaginatedList(IQueryable<T> source, int pageIndex, int pageSize)
{
PageIndex = pageIndex;
PageSize = pageSize;
TotalCount = source.Count();
TotalPages = (int)Math.Ceiling(TotalCount / (double)PageSize);
this.AddRange(source.Skip(PageIndex * PageSize).Take(PageSize));
}
Does the code above somehow doesn't need a new query to be created for the IQueryable source object? Was a temporary object created?
Edit
It's strange, but to get it to work I have to do the following:
IQueryable<ActivityLogEntry> log = activityRepo.GetPointTransaction(userID).
Where(x => x.PointsEarned == 50);
return log.ToList();
The following will not work:
var log = = activityRepo.GetPointTransaction(userID);
log.Where( x => x.PointsEarned == 50);
return log.ToList();
There is no error message, just that the where clause seems to be ignored (it is also returning all data which PointsEarned is not 50)

Your entries is of IQueryable type, that's enough and you can add any number of clauses before fetching the data, e.g. before calling the ToList() function.
It doesn't execute the SQL code, just an expression tree will be created until you fetch the whole data with one of the existing methods (again, e.g. the ToList() function).
var query = context.Where(x=>x.id == test);
query = query.Where(anotherCondition1);
query = query.Where(anotherCondition2);
...
var result = query.ToList();
it's equal to
var result = context.Where(x=>x.id == test)
.Where(anotherCondition1)
.Where(anotherCondition2)
....
.ToList()
This is called deferred execution, for more details see the MSDN blog post on LINQ and Deferred Execution.

You do need to create a new object. IQueryable is immutable. Don't worry this is how you are supposed to do it. This is how the queries are formed internally. All the extension methods like "Where" don't actually change the object. They just return a new one.
The code that you claim works should not work. The method doesn't even have a type.

i mean you can write this sample :
opportunites = from opp in oppDC.Opportunities
join org in oppDC.Organizations on opp.OrganizationID equals org.OrgnizationID
select new
{
opp.OpportunityID,
opp.Title,
opp.PostedBy,
opp.Address1,
opp.CreatedDate,
org.OrganizationName
};
if(condition)
{
opportunites = opportunites.Where(opp => opp.Title.StartsWith(title));
}
//------Other Condition you need
if(!String.IsNullOrEmpty(title))
{
opportunites = opportunites.Where(.....);
}
if(!String.IsNullOrEmpty(name))
{
opportunites = opportunites.Where(.....);
}

As others have pointed out, you do not need a new object. Your syntax for OrderByDescending is wrong though, you need to specify the key selector.
var entries = GetPointTransaction(1);
return entries.Where(x => x.When >= start && w.When <= end).OrderbyDescending(x => x.When);

How can I conditionally apply a Linq operator?

We're working on a Log Viewer. The use will have the option to filter by user, severity, etc. In the Sql days I'd add to the query string, but I want to do it with Linq. How can I conditionally add where-clauses?

if you want to only filter if certain criteria is passed, do something like this
var logs = from log in context.Logs
select log;
if (filterBySeverity)
logs = logs.Where(p => p.Severity == severity);
if (filterByUser)
logs = logs.Where(p => p.User == user);
Doing so this way will allow your Expression tree to be exactly what you want. That way the SQL created will be exactly what you need and nothing less.

If you need to filter base on a List / Array use the following:
public List<Data> GetData(List<string> Numbers, List<string> Letters)
{
if (Numbers == null)
Numbers = new List<string>();
if (Letters == null)
Letters = new List<string>();
var q = from d in database.table
where (Numbers.Count == 0 || Numbers.Contains(d.Number))
where (Letters.Count == 0 || Letters.Contains(d.Letter))
select new Data
{
Number = d.Number,
Letter = d.Letter,
};
return q.ToList();
}

I ended using an answer similar to Daren's, but with an IQueryable interface:
IQueryable<Log> matches = m_Locator.Logs;
// Users filter
if (usersFilter)
matches = matches.Where(l => l.UserName == comboBoxUsers.Text);
// Severity filter
if (severityFilter)
matches = matches.Where(l => l.Severity == comboBoxSeverity.Text);
Logs = (from log in matches
orderby log.EventTime descending
select log).ToList();
That builds up the query before hitting the database. The command won't run until .ToList() at the end.

I solved this with an extension method to allow LINQ to be conditionally enabled in the middle of a fluent expression. This removes the need to break up the expression with if statements.
.If() extension method:
public static IQueryable<TSource> If<TSource>(
this IQueryable<TSource> source,
bool condition,
Func<IQueryable<TSource>, IQueryable<TSource>> branch)
{
return condition ? branch(source) : source;
}
This allows you to do this:
return context.Logs
.If(filterBySeverity, q => q.Where(p => p.Severity == severity))
.If(filterByUser, q => q.Where(p => p.User == user))
.ToList();
Here's also an IEnumerable<T> version which will handle most other LINQ expressions:
public static IEnumerable<TSource> If<TSource>(
this IEnumerable<TSource> source,
bool condition,
Func<IEnumerable<TSource>, IEnumerable<TSource>> branch)
{
return condition ? branch(source) : source;
}

When it comes to conditional linq, I am very fond of the filters and pipes pattern.
http://blog.wekeroad.com/mvc-storefront/mvcstore-part-3/
Basically you create an extension method for each filter case that takes in the IQueryable and a parameter.
public static IQueryable<Type> HasID(this IQueryable<Type> query, long? id)
{
return id.HasValue ? query.Where(o => i.ID.Equals(id.Value)) : query;
}

Doing this:
bool lastNameSearch = true/false; // depending if they want to search by last name,
having this in the where statement:
where (lastNameSearch && name.LastNameSearch == "smith")
means that when the final query is created, if lastNameSearch is false the query will completely omit any SQL for the last name search.

Another option would be to use something like the PredicateBuilder discussed here.
It allows you to write code like the following:
var newKids = Product.ContainsInDescription ("BlackBerry", "iPhone");
var classics = Product.ContainsInDescription ("Nokia", "Ericsson")
.And (Product.IsSelling());
var query = from p in Data.Products.Where (newKids.Or (classics))
select p;
Note that I've only got this to work with Linq 2 SQL. EntityFramework does not implement Expression.Invoke, which is required for this method to work. I have a question regarding this issue here.

It isn't the prettiest thing but you can use a lambda expression and pass your conditions optionally. In TSQL I do a lot of the following to make parameters optional:
WHERE Field = #FieldVar OR #FieldVar IS NULL
You could duplicate the same style with a the following lambda (an example of checking authentication):
MyDataContext db = new MyDataContext();
void RunQuery(string param1, string param2, int? param3){
Func checkUser = user =>
((param1.Length > 0)? user.Param1 == param1 : 1 == 1) &&
((param2.Length > 0)? user.Param2 == param2 : 1 == 1) &&
((param3 != null)? user.Param3 == param3 : 1 == 1);
User foundUser = db.Users.SingleOrDefault(checkUser);
}

I had a similar requirement recently and eventually found this in he MSDN.
CSharp Samples for Visual Studio 2008
The classes included in the DynamicQuery sample of the download allow you to create dynamic queries at runtime in the following format:
var query =
db.Customers.
Where("City = #0 and Orders.Count >= #1", "London", 10).
OrderBy("CompanyName").
Select("new(CompanyName as Name, Phone)");
Using this you can build a query string dynamically at runtime and pass it into the Where() method:
string dynamicQueryString = "City = \"London\" and Order.Count >= 10";
var q = from c in db.Customers.Where(queryString, null)
orderby c.CompanyName
select c;

You can create and use this extension method
public static IQueryable<TSource> WhereIf<TSource>(this IQueryable<TSource> source, bool isToExecute, Expression<Func<TSource, bool>> predicate)
{
return isToExecute ? source.Where(predicate) : source;
}

Just use C#'s && operator:
var items = dc.Users.Where(l => l.Date == DateTime.Today && l.Severity == "Critical")
Edit: Ah, need to read more carefully. You wanted to know how to conditionally add additional clauses. In that case, I have no idea. :) What I'd probably do is just prepare several queries, and execute the right one, depending on what I ended up needing.

You could use an external method:
var results =
from rec in GetSomeRecs()
where ConditionalCheck(rec)
select rec;
...
bool ConditionalCheck( typeofRec input ) {
...
}
This would work, but can't be broken down into expression trees, which means Linq to SQL would run the check code against every record.
Alternatively:
var results =
from rec in GetSomeRecs()
where
(!filterBySeverity || rec.Severity == severity) &&
(!filterByUser|| rec.User == user)
select rec;
That might work in expression trees, meaning Linq to SQL would be optimised.

Well, what I thought was you could put the filter conditions into a generic list of Predicates:
var list = new List<string> { "me", "you", "meyou", "mow" };
var predicates = new List<Predicate<string>>();
predicates.Add(i => i.Contains("me"));
predicates.Add(i => i.EndsWith("w"));
var results = new List<string>();
foreach (var p in predicates)
results.AddRange(from i in list where p.Invoke(i) select i);
That results in a list containing "me", "meyou", and "mow".
You could optimize that by doing the foreach with the predicates in a totally different function that ORs all the predicates.