We Keep Coding

C# - How to use Expression<Func> with parametrs in LINQ where

Task: I need to give some expression with parameters into LINQ's where to get some data from database, but have an error above
This example of working expression:
var shopExp = GetPersonForShop(PersonTypeIds.Director, new Guid("adda423f-8c38-40e0-9f39-6deceb787bc0")); // id
Where(shopExp)
But i need assign id dynamically, but got error above :
_repository.Persons
.Where(GetPersonForShop(PersonTypeIds.Director, person.PersonId)
And got error:
{"Unable to cast object of type 'System.Linq.Expressions.InstanceMethodCallExpression2' to type 'System.Linq.Expressions.LambdaExpression'."}
How does function for where(linq) look:
private Expression<Func<Person, bool>> GetPersonForShop(PersonTypeIds personTypeId, Guid personId)
{
return person => person .PeronTypeId== (int) personTypeId && person .PersonId == personId;
}
This is approximate look like out production, just change names of parametrs code
How can I add expression with parameters to Where clause??

Lambda expressions use => notation. Try something like this:
var idToFind = new Guid("adda423f-8c38-40e0-9f39-6deceb787bc0");
var result = _repository.Persons
.Where(p => p.TypeId == PersonTypeIds.Director && p.PersonId == idToFind);
In this expression, p represents each Person record in the Persons table, compared one-by-one using the boolean expression that follows it.
Depending on your datasource, the comparison for each p will either be done by .NET in memory, or it will happen inside your database using a SQL WHERE clause which is constructed from the boolean expression. The last would be optimal because it would mean that not the entire Persons table has to be transferred into .NET memory before comparison can take place.
Update - To apply the same condition multiple times without repeating it in your code, while still keeping the advantages of LINQ to SQL translation intact, you can put the condition in an Expression<Func<Person, bool>> object and then use that multiple times:
Expression<Func<Person, bool>> expression =
p => p.TypeId == PersonTypeIds.Director && p.PersonId == idToFind;
var result1 = datasource1.Where(expression);
var result2 = datasource2.Where(expression);
var result3 = datasource3.Where(expression);
Or through a method that produces the Expression object:
var result1 = datasource1.Where(GetExpression(idToFind));
var result2 = datasource2.Where(GetExpression(idToFind));
var result3 = datasource3.Where(GetExpression(idToFind));
public Expression<Func<Person, bool>> GetExpression(Guid idToFind)
{
return p => p.TypeId == PersonTypeIds.Director && p.PersonId == idToFind;
}
Or alternatively you can use a helper method:
var result1 = FilterByTypeAndId(datasource1, idToFind);
var result2 = FilterByTypeAndId(datasource2, idToFind);
var result3 = FilterByTypeAndId(datasource3, idToFind);
public IQueryable<Person> FilterByTypeAndId(IQueryable<Person> datasource, Guid idToFind)
{
return datasource.Where(p => p.TypeId == PersonTypeIds.Director && p.PersonId == idToFind);
}

based on the previous response, I am going to give you a few alternatives and suggestions.
var idToFind = new Guid("adda423f-8c38-40e0-9f39-6deceb787bc0");
var result = _repository
.Persons
.Where(p => p.TypeId == PersonTypeIds.Director)
.Where(p => p.PersonId == idToFind)
.ToList();
First is doing the where clause in 2 steps and then, adding the ToList(), with the ToList(), you will deal with collections and LINQ that is pretty useful. And by doing the where clause in 2 steps, is more for readable purposes.

Check if field contains any item from collection in LinqToSql

I have next query
var query = (from titles in db.Titles
join ratings in db.Ratings on titles.Rating equals ratings.Rating1
join synopsis in db.Synopsis on titles.Certificate equals synopsis.Certificate into items
from item in items.DefaultIfEmpty()
where titles.Rating_Release_Date >= DateTime.Parse(fromDate) &&
titles.Certificate < 1000000 &&
platformElements.Any(r => titles.Platforms.Contains(r)) && //here i get error Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator.
ratingElements.Any(r => titles.Rating.Contains(r))
orderby titles.Rating_Release_Date descending, titles.Submission_Title
select new {...}
But i get error when i try to check if field (titles.Platforms) contain any item from collection.
Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator.
And now i don't know how to make where-clause correctly.
First idea - in loop check if Platforms contains item from list, but also don't know how to implement it.
I found this article How do I use LINQ Contains(string[]) instead of Contains(string) but i don't think that custom extension will work in my case. Other answers is like this : platformElements.Contains(titles.Platforms) which isn't what i need.
Type of titles.Platforms is string.
Type of platformElements is string[];

I didn't compile it, but it should be compilable.
New class PredicateBuilder:
public static class PredicateBuilder
{
public static Expression<Func<T, bool>> False<T>() {
return f => false;
}
public static Expression<Func<T, bool>> Or<T>(this Expression<Func<T, bool>> expr1, Expression<Func<T, bool>> expr2) {
var invokedExpr = Expression.Invoke(expr2, expr1.Parameters.Cast<Expression>());
return Expression.Lambda<Func<T, bool>>(Expression.OrElse(expr1.Body, invokedExpr), expr1.Parameters);
}
}
New function:
public IQueryable<Title> FilterByPlatforms(this IQueryable<Title> titles, string[] platformElements) {
var predicate = PredicateBuilder.False<Title>();
foreach (string platformElement in platformElements) {
string temp = platformElement;
predicate = predicate.Or(t => t.Platforms.Contains(temp));
}
return titles.Where(predicate);
}
Modified query:
var query = (from titles in db.Titles.FilterByPlatforms(platformElements)
join ratings in db.Ratings on titles.Rating equals ratings.Rating1
join synopsis in db.Synopsis on titles.Certificate equals synopsis.Certificate into items
from item in items.DefaultIfEmpty()
where titles.Rating_Release_Date >= DateTime.Parse(fromDate) &&
titles.Certificate < 1000000 &&
// platformElements.Any(r => titles.Platforms.Contains(r)) && //here i get error Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator.
ratingElements.Any(r => titles.Rating.Contains(r))
orderby titles.Rating_Release_Date descending, titles.Submission_Title
select new {...}
Maybe for ratingElements you should do the same.
Source: http://www.albahari.com/nutshell/predicatebuilder.aspx

LINQ to Entities equivalent of sql "TOP(n) WITH TIES"

I have been searcing for LINQ equivalent of WITH TIES in sql server lately, I came across a couple things, which couldn't proove to be useful.
I know this question was asked before and has an accepted answer, but it doesn't work the way with ties does. The solution using GroupBy() doesn't result as expected for TOP(3) WITH TIES considering a data set consisting of {3 2 2 1 1 0} the result set will be {3 2 2 1 1} where it should be {3 2 2}
Using the following sample data (taken from this question):
CREATE TABLE Person
(
Id int primary key,
Name nvarchar(50),
Score float
)
INSERT INTO Person VALUES (1, 'Tom',8.9)
INSERT INTO Person VALUES (2, 'Jerry',8.9)
INSERT INTO Person VALUES (3, 'Sharti',7)
INSERT INTO Person VALUES (4, 'Mamuzi',9)
INSERT INTO Person VALUES (5, 'Kamala',9)
Traditional OrderByDescending(p => p.Score).Take(3) will result with: Mamuzi, Kamala and one of Tom (or Jerry) where it should include BOTH
I know there is no built-in equivalent of it and i've found a way to implement it. I don't know if it is the best way to do it and open for alternative solutions.

var query = (from q in list.OrderByDescending(s => s.Score).Take(3).Select(s => s.Score).Distinct()
from i in list
where q == i.Score
select i).ToList();
Edit:
#Zefnus
I wasn't sure in which order you wanted it but to change the order you can put a OrderBy(s => s.Score) between select i and ToList()
I don't have the possibility to check what sql statement my linq clause would produce. But your answer is much better i think. And your question was also really good. I never thought about top with ties in linq. ;)
Basically it only takes top 3 scores from the first list and compares them with the whole list and i takes only those scores which are equal to the scores of the first list.

Do not use IEnumerable<T> with anything touching a database!
A solution aimed at LinqToSql and LinqToEntities should not be using IEnumerable<T>. Your current self answer will result in every single person being selected from the database and then being queried in memory using LinqToObjects.
To make a solution that is translated to SQL and executed by the database you have to use IQueryable<T> and Expressions instead.
public static class QueryableExtensions
{
public static IQueryable<T> TopWithTies<T, TComparand>(this IQueryable<T> source, Expression<Func<T, TComparand>> topBy, int topCount)
{
if (source == null) throw new ArgumentNullException("source");
if (topBy == null) throw new ArgumentNullException("topBy");
if (topCount < 1) throw new ArgumentOutOfRangeException("topCount", string.Format("topCount must be greater than 0, was {0}", topCount));
var topValues = source.OrderBy(topBy)
.Select(topBy)
.Take(topCount);
var queryableMaxMethod = typeof(Queryable).GetMethods()
.Single(mi => mi.Name == "Max" &&
mi.GetParameters().Length == 1 &&
mi.IsGenericMethod)
.MakeGenericMethod(typeof(TComparand));
var lessThanOrEqualToMaxTopValue = Expression.Lambda<Func<T, bool>>(
Expression.LessThanOrEqual(
topBy.Body,
Expression.Call(
queryableMaxMethod,
topValues.Expression)),
new[] { topBy.Parameters.Single() });
var topNRowsWithTies = source.Where(lessThanOrEqualToMaxTopValue)
.OrderBy(topBy);
return topNRowsWithTies;
}
public static IQueryable<T> TopWithTiesDescending<T, TComparand>(this IQueryable<T> source, Expression<Func<T, TComparand>> topBy, int topCount)
{
if (source == null) throw new ArgumentNullException("source");
if (topBy == null) throw new ArgumentNullException("topBy");
if (topCount < 1) throw new ArgumentOutOfRangeException("topCount", string.Format("topCount must be greater than 0, was {0}", topCount));
var topValues = source.OrderByDescending(topBy)
.Select(topBy)
.Take(topCount);
var queryableMinMethod = typeof(Queryable).GetMethods()
.Single(mi => mi.Name == "Min" &&
mi.GetParameters().Length == 1 &&
mi.IsGenericMethod)
.MakeGenericMethod(typeof(TComparand));
var greaterThanOrEqualToMinTopValue = Expression.Lambda<Func<T, bool>>(
Expression.GreaterThanOrEqual(
topBy.Body,
Expression.Call(queryableMinMethod,
topValues.Expression)),
new[] { topBy.Parameters.Single() });
var topNRowsWithTies = source.Where(greaterThanOrEqualToMinTopValue)
.OrderByDescending(topBy);
return topNRowsWithTies;
}
}
This creates queries of the following form:
SELECT [t0].[Id], [t0].[Name], [t0].[Score]
FROM [Person] AS [t0]
WHERE [t0].[Score] >= ((
SELECT MIN([t2].[Score])
FROM (
SELECT TOP (3) [t1].[Score]
FROM [Person] AS [t1]
ORDER BY [t1].[Score] DESC
) AS [t2]
))
ORDER BY [t0].[Score] DESC
That query is only about 50% worse than the baseline query:
SELECT TOP (3) WITH TIES
[t0].[Id],
[t0].[Name],
[t0].[Score]
FROM
[Person] AS [t0]
ORDER BY [t0].[Score] desc
With a data set consisting of your original 5 records and an additional 10000 records all with scores less than the original both of these are more or less instant (less than 20 milliseconds).
The IEnumerable<T> approach took a whole 2 minutes!
If the expression building and reflection seems scary the same thing can be achieved with a join:
public static IQueryable<T> TopWithTiesDescendingJoin<T, TComparand>(this IQueryable<T> source, Expression<Func<T, TComparand>> topBy, int topCount)
{
if (source == null) throw new ArgumentNullException("source");
if (topBy == null) throw new ArgumentNullException("topBy");
if (topCount < 1) throw new ArgumentOutOfRangeException("topCount", string.Format("topCount must be greater than 0, was {0}", topCount));
var orderedByValue = source.OrderByDescending(topBy);
var topNValues = orderedByValue.Select(topBy).Take(topCount).Distinct();
var topNRowsWithTies = topNValues.Join(source, value => value, topBy, (x, row) => row);
return topNRowsWithTies.OrderByDescending(topBy);
}
With the following query as the result (with about the same performance):
SELECT [t3].[Id], [t3].[Name], [t3].[Score]
FROM (
SELECT DISTINCT [t1].[Score]
FROM (
SELECT TOP (3) [t0].[Score]
FROM [Person] AS [t0]
ORDER BY [t0].[Score] DESC
) AS [t1]
) AS [t2]
INNER JOIN [Person] AS [t3] ON [t2].[Score] = [t3].[Score]
ORDER BY [t3].[Score] DESC

Another solution - which probably is not as efficient as the other solution - is to get TOP(3) Scores and get the rows with Score values contained in the TOP(3).
We can use Contains() as follows;
orderedPerson = datamodel.People.OrderByDescending(p => p.Score);
topPeopleList =
(
from p in orderedPerson
let topNPersonScores = orderedPerson.Take(n).Select(p => p.Score).Distinct()
where topNPersonScores.Contains(p.Score)
select p
).ToList();
What's good about this implementation is that it's extension method TopWithTies() can be implemented easly as;
public static IEnumerable<T> TopWithTies<T, TResult>(this IEnumerable<T> enumerable, Func<T, TResult> selector, int n)
{
IEnumerable<T> orderedEnumerable = enumerable.OrderByDescending(selector);
return
(
from p in orderedEnumerable
let topNValues = orderedEnumerable.Take(n).Select(selector).Distinct()
where topNValues.Contains(selector(p))
select p
);
}

I think that maybe you can do something like:
OrderByDescending(p => p.Score).Skip(2).Take(1)
Count the number of occurrences of this element, and then:
OrderByDescending(p => p.Score).Take(2 + "The select with the number of occurrences for the third element")
I think that maybe this works ;)
It´s only an idea!

I've found a solution taking the Score field value of the Nth row (3rd row in this case) using .Skip(n-1).Take(1) and selecting all rows with score value greater or equal to that as follows:
qryPeopleOrderedByScore = datamodel.People.OrderByDescending(p => p.Score);
topPeopleList =
(
from p in qryPeopleOrderedByScore
let lastPersonInList = qryPeopleOrderedByScore.Skip(2).Take(1).FirstOrDefault()
where lastPersonInList == null || p.Score >= lastPersonInList.Score
select p
).ToList();

Linq to Sql any keyword search query

I have a case in my application where the user can search for a list of terms. The search needs to make three passes in the following order:
One for an exact match of what they entered. Done, easy.
One where all the words (individually) match. Done, also easy.
One where any of the words match...how?
Essentially, how do I, in Linq to Sql, tell it to do this:
select * from stuff s where s.Title like '%blah%' || s.Title like '%woo&' || s.Title like '%fghwgads%' || s.Title like...
And so on?

This might be a tough one... I think you'd have to write your own operator.
(Update: Yep, I tested it, it works.)
public static class QueryExtensions
{
public static IQueryable<TEntity> LikeAny<TEntity>(
this IQueryable<TEntity> query,
Expression<Func<TEntity, string>> selector,
IEnumerable<string> values)
{
if (selector == null)
{
throw new ArgumentNullException("selector");
}
if (values == null)
{
throw new ArgumentNullException("values");
}
if (!values.Any())
{
return query;
}
var p = selector.Parameters.Single();
var conditions = values.Select(v =>
(Expression)Expression.Call(typeof(SqlMethods), "Like", null,
selector.Body, Expression.Constant("%" + v + "%")));
var body = conditions.Aggregate((acc, c) => Expression.Or(acc, c));
return query.Where(Expression.Lambda<Func<TEntity, bool>>(body, p));
}
}
Then you could call this with:
string[] terms = new string[] { "blah", "woo", "fghwgads" };
var results = stuff.LikeAny(s => s.Title, terms);
P.S. You'll need to add the System.Linq.Expressions and System.Data.Linq.SqlClient namespaces to your namespaces for the QueryExtensions class.

How can I conditionally apply a Linq operator?

We're working on a Log Viewer. The use will have the option to filter by user, severity, etc. In the Sql days I'd add to the query string, but I want to do it with Linq. How can I conditionally add where-clauses?

if you want to only filter if certain criteria is passed, do something like this
var logs = from log in context.Logs
select log;
if (filterBySeverity)
logs = logs.Where(p => p.Severity == severity);
if (filterByUser)
logs = logs.Where(p => p.User == user);
Doing so this way will allow your Expression tree to be exactly what you want. That way the SQL created will be exactly what you need and nothing less.

If you need to filter base on a List / Array use the following:
public List<Data> GetData(List<string> Numbers, List<string> Letters)
{
if (Numbers == null)
Numbers = new List<string>();
if (Letters == null)
Letters = new List<string>();
var q = from d in database.table
where (Numbers.Count == 0 || Numbers.Contains(d.Number))
where (Letters.Count == 0 || Letters.Contains(d.Letter))
select new Data
{
Number = d.Number,
Letter = d.Letter,
};
return q.ToList();
}

I ended using an answer similar to Daren's, but with an IQueryable interface:
IQueryable<Log> matches = m_Locator.Logs;
// Users filter
if (usersFilter)
matches = matches.Where(l => l.UserName == comboBoxUsers.Text);
// Severity filter
if (severityFilter)
matches = matches.Where(l => l.Severity == comboBoxSeverity.Text);
Logs = (from log in matches
orderby log.EventTime descending
select log).ToList();
That builds up the query before hitting the database. The command won't run until .ToList() at the end.

I solved this with an extension method to allow LINQ to be conditionally enabled in the middle of a fluent expression. This removes the need to break up the expression with if statements.
.If() extension method:
public static IQueryable<TSource> If<TSource>(
this IQueryable<TSource> source,
bool condition,
Func<IQueryable<TSource>, IQueryable<TSource>> branch)
{
return condition ? branch(source) : source;
}
This allows you to do this:
return context.Logs
.If(filterBySeverity, q => q.Where(p => p.Severity == severity))
.If(filterByUser, q => q.Where(p => p.User == user))
.ToList();
Here's also an IEnumerable<T> version which will handle most other LINQ expressions:
public static IEnumerable<TSource> If<TSource>(
this IEnumerable<TSource> source,
bool condition,
Func<IEnumerable<TSource>, IEnumerable<TSource>> branch)
{
return condition ? branch(source) : source;
}

When it comes to conditional linq, I am very fond of the filters and pipes pattern.
http://blog.wekeroad.com/mvc-storefront/mvcstore-part-3/
Basically you create an extension method for each filter case that takes in the IQueryable and a parameter.
public static IQueryable<Type> HasID(this IQueryable<Type> query, long? id)
{
return id.HasValue ? query.Where(o => i.ID.Equals(id.Value)) : query;
}

Doing this:
bool lastNameSearch = true/false; // depending if they want to search by last name,
having this in the where statement:
where (lastNameSearch && name.LastNameSearch == "smith")
means that when the final query is created, if lastNameSearch is false the query will completely omit any SQL for the last name search.

Another option would be to use something like the PredicateBuilder discussed here.
It allows you to write code like the following:
var newKids = Product.ContainsInDescription ("BlackBerry", "iPhone");
var classics = Product.ContainsInDescription ("Nokia", "Ericsson")
.And (Product.IsSelling());
var query = from p in Data.Products.Where (newKids.Or (classics))
select p;
Note that I've only got this to work with Linq 2 SQL. EntityFramework does not implement Expression.Invoke, which is required for this method to work. I have a question regarding this issue here.

It isn't the prettiest thing but you can use a lambda expression and pass your conditions optionally. In TSQL I do a lot of the following to make parameters optional:
WHERE Field = #FieldVar OR #FieldVar IS NULL
You could duplicate the same style with a the following lambda (an example of checking authentication):
MyDataContext db = new MyDataContext();
void RunQuery(string param1, string param2, int? param3){
Func checkUser = user =>
((param1.Length > 0)? user.Param1 == param1 : 1 == 1) &&
((param2.Length > 0)? user.Param2 == param2 : 1 == 1) &&
((param3 != null)? user.Param3 == param3 : 1 == 1);
User foundUser = db.Users.SingleOrDefault(checkUser);
}

I had a similar requirement recently and eventually found this in he MSDN.
CSharp Samples for Visual Studio 2008
The classes included in the DynamicQuery sample of the download allow you to create dynamic queries at runtime in the following format:
var query =
db.Customers.
Where("City = #0 and Orders.Count >= #1", "London", 10).
OrderBy("CompanyName").
Select("new(CompanyName as Name, Phone)");
Using this you can build a query string dynamically at runtime and pass it into the Where() method:
string dynamicQueryString = "City = \"London\" and Order.Count >= 10";
var q = from c in db.Customers.Where(queryString, null)
orderby c.CompanyName
select c;

You can create and use this extension method
public static IQueryable<TSource> WhereIf<TSource>(this IQueryable<TSource> source, bool isToExecute, Expression<Func<TSource, bool>> predicate)
{
return isToExecute ? source.Where(predicate) : source;
}

Just use C#'s && operator:
var items = dc.Users.Where(l => l.Date == DateTime.Today && l.Severity == "Critical")
Edit: Ah, need to read more carefully. You wanted to know how to conditionally add additional clauses. In that case, I have no idea. :) What I'd probably do is just prepare several queries, and execute the right one, depending on what I ended up needing.

You could use an external method:
var results =
from rec in GetSomeRecs()
where ConditionalCheck(rec)
select rec;
...
bool ConditionalCheck( typeofRec input ) {
...
}
This would work, but can't be broken down into expression trees, which means Linq to SQL would run the check code against every record.
Alternatively:
var results =
from rec in GetSomeRecs()
where
(!filterBySeverity || rec.Severity == severity) &&
(!filterByUser|| rec.User == user)
select rec;
That might work in expression trees, meaning Linq to SQL would be optimised.

Well, what I thought was you could put the filter conditions into a generic list of Predicates:
var list = new List<string> { "me", "you", "meyou", "mow" };
var predicates = new List<Predicate<string>>();
predicates.Add(i => i.Contains("me"));
predicates.Add(i => i.EndsWith("w"));
var results = new List<string>();
foreach (var p in predicates)
results.AddRange(from i in list where p.Invoke(i) select i);
That results in a list containing "me", "meyou", and "mow".
You could optimize that by doing the foreach with the predicates in a totally different function that ORs all the predicates.