We Keep Coding

Creating a LINQ query based on two Lists and object array for testing parameters

I'm trying to separate out as much as possible an android from the business logic in order to speed things up.
Dotted around the code I have a piece of LINQ that looks like this
var jobItemDoneTest = JobItemsData.GetJobIfDone(wheelpos, theOrder.GetOrderItemStockItemID);
var jobsDoneList = new JobItemsData(theOrder.OrderData.jobItemsID).JobItemDone;
var stillToDo = theOrder
.OrderItemsData
.Where(p => jobsDoneList.All(p2 => p2.orderItemStockItemID != p.orderItemStockItemID))
.Where(t => !t.description.Contains("2hr"))
.Where(t => !t.description.Contains("Staff"))
.ToList();
In other words, there is a comparison between two lists of to filter out some results.
What I'm trying to do is remove the instances of this to create a generic method in a business object class.
So far, I've got this
public List<T> GetWorkStillToDo<T, U>(List<T> orderItems, List<U> jobItems, params object[] searchList1, string searchList2)
{
var stillToDo = orderItems.Where(p=>jobItems.Add(p2=>p2.orderItemStockItemID != p.orderItemStockItemID);
}
The problem is that if I want to search on different properties for p=> and p2=> and then filter on the where conditions, I'm getting lost and can't think of a way to iterate from n = 1 to n in the searchList object array and include them in a LINQ
Is
p2=>searchList[0].ToString() != searchList2
permitted within LINQ and how can I create the Where part of the query?

Where clauses in LINQ are a Func<object, bool> object where object can be a generic if you have well-defined rules around them. The construction syntax is just a normal lambda. So:
Func<object, bool> where = obj => obj.Propery == delta;
return someEnumOrList.Where(where);
Wrapping this into a parameter lets you ad-hoc query against your items. You may need to provide an extension method, something like a WhereAll that takes an array of Func<> objects and checks them all. One idea for that would be:
public static IEnumerable<T> WhereAll(this IEnumerable<T> enumerable, Func<object, bool>[] clauses)
{
var result = enumerable;
foreach(var c in clauses)
{
result = result.Where(c);
}
return result;
}

How to reuse LINQ subqueries

I am writing a query which contains several join operations. I want to write this query in a function so that I can call this function everytime I need this query. My function and the query are as follows. Since I have several join operations and I don't want to define a complex return type, I keep the return type just IQueryable.
private IQueryable getMySubQuery(MyContext db)
{
var query = db.Orders
.Join( ... )
.Join( ... )
.Join( ... );
return query;
}
public IQueryable <MyType> getData()
{
var db = ...
...
...
var query = getMySubQuery(db)
.Select( /// ERROR ???
return query;
}
I am getting an error: System.Linq.IQueryable doesn't contain a definition for Select.
I understand that if I define the return type of getMySubQuery() method as IQueryable <SomeType>, it will solve the problem. But the problem is I have to define a very complex type containing 50 fields. I don't want to do the projection in getMySubQuery() function. Is there any way I can solve it without creating the complex type? I want to use IQueryable rather than IEnumerable?

If this sample code is truly representative of your actual code then the problem is you're returning an IQueryable, not an IQueryable<T>. The reason you're doing that is because you're trying to use an anonymous type as the result of getMySubQuery, but it's not possible to share anonymous types across method boundaries. If you truly want to do this, you need to create a type that represents the data that is currently being returned as an anonymous type and then make sure you change getMySubQuery to return IQueryable<T> of that type.

Try this: var query = getMySubQuery(db).Tolis()
.Select( /// No ERROR!

Lambda expression for querying an array of KeyValuePairs using Expression<Func<T, bool>>

I have an array of KeyValuePairs like this:
KeyValuePair<long,int>[] orderItems;
The 'long' component of this KeyValuePair corresponds to an Id (primary key) value on a database table.
Using the Entity Framework, I have defined a repository which provides the following method on its interface:
IEnumerable<T> GetMany(Expression<Func<T, bool>> where);
The above repository method allows me to query the database using lambda expressions. For example, if I want to query the database for all rows where the 'Category' column equals 'Cameras' I can say this:
var results = repository.GetMany(a => a.Category.Contains("Cameras")).ToList();
What I want to query for is all rows where the Id (primary key) is equal to the Key value from any element of the array of KeyValuePairs. So if the list of KeyValuePairs is a list of product Ids and their quantities , I just want to query the database for all products whose Id is in my array.
Can anyone venture an appropriate lambda expression?
**EDIT TO ORIGINAL
The suggestion to use:
var results = repository.GetMany(a => keys.Any(x => x == a.Id)).ToList();
is a good one and will work if I first create an array of the keys from the KeyValuePair array. So something like this:
long[] pids = new long[orderItems.Length];
for (int i = 0; i < orderItems.Length; i++ )
{
pids[0] = orderItems[i].Key;
}
var products = productRepository.GetMany(a => pids.Any(x => x == a.Id)).ToList();
This shows that the 'Any' clause is supported by EF. I still can't get this to work without using the array of longs though. Kudos to anyone clever enough to give a solution that doesn't require me to extract the 'key' value from the KeyValuePair into an array first.
If I use the keys.Any I get the following exception:
"Unable to create a constant value of type 'System.Collections.Generic.IEnumerable`1'. Only primitive types (for instance Int32, String and Guid) are supported in this context"

Try this:
var keys = orderItems.Select(x => x.Key).ToList();
var results = repository.GetMany(a => keys.Any(x => x == a.Id))
.ToList();

it's hard to use the GetMany-Method while having the EF do it's work with the Database in this case.
But you should have a query-Object named after your entities - I take here Categories (as a queryable object of a Type with an field/property Id) as a example. Then you can do:
var query = context.Categories;
foreach(var pair in orderedItems)
query = query.Union(context.Where(categorie => categorie.Id = pair.Key));
and finally using the query:
var results = query.ToList();
PS: to use this with your repository pattern you can just expose this queryable-Collection as IQueryable within your repository interface (or make a inherited interface from your generic repository for example ICategoryRepository)

Scope of Collections used in LINQ Predicates

I really like PredicateBuilder. It allows me to build all sorts of queries very dynamically. The predicate variable can be passed around to different objects and they can add onto it with values they know about, etc. Except when I am needing to use a .Contains on a hashed collection. Bzzt! Crash and burn.
For instance (example/pseudo code, this may or may not compile/run):
protected Expression<Func<MyClass, bool>> GetWherePredicate()
{
string[] selectedValues = Request.Form.GetValues("checkbox1") ?? new string[0];
HashSet<int> selectedIDs = new HashSet<int>(selectedValues.Cast<int>());
Expression<Func<MyClass, bool>> predicate = PredicateBuilder.True<MyClass>();
predicate = predicate.And(s => selectedIDs.Contains(s.ID));
return predicate;
}
protected void Retrieve()
{
Expression<Func<MyClass, bool>> predicate = GetWherePredicate();
IEnumerable<MyClass> retrievedValues = MyDataContext.GetTable<MyClass>.Where(predicate);
}
When I try to do that, I get a NotSupportedException: Method 'Boolean Contains(Int32)' has no supported translation to SQL due to the selectedIDs HashSet not being in scope. If I do this all in the same method, then it works fine.
I need to know the right way to get my predicate there to resolve or compile or whatever so that it can be used in a different scope from where the HashSet is declared. Any help?
UPDATE: I had this pretty wrong. The code below works fine, so there is no scope conflict. Thanks Jay.
string[] selectedValues = Request.Form.GetValues("checkbox1") ?? new string[0];
Expression<Func<MyClass, bool>> predicate = PredicateBuilder.True<MyClass>();
predicate = predicate.And(s => selectedValues.Contains(s.ID.ToString()));

From the exception you cite, it seems unlikely that scope is a factor here.
My answer to this is that you need to declare selectedIDs as IEnumerable<int> instead of HashSet<int> (or just cast it before calling Contains(), but that doesn't account for it working when all in the same method, so I'm unsure.
Without seeing any actual code that is exhibiting this behaviour, it will be difficult to troubleshoot any further.

Do not get razzle-dazzled by the name Contains... there are many methods that are named that, and not many have supported translations into SQL.
Enumerable.Contains<T> and List<T>.Contains are methods with supported translations.
HashSet<T>.Contains is a method that has no supported translation.
There are many resolutions, but I recommend:
IEnumerable<string> selectedValueQuery =
Request.Form.GetValues("checkbox1") ?? Enumerable.Empty<string>();
List<string> selectedIds = selectedValueQuery
.Cast<int>()
.Distinct()
.ToList();
Although a simpler solution might be:
IEnumerable<int> selectedIDs = new HashSet<int>(selectedValues.Cast<int>());
Edit: It's not about concrete implementation of Contains at all. It's about whether the linqtosql query provider can identify the method and translate it into an IN (list) sql expression. The recognition code looks at the type of the parameter used in the expression. The recognition code does not use polymorphism/implementation nor does it walk the inheritence tree looking for other possibilities.
List<int> myList = new List<int>(){1, 2, 3};
IList<int> myIList = myList;
IEnumerable<int> myIEnumerable = myList;
//works by List<T>.Contains()
db.Customers.Where(c => myList.Contains(c.CustomerID));
//doesn't work, no translation for IList<T>.Contains
db.Customers.Where(c => myIList.Contains(c.CustomerID));
//works by Enumerable.Contains<T>()
db.Customers.Where(c => myIEnumerable.Contains(c.CustomerID));
//works by Enumerable.Contains<T>()
db.Customers.Where(c => Enumerable.Contains(myIEnumerable, c.CustomerID));
Even though these parameters reference the same instance, different translation behaviors occur because the type of the parameter is different.
.Contains() is not called as it is translated, therefore its implementation is irrelevant. It could throw NotImplementedException or return true;

C# - AsEnumerable Example

What is the exact use of AsEnumerable? Will it change non-enumerable collection to enumerable
collection?.Please give me a simple example.

From the "Remarks" section of the MSDN documentation:
The AsEnumerable<TSource> method has no effect
other than to change the compile-time
type of source from a type that
implements IEnumerable<T> to
IEnumerable<T> itself.
AsEnumerable<TSource> can be used to choose
between query implementations when a
sequence implements IEnumerable<T> but also has a different set
of public query methods available. For
example, given a generic class Table
that implements IEnumerable<T> and has its own methods such
as Where, Select, and SelectMany, a
call to Where would invoke the public
Where method of Table. A Table type
that represents a database table could
have a Where method that takes the
predicate argument as an expression
tree and converts the tree to SQL for
remote execution. If remote execution
is not desired, for example because
the predicate invokes a local method,
the AsEnumerable<TSource>
method can be used to hide the custom
methods and instead make the standard
query operators available.

If you take a look in reflector:
public static IEnumerable<TSource> AsEnumerable<TSource>(this IEnumerable<TSource> source)
{
return source;
}
It basically does nothing more than down casting something that implements IEnumerable.

Nobody has mentioned this for some reason, but observe that something.AsEnumerable() is equivalent to (IEnumerable<TSomething>) something. The difference is that the cast requires the type of the elements to be specified explicitly, which is, of course, inconvenient. For me, that's the main reason to use AsEnumerable() instead of the cast.

AsEnumerable() converts an array (or list, or collection) into an IEnumerable<T> of the collection.
See http://msdn.microsoft.com/en-us/library/bb335435.aspx for more information.
From the above article:
The AsEnumerable<TSource>(IEnumerable<TSource>) method has no
effect other than to change the compile-time type of source from a type
that implements IEnumerable<T> to IEnumerable<T> itself.

After reading the answers, i guess you are still missing a practical example.
I use this to enable me to use linq on a datatable
var mySelect = from table in myDataSet.Tables[0].AsEnumerable()
where table["myColumn"].ToString() == "Some text"
select table;

AsEnumerable can only be used on enumerable collections. It just changes the type of the collection to IEnumerable<T> to access more easily the IEnumerable extensions.

No it doesn't change a non-enumerable collection to an enumerable one. What is does it return the collection back to you as an IEnumerable so that you can use it as an enumerable. That way you can use the object in conjunction with IEnumerable extensions and be treated as such.

Here's example code which may illustrate LukeH's correct explanation.
IEnumerable<Order> orderQuery = dataContext.Orders
.Where(o => o.Customer.Name == "Bob")
.AsEnumerable()
.Where(o => MyFancyFilterMethod(o, MyFancyObject));
The first Where is Queryable.Where, which is translated into sql and run in the database (o.Customer is not loaded into memory).
The second Where is Enumerable.Where, which calls an in-memory method with an instance of something I don't want to send into the database.
Without the AsEnumerable method, I'd have to write it like this:
IEnumerable<Order> orderQuery =
((IEnumerable<Order>)
(dataContext.Orders.Where(o => o.Customer.Name == "Bob")))
.Where(o => MyFancyFilterMethod(o, MyFancyObject));
Or
IEnumerable<Order> orderQuery =
Enumerable.Where(
dataContext.Orders.Where(o => o.Customer.Name == "Bob"),
(o => MyFancyFilterMethod(o, MyFancyObject));
Neither of which flow well at all.

static void Main()
{
/*
"AsEnumerable" purpose is to cast an IQueryable<T> sequence to IEnumerable<T>,
forcing the remainder of the query to execute locally instead of on database as below example so it can hurt performance. (bind Enumerable operators instead of Queryable).
In below example we have cars table in SQL Server and are going to filter red cars and filter equipment with some regex:
*/
Regex wordCounter = new Regex(#"\w");
var query = dataContext.Cars.Where(car=> article.Color == "red" && wordCounter.Matches(car.Equipment).Count < 10);
/*
SQL Server doesn’t support regular expressions therefore the LINQ-to-db providers will throw an exception: query cannot be translated to SQL.
TO solve this firstly we can get all cars with red color using a LINQ to SQL query,
and secondly filtering locally for Equipment of less than 10 words:
*/
Regex wordCounter = new Regex(#"\w");
IEnumerable<Car> sqlQuery = dataContext.Cars
.Where(car => car.Color == "red");
IEnumerable<Car> localQuery = sqlQuery
.Where(car => wordCounter.Matches(car.Equipment).Count < 10);
/*
Because sqlQuery is of type IEnumerable<Car>, the second query binds to the local query operators,
therefore that part of the filtering is run on the client.
With AsEnumerable, we can do the same in a single query:
*/
Regex wordCounter = new Regex(#"\w");
var query = dataContext.Cars
.Where(car => car.Color == "red")
.AsEnumerable()
.Where(car => wordCounter.Matches(car.Equipment).Count < 10);
/*
An alternative to calling AsEnumerable is ToArray or ToList.
*/
}

The Enumerable.AsEnumerable method can be used to hide a type's custom implementation of a standard query operator
Consider the following example. we have a custom List called MyList
public class MyList<T> : List<T>
{
public string Where()
{
return $"This is the first element {this[0]}";
}
}
MyList has a method called Where which is Enumerable.Where() exact same name. when I use it, actually I am calling my version of Where, not Enumerable's version
MyList<int> list = new MyList<int>();
list.Add(4);
list.Add(2);
list.Add(7);
string result = list.Where();
// the result is "This is the first element 4"
Now how can I find the elements which are less than 5 with the Enumerable's version of Where?
The answer is: Use AsEnumerable() method and then call Where
IEnumerable<int> result = list.AsEnumerable().Where(e => e < 5);
This time the result contains the list of elements that are less than 5