We Keep Coding

QueryOver: select ... where property in (...)

I try to use queryover to represent the following sql:
select * from Table1 t1 where t1.foreign_key in (select t2.Id from Table2 t2 where (...))
So I created a subquery for the inner select statement like this:
var sq = QueryOver.Of<Table2>().Where(...).Select(c => c.Id);
However when I cannot use this subquery in the following query:
var query = QueryOver.Of<Table1>().WithSubquery.
WhereProperty(t1 = t1.foreign_key).In(contactSubQuery);
I think the problem is that QueryOver expects a subquery over Table1 instead of Table2 in contactSubQuery, but then I cannot access the required properties of Table2. In How do I express a query containing a WHERE..IN subquery using NHibernate's QueryOver API? a similar problem is addressed (using JoinAlias), but I cannot see how to apply that solution for my case. Thanks for any help!
SOLUTION:
Thanks alot #Radim, you were almost right. I was already using
Queryover.Of<T>()
in the query but the problem was that I was assigning it to a IQueryOver variable (since we have a no var-keyword styleguide in our company). After I assigned it to var it compiled. Since I did not expect this to cause the problem at all I simplified every variable to var in the question, so the posted code should actually already have worked lol... I checked the type and simply changed the query to (in accordance with the no-var rule):
QueryOver<Table1> = QueryOver.Of<Table1>()
.WithSubquery
.WhereProperty(t1 => t1.foreign_key)
// won't compile, because passed is IQueryOver<T,T>,
// not the QueryOver<U>
.In(subquery)
where before I had...
IQueryOver<Table1, Table1> = ...
Again, thanks alot for the help!

You are almost there, just the syntax is not like this:
var query = QueryOver.Of<Table1>().WithSubquery.
WhereProperty(t1 = t1.foreign_key).IsIn(contactSubQuery);
but:
// subquery "sq"
var sq = QueryOver.Of<Table2>().Where(...).Select(c => c.Id);
var query = QueryOver.Of<Table1>()
.WithSubquery
.WhereProperty(t1 => t1.foreign_key)
.In(sq) // instead of .IsIn(contactSubQuery)
...
Because .IsIn() is a general extension method:
/// <summary>
/// Apply an "in" constraint to the named property
/// Note: throws an exception outside of a QueryOver expression
///
/// </summary>
public static bool IsIn(this object projection, ICollection values);
while .In() is the method of the returned result "QueryOverSubqueryPropertyBuilderBase" (the result of the .WhereProperty() call)
Also, be sure, that the passed argument into .In(subquery) is a QueryOver.Of<T>(). For example this is wrong:
var subquery = session.QueryOver<T>(); // it is named subquery
// but it is not of a type QueryOver<T>, but of a type
// IQueryOver<T, T>
// which is not what is expected here
var query = QueryOver.Of<Table1>()
.WithSubquery
.WhereProperty(t1 => t1.foreign_key)
// won't compile, because passed is IQueryOver<T,T>,
// not the QueryOver<U>
.In(subquery)
...
And that will produce the:
Error 1 The type arguments for method 'NHibernate.Criterion.Lambda.QueryOverSubqueryBuilderBase<NHibernate.IQueryOver<>....
cannot be inferred from the usage. Try specifying the type arguments explicitly.

IQueryable to List

I'm using System.Linq.Dynamic to allow me to dynamically select a list of fields from a query like this:
finalQuery = query.Select(string.Format("new({0})", string.Join(",", selectors)));
Where selectors is just a List<string> with all the fields I want. This works great, but this version of the extension method Select returns an IQueryable. Not this is not IQueryable<T>. If I have an IQueryable<T> I can simply do a .ToList() to convert it into a list and force the query to actually run on the database, but with the non-generic IQueryable, that method doesn't exists.
This is because ToList is inherited from IEnumerable<T> which IQueryable<T> inherits and IQueryable, obviously, doesn't.
So what's the most efficient way to get an IQueryable to execute the query and give me back a list? I can do this:
List<object> rtn = new List<object>();
foreach (var o in finalQuery)
{
rtn.Add(o);
}
But it seems like their ought to be an easier way.
Edit: In response to suggestions, I tried both:
finalQuery.Cast<object>().ToList();
and:
finalQuery.Cast<dynamic>().ToList();
Which both give NotSupportedExceptions with the message:
Unable to cast the type 'DynamicClass1' to type 'System.Object'. LINQ to Entities
only supports casting EDM primitive or enumeration types.

This appears to be a limitation in the way LINQ to Entities translates IQueryable.Cast with anonymous types. You can work around this by using it as an IEnumerable (your working example does this). This causes the code to do the cast in the .NET runtime after it's retrieved from the DB, instead of trying to handle it in the DB engine. E.g.
IEnumerable finalQuery = query.Select(string.Format("new({0})",
string.Join(",", selectors)));
var result = finalQuery.Cast<dynamic>().ToList();
Or
public static IList<T> CastToList<T>(this IEnumerable source)
{
return new List<T>(source.Cast<T>());
}
var finalQuery = query.Select(string.Format("new({0})",
string.Join(",", selectors)));
var result = finalQuery.CastToList<dynamic>();

Casting errors when attempting to return an IQueryable<MyType>

I have a query that ought to return an IQueryable<MyType>. The code looks like this:
public IQueryable<MyType> GetFooList()
{
var query = (from x in dbContext.TableX
join y in dbContext.TableY on x.our_id equals y.our_id
join z in dbContext.TableZ on y.our_id equals z.our_id
join a in dbContext.TableA on z.other_id equals a.other_id
where !string.IsNullOrEmpty(x.status)
select new
{
(fields....)
})
.AsQueryable();
IQueryable<MyType> result = (IQueryable<MyType>) query;
return result;
}
In the calling controller actions, I want to filter this list for values specified at run time; the parameters to filter for will differ between the various calling actions. E.g.:
List<MyType> FooList = Interface.GetFooList()
.Where( specific conditions )
.ToList();
On the line setting result, an exception gets raised:
Invalid Cast Exception was unhandled by user code
Unable to cast object of type
'System.Data.Entity.Infrastructure.DbQuery'1[<>f__AnonymousType9'9[System.String,System.Nullable`1[System.DateTime],System.String,System.String,System.String,System.String,System.Int32,System.Nullable'1[System.DateTime],System.Nullable'1[System.DateTime]]]'
to type 'System.Linq.IQueryable'1[MyType]'.
So I figured this was a casting problem, and added .Cast<MyType>() before the call to AsQueryable(). This generates a different error:
Unable to cast the type 'Anonymous type' to type 'MyType'. LINQ to
Entities only supports casting EDM primitive or enumeration types.
And if I don't do any casting, these errors get raised in the calling actions instead of the Entity Frameworks accessor.
I've tried the suggestions in all the linked "Similar Questions", to no avail -- the errors keep going back and forth. I even tried including .Select(obj => new MyType() {fields...} ) to get away from the anonymous type. That didn't work either.
I feel like I'm missing something subtly obvious.
Edited to add
I updated the code to select a type: select new MyType() {fields...}. This worked properly. Then the calling method threw a NotSupportedException, on the line where I filter the results and make a list from the query:
The entity or complex type 'MyType' cannot be constructed in a
LINQ to Entities query.
ETA2
I copied the EF table properties to a new class, MyTypeDTO. I replaced all use of MyType with MyTypeDTO. I got this error:
The specified type member 'our_id' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.
Here's the property in the DTO:
public int our_id { get; set; }
So I removed the get/set, rebuilt, and reran. Nope, I got the same error.

You are correct, the reason why all of your casts have failed is that an anonymous type (i.e. the thing you create with select new {...} construct) cannot be cast to a named type.
I even tried including .Select(obj => new MyType() {fields...} ) to get away from the anonymous type. That didn't work either.
You were on the right track - this should work, assuming that MyType has the appropriate setters:
var query = from x in dbContext.TableX
join y in dbContext.TableY on x.our_id equals y.our_id
join z in dbContext.TableZ on y.our_id equals z.our_id
join a in dbContext.TableA on z.other_id equals a.other_id
where !string.IsNullOrEmpty(x.status)
select new MyType
{
MyTypeField1 = a.Column1
, MyTypeField2 = z.Column3
, // ...and so on
}; // No need to convert to IQueryable - this should produce the right type
return result;
The entity or complex type 'MyType' cannot be constructed in a LINQ to Entities query.
This is because MyType is a mapped entity. You should not be creating projections that return mapped entities, so EF correctly restricts your ability to do so. You should be able to project into a non-mapped type (to create a so-called DTO - data transfer object).
Define a class MyTypeDto which has the properties of MyClass, but no methods or mappings. Use MyClassDto in the definition of your method. This should fix the problem.

The problem was incredibly subtle. One of the fields in the query's anonymous type did not exist in MyType. I discovered this while I was creating a view model to use in this method.
For what it's worth, the view model works just fine. Eg:
public IQueryable<MyViewModel> GetFooList(int parameter)
{
var query = (from x in dbContext.TableX
join y in dbContext.TableY on x.our_id equals y.our_id
join z in dbContext.TableZ on y.our_id equals z.our_id
join a in dbContext.TableA on z.other_id equals a.other_id
where x.their_id == parameter
select new MyViewModel()
{
field1 = x.field1,
field2 = y.field2,
field3 = z.field3 //<= this field did not exist in MyType
}
).AsQueryable();
return query;
}

Insert Linq query part inside existing query

Having the following variable which stores an IQueryable:
var mainQuery = session
.Query<Employee>()
.Select(e => new
{
e.Name,
e.Address
});
And a method which takes that IQueryable as parameter.
public DataTable GetData(IQueryable query)
{
...
}
How can I write code inside GetData() that adds OrderBy() before the Select()?
Final value of query should look like it was built using following Linq expression:
var query = session
.Query<Employee>()
.OrderBy(e => e.Age)
.Select(e => new
{
e.Name,
e.Address
});
This is required because if I add the OrderBy() after Select() I will only be able to sort by the members of the anonymous type (name, address). If an employee would also have and age, I could not sort by it when placing OrderBy() after Select().
Thanks for your help!
UPDATE:
GetData has no knowledge of the structure of the query except that it ends with a Select.
And it has to have that exact signature public DataTable GetData(IQueryable query) - no extra parameters.
What is required it to modify the existing query inside the method and add OrderBy before the Select.
When there will be a correct answer I will accept and vote for it.

Why not just apply the select() after the call to GetData()?
var mainQuery = session.Query<Employee>();
this.GetData(mainQuery);
mainQuery.OrderBy(x => x.Age)
.Select(x => new
{
x.Name,
x.Address
});
Linq expression trees are immutable (source). You have to create a copy of the tree in parts to modify it.
Update: Keep in mind your pattern is trying to separate data access from presentation (or at least that is how it reads). Ordering is a matter of presentation. You may have multiple clients wanting to use GetData to fetch data but each of those clients might want the data to be sorted differently. Your original query projected after the call to GetData anyway so it makes sense to order with the projection.
Now if you REALLY want to order inside the method without changing its contract, you have to walk the expression tree of the linq query and rebuild it from scratch injecting the ordering in the right place. Expression trees are immutable and cannot be modified in-place.

Consider creating ViewModel or DTO for Employee and not to pass anonymous objects around. But anyway you can pass selector into GetData and apply it after sorting Employee
var mainQuery = session.Query<Employee>();
GetData(mainQuery, e => new { e.Name, e.Address });
//object is used because you create anonymous objects and pass them around
public DataTable GetData(IQueryable query,
Expression<Func<Employee, object>> selector)
{
return query.OrderBy(e => e.Age)
.Select(selector)
//...
}

How to create C# LambdaExpression that returns anonymous type for SelectMany resultSelector

I'm building a dynamic query that can have n of Where method calls and n of SelectMany calls dependent upon user input. For example I may have:
var qZ = entityContext.TableA
.SelectMany(a=>a.TableB, (a,t)=>new{a,t} )
.Where(a=>a.t.FieldID==21)
.Where(a=> EntityFunctions.Left(a.t.Value,1)=="p")
.SelectMany(a=>a.a.TableC, (a,t)=>new{a,t} )
.Where(a=>a.t.FieldID==22)
.Where(a=> a.a.t.Value=="Peter" && a.t.Value=="Pan")
.Where(a=> a.a.a.TypeID==3)
.Select(a=> new{ a.a.a.ItemID }
).Distinct();
In the method I'm writing, I use helper methods that return an IQueryable as seen in the return line below.
return query.Provider.CreateQuery(
Expression.Call(typeof(Queryable),
"Where",
new Type[] {query.ElementType},
query.Expression, predicateLambda)
);
I'm able to create LambdaExpressions for all of the various query attribute-value pairs required, but I am unable to create one for the resultSelector of Queryable.SelectMany.
How can we create (a,t) => new{a=a, t=t} in an expression tree? Or How do we accomplish the same result as the .SelectMany above using Expression.Call like below?
Expression.Call(typeof(Queryable),
"SelectMany",
????????,
????????
);
I've tried using the SelectMany overload that doesn't require the resultSelector which works to some degree, however, I don't know how to reference the properties of t in subsequent method calls.
I've found this lambda expression ((a,t) => new{a=a, t=t}) associated with SelectMany all over the web, but I can't find any example of how to convert it to an expression tree.
UPDATE:
Let's reframe the question. I can pass the lambda like this
var q = entityContext.TableA.AsQueryable();
var q1 = Queryable.SelectMany(q, a => a.TableB, (a, t) => new { a = a, t = t });
var q2 = Queryable.Where(q1,a=>a.t.FieldID==22);
That works, however, since I don't know ahead of time how many SelectMany need to be called and since each call changes to anonymous type of the IQueriable, is there a way to cast (and re-cast) the anonymous type to a single variable? This way I can loop through and apply whatever method necessary to the variable and then enumerate to get the results once the query is built. Something like:
var q = entityContext.TableA..AsQueryable();
q = Queryable.SelectMany(q, a => a.TableB, (a, t) => new { a = a, t = t });
q = Queryable.Where(q,a=>a.t.FieldID==22);
(BTW: This doesn't work)

The way that I ended up resolving this required a paradigm shift. The first query above was based upon the fact that I learned to write queries by joining all the tables I needed together to give me access to filter on and select fields in those tables.
SelectMany() creates joins and the box around my thinking at the time required that if I need to filter on a specific column in a table, I had to join that table to my query. This in turn changed the type of my IQueryable resulting in my not being able to predict the Type of the IQueryable at design time.
Answer:
Step 1: Set the type of IQueryable to the output type it needs to return. In the case above, the result was always IQueryable.
Step 2: Utilize Expressions to dynamically create the WHERE predicate, including any and all tables necessary to create the proper filter. This always returns Expression> and all of the other variables we easily accounted for. And rememeber, in EF it isn't necessary to join table outside of Where() if they are only needed in Where().