We Keep Coding

Generic Linq IN sub query expression

I am trying to find a way to create the Linq Expression tree for a sub-query in-clause.
SELECT * FROM x WHERE X.Id IN (SELECT XId FROM Y)
Say I have this query, I need a way to create the System.Linq.Expression of the where clause. I have a generic repository base solution for numerous Entities and I am creating the expression tree dynamically for other filters and I would like to add this to the already created expression tree.
I know I can do something like below, but it isn't generic enough.
var xIds = Y.Select(x => x.XId).ToList();
var final = X.Where(x => xIds.Contains(x.Id)).ToList();
But I need Expression tree that this would create.
I played around with this link: Creating a Linq expression dynamically containing a subquery and it doesn't seem to work. I might be missing something though, been staring at this for a while.
Thanks!

Getting only specific columns in entity framework lambda join

I am surprised I cannot find a solution for this on the web, but wording the search terms was a bit difficult. The question I have is about generating entity SQL that only returns the needed columns in a group join using Lambda syntax.
The following is a "toy" example. I am not joining on two entities, rather on an enumerated list and an entity. And tunnelling is not an acceptable answer. I need to apply this to a much larger problem using a group join and select many.
var result1 = clientprofiles.Join(Context.Adjusters,
c => c.AdjusterId,
a => a.AdjusterId,
(c, a) => new {a.ClientAccountId}).ToList();
Using Julie Lehrman's Entity profiler, I see that the query is being generated to select every record in the rows that meet the join criteria. How do I pare it down so it only selects the ClientAccountId field in this example?

You can project a set of columns on any select from the context, so in your case you can constrain the Context.Adjusters parameter by using
Context.Adjusters.Select(a=> new { a.ClientAccountId })
to constrain the query to just the single column

Linq2Entities, many to many and dynamic where clause

I'm fairly new to Linq and struggling using dynamic where over a many to many relationship.
Database tables are like so:
Products <-> Products_SubCategories <-> SubCategories
with Products_SubCategories being a link table.
My full linq statement is
db.Products.Where("it.SubCategories.SubCategoryID = 2")
.Include("SubCategories")
.OrderBy(searchOrderBy)
.Skip(currentPage * pageSize)
.Take(pageSize)
.ToList()
.ForEach(p => AddResultItem(items, p));
So ignoring everything bar the Where() I'm just trying to pull out all products which are linked to sub category ID 2, this fails with
To extract properties out of collections, you must use a sub-query to iterate over the collection., near multipart identifier, line 8, column 1.
I think using the SQL-esque syntax I can do a subquery as per this link. However I'm not sure how to do that in the lambda / chaining syntax.
This is the start of a search function and I would like to build up the where string dynamically, as I have with the searchOrderBy string to avoid a large SELECT CASE. Products is linked to another table via a link table that I will need to include once I understand how to do this example.
Any help would be much appreciated!
Thanks

This is wrong:
db.Products.Where("it.SubCategories.SubCategoryID = 2")
SubCategories is a list. It does not have a property called SubCategoryID. Rather, it contains a group of entities which each have a property called SubCategoryID. That's a critical distinction.
When you run into a situation where you don't know how to proceed in there are multiple problems, it is good to break the problem down into several, smaller problems.
Let's start by removing the dynamic query. It will be easier to solve the problem with a non-dynamic query. Once you've done that, you can go back and make it dynamic again.
So start by using the non-dynamic syntax. Type something like this in Visual Studio, and see what IntelliSense does for you:
db.Products.Where(p => p.SubCategories.
You will quickly see that there is no SubCategoryID property. Instead, you will see a bunch of LINQ API methods for working with lists. If you know LINQ well, you will recognize that the Any() method is what you want here:
db.Products.Where(p => p.SubCategories.Any(sc => sc.SubCategoryID == 2))
Go ahead and run that query. Does it work? If so, you can move ahead to making it dynamic. I'm no ESQL expert, but I'd start with something along the lines of:
db.Products.Where("EXISTS(SELECT SC FROM it.SubCategories AS SC WHERE SC.SubCategoryID = 2");
As an aside, I use MS Dynamic Query ("Dynamic LINQ") for this sort of thing rather than Query Builder, as it's more testable.

It worked for me.
db.Products.Where("SubCategories.Any(SubCategoryID = 2)")

Linq, emulating joins, and the Include method

I'm looking into an issue that is related to...
Join and Include in Entity Framework
Basically the following query returns the list of "Property" objects the current user has permissions ( ACLs ) to view.
IQueryable<Property> currPropList
= from p in ve.Property
.Include("phyAddress")
.Include("Contact")
from a in ve.ACLs
from u in ve.Account
from gj in ve.ObjectGroupJoin
where u.username == currUsername // The username
&& (a.Account.id == u.id // The ACLs
&& a.objType == (int)ObjectType.Group)
&& (gj.ObjectGroup.id == a.objId // The groups
&& gj.objId == p.id) // The properties
select p;
The query returns the correct list of properties and in large works fine.
But the "Include" calls in the linq query above does not load the objects. If I call "Load()" explicitly after the LINQ query then the objects load.
The related SO question suggested that there may be a conflict between the "Include" call and the where clause. How can that be the case?
But at any rate, how can I restructure this query to load the "phyAddress" and "Contract" members? Specifically, I'd only like to load the members on returned objects, not all the "phyAddress" and "Contact" objects in the database.
Thanks.
Edit
I've tracked down the issue to the use of multiple from clauses
This works...
IQueryable<Property> currPropList
= from p in ve.Property
.Include("phyAddress")
select p;
And the "phyAddress" member is loaded.
But this doesn't work...
IQueryable<Property> currPropList
= from p in ve.Property
.Include("phyAddress")
from a in ve.ACLs
select p;
Basically the Include call is ignored when there are multiple from clauses. Does anyone know of a work around for this?
Edit 2
One workaround is to cast the IQueryable result as a ObjectQuery and get the include off of that. But I would like to prevent the second roundtrip to the database I am assuming this causes.
Eg. This works....
IQueryable<Property> currPropList
= ((from p in ve.Property
from a in ve.ACLs
select p) as ObjectQuery<Property>).Include("phyAddress");
Is there a way to do this with only a single query?
Edit 3
No second query because of deferred execution [ http://blogs.msdn.com/charlie/archive/2007/12/09/deferred-execution.aspx. So edit 2 would be the solution.

This is a known issue with Include... if you do something that changes the shape of the query (i.e. from from) then the Include is lost there are simple enough workarounds though:
you can wrap the include around the query, see Tip 22 - How to make include really include.
or you can get everything you need in the select clause and let relationship fixup do the job for you. i.e.
var x = from p in ve.Property
from a in ve.ACLs
select new {p,p.phyAddress};
var results = x.AsEnumerable().Select(p => p.p);
Now results is an enumeration of property entities, but each one has it's phyAddress loaded, as a side-effect of the initial request for the phyAddress, and EF's relationship fixup.

Select clause containing non-EF method calls

I'm having trouble building an Entity Framework LINQ query whose select clause contains method calls to non-EF objects.
The code below is part of an app used to transform data from one DBMS into a different schema on another DBMS. In the code below, Role is my custom class unrelated to the DBMS, and the other classes are all generated by Entity Framework from my DB schema:
// set up ObjectContext's for Old and new DB schemas
var New = new NewModel.NewEntities();
var Old = new OldModel.OldEntities();
// cache all Role names and IDs in the new-schema roles table into a dictionary
var newRoles = New.roles.ToDictionary(row => row.rolename, row => row.roleid);
// create a list or Role objects where Name is name in the old DB, while
// ID is the ID corresponding to that name in the new DB
var roles = from rl in Old.userrolelinks
join r in Old.roles on rl.RoleID equals r.RoleID
where rl.UserID == userId
select new Role { Name = r.RoleName, ID = newRoles[r.RoleName] };
var list = roles.ToList();
But calling ToList gives me this NotSupportedException:
LINQ to Entities does not recognize
the method 'Int32
get_Item(System.String)' method, and
this method cannot be translated into
a store expression
Sounds like LINQ-to-Entities is barfing on my call to pull the value out of the dictionary given the name as a key. I admittedly don't understand enough about EF to know why this is a problem.
I'm using devart's dotConnect for PostgreSQL entity framework provider, although I assume at this point that this is not a DBMS-specific issue.
I know I can make it work by splitting up my query into two queries, like this:
var roles = from rl in Old.userrolelinks
join r in Old.roles on rl.RoleID equals r.RoleID
where rl.UserID == userId
select r;
var roles2 = from r in roles.AsEnumerable()
select new Role { Name = r.RoleName, ID = newRoles[r.RoleName] };
var list = roles2.ToList();
But I was wondering if there was a more elegant and/or more efficient way to solve this problem, ideally without splitting it in two queries.
Anyway, my question is two parts:
First, can I transform this LINQ query into something that Entity Framework will accept, ideally without splitting into two pieces?
Second, I'd also love to understand a little about EF so I can understand why EF can't layer my custom .NET code on top of the DB access. My DBMS has no idea how to call a method on a Dictionary class, but why can't EF simply make those Dictionary method calls after it's already pulled data from the DB? Sure, if I wanted to compose multiple EF queries together and put custom .NET code in the middle, I'd expect that to fail, but in this case the .NET code is only at the end, so why is this a problem for EF? I assume the answer is something like "that feature didn't make it into EF 1.0" but I am looking for a bit more explanation about why this is hard enough to justify leaving it out of EF 1.0.

The problem is that in using Linq's delayed execution, you really have to decide where you want the processing and what data you want to traverse the pipe to your client application. In the first instance, Linq resolves the expression and pulls all of the role data as a precursor to
New.roles.ToDictionary(row => row.rolename, row => row.roleid);
At that point, the data moves from the DB into the client and is transformed into your dictionary. So far, so good.
The problem is that your second Linq expression is asking Linq to do the transform on the second DB using the dictionary on the DB to do so. In other words, it is trying to figure out a way to pass the entire dictionary structure to the DB so that it can select the correct ID value as part of the delayed execution of the query. I suspect that it would resolve just fine if you altered the second half to
var roles = from rl in Old.userrolelinks
join r in Old.roles on rl.RoleID equals r.RoleID
where rl.UserID == userId
select r.RoleName;
var list = roles.ToDictionary(roleName => roleName, newRoles[roleName]);
That way, it resolves your select on the DB (selecting just the rolename) as a precursor to processing the ToDictionary call (which it should do on the client as you'd expect). This is essentially exactly what you are doing in your second example because AsEnumerable is pulling the data to the client before using it in the ToList call. You could as easily change it to something like
var roles = from rl in Old.userrolelinks
join r in Old.roles on rl.RoleID equals r.RoleID
where rl.UserID == userId
select r;
var list = roles.AsEnumerable().Select(r => new Role { Name = r.RoleName, ID = newRoles[r.RoleName] });
and it'd work out the same. The call to AsEnumerable() resolves the query, pulling the data to the client for use in the Select that follows it.
Note that I haven't tested this, but as far as I understand Entity Framework, that's my best explanation for what's going on under the hood.

Jacob is totally right.
You can not transform the desired query without splitting it in two parts, because Entity Framework is unable to translate the get_Item call into the SQL query.
The only way is to write the LINQ to Entities query and then write a LINQ to Objects query to its result, just as Jacob advised.
The problem is Entity-Framework-specific one, it does not arise from our implementation of the Entity Framework support.