We Keep Coding

Linq query with nested select(s)

Trying to figure out the right way to express the following SQL in a linq statement and getting nowhere.
select messageid, comment
from Message
where isactive = 1 and messageid in
(
select messageid
from org.Notification
where NotificationTypeId = 7 and orguserid in
(
select OrgUserId
from org.OrgUser
where OrgId in
(
select ChildOrgId from OrgRelation
where ParentOrgId = 10001
)
)
group by messageid
)
This works as-is in SQL Server, but I want to use this query in C# (E-F) code, and I seem to be running around in circles.

Apparently Notification objects have a messageId. Several Notification objects may have the same value of MessageId.
I want the Messages that have an Id that equal one of the
MessageId values of a sub-selection of all Notification objects.
First I'll show you a LINQ query that solves your problem, then I'll tell you something about entity framework that would make this kind of LINQ queries easier to understand and maintain.
Direct Solution
(1) Select the notifications of whom the messages should be fetched:
IQueryable<Notification> notificationsToUse = org.Notifications
.Where(notification => notification.TypeId == 7
&& ....);
This is your inner select. I'm not sure about the relations between Notifications, OrgUsers and OrgRelations. But that is outside this question.
(2) Extract all used MessageIds of these Notifications
IQueryable<int> messageIdsUsedByNotificationsToUse = notificationsToUse
.Select(notification => notification.MessageId)
// remove duplicates:
.Distinct();
(3) Fetch all active messages with an Id in `messageIdsUsedByNotificationsToUse
IQueryable<Message> fetchedActiveMessages = org.Messages
.Where(message => message.IsActive
&& messageIdsUsedByNotificationsToUse.Contains(message.Id));
(4) You don't want the complete message, you only want the MessageId and the Comment:
var result = fetchedActiveMessages.Select(message => new
{
MessageId = message.Id,
Comment = message.Comment,
});
TODO: if desired: make one big LINQ statement.
Until now you haven't accessed the database yet. I only changed the Expression in the IQueryable. Making it one big LINQ statement won't increase performance very much and I doubt whether it would improve readability and maintainability.
Solution using possibilities of entity framework
It seems there is a one-to-many relation between Message and Notification: Every Message has zero or more Notifications, every Notification belongs to exactly one Message, using the foreign key MessageId.
If you stuck to the entity framework code-first conventions, you designed your classes similar to the following. (emphasizing the one-to-many relation):
class Message
{
public int Id {get; set;}
// every Message has zero or more Notifications (one-to-many)
public virtual ICollection<Notification> Notifications {get; set;}
... // other properties
}
class Notification
{
public int Id {get; set;}
// every Notifications belongs to exactly one Message using foreign key
public int MessageId {get; set;}
public virtual Message Message {get; set;}
... // other properties
}
class MyDbContext : DbContext
{
public DbSet<Message> Messages {get; set;}
public DbSet<Notification> Notifications {get; set;}
}
This is all entity framework needs to know that you planned a one-to-many relation between Messages and Notifications. Entity framework knows which properties you intended to be the primary keys and the foreign keys, and it knows about the relation between the tables.
Sometimes there are good reasons to deviate from the conventions. This needs to be solved using attributes or fluent API.
The important thing is the structure with the virutal ICollection from Message to Notification and the virtual reference back from Notification to the Message that it belongs to.
If you've designed your classes like this, your query will be a piece of cake:
(1) Select the notifications you want to use:
IQueryable<Notification> notificationsToUse = ... same as above
(2) Now you can select the messages belonging to these Notifications directly:
var result = notificationsToUse.Select(notification => notification.Message)
Because every notification belongs to exactly one message, I'm certain there are no duplicates.
Continuing: only the MessageId and the Comment of the active messages
.Where(message => message.IsActive)
.Select(message => new
{
MessageId = message.Id,
Comment = message.Comment,
});
I wasn't sure about the relations between Notifications, OrgUsers and OrgRelations. If you design your classes such that they represent a proper one-to-many or many-to-many relation, then even expression (1) will be much simpler.

You can break down your query into 3 different parts as shown below -
Filter out the OrgUserIds first
var filteredOrgUserIds = dc.OrgUsers.Where(u => dc.OrgRelations.Where(o
=>o.ParentOrgId == 10001).Select(d =>
d.ChildOrgId).ToList().Contains(u.OrgId))
.Select(uid => uid.OrgUserId)
.Distinct()
.ToList();
Using the filteredOrgUserIds to filter out the notifications.
var filteredNotifications = dc.Notifications.Where(n =>
n.NotificationTypeId == 7 && filteredOrgUserIds.Contains(n.OrgUserId))
.Select(m => m.messageid)
.Distinct()
.ToList();
Lastly filter out the messages by looking at the filteredNotifications.
m.isactive == 1 && filteredNotifications.Contains(m.messageid))
.Select(m => new { message = m.messageid, comment = m.comment })
.Distinct()
.ToList();
This should work.Please try this once from your end and let us know if this helps.

If you need to use the IN clause you shoud use the Contains method.

From my Recipe for converting SQL to LINQ to SQL:
Translate subselects as separately declared variables.
Translate each clause in LINQ clause order, translating monadic and aggregate operators (DISTINCT, TOP, MIN, MAX etc) into functions applied to the whole LINQ query.
SELECT fields must be replaced with select new { ... } creating an anonymous object with all the desired fields or expressions.
Translate IN to .Contains() and NOT IN to !...Contains().
Noting that your SQL query is using GROUP BY when it should use DISTINCT, and then applying the recipe rules:
var childOrgs = from or in OrgRelation where or.ParentOrgId == 1001 select or.ChildOrgId;
var orgUsers = from ou in OrgUser where childOrgs.Contains(ou.OrgId) select ou.OrgUserId;
var notificationMessages = (from n in Notification
where n.NotificationTypeId == 7 && orgUsers.Contains(n.orguserid)
select n.messageid).Distinct();
var ans = from m in Message
where m.isactive == 1 && notificationMessages.Contains(m.messageid)
select new { m.messageid, m.comment };

Expressions: Breaking up a complex linq query

So I have a linq query which has several joins and where clauses, and in the projection it accesses one of the joins. So, I want to break this query up to make it reusable and have tried all sorts of things but can't get it to work. I've put the main query with the joins into another class and returned it as an IQueryable... but then because you have to perform a select at the end, the Join objects are no longer available to the projection later on.
I put the where clauses as expressions into another class, but again have problems when I want to access the joins and place a 'where' onto one of the joins... since they are no longer available to the where clauses.
So, my question is... is it even possible to do something like this? Can you rewrite the main part of the query (select * from x join y join z) as expressions and place into another file, such that all parts of that query can be accessible later on.
I've heard that linq is just a series of expressions... but maybe someone with more knowledge can confirm how one could do something like this?
An example would be:
var data2 = QueryExtensions.GetEmployeeMeetings(_context, id)
.Where(Expressions.Meetings())
Then in the QueryExtensions class you would have
public static IQueryable<table1> GetEmployeeMeetings(EmployeeEntities _context, long id)
{
return (from t in _context.table1
join ed in _context.table2 on t.ID equals ed.ID
join edt in _context.table3 on ed.ID equals edt.ID
select t);
}
Finally, in the Expressions class
public static Expression<Func<table3, bool>> Meetings()
{
return edt => edt.ID == 5;
}
So, unfortunately this doesn't work, because edt is no longer available to the Where lambda that has been created (because in the main query you selected 't' which was just the table1. Obviously I don't want to return ALL the data just to be able to do something like this... I would just like to know if it is possible to break up queries like this, and then put them back together?

Ok I think I just managed to figure this out by creating a join class of those three tables (as below) and then returning that instead of just 't'... seemed to work ok
public class Jointclass
{
public table1 c1 { get; set; }
public table2 c2 { get; set; }
public table3 c3 { get; set; }
}

NHibernate 3.3 Right Join QueryOver One-to-One Relationship

So, this is the situation. I have a C# project that uses NHibernate 3.3 (upgrading is not an option), with the following classes:
public class Person{
public List<PersonAddress> PersonAddresses {get;set;}
}
public class NaturalPerson : Person{
public NaturalProperty NaturalProperty {get;set;}
}
public class LegalPerson : Person{
public LegalProperty LegalProperty {get;set;}
}
public class Quarantine{
public Person QuarantinePerson {get;set;}
public QuarantineProperty QuarantineProperty {get;set;}
}
What I need to do is to obtain Persons that may or not "be in quarantine" and that satisfy different conditions that involve Quarantine's properties, and legal or natural properties. Basically what I need is a Left Join from Persons, or a Right Join from Quarantines:
Select *
From Quarantine q
Right Join Persons p on p.ID = q.QuarantinePersonID
Left Join NaturalPersons np on p.ID = np.ID
Left Join LegalPersons lp on p.ID = lp.ID
Where q.Property = 1
and np.Property = 1
This is what I have now. I have managed to do the "from" clause that I need, but I'm having serious problems with the "select" and "where" clauses:
Person p = null;
Quarantine q = null;
var results = this.Session.QueryOver<Quarantine>(() => q)
.JoinQueryOver<Person>(() => q.QuarantinePerson, () => p, JoinType.RightOuterJoin)
.SelectList(list => list
.Select(() => p.Id))
.TransformUsing(Transformers.AliasToBean<Person>())
.List<Person>();
This code generates the following query:
`SELECT p1_.PersonID as y0_
FROM BUP.Quarantines this_
right outer join BUP.Persons p1_
on this_.QuarantinePersonID = p1_.PersonID
left outer join BUP.LegalPersons p1_1_
on p1_.PersonID = p1_1_.PersonID
left outer join BUP.NaturalPersons p1_2_
on p1_.PersonID = p1_2_.PersonID`
As I said, the "from" clause is OK. Now the problem is with the Select and Where clauses.
The main problem with the Select is that I need the whole Person object, wether it's a legal or a natural person. I have tried removing the .SelectList from the query, but it throws a PropertyNotFoundException: "Could not find a setter for property 'p' in class 'Person'".
And the problem with the Where clause is that i don't know how to add conditions based on NaturalPerson and LegalPerson's properties. I have no problem filtering Quarantine and Person properties, but haven't yet succedeed to do the same with the other classes.
Also, this query should be as performant as possible, because timeout is a serious problem. I have managed to do other solutions with subqueries and such, but they took too long.
Any help, with any of the two issues, will be very appreciated!
Thanks!!

You can use subqueries mixing with future to execute both queries in one row trip:
var subquery = QueryOver.Of<Quarantine>()
.Select(x => x.QuarantinePerson.Id);
var naturalPersons = Session.QueryOver<NaturalPerson>()
.WithSubquery.WhereProperty(x => x.Id).NotIn(subquery)
//.Where(x => x.NaturalProperty == somehing)
.Future();
var legalPersons = Session.QueryOver<LegalPerson>()
.WithSubquery.WhereProperty(x => x.Id).NotIn(subquery)
//.Where(x => x.LegalProperty == somehing)
.Future();
var persons = naturalPersons.Cast<Person>().Union(legalPersons);

IQueryable Linq query with multiple joins at inner level

EDIT: forgot to say I'm using Fluent NHibernate, even though the tag could hint about it anyway.
I have these entity classes:
class OuterLevel
{
ICollection<MidLevel> mid_items;
... other properties
}
class MidLevel
{
OuterLevel parent;
Inner1 inner1;
Inner2 inner2;
... other properties
}
class Inner1
{
int id;
string description;
}
class Inner2
{
int id;
string description;
}
I need to build a Linq query that returns a list of OuterLevel objects with all children populated properly.
Supposing all mappings are correct and working, the hard part I'm finding here is that the resulting query should be something like
SELECT * FROM OuterLevelTable OLT INNER JOIN MidLevelTable MLT ON (MLT.parentID = OLT.ID) INNER JOIN
Inner1Table ON (MLT.Inner1ID = Inner1Table.ID) INNER JOIN
Inner2Table ON (MLT.Inner2ID = Inner2Table.ID)
WHERE (Inner1Table.someproperty1 = somevalue1) AND (Inner2Table.someproperty2 = somevalue2)
The main problem is that two joins start from MidLevel object downward the hierarchy, so I cannot figure out which Fetch and FetchMany combination can be used without having the resulting query join two times the MidLevelTable, such as the following does:
return All().FetchMany(x => x.mid_items).ThenFetch(x => x.inner1).FetchMany(x => x.mid_items).ThenFetch(x => x.inner2);
I would like to return a IQueryable that can be further filtered, so I would prefer avoiding Query and QueryOver.
Thanks in advance,
Mario

what you want is not possible because when you filter on the joined tables the resulting records are not enough to populate the collections anyway. You better construct the query in one place to further tune it or set the collection batch size to get down SELECT N+1.

Entity Framework with Include / Joining Issues

I've implemented an Table Inheritance functionality demonstrated at http://www.sqlteam.com/article/implementing-table-inheritance-in-sql-server.
All the foreign keys and constraints are in place.
Now I am using Entity Framework to pull back my People, Students, Teachers, and Parents where the model looks something like the following (without all the EF specific attributes etc).
public partial class People : EntityObject
{
public guid PeopleID { get; set; }
public int Age { get; set; } /Added for an example query
public PeopleParent Parent { get; set; }
public PeopleStudent Student { get; set; }
public PeopleTeacher Teacher { get; set; }
}
Now I need to get all People regardless of type, who are 25 years old, no more than 100 records, and I want to include all the referenced data. I create my EF Query like:
IQueryable<People> query = Entities.People.Include("PeopleParent")
.Include("PeopleStudent")
.Include("PeopleTeacher");
query.Where(x => x.Age == 25)
.Take(100);
IEnumerable<People> results = query.ToList();
Seems simple enough, but whatever table/entityset I've set to include first creates an INNER JOIN instead of the LEFT OUTER JOIN, which is not producing the right results.
Generated TSQL (incorrect for my needs):
SELECT
[Limit1].[C1] AS [C1],
<A bunch of Limit1 Columns>
FROM (
SELECT TOP (100)
[Extent1].[PeopleID] AS [PeopleID],
<A bunch of Extent1 Columns>
[Extent2].[PeopleID] AS [PeopleID1],
<A bunch of Extent2 Columns>
[Extent3].[PeopleID] AS [PeopleID2],
<A bunch of Extent3 Columns>
[Extent4].[PeopleID] AS [PeopleID3],
<A bunch of Extent4 Columns>
1 AS [C1]
FROM [rets].[People] AS [Extent1]
INNER JOIN [rets].[PeopleParent] AS [Extent2]
ON [Extent1].[PeopleID] = [Extent2].[PeopleID]
LEFT OUTER JOIN [rets].[PeopleStudent] AS [Extent3]
ON [Extent1].[PeopleID] = [Extent3].[PeopleID]
LEFT OUTER JOIN [rets].[PeopleTeacher] AS [Extent4]
ON [Extent1].[PeopleID] = [Extent4].[PeopleID]
) AS [Limit1]
Why is the first Include used as an INNER JOIN, and is there a soluion to my problem?
** UPDATE 1**
Assuming I use Ladislav Mrnka's Answer, there are two additional requirements due to the significant change in Linq and Lambda querying.
Question: How do I search for specific People which have specific properties?
(All Students with a Grade of "A")
Answer:
context.People.OfType<Student>().Where(s => s.Grade == "A");
Question: How do I search for any People that have a specific property?
(All Students or Teachers who's PrimaryFocus = "Math")
Answer:
List<People> result = new List<People>();
result.AddRange(context.People.OfType<Student>()
.Where(x => x.PrimaryFocus == "Math")
.ToList());
result.AddRange(context.People.OfType<Teacher>()
.Where(x => x.PrimaryFocus == "Math")
.ToList());

The obvious solution for you should be using native EF support for inheritance. In your case TPT inheritance. Once you have inheritance you will simply call:
IEnumerable<People> results = Entities.People
.Where(x => x.Age == 25)
.Take(100)
.ToList();
And it will return you instances of Student, Teachers, Parents etc.
In your solution the only advice is check that the relation is optional (1 - 0..1) - . If it is required it will use INNER JOIN. If it is optional and it still uses INNER JOIN it can be some bug or other problem in your model.