EDIT: forgot to say I'm using Fluent NHibernate, even though the tag could hint about it anyway.
I have these entity classes:
class OuterLevel
{
ICollection<MidLevel> mid_items;
... other properties
}
class MidLevel
{
OuterLevel parent;
Inner1 inner1;
Inner2 inner2;
... other properties
}
class Inner1
{
int id;
string description;
}
class Inner2
{
int id;
string description;
}
I need to build a Linq query that returns a list of OuterLevel objects with all children populated properly.
Supposing all mappings are correct and working, the hard part I'm finding here is that the resulting query should be something like
SELECT * FROM OuterLevelTable OLT INNER JOIN MidLevelTable MLT ON (MLT.parentID = OLT.ID) INNER JOIN
Inner1Table ON (MLT.Inner1ID = Inner1Table.ID) INNER JOIN
Inner2Table ON (MLT.Inner2ID = Inner2Table.ID)
WHERE (Inner1Table.someproperty1 = somevalue1) AND (Inner2Table.someproperty2 = somevalue2)
The main problem is that two joins start from MidLevel object downward the hierarchy, so I cannot figure out which Fetch and FetchMany combination can be used without having the resulting query join two times the MidLevelTable, such as the following does:
return All().FetchMany(x => x.mid_items).ThenFetch(x => x.inner1).FetchMany(x => x.mid_items).ThenFetch(x => x.inner2);
I would like to return a IQueryable that can be further filtered, so I would prefer avoiding Query and QueryOver.
Thanks in advance,
Mario
what you want is not possible because when you filter on the joined tables the resulting records are not enough to populate the collections anyway. You better construct the query in one place to further tune it or set the collection batch size to get down SELECT N+1.
Related
I have a query below which is supposed to group the result by Id, EntityName, DocType, Jurisdiction. For each group the query also returns the ProductList items.
At the moment if the group contains one or more than one product, Then i can see the result giving out a group with a combination of Id,EntityName,DocType,Jurisdiction and ProductList, However if the result doesnt contain products for a particular group i do not see the group at all. What i would like to do is show the groups even if does not have any products in its group. So if the count of ProductList is zero, i would like to set
ProductList= new List NettingAgreementProductDto. Any input would be highly appreciated.
var result = from nae in nettingAgreementEntities.Result
join no in nettingOpinions.Result
on nae.EntityId equals no.EntityId
join np in nettingProducts.Result
on no.ProductId equals np.Id
group np by new
{ nae.EntityId,
nae.EntityName,
nae.DocType,
nae.Jurisdiction
} into g
select new NettingAgreementEntityDto
{
Id = g.Key.EntityId,
EntityName = g.Key.EntityName,
DocType = g.Key.DocType,
Jurisdiction = g.Key.Jurisdiction,
ProductList = g.Select(x => new
NettingAgreementProductDto
{
Id = x.Id,
Name = x.Name
}).ToList()
};
To recap from the comments, currently your query is using Inner Join for associating NettingAgreementEntity with NettingAgreementProducts. This not only multiplies the result set (and thus requires you to use GroupBy after), but also filters out the NettingAgreementEntity without NettingAgreementProducts.
You can achieve the goal by switching to Group Join (or Left Outer Join + GroupBy).
But why entering all these complications. EF navigation properties allow you to almost forget about manual joins, and also allow you to easily see the multiplicity, thus whether you need to group the result or not.
So what I would suggest is to add the currently missing collection navigation property to your NettingAgreementEntity class:
public class NettingAgreementEntity
{
// ...
public virtual ICollection<NettingOpinion> Opinions { get; set; }
}
Optionally do the same for NettingAgreementProduct in case in the future you need something similar for products (it's a many-to-many relationship and should be able to be queried from both sides).
Also I would rename the NettingOpinion class navigation properties NettingAgreementProductNavigation and NettingAgreementEntityNavigation to something shorter, for instance Product and Entity. These names (as well as the names of the collection navigation properties) do not affect the database schema, but IMHO provide better readability.
Once you have that, you'll see that the desired LINQ query is a matter of simple Selects which convert entity class to DTO and let EF query translator produce the necessary joins for you:
var result = db.Set<NettingAgreementEntity>()
.Selec(nae => new NettingAgreementEntityDto
{
Id = nae.EntityId,
EntityName = nae.EntityName,
DocType = nae.DocType,
Jurisdiction = nae.Jurisdiction,
ProductList = nae.Opinions
.Select(no => new NettingAgreementProductDto
{
no.Product.Id,
no.Product.Name,
}).ToList(),
});
So I have a linq query which has several joins and where clauses, and in the projection it accesses one of the joins. So, I want to break this query up to make it reusable and have tried all sorts of things but can't get it to work. I've put the main query with the joins into another class and returned it as an IQueryable... but then because you have to perform a select at the end, the Join objects are no longer available to the projection later on.
I put the where clauses as expressions into another class, but again have problems when I want to access the joins and place a 'where' onto one of the joins... since they are no longer available to the where clauses.
So, my question is... is it even possible to do something like this? Can you rewrite the main part of the query (select * from x join y join z) as expressions and place into another file, such that all parts of that query can be accessible later on.
I've heard that linq is just a series of expressions... but maybe someone with more knowledge can confirm how one could do something like this?
An example would be:
var data2 = QueryExtensions.GetEmployeeMeetings(_context, id)
.Where(Expressions.Meetings())
Then in the QueryExtensions class you would have
public static IQueryable<table1> GetEmployeeMeetings(EmployeeEntities _context, long id)
{
return (from t in _context.table1
join ed in _context.table2 on t.ID equals ed.ID
join edt in _context.table3 on ed.ID equals edt.ID
select t);
}
Finally, in the Expressions class
public static Expression<Func<table3, bool>> Meetings()
{
return edt => edt.ID == 5;
}
So, unfortunately this doesn't work, because edt is no longer available to the Where lambda that has been created (because in the main query you selected 't' which was just the table1. Obviously I don't want to return ALL the data just to be able to do something like this... I would just like to know if it is possible to break up queries like this, and then put them back together?
Ok I think I just managed to figure this out by creating a join class of those three tables (as below) and then returning that instead of just 't'... seemed to work ok
public class Jointclass
{
public table1 c1 { get; set; }
public table2 c2 { get; set; }
public table3 c3 { get; set; }
}
I am projecting LINQ to SQL results to strongly typed classes: Parent and Child. The performance difference between these two queries is large:
Slow Query - logging from the DataContext shows that a separate call to the db is being made for each parent
var q = from p in parenttable
select new Parent()
{
id = p.id,
Children = (from c in childtable
where c.parentid = p.id
select c).ToList()
}
return q.ToList() //SLOW
Fast Query - logging from the DataContext shows a single db hit query that returns all required data
var q = from p in parenttable
select new Parent()
{
id = p.id,
Children = from c in childtable
where c.parentid = p.id
select c
}
return q.ToList() //FAST
I want to force LINQ to use the single-query style of the second example, but populate the Parent classes with their Children objects directly. otherwise, the Children property is an IQuerierable<Child> that has to be queried to expose the Child object.
The referenced questions do not appear to address my situation. using db.LoadOptions does not work. perhaps it requires the type to be a TEntity registered with the DataContext.
DataLoadOptions options = new DataLoadOptions();
options.LoadWith<Parent>(p => p.Children);
db.LoadOptions = options;
Please Note: Parent and Child are simple types, not Table<TEntity> types. and there is no contextual relationship between Parent and Child. the subqueries are ad-hoc.
The Crux of the Issue: in the 2nd LINQ example I implement IQueriable statements and do not call ToList() function and for some reason LINQ knows how to generate one single query that can retrieve all the required data. How do i populate my ad-hoc projection with the actual data as is accomplished in the first query? Also, if anyone could help me better-phrase my question, I would appreciate it.
It's important to remember that LINQ queries rely in deferred execution. In your second query you aren't actually fetching any information about the children. You've created the queries, but you haven't actually executed them to get the results of those queries. If you were to iterate the list, and then iterate the Children collection of each item you'd see it taking as much time as the first query.
Your query is also inherently very inefficient. You're using a nested query in order to represent a Join relationship. If you use a Join instead the query will be able to be optimized appropriately by both the query provider as well as the database to execute much more quickly. You may also need to adjust the indexes on your database to improve performance. Here is how the join might look:
var q = from p in parenttable
join child in childtable
on p.id equals child.parentid into children
select new Parent()
{
id = p.id,
Children = children.ToList(),
}
return q.ToList() //SLOW
The fastest way I found to accomplish this is to do a query that returns all the results then group all the results. Make sure you do a .ToList() on the first query, so that the second query doesn't do many calls.
Here r should have what you want to accomplish with only a single db query.
var q = from p in parenttable
join c in childtable on p.id equals c.parentid
select c).ToList();
var r = q.GroupBy(x => x.parentid).Select(x => new { id = x.Key, Children=x });
You must set correct options for your data load.
options.LoadWith<Document>(d => d.Metadata);
Look at this
P.S. Include for the LINQToEntity only.
The second query is fast precisely because Children is not being populated.
And the first one is slow just because Children is being populated.
Choose the one that fits your needs best, you simply can't have their features together!
EDIT:
As #Servy says:
In your second query you aren't actually fetching any information about the children. You've created the queries, but you haven't actually executed them to get the results of those queries. If you were to iterate the list, and then iterate the Children collection of each item you'd see it taking as much time as the first query.
I have a fairly complicated join query that I use with my database. Upon running it I end up with results that contain an baseID and a bunch of other fields. I then want to take this baseID and determine how many times it occurs in a table like this:
TableToBeCounted (One to Many)
{
baseID,
childID
}
How do I perform a linq query that still uses the query I already have and then JOINs the count() with the baseID?
Something like this in untested linq code:
from k in db.Kingdom
join p in db.Phylum on k.KingdomID equals p.KingdomID
where p.PhylumID == "Something"
join c in db.Class on p.PhylumID equals c.PhylumID
select new {c.ClassID, c.Name};
I then want to take that code and count how many orders are nested within each class. I then want to append a column using linq so that my final select looks like this:
select new {c.ClassID, c.Name, o.Count()}//Or something like that.
The entire example is based upon the Biological Classification system.
Assume for the example that I have multiple tables:
Kingdom
|--Phylum
|--Class
|--Order
Each Phylum has a Phylum ID and a Kingdom ID. Meaning that all phylum are a subset of a kingdom. All Orders are subsets of a Class ID. I want to count how many Orders below to each class.
select new {c.ClassID, c.Name, (from o in orders where o.classId == c.ClassId select o).Count()}
Is this possible for you? Best I can do without knowing more of the arch.
If the relationships are as you describe:
var foo = db.Class.Where(c=>c.Phylum.PhylumID == "something")
.Select(x=> new { ClassID = x.ClassID,
ClassName = x.Name,
NumOrders= x.Order.Count})
.ToList();
Side question: why are you joining those entities? Shouldn't they naturally be FK'd, thereby not requiring an explicit join?
Can you return IQueryable which is composed of two or more different subclasses ? Here's my attempt to show what I mean. It throws the error:
System.NotSupportedException: Types in
Union or Concat have members assigned
in different order..
var a = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.ListingID != null
select new OrderItemA {
// etc
} as OrderItem;
var b = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.AdID != null
select new OrderItemB {
//etc
} as OrderItem;
return a.Concat<OrderItem>(b);
Try doing the concat on IEnumerable instead of IQueryable:
return a.AsEnumerable().Concat(b.AsEnumerable());
If you need an IQueryable result you could do this:
return a.AsEnumerable().Concat(b.AsEnumerable()).AsQueryable();
Doing this will force the concat to happen in-memory instead of in SQL, and any additional operations will also happen in-memory (LINQ To Objects).
However, unlike the .ToList() example, the execution should still be deferred (making your data lazy loaded).
My guess is that this is because you are using LINQ in an LINQ-to-SQL context.
So using Concat means that LINQ2SQL will need to join both query into a SQL UNION query which might be where the System.NotSupportedException originated from.
Can you try this:
return a.ToList().Concat<OrderItem>(b.ToList());
And see if it make any difference?
What the above does is that it executes the query twice and then concatenate them in-memory instead of hot-off-SQL as to avoid the query translation problem.
It might not be the ideal solution, but if this work, my assumption is probably correct, that it's a query translation problem:
More information about Union and Concat translation to SQL:
http://blog.benhall.me.uk/2007/08/linq-to-sql-difference-between-concat.html
http://msdn.microsoft.com/en-us/library/bb399342.aspx
http://msdn.microsoft.com/en-us/library/bb386979.aspx
Hope this helps.
Interestingly after reading your post and a bit of testing, I realized that what your actually doing does seem to work just fine for me given that the projection part you show as ellipsis in both of your queries match. You see, LINQ to SQL appears to construct the underlying projection for the SQL select command based off of the property assignment statements as opposed to the actual type being materialized so as long as both sides have the same number, type, and order (not sure about this) of member assignments the UNION query should be valid.
My solution that I've been working with is to create a property on my DataContext class which acts much like a SQL View in that it allows me to write a query (in my case a Union between two different tables) and then use that query as if it is itself like a table when composing read-only select statements.
public partial class MyDataContext
{
public IQueryable<MyView> MyView
{
get
{
var query1 =
from a in TableA
let default_ColumnFromB = (string)null
select new MyView()
{
ColumnFromA = a.ColumnFromA,
ColumnFromB = default_ColumnFromB,
ColumnSharedByAAndB = a.ColumnSharedByAAndB,
};
var query2 =
from a in TableB
let default_ColumnFromA = (decimal?)null
select new MyView()
{
ColumnFromA = default_ColumnFromA,
ColumnFromB = b.ColumnFromB,
ColumnSharedByAAndB = b.ColumnSharedByAAndB,
};
return query1.Union(query2);
}
}
}
public class MyView
{
public decimal? ColumnFromA { get; set; }
public string ColumnFromB { get; set; }
public int ColumnSharedByAAndB { get; set; }
}
Notice two key things:
First of all the projection formed by the queries which make up both halves of the Union have the same number, type, and order of columns. Now LINQ may require the order to be the same (not sure about this) but it is definitely true that SQL does for a UNION and we can be sure that LINQ will require at least the same type and number of columns and these "columns" are known by the member assignments and not from the properties of the type you are instantiating in your projection.
Secondly LINQ currently doesn't allow for multiple constants to be used within a projections for queries which formulate a Concat or Union and from my understanding this is mainly because these two separate queries are separately optimized before the Union operation is processed. Normally LINQ to SQL is smart enough to realize that if you have a constant value which is only being used in the projection, then why send it to SQL just to have it come right back the way it was instead of tacking it on as a post process after the raw data comes back from SQL Server. Unfortunately the problem here is that this is a case of LINQ to SQL being to smart for it's own good, as it optimizes each individual query too early in the process. The way I've found to work around this is to use the let keyword to form a range variable for each value in the projection which will be materialized by getting it's value from a constant. Somehow this tricks LINQ to SQL into carrying these constants through to the actual SQL command which keeps all expected columns in the resulting UNION. More on this technique can be found here.
Using this techinque I at least have something reusable so that no matter how complex or ugly the actual Union can get, especially with the range variables, that in your end queries you can write queries to these pseudo views such as MyView and deal with the complexity underneath.
Can you do the projection after the concat?
// construct the query
var a = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.ListingID != null
select new {
type = "A"
item = oi
}
var b = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.AdID != null
select new {
type = "B"
item = oi
}
var temp = a.Concat<OrderItem>(b);
// create concrete types after concatenation
// to avoid inheritance issue
var result = from oi in temp
select (oi.type == "A"
? (new OrderItemA {
// OrderItemA projection
} as OrderItem)
: (new OrderItemB {
// OrderItemB projection
} as OrderItem)
);
return result
Not sure if the ternary operator works in LINQ2SQL in the above scenario but that might help avoid the inheritance issue.