We Keep Coding

LINQ method to load child Entity that includes some of Parent's navigational properties and only selected Grand Child 's records

I've three tables Site_Report Report_Asset and Asset_Calcert with One-to-Many relationship between them, as shown.
I want to load a child entity as a model i.e Report_Asset which should include some of the parent's properties. And also, from grand child's collection navigation property, I want to load single record using a condition.
First Attempt - This leads to an error.
Report_Asset model;
model = ctx.Report_Asset
.Include(i => i.Site_Report)
.Include(i => i.Site_Report.Handled_By)
.Include(i => i.Site_Report.Published_By)
.Include(i => i.Asset_Calcerts.Select(b => b.asset_calcert_id == assetCalcertId))
.FirstOrDefault();
ERROR: The Include path expression must refer to a navigation property defined on the type. Use dotted paths for reference navigation properties and the Select operator for collection navigation properties.
Second Attempt - By starting form grandchild Entity and including the parents. This approach does load the result set, but it only contains the child entity i.e Report_Asset and no record from Asset_Calcert collection and null for Site_Report.
Report_Asset model;
model = ctx.Asset_Calcert.Include(i => i.Report_Asset)
.Include(i => i.Report_Asset.Site_Report.Handled_By)
.Include(i => i.Report_Asset.Site_Report.Published_By)
.Where(i => i.asset_calcert_id == assetCalcertId)
.Select(i => i.Report_Asset).FirstOrDefault();
I've set Configuration.LazyLoadingEnabled = false in DbContext().
Need direction, thanks

It looks like you want to query and return an entity, it's parent, and then a sub-set of it's children. You can use EF to query this, but only by reducing your query into a desired structure. If you return a Report_Asset, it can include a reference to it's parent easily enough, but it will always reference it's complete set of children, EF does not filter child sets at the entity level.
It's best to look at exactly what fields you want from each entity and select a structure containing just those, but at a very basic level, this should give you what you want:
var model = ctx.Report_Asset
.Select( x = > new
{
Report_Asset = x,
Site_Report = x.Site_Report,
Handled_By = x.Site_Report.Handled_By,
Published_By = x.Site_Report.Published_By,
Asset_Calcerts = x.Assert_Calcerts.Where(c => c.asset_calcert_id == assetCalcertId).ToList()
}).FirstOrDefault();
The calcert check looks to be looking for a specific, single cert, so this might be better as:
var model = ctx.Report_Asset
.Select( x = > new
{
Report_Asset = x,
Site_Report = x.Site_Report,
Handled_By = x.Site_Report.Handled_By,
Published_By = x.Site_Report.Published_By,
Asset_Calcert = x.Assert_Calcerts.SingleOrDefailt(c => c.asset_calcert_id == assetCalcertId)
}).FirstOrDefault();
If you were looking to load only report assets that had a matching Calcert, with the parent and the applicable calcert:
var model = ctx.Report_Asset
.Where(x => x.Asset_Calcerts.Any(c => c.asset_calcert_id == assetCalcertId))
.Select( x = > new
{
Report_Asset = x,
Site_Report = x.Site_Report,
Handled_By = x.Site_Report.Handled_By,
Published_By = x.Site_Report.Published_By,
Asset_Calcert = x.Asset_Calcerts.SingleOrDefailt(c => c.asset_calcert_id == assetCalcertId)
}).FirstOrDefault();
In all cases I would recommend adding an OrderBy clause before the FirstOrDefault to ensure a predictable order is used.
The caveat of this approach is that if you access the returned model's entities and start drilling down through their references, you will still trigger lazy loads from the database. For instance if I use:
model.Report_Asset.Asset_Calcerts
for instance, this will still lazy load the related entities for that report asset, and list all calcerts for that asset. The filtered set/match is eager loaded into the model.Asset_Calcert(s), not the returned Model_Asset entity.
Generally though you are better off using Select to just retrieve the properties you care about from the various entities, rather than attempting to select entire entities or entity graphs. This forgoes the need for using Include, just tell EF exactly what you want from the entity graph and it will build an SQL statement to efficiently retrieve just that information.

when using more then 1 level it is better to use the string option.
if you dont want to use string you will need to do select for the levels greater then 1.
Report_Asset model;
model = ctx.Report_Asset
.Include("Site_Report")
.Include("Site_Report.Handled_By")
.Include("Site_Report.Published_By")
.Include("Asset_Calcerts")
.FirstOrDefault(x => x.Asset_Calcerts.Any(y => y.asset_calcert_id == assetCalcertId);

Entity Framework Include() is not working within complex query

Consider following LINQ query:
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select new
{
ItemProp1 = obj,
ItemProp2 = obj.NavProp2.Any(n => n.Active)
}).SingleOrDefault();
This runs as expected, but item.ItemProp1.NavProp1 is NULL.
As it explains here this is because of the query actually changes after using Include(). but the question is what is the solution with this situation?
Edit:
When I change the query like this, every things works fine:
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select obj).SingleOrDefault();
Regarding to this article I guess what the problem is... but the solution provided by author not working in my situation (because of using anonymous type in final select rather than entity type).

As you mentioned, Include is only effective when the final result of the query consists of the entities that should include the Include-d navigation properties.
So in this case Include has effect:
var list = _db.SampleEntity.Include(s => s.NavProp1).ToList();
The SQL query will contain a JOIN and each SampleEntity will have its NavProp1 loaded.
In this case it has no effect:
var list = _db.SampleEntity.Include(s => s.NavProp1)
.Select(s => new { s })
.ToList();
The SQL query won't even contain a JOIN, EF completely ignores the Include.
If in the latter query you want the SampleEntitys to contain their NavProp1s you can do:
var list = _db.SampleEntity
.Select(s => new { s, s.NavProp1 })
.ToList();
Now Entity Framework has fetched SampleEntitys and NavProp1 entities from the database separately, but it glues them together by a process called relationship fixup. As you see, the Include is not necessary to make this happen.
However, if Navprop1 is a collection, you'll notice that...
var navprop1 = list.First().s.Navprop1;
...will still execute a query to fetch Navprop1 by lazy loading. Why is that?
While relationship fixup does fill Navprop1 properties, it doesn't mark them as loaded. This only happens when Include loaded the properties. So now we have SampleEntity all having their Navprop1s, but you can't access them without triggering lazy loading. The only thing you can do to prevent this is
_db.Configuration.LazyLoadingEnabled = false;
var navprop1 = list.First().s.Navprop1;
(or by preventing lazy loading by disabling proxy creation or by not making Navprop1 virtual.)
Now you'll get Navprop1 without a new query.
For reference navigation properties this doesn't apply, lazy loading isn't triggered when it's enabled.
In Entity Framework core, things have changed drastically in this area. A query like _db.SampleEntity.Include(s => s.NavProp1).Select(s => new { s }) will now include NavProp1 in the end result. EF-core is smarter in looking for "Includable" entities in the end result. Therefore, we won't feel inclined to shape a query like Select(s => new { s, s.NavProp1 }) in order to populate the navigation property. Be aware though, that if we use such a query without Include, lazy loading will still be triggered when s.NavProp1 is accessed.

I know this will probably get a few laughs, but don't forget the obvious like i just did. The row in the database didn't actually have a foreign key reference! I should have checked the dam data first before thinking EF Include wasn't working! Grrr. 30 minutes of my life I won't get back.

If your model is defined properly it should work without any problems.
using System.Data.Entity;
var item = _db.SampleEntity
.Include(p => p.NavigationProperty)
.Select(p => new YourModel{
PropertyOne = p.Something,
PropertyTwo = p.NavigationProperty.Any(x => x.Active)
})
.SingleOrDefault(p => p.Something == true);

How did you find that item.ItemProp1.NavProp1 is null. EF uses proxies to load all required properties when you try to access it.
What about
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select obj).SingleOrDefault();
Assert.IsNotNull(obj.NavProp1);
Assert.IsNotNull(obj.NavProp2);
You can also try with
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select new
{
ItemProp1 = obj,
NavProp1 = obj.NavProp1,
ItemProp2 = obj.NavProp2.Any(n => n.Active)
}).SingleOrDefault();
Assert.IsNotNull(item.NavProp1)
Of course I assume that you don't have any problems with EF navigation property mappings.

how to compare values from a list in Linq to Entity (any / contains)

There is a list, policiesToDelete of entity class, MonitoringRelations. Out of this list I have selected two elements and construed a new list:
var policyKeysToDelete = policiesToDelete
.Select(r => new {r.PolicyId, r.GroupId})
.ToList();
Now, I have a query where I want to compare elements from policyKeysToDelete list.
var objectsToDelete = (from p in storageContext.MonitoringRelations
where policyKeysToDelete
.Any(x => x == new {p.PolicyId, p.GroupId})
select p)
.ToList();
The problem: the query above throws this exception:
NotSupportedException: Unable to create a constant value of type 'Anonymous type'. Only primitive types or enumeration types are supported in this context.
I have tried changing the anonymous list to a list<tuple<PolicyId, GroupId>> , but that also didn't help, throwing the almost same exception. I tried using Contains in place of Any but that also didn't help.
Any idea how can I solve this problem?

EF cannot translate a list of complex objects into the SQL query. What EF can do, is translate a list of simple values into SQL when you use it with the .Contains method.
So if you extract a list of PolicyId's and a list of GroupId's from the policyKeysToDelete, and use it to select as much as you can with EF, then you can do the full check in the resultset which is then in-memory, using Linq-to-objects.
Warning: you are extracting too much from the database, so depending on the amount of data, a different solution might be better.
var policyKeysToDelete = policiesToDelete
.Select(r => new {r.PolicyId, r.GroupId})
.ToList();
// List of values types, which can be translated to SQL
var policyIds = policyKeysToDelete.Select(x => x.PolicyId).ToList();
var groupIds = policyKeysToDelete.Select(x => x.GroupId).ToList();
var objectsToDelete = storageContext.MonitoringRelations
// Do the part that we can do in the database, which is select the records
// which have an corresponding PolicyId or GroupId
.Where(x => policyIds.Contains(x.PolicyId) || groupIds.Contains(x.GroupId))
// Use this method to indicate that whatever follows after should not be
// translated to SQL
.AsEnumerable()
// Do the full check in-memory
.Where(x => policyKeysToDelete
.Any(y => x.PolicyId == y.PolicyId && x.GroupId == y.GroupId)
)
.ToList();

Multiple Include and Where Clauses Linq

I have a database where I'm wanting to return a list of Clients.
These clients have a list of FamilyNames.
I started with this
var query = DbContext.Clients.Include(c => c.FamilyNames).ToList() //returns all clients, including their FamilyNames...Great.
But I want somebody to be able to search for a FamilyName, ifany results are returned, then show the clients to the user.
so I did this...
var query = DbContext.Clients.Include(c => c.FamilyNames.Where(fn => fn.familyName == textEnteredByUser)).ToList();
I tried...
var query = DbContext.Clients.Include(c => c.FamilyNames.Any(fn => fn.familyName == textEnteredByUser)).ToList();
and...
var query = DbContext.FamilyNames.Include(c => c.Clients).where(fn => fn.familyname == textEnteredByUser.Select(c => c.Clients)).ToList();
What I would like to know (obviously!) is how I could get this to work, but I would like it if at all possible to be done in one query to the database. Even if somebody can point me in the correct direction.
Kind regards

In Linq to Entities you can navigate on properties and they will be transformed to join statements.
This will return a list of clients.
var query = DbContext.Clients.Where(c => c.FamilyNames.Any(fn => fn == textEnteredByUser)).ToList();
If you want to include all their family names with eager loading, this should work:
var query = DbContext.Clients.Where(c => c.FamilyNames.Any(fn => fn == textEnteredByUser)).Include(c => c.FamilyNames).ToList();
Here is some reference about loading related entities if something doesn't work as expected.

You can use 'Projection', basically you select just the fields you want from any level into a new object, possibly anonymous.
var query = DbContext.Clients
.Where(c => c.FamilyNames.Any(fn => fn == textEnteredByUser))
// only calls that can be converted to SQL safely here
.Select(c => new {
ClientName = c.Name,
FamilyNames = c.FamilyNames
})
// force the query to be materialized so we can safely do other transforms
.ToList()
// convert the anon class to what we need
.Select(anon => new ClientViewModel() {
ClientName = anon.ClientName,
// convert IEnumerable<string> to List<string>
FamilyNames = anon.FamilyNames.ToList()
});
That creates an anonymous class with just those two properties, then forces the query to run, then performs a 2nd projection into a ViewModel class.
Usually I would be selecting into a ViewModel for passing to the UI, limiting it to just the bare minimum number of fields that the UI needs. Your needs may vary.

Using Count with Take with LINQ

Is there a way to get the whole count when using the Take operator?

You can do both.
IEnumerable<T> query = ...complicated query;
int c = query.Count();
query = query.Take(n);
Just execute the count before the take. this will cause the query to be executed twice, but i believe that that is unavoidable.
if this is in a Linq2SQL context, as your comment implies then this will in fact query the database twice. As far as lazy loading goes though it will depend on how the result of the query is actually used.
For example: if you have two tables say Product and ProductVersion where each Product has multiple ProductVersions associated via a foreign key.
if this is your query:
var query = db.Products.Where(p => complicated condition).OrderBy(p => p.Name).ThenBy(...).Select(p => p);
where you are just selecting Products but after executing the query:
var results = query.ToList();//forces query execution
results[0].ProductVersions;//<-- Lazy loading occurs
if you reference any foreign key or related object that was not part of the original query then it will be lazy loaded in. In your case, the count will not cause any lazy loading because it is simply returning an int. but depending on what you actually do with the result of the Take() you may or may not have Lazy loading occur. Sometimes it can be difficult to tell if you have LazyLoading ocurring, to check you should log your queries using the DataContext.Log property.

The easiest way would be to just do a Count of the query, and then do Take:
var q = ...;
var count = q.Count();
var result = q.Take(...);

It is possible to do this in a single Linq-to-SQL query (where only one SQL statement will be executed). The generated SQL does look unpleasant though, so your performance may vary.
If this is your query:
IQueryable<Person> yourQuery = People
.Where(x => /* complicated query .. */);
You can append the following to it:
var result = yourQuery
.GroupBy (x => true) // This will match all of the rows from your query ..
.Select (g => new {
// .. so 'g', the group, will then contain all of the rows from your query.
CountAll = g.Count(),
TakeFive = g.Take(5),
// We could also query for a max value.
MaxAgeFromAll = g.Max(x => x.PersonAge)
})
.FirstOrDefault();
Which will let you access your data like so:
// Check that result is not null before access.
// If there are no records to find, then 'result' will return null (because of the grouping)
if(result != null) {
var count = result.CountAll;
var firstFiveRows = result.TakeFive;
var maxPersonAge = result.MaxAgeFromAll;
}