We Keep Coding

LINQ Query - Proper "Where" clause to select only children that meet a condition

I have an IQueryable of a complex EF model, let's call it GeneralForm. This GeneralForm entity aggregates a member called Section. The Section contains a list of FormFields and each FormField has a name. I want to select only the FormFields whose names are in a list of given names.
IQueryable<GeneralForm> query = InitializeMyQuery();
What is the correct "Where" clause to do so. something like this:
if (criteria.FormFieldNames.Any())
{
query = query.Where(gf => gf.Section.FormFields.Where(x => criteria.FormFieldNames.Contains(x.FormField.FieldName)).Any());
}
does not work, as it still retrieves all FormFields, not just the ones I want.
Any suggestion would be highly appreciated.
Thanks,
Ed
Edit 1: This is how the query is built (for privacy reason, I renamed some entities and I also removed the ones that do not really pertain to the issue I am trying to resolve):
query = (from genFormEntry in _context.GeneralForms
.Include(r => r.Sections)
.Include(r => r.Form.FormFields)
.Include(r => r.Form.FormFields.Select(x => x.FormField))
select genFormEntry);

This query retrieves Sections that have any matching form name. It doesn't do any filtering on the FormField side.
You may try to join those tables manually, or depending on your Ef version, you can try using filtered includes:
if (criteria.FormFieldNames.Any())
{
query = query
.Include(gf => gf.Section.FormFields.Where(x => criteria.FormFieldNames.Contains(x.FormField.FieldName)) // Include the FormFields that match the criteria
.Where(gf => gf.Section.FormFields.Where(x => criteria.FormFieldNames.Contains(x.FormField.FieldName)).Any());
}
Edit:
As Ef 6.1 doesn't support filtered includes. Only two options left. 1 is mentioned above which is manual linq joins (which is pretty ugly and not versatile) and the other is to rewrite the query like below :
// guessing navigation property names here.
query = _context.FormFields.Include(r => r.Form.Section.GeneralForm);
// and later in your code
if (criteria.FormFieldNames.Any())
{
query = query.Where(f => criteria.FormFieldNames.Contains(f.FieldName));
}

Ef returning null, but entity exists

I'm trying to load some user info in code, but EF returns null.
foreach (var user in allAutoUsers)
{
Wallet wallet = db.CabinetUsers
.Find(user.IdCabinetUser)?
.Wallets
.FirstOrDefault(x => x.TypeCurrency == currency);
}
Variable user reporting 1 wallet, but when I'm trying to get it in code above it returns null.
Are there some ways to solve this problem?

Read more into Linq expressions rather than relying on Find. You can be running into issues if your relationships between entities are not defined as virtual which would prevent EF from lazy loading them.
The issue with using .Find() is that it will return the entity if it exists, however, attempting to access any related property that the DbContext isn't already aware of will require a lazy load call. This can be easily missed if lazy loading is disabled, or the member isn't virtual, and it can be a performance issue when it is enabled.
Instead, Linq can allow you to query through the object graph directly to get what you want:
foreach (var user in allAutoUsers)
{
Wallet wallet = db.CabinetUsers
.Where(x => x.IdCabinetUser == user.IdCabinetUser)
.SelectMany(x => x.Wallets)
.SingleOrDefault(x => x.TypeCurrency == currency);
// do stuff with wallet...
}
This assumes that there will be only 1 wallet for the specified currency per user. When expecting 0 or 1, use SingleOrDefault. Use FirstOrDefault only when you are expecting 0 or many, want the "first" one and have specified an OrderBy clause to ensure the first item is predictable.
This will result in a query per user. To accomplish with 1 query for all users:
var userIds = allAutoUsers.Select(x => x.IdCabinetUser).ToList();
var userWallets = db.CabinetUsers
.Where(x => userIds.Contains(x.IdCabinetUser))
.Select(x => new
{
x.IdCabinetUser,
Wallet = x.SelectMany(x => x.Wallets)
.SingleOrDefault(x => x.TypeCurrency == currency);
}).ToList();
From this I would consider expanding the wallets SelectMany with a Select for the details in the Wallet you actually care about rather than a reference to the entire wallet entity. This has the benefit of speeding up the query, reducing memory use, and avoids the potential for lazy load calls tripping things up if Wallet references any other entities that get touched later on.
For example if you only need the IdWallet, WalletName, TypeCurrency, and Balance:
// replace this line from above...
Wallet = x.SelectMany(x => x.Wallets)
// with ...
Wallet = x.SelectMany(x => x.Wallets.Select(w => new
{
w.IdWallet,
w.WalletName,
w.TypeCurrency,
w.Ballance
}) // ...
From there you can foreach to your heart's content without extra queries:
foreach ( var userWallet in userWallets)
{
// do stuff with userWallet.Wallet and userWallet.IdCabinetUser.
}
If you want to return the wallet details to a calling method or view or such, then you cannot use an anonymous type for that ( new { } ). Instead you will need to define a simple class for the data you want to return and use Select into that. I.e. new WalletDTO { IdWallet = w.IdWallet, //... } Even if you use Entities, it is recommended to reduce these into DTOs or ViewModels rather than returning entities. Entities should not "live" longer than the DbContext that spawned them otherwise you get all kinds of nasty behaviour popping up like ObjectDisposedException and serialization exceptions.

LINQ method to load child Entity that includes some of Parent's navigational properties and only selected Grand Child 's records

I've three tables Site_Report Report_Asset and Asset_Calcert with One-to-Many relationship between them, as shown.
I want to load a child entity as a model i.e Report_Asset which should include some of the parent's properties. And also, from grand child's collection navigation property, I want to load single record using a condition.
First Attempt - This leads to an error.
Report_Asset model;
model = ctx.Report_Asset
.Include(i => i.Site_Report)
.Include(i => i.Site_Report.Handled_By)
.Include(i => i.Site_Report.Published_By)
.Include(i => i.Asset_Calcerts.Select(b => b.asset_calcert_id == assetCalcertId))
.FirstOrDefault();
ERROR: The Include path expression must refer to a navigation property defined on the type. Use dotted paths for reference navigation properties and the Select operator for collection navigation properties.
Second Attempt - By starting form grandchild Entity and including the parents. This approach does load the result set, but it only contains the child entity i.e Report_Asset and no record from Asset_Calcert collection and null for Site_Report.
Report_Asset model;
model = ctx.Asset_Calcert.Include(i => i.Report_Asset)
.Include(i => i.Report_Asset.Site_Report.Handled_By)
.Include(i => i.Report_Asset.Site_Report.Published_By)
.Where(i => i.asset_calcert_id == assetCalcertId)
.Select(i => i.Report_Asset).FirstOrDefault();
I've set Configuration.LazyLoadingEnabled = false in DbContext().
Need direction, thanks

It looks like you want to query and return an entity, it's parent, and then a sub-set of it's children. You can use EF to query this, but only by reducing your query into a desired structure. If you return a Report_Asset, it can include a reference to it's parent easily enough, but it will always reference it's complete set of children, EF does not filter child sets at the entity level.
It's best to look at exactly what fields you want from each entity and select a structure containing just those, but at a very basic level, this should give you what you want:
var model = ctx.Report_Asset
.Select( x = > new
{
Report_Asset = x,
Site_Report = x.Site_Report,
Handled_By = x.Site_Report.Handled_By,
Published_By = x.Site_Report.Published_By,
Asset_Calcerts = x.Assert_Calcerts.Where(c => c.asset_calcert_id == assetCalcertId).ToList()
}).FirstOrDefault();
The calcert check looks to be looking for a specific, single cert, so this might be better as:
var model = ctx.Report_Asset
.Select( x = > new
{
Report_Asset = x,
Site_Report = x.Site_Report,
Handled_By = x.Site_Report.Handled_By,
Published_By = x.Site_Report.Published_By,
Asset_Calcert = x.Assert_Calcerts.SingleOrDefailt(c => c.asset_calcert_id == assetCalcertId)
}).FirstOrDefault();
If you were looking to load only report assets that had a matching Calcert, with the parent and the applicable calcert:
var model = ctx.Report_Asset
.Where(x => x.Asset_Calcerts.Any(c => c.asset_calcert_id == assetCalcertId))
.Select( x = > new
{
Report_Asset = x,
Site_Report = x.Site_Report,
Handled_By = x.Site_Report.Handled_By,
Published_By = x.Site_Report.Published_By,
Asset_Calcert = x.Asset_Calcerts.SingleOrDefailt(c => c.asset_calcert_id == assetCalcertId)
}).FirstOrDefault();
In all cases I would recommend adding an OrderBy clause before the FirstOrDefault to ensure a predictable order is used.
The caveat of this approach is that if you access the returned model's entities and start drilling down through their references, you will still trigger lazy loads from the database. For instance if I use:
model.Report_Asset.Asset_Calcerts
for instance, this will still lazy load the related entities for that report asset, and list all calcerts for that asset. The filtered set/match is eager loaded into the model.Asset_Calcert(s), not the returned Model_Asset entity.
Generally though you are better off using Select to just retrieve the properties you care about from the various entities, rather than attempting to select entire entities or entity graphs. This forgoes the need for using Include, just tell EF exactly what you want from the entity graph and it will build an SQL statement to efficiently retrieve just that information.

when using more then 1 level it is better to use the string option.
if you dont want to use string you will need to do select for the levels greater then 1.
Report_Asset model;
model = ctx.Report_Asset
.Include("Site_Report")
.Include("Site_Report.Handled_By")
.Include("Site_Report.Published_By")
.Include("Asset_Calcerts")
.FirstOrDefault(x => x.Asset_Calcerts.Any(y => y.asset_calcert_id == assetCalcertId);

Include() does not work

var x = connection.Set<Team>()
.Include(t => t.Level)
.Select(t =>
new {Team = t, LevelForTesting = t.Level})
.ToList()
Why I don't get the object x[0].Team.Level (I have 'null') but get the object x[0].LevelForTesting? How can I change this code to get x[0].Team.Level? Thanks.

You are throwing away the results of your eager loading by the anonymous select. Just drop the select and you will be able to access the Level in your list of Team:
var x = connection.Set<Team>().Include(t => t.Level).ToList();
var level = x[0].Level;
To get a better understanding of lazy/eager loading you should read this. Basically eager loading populates the specified navigation properties of your list of entities.

How to do join with objects in Entity Framework

So I am converting a old project with ordinary SQL queries to a ORM using the Entity Framework. So I have created database model like this:
So I had this old query which I want to translate to a linq expression
SELECT UGLINK.USERNAME
FROM GMLINK
INNER JOIN UGLINK
ON GMLINK.GROUPID = UGLINK.GROUPID
WHERE (((GMLINK.MODULEID)=%ID%))
And the problem I have is that I can't figure out how to do a join query using the objects.
Instead I have to go though the properties like this (which seems to be working):
// So this is one of the module objects that is located in a listView in the GUI
Module m = ModuleList.selectedItem as Module;
/* Now I want to fetch all the User objects that,
* via a group, is connected to a certain module */
var query = context.gmLink
.Join(context.ugLink,
gmlink => gmlink.GroupId,
uglink => uglink.GroupId,
(gmlink, uglink) => new { gmLink = gmlink, ugLink = uglink })
.Where(gmlink => gmlink.gmLink.ModuleId == m.ModuleId)
.Select(x => x.ugLink.User);
So as I said this works, but as you see I kind of have to connect the modules via the link tables properties .GroupId and .ModuleId and so on. Instead I would like to go through the objects created by EF.
I wanted to write a question a bit like this, but can't figure out how to do it, is it at all possible?
var query = context.User
.Select(u => u.ugLink
.Select(uglink => uglink.Group.gmLink
.Where(gmLink => gmLink.Module == m)));

This should be working:
var query = context.gmLink
.Where(gmlink => gmlink.ModuleId == m.ModuleId)
.SelectMany(gmlink => gmlink.Group.ugLink)
.Select(uglink => uglink.User);
It's impossible to filter gmLinks using .Where(gmlink => gmlink.Module == m) in EF, so this comparison needs to be done using identifiers. Another option is .Where(gmlink => gmlink.Module.ModuleId == m.ModuleId)

If you have lazy loading enabled, you do not need to apply specific join notation (you can access the navigation properties directly) - but the queries that are ran against SQL are inefficient (generally the results are returned in a number of different select statements).
My preference is to disable lazy loading on the context, and use .Include() notation to join tables together manually, resulting in generally more efficient queries. .Include() is used to explicitly join entities in Entity Framework.
Join() is misleading, and not appropriate for joining tables in EF.
So, to replicate this statement:
SELECT UGLINK.USERNAME
FROM GMLINK
INNER JOIN UGLINK
ON GMLINK.GROUPID = UGLINK.GROUPID
WHERE (((GMLINK.MODULEID)=%ID%))
You would use the following:
var query = context.gmLink
.Include(x => x.Group.gmLink)
.Where(x => x.ModuleId == myIdVariable)
.Select(x => new {
UserName = x.Group.ugLink.UserName
});
Assuming that your navigation properties are correctly set up. I have not tested this, so I'm not 100% on the syntax.
You should really run SQL profiler while you write and run LINQ to Entity queries against your database, so you can understand what's actually being generated and run against your database. A lot of the time, an EF query may be functioning correctly, but you may experience performance issues when deployed to a production system.
This whitepaper might help you out.

I haven't tested it, but something like this:
var users = context.User
.Where(x => x.ugLink
.Any(y => context.gmLink
.Where(z => z.ModuleId == m)
.Select(z => z.GroupId)
.Contains(y.GroupId)
)
)
.ToList();