So I am converting a old project with ordinary SQL queries to a ORM using the Entity Framework. So I have created database model like this:
So I had this old query which I want to translate to a linq expression
SELECT UGLINK.USERNAME
FROM GMLINK
INNER JOIN UGLINK
ON GMLINK.GROUPID = UGLINK.GROUPID
WHERE (((GMLINK.MODULEID)=%ID%))
And the problem I have is that I can't figure out how to do a join query using the objects.
Instead I have to go though the properties like this (which seems to be working):
// So this is one of the module objects that is located in a listView in the GUI
Module m = ModuleList.selectedItem as Module;
/* Now I want to fetch all the User objects that,
* via a group, is connected to a certain module */
var query = context.gmLink
.Join(context.ugLink,
gmlink => gmlink.GroupId,
uglink => uglink.GroupId,
(gmlink, uglink) => new { gmLink = gmlink, ugLink = uglink })
.Where(gmlink => gmlink.gmLink.ModuleId == m.ModuleId)
.Select(x => x.ugLink.User);
So as I said this works, but as you see I kind of have to connect the modules via the link tables properties .GroupId and .ModuleId and so on. Instead I would like to go through the objects created by EF.
I wanted to write a question a bit like this, but can't figure out how to do it, is it at all possible?
var query = context.User
.Select(u => u.ugLink
.Select(uglink => uglink.Group.gmLink
.Where(gmLink => gmLink.Module == m)));
This should be working:
var query = context.gmLink
.Where(gmlink => gmlink.ModuleId == m.ModuleId)
.SelectMany(gmlink => gmlink.Group.ugLink)
.Select(uglink => uglink.User);
It's impossible to filter gmLinks using .Where(gmlink => gmlink.Module == m) in EF, so this comparison needs to be done using identifiers. Another option is .Where(gmlink => gmlink.Module.ModuleId == m.ModuleId)
If you have lazy loading enabled, you do not need to apply specific join notation (you can access the navigation properties directly) - but the queries that are ran against SQL are inefficient (generally the results are returned in a number of different select statements).
My preference is to disable lazy loading on the context, and use .Include() notation to join tables together manually, resulting in generally more efficient queries. .Include() is used to explicitly join entities in Entity Framework.
Join() is misleading, and not appropriate for joining tables in EF.
So, to replicate this statement:
SELECT UGLINK.USERNAME
FROM GMLINK
INNER JOIN UGLINK
ON GMLINK.GROUPID = UGLINK.GROUPID
WHERE (((GMLINK.MODULEID)=%ID%))
You would use the following:
var query = context.gmLink
.Include(x => x.Group.gmLink)
.Where(x => x.ModuleId == myIdVariable)
.Select(x => new {
UserName = x.Group.ugLink.UserName
});
Assuming that your navigation properties are correctly set up. I have not tested this, so I'm not 100% on the syntax.
You should really run SQL profiler while you write and run LINQ to Entity queries against your database, so you can understand what's actually being generated and run against your database. A lot of the time, an EF query may be functioning correctly, but you may experience performance issues when deployed to a production system.
This whitepaper might help you out.
I haven't tested it, but something like this:
var users = context.User
.Where(x => x.ugLink
.Any(y => context.gmLink
.Where(z => z.ModuleId == m)
.Select(z => z.GroupId)
.Contains(y.GroupId)
)
)
.ToList();
Related
I have an IQueryable of a complex EF model, let's call it GeneralForm. This GeneralForm entity aggregates a member called Section. The Section contains a list of FormFields and each FormField has a name. I want to select only the FormFields whose names are in a list of given names.
IQueryable<GeneralForm> query = InitializeMyQuery();
What is the correct "Where" clause to do so. something like this:
if (criteria.FormFieldNames.Any())
{
query = query.Where(gf => gf.Section.FormFields.Where(x => criteria.FormFieldNames.Contains(x.FormField.FieldName)).Any());
}
does not work, as it still retrieves all FormFields, not just the ones I want.
Any suggestion would be highly appreciated.
Thanks,
Ed
Edit 1: This is how the query is built (for privacy reason, I renamed some entities and I also removed the ones that do not really pertain to the issue I am trying to resolve):
query = (from genFormEntry in _context.GeneralForms
.Include(r => r.Sections)
.Include(r => r.Form.FormFields)
.Include(r => r.Form.FormFields.Select(x => x.FormField))
select genFormEntry);
This query retrieves Sections that have any matching form name. It doesn't do any filtering on the FormField side.
You may try to join those tables manually, or depending on your Ef version, you can try using filtered includes:
if (criteria.FormFieldNames.Any())
{
query = query
.Include(gf => gf.Section.FormFields.Where(x => criteria.FormFieldNames.Contains(x.FormField.FieldName)) // Include the FormFields that match the criteria
.Where(gf => gf.Section.FormFields.Where(x => criteria.FormFieldNames.Contains(x.FormField.FieldName)).Any());
}
Edit:
As Ef 6.1 doesn't support filtered includes. Only two options left. 1 is mentioned above which is manual linq joins (which is pretty ugly and not versatile) and the other is to rewrite the query like below :
// guessing navigation property names here.
query = _context.FormFields.Include(r => r.Form.Section.GeneralForm);
// and later in your code
if (criteria.FormFieldNames.Any())
{
query = query.Where(f => criteria.FormFieldNames.Contains(f.FieldName));
}
For the life of me I am unable to google my way out of this one.
I have 2 tables within a database
1. Computers
2. UserLogins
Essentially, I'm trying to get the latest login entry from the "UserLogins" table, and join it with the corresponding entry in the "Computers" table.
This sounds simple enough, but I haven't sat through enough LINQ/EF Core courses yet to figure out how to do this correctly it seems.
Here is some SQL that I know functions how I expect it to:
SELECT * FROM ComputerInfo
LEFT JOIN (
SELECT LoginID, UserID, l.ComputerName, IpAddress, l.LoginTime FROM UserLogins as l
INNER JOIN (
SELECT ComputerName, MAX(LoginTime) as LoginTime
FROM UserLogins
GROUP BY ComputerName) as max on max.ComputerName = l.ComputerName and max.LoginTime = l.LoginTime
) as toplogin on toplogin.ComputerName = ComputerInfo.ComputerName
For reference, I am going to be implementing this in my Controller.cs class, and I am using :
EF Core (3.1.2)
ASP.NET Core (3.1)
I do have a couple queries I was experimenting with that return the results, but I can't join them without errors:
var computerQuery = _context.ComputerInfo
.OrderBy(on => on.ComputerName)
var userQuery = _context.UserLogins
.Select(p => p.ComputerName)
.Distinct()
.Select(id => _context.UserLogins
.OrderByDescending(p => p.LoginTime)
.FirstOrDefault(p => p.ComputerName == id))
.ToListAsync();
So I kind of found a shotty way to get this done I think. Not sure if it is correct but here is what I came up with:
I Created a new class called "ComputerInfoFull" which basically was just "ComputerInfo" && "UserLogins" combined, and used this for the linq query:
var initial = from computerInfo in _context.ComputerInfo
from userInfo in _context.UserLogins
.Where(o => o.ComputerName == computerInfo.ComputerName)
.OrderByDescending(o => o.LoginTime).Take(1)
select new ComputerInfoFull(computerInfo, userInfo);
I'm very sure there is a cleaner Lambda way of writing this, but I can't figure out how to make it work right. Too much stuff going on for my tiny brain to handle lol. If anyone has any ideas on how I can make this cleaner please let me know so I can learn.
I have to put a complex query on your database. But the query ends at 8000 ms. Do I do something wrong? I use .net 1.1 and Entity Framework core 1.1.2 version.
var fol = _context.UserRelations
.Where(u => u.FollowerId == id && u.State == true)
.Select(p => p.FollowingId)
.ToArray();
var Votes = await _context.Votes
.OrderByDescending(c => c.CreationDate)
.Skip(pageSize * pageIndex)
.Take(pageSize)
.Where(fo => fol.Contains(fo.UserId))
.Select(vote => new
{
Id = vote.Id,
VoteQuestions = vote.VoteQuestions,
VoteImages = _context.VoteMedias.Where(m => m.VoteId == vote.Id)
.Select(k => k.MediaUrl.ToString()),
Options = _context.VoteOptions.Where(m => m.VoteId == vote.Id).Select( ques => new
{
OptionsID = ques.Id,
OptionsName = ques.VoteOption,
OptionsCount = ques.VoteRating.Count(cout => cout.VoteOptionsId == ques.Id),
}),
User = _context.Users.Where(u => u.Id == vote.UserId).Select(usr => new
{
Id = usr.Id,
Name = usr.UserProperties.Where(o => o.UserId == vote.UserId).Select(l => l.Name.ToString())
.First(),
Surname = usr.UserProperties.Where(o => o.UserId == vote.UserId)
.Select(l => l.SurName.ToString()).First(),
ProfileImage = usr.UserProfileImages.Where(h => h.UserId == vote.UserId && h.State == true)
.Select(n => n.ImageUrl.ToString()).First()
}),
NextPage = nextPage
}).ToListAsync();
Have a look at the SQL queries you generate to the server (and results of this queries). For SQL Server the best option is SQL Server Profiler, there are ways for other servers too.
you create two queries. First creates fol array and then you pass it into the second query using Contains. Do you know how this works? You probably generate query with as many parameters as many items you have in the array. It is neither pretty or efficient. It is not necessary here, merge it into the main query and you would have only one parameter.
you do paginating before filtering, is this really the way it should work? Also have a look at other ways of paginating based on filtering by ids rather than simple skipping.
you do too much side queries in one query. When you query three sublists of 100 items each, you do not get 300 rows. To get it in one query you create join and get actually 100*100*100 = 1000000 rows. Unless you are sure the frameworks can split it into multiple queries (probably can not), you should query the sublists in separate queries. This would be probably the main performance problem you have.
please use singular to name tables, not plural
for performance analysis, indexes structure and execution plan are vital information and you can not really say much without them
As noted in the comments, you are potentially executing 100, 1000 or 10000 queries. For every Vote in your database that matches the first result you do 3 other queries.
For 1000 votes which result from the first query you need to do 3000 other queries to fetch the data. That's insane!
You have to use EF Cores eager loading feature to fetch this data with very few queries. If your models are designed well with relations and navigation properties its easy.
When you load flat models without a projection (using .Select), you have to use .Include to tell EF Which other related entities it should load.
// Assuming your navigation property is called VoteMedia
await _context.Votes.
.Include(vote => vote.VoteMedia)
...
This would load all VoteMedia objects with the vote. So extra query to get them is not necessary.
But if you use projects, the .Include calls are not necessary (in fact they are even ignored, when you reference navigation properties in the projection).
// Assuming your navigation property is called VoteMedia
await _context.Votes.
.Include(vote => vote.VoteMedia)
...
.Select( vote => new
{
Id = vote.Id,
VoteQuestions = vote.VoteQuestions,
// here you reference to VoteMedia from your Model
// EF Core recognize that and will load VoteMedia too.
//
// When using _context.VoteMedias.Where(...), EF won't do that
// because you directly call into the context
VoteImages = vote.VoteMedias.Where(m => m.VoteId == vote.Id)
.Select(k => k.MediaUrl.ToString()),
// Same here
Options = vote.VoteOptions.Where(m => m.VoteId == vote.Id).Select( ques => ... );
}
I have a fairly complex sql statement with joins. Here is my code using EF:
var MicrositeResponseAdors = _context.MicrositeResponseAdor
.Include(user => user.UserTracking)
.ThenInclude(tracking => tracking.AddTradeIn)
.Include(user => user.UserTracking)
.ThenInclude(tracking => tracking.TrackingFinance)
.Include(user => user.UserTracking)
.ThenInclude(tracking => tracking.SearchTracking)
.Include(user => user.UserTracking)
.ThenInclude(tracking => tracking.TrackingType)
.Include(user => user.UserTracking)
.ThenInclude(tracking => tracking.Vehicle)
.FirstOrDefault(m => m.Id == id && m.FirstName.ToLower() != "test" && m.LastName.ToLower() != "test");
I need to add another join to a table that has no primary key, which Entity Framework doesn't allow. I wasn't even able to fake it. So I was told to make something more custom using what I believe is called LINQ?
I cannot figure out how to assign my data values with the original EF entity classes. Here is what I have so far:
var MicrositeResponseAdors = (from p in _context.MicrositeResponseAdor
join c in _context.UserTracking on p.Id equals c.UserTrackingId
into pc
select new {
MicrositeResponseAdor = p, MicrositeResponseAdor.UserTracking = pc
}
).FirstOrDefault();
This gives:
Invalid anonymous type member declarator.
I think that is because of the first part of MicrositeResponseAdor.UserTracking. I am not sure how to assign UserTracking under the MicrositeResponseAdor.
The error you've given is a C# compilation error, as you're trying to name a property MicrositeResponseAdor.UserTracking. Property names can't contain dots.
This would at least compile:
var micrositeResponseAdor = (
from p in _context.MicrositeResponseAdor
join c in _context.UserTracking on p.Id equals c.UserTrackingId into pc
select new {
MicrositeResponseAdor = p,
UserTracking = pc
}
).FirstOrDefault();
This will make your UserTracking appear in two places:
micrositeResponseAdor.UserTracking
micrositeResponseAdor.MicrositeResponseAdor.UserTracking
It's in the first because you've put it there, and it's in the second because there's a relationship between the entities anyway, so EF is conveniently populating it if it has loaded it in that context (and it has, because you asked for it explicitly).
So you don't have to assign loaded navigational properties in your query, you just have to make sure the related entities are loaded, either with a manual join, an Include or even a second query.
The question is why you do this at all. You've already had the Include version. The join variant is just more verbose, but it doesn't add anything.
Not that EF will always have a key on every entity, whether the database thinks it is one or not. Technically, that's unrelated Include vs join though.
Consider following LINQ query:
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select new
{
ItemProp1 = obj,
ItemProp2 = obj.NavProp2.Any(n => n.Active)
}).SingleOrDefault();
This runs as expected, but item.ItemProp1.NavProp1 is NULL.
As it explains here this is because of the query actually changes after using Include(). but the question is what is the solution with this situation?
Edit:
When I change the query like this, every things works fine:
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select obj).SingleOrDefault();
Regarding to this article I guess what the problem is... but the solution provided by author not working in my situation (because of using anonymous type in final select rather than entity type).
As you mentioned, Include is only effective when the final result of the query consists of the entities that should include the Include-d navigation properties.
So in this case Include has effect:
var list = _db.SampleEntity.Include(s => s.NavProp1).ToList();
The SQL query will contain a JOIN and each SampleEntity will have its NavProp1 loaded.
In this case it has no effect:
var list = _db.SampleEntity.Include(s => s.NavProp1)
.Select(s => new { s })
.ToList();
The SQL query won't even contain a JOIN, EF completely ignores the Include.
If in the latter query you want the SampleEntitys to contain their NavProp1s you can do:
var list = _db.SampleEntity
.Select(s => new { s, s.NavProp1 })
.ToList();
Now Entity Framework has fetched SampleEntitys and NavProp1 entities from the database separately, but it glues them together by a process called relationship fixup. As you see, the Include is not necessary to make this happen.
However, if Navprop1 is a collection, you'll notice that...
var navprop1 = list.First().s.Navprop1;
...will still execute a query to fetch Navprop1 by lazy loading. Why is that?
While relationship fixup does fill Navprop1 properties, it doesn't mark them as loaded. This only happens when Include loaded the properties. So now we have SampleEntity all having their Navprop1s, but you can't access them without triggering lazy loading. The only thing you can do to prevent this is
_db.Configuration.LazyLoadingEnabled = false;
var navprop1 = list.First().s.Navprop1;
(or by preventing lazy loading by disabling proxy creation or by not making Navprop1 virtual.)
Now you'll get Navprop1 without a new query.
For reference navigation properties this doesn't apply, lazy loading isn't triggered when it's enabled.
In Entity Framework core, things have changed drastically in this area. A query like _db.SampleEntity.Include(s => s.NavProp1).Select(s => new { s }) will now include NavProp1 in the end result. EF-core is smarter in looking for "Includable" entities in the end result. Therefore, we won't feel inclined to shape a query like Select(s => new { s, s.NavProp1 }) in order to populate the navigation property. Be aware though, that if we use such a query without Include, lazy loading will still be triggered when s.NavProp1 is accessed.
I know this will probably get a few laughs, but don't forget the obvious like i just did. The row in the database didn't actually have a foreign key reference! I should have checked the dam data first before thinking EF Include wasn't working! Grrr. 30 minutes of my life I won't get back.
If your model is defined properly it should work without any problems.
using System.Data.Entity;
var item = _db.SampleEntity
.Include(p => p.NavigationProperty)
.Select(p => new YourModel{
PropertyOne = p.Something,
PropertyTwo = p.NavigationProperty.Any(x => x.Active)
})
.SingleOrDefault(p => p.Something == true);
How did you find that item.ItemProp1.NavProp1 is null. EF uses proxies to load all required properties when you try to access it.
What about
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select obj).SingleOrDefault();
Assert.IsNotNull(obj.NavProp1);
Assert.IsNotNull(obj.NavProp2);
You can also try with
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select new
{
ItemProp1 = obj,
NavProp1 = obj.NavProp1,
ItemProp2 = obj.NavProp2.Any(n => n.Active)
}).SingleOrDefault();
Assert.IsNotNull(item.NavProp1)
Of course I assume that you don't have any problems with EF navigation property mappings.