I am relatively new to Entity Framework 6.0 and I have come across a situation where I want to execute a query in my C# app that would be similar to this SQL Query:
select * from periods where id in (select distinct periodid from ratedetails where rateid = 3)
Is it actually possible to execute a query like this in EF or would I need to break it into smaller steps?
Assuming that you have in your Context class:
DbSet<Period> Periods...
DbSet<RateDetail> RateDetails...
You could use some Linq like this:
var distincts = dbContext.RateDetails
.Where(i => i.rateId == 3)
.Select(i => i.PeriodId)
.Distinct();
var result = dbContext.Periods
.Where(i => i.Id)
.Any(j => distincts.Contains(j.Id));
Edit: Depending on your entities, you will probably need a custom Comparer for Distinct(). You can find a tutorial here, and also here
or use some more Linq magic to split the results.
Yes, this can be done but you should really provide a better example for your query. You are already providing a bad starting point there. Lets use this one:
SELECT value1, value2, commonValue
FROM table1
WHERE EXISTS (
SELECT 1
FROM table2
WHERE table1.commonValue = table2.commonValue
// include some more filters here on table2
)
First, its almost always better to use EXISTS instead of IN.
Now to turn this into a Lambda would be something like this, again you provided no objects or object graph so I will just make something up.
DbContext myContext = this.getContext();
var myResults = myContext.DbSet<Type1>().Where(x => myContext.DbSet<Type2>().Any(y => y.commonValue == x.commonValue)).Select(x => x);
EDIT - updated after you provided the new sql statement
Using your example objects this would produce the best result. Again, this is more efficient than a Contains which translates to an IN clause.
Sql you really want:
SELECT *
FROM periods
WHERE EXISTS (SELECT 1 FROM ratedetails WHERE rateid = 3 AND periods.id = ratedetails.periodid)
The Lamda statement you are after
DbContext myContext = this.getContext();
var myResults = myContext.DbSet<Periods>()
.Where(x => myContext.DbSet<RateDetails>().Any(y => y.periodid == x.id && y.rateid == 3))
.Select(x => x);
Here is a good starting point for learning about lamda's and how to use them.
Lambda Expressions (C# Programming Guide).
this is your second where clause in your query
var priodidList=ratedetails.where(x=>x.rateid ==3).DistinctBy(x=>x.rateid);
now for first part of query
var selected = periods.Where(p => p.id
.Any(a => priodidList.Contains(a.periodid ))
.ToList();
Related
The following LINQ query in EF (EFCore, v2.2.1)
var x = context.Exchange
.Include(q => q.Input)
.Where(q => q.InputId != 1&&
q.Input.CreatedOnUtc > DateTime.Parse("2019-11-25") &&
q.Input.UserId == 2 &&
q.BotConversationId == 3)
.Distinct()
.OrderBy(q => q.Input.CreatedOnUtc)
.FirstOrDefault()
Ends up giving the profiled SQL results (simplified)
select * from (
select distinct e.*
from Exchange e, ExchangeInput i
where e.InputId = i.InputId
and e.InputId <> 1
and i.UserId = 2
and e.BotConversationId = 3
)
select * from ExchangeInput
Why does it need to do two separate queries? The second query being horrendous when ExchangeInput might have millions of rows.
Surely, this would suffice:
select * from (
select distinct e.*, i.CreatedOnUtc
from Exchange e, ExchangeInput i
where e.InputId = i.InputId
and e.InputId <> 1
and i.UserId = 2
and e.BotConversationId = 3
) a
order by a.CreatedOnUtc
Also - putting the Distinct after the order by gives only 1 query as I would expect.
Fixing the problem is easy enough. Adding a .Select(...) before the .Distinct or removing the .Distinct() will do it. But the initial, poorly performing code, doesn't seem immediately problematic when reviewing it.
I would start by suggesting that calling Distinct() before a FirstOrDefault() is unnecessary. The first row in a "non-distinct" query should always be the same as a "distinct" query when you have an OrderBy! As you mentioned in your last sentence, it seems that removing the Distinct() should only create one query.
Separate to your question, I would also suggest calculating DateTime.Parse("2019-11-25") outside of the query. That should allow you to pass it to the database server as a parameter and that might make your query even more efficient.
All in all, I would try:
var dateFilter = DateTime.Parse("2019-11-25");
var x = context.Exchange
.Include(q => q.Input)
.Where(q => q.InputId != 1 &&
q.Input.CreatedOnUtc > dateFilter &&
q.Input.UserId == 2 &&
q.BotConversationId == 3)
.OrderBy(q => q.Input.CreatedOnUtc)
.FirstOrDefault()
I was really excited to hear that EF Core 2.1 will be supporting group by translations. I downloaded the preview and started testing it out but found that I am still not getting group by translations in a number of places.
In the code snippet below, the query for TotalFlagCases will prevent the group by translation from working. Is there anyway that I can rewrite this so that I can have group by support? Or perhaps another approach that I can take?
There are a lot of rows in this table and I don't want .NET to have to load all of these rows. I use row level data as well, but only about 15 records at a time.
var headerTask = (from c in cases
group c by 1
into g
select new CaseHeader
{
TotalCases = g.Count(),
// ... A number of other aggregates
TotalFlagCases = g.Where(a => a.Flag).Sum(b => 1),
})
.DefaultIfEmpty(new CaseHeader()).FirstAsync();
There is a way to do a conditional sum in this version of EF Core. Provided code is not going to be translated into desired SQL with GROUP BY but maybe some future version will support it this way. For now you can try something like this:
var headerTask = cases
.Select(c => new
{
c.Flag,
c.YourKey,
//other properties
})
.GroupBy(c => c.YourKey, (k, g) => new CaseHeader
{
TotalCases = g.Count(),
// ... A number of other aggregates
TotalFlagCases = g.Sum(b => a.Flag ? 1 : 0)
});
When you project your entity into an anonymous type and then group it and use conditional operator in an aggregate function it will be translated into SQL with GROUP BY and aggregates like:
SELECT COUNT(*) AS [TotalCases], SUM(CASE
WHEN [c].[Flag] = 1
THEN 1 ELSE 0
END) AS [TotalFlagCases]
FROM [Cases] AS [c]
GROUP BY [c].[YourKey]
When you do not project it to an anonymous type so when there is the above Select function missing it will not be translated into SQL with GROUP BY. It looks like the query translator for this prerelease does not support it or it's a bug.
I have to put a complex query on your database. But the query ends at 8000 ms. Do I do something wrong? I use .net 1.1 and Entity Framework core 1.1.2 version.
var fol = _context.UserRelations
.Where(u => u.FollowerId == id && u.State == true)
.Select(p => p.FollowingId)
.ToArray();
var Votes = await _context.Votes
.OrderByDescending(c => c.CreationDate)
.Skip(pageSize * pageIndex)
.Take(pageSize)
.Where(fo => fol.Contains(fo.UserId))
.Select(vote => new
{
Id = vote.Id,
VoteQuestions = vote.VoteQuestions,
VoteImages = _context.VoteMedias.Where(m => m.VoteId == vote.Id)
.Select(k => k.MediaUrl.ToString()),
Options = _context.VoteOptions.Where(m => m.VoteId == vote.Id).Select( ques => new
{
OptionsID = ques.Id,
OptionsName = ques.VoteOption,
OptionsCount = ques.VoteRating.Count(cout => cout.VoteOptionsId == ques.Id),
}),
User = _context.Users.Where(u => u.Id == vote.UserId).Select(usr => new
{
Id = usr.Id,
Name = usr.UserProperties.Where(o => o.UserId == vote.UserId).Select(l => l.Name.ToString())
.First(),
Surname = usr.UserProperties.Where(o => o.UserId == vote.UserId)
.Select(l => l.SurName.ToString()).First(),
ProfileImage = usr.UserProfileImages.Where(h => h.UserId == vote.UserId && h.State == true)
.Select(n => n.ImageUrl.ToString()).First()
}),
NextPage = nextPage
}).ToListAsync();
Have a look at the SQL queries you generate to the server (and results of this queries). For SQL Server the best option is SQL Server Profiler, there are ways for other servers too.
you create two queries. First creates fol array and then you pass it into the second query using Contains. Do you know how this works? You probably generate query with as many parameters as many items you have in the array. It is neither pretty or efficient. It is not necessary here, merge it into the main query and you would have only one parameter.
you do paginating before filtering, is this really the way it should work? Also have a look at other ways of paginating based on filtering by ids rather than simple skipping.
you do too much side queries in one query. When you query three sublists of 100 items each, you do not get 300 rows. To get it in one query you create join and get actually 100*100*100 = 1000000 rows. Unless you are sure the frameworks can split it into multiple queries (probably can not), you should query the sublists in separate queries. This would be probably the main performance problem you have.
please use singular to name tables, not plural
for performance analysis, indexes structure and execution plan are vital information and you can not really say much without them
As noted in the comments, you are potentially executing 100, 1000 or 10000 queries. For every Vote in your database that matches the first result you do 3 other queries.
For 1000 votes which result from the first query you need to do 3000 other queries to fetch the data. That's insane!
You have to use EF Cores eager loading feature to fetch this data with very few queries. If your models are designed well with relations and navigation properties its easy.
When you load flat models without a projection (using .Select), you have to use .Include to tell EF Which other related entities it should load.
// Assuming your navigation property is called VoteMedia
await _context.Votes.
.Include(vote => vote.VoteMedia)
...
This would load all VoteMedia objects with the vote. So extra query to get them is not necessary.
But if you use projects, the .Include calls are not necessary (in fact they are even ignored, when you reference navigation properties in the projection).
// Assuming your navigation property is called VoteMedia
await _context.Votes.
.Include(vote => vote.VoteMedia)
...
.Select( vote => new
{
Id = vote.Id,
VoteQuestions = vote.VoteQuestions,
// here you reference to VoteMedia from your Model
// EF Core recognize that and will load VoteMedia too.
//
// When using _context.VoteMedias.Where(...), EF won't do that
// because you directly call into the context
VoteImages = vote.VoteMedias.Where(m => m.VoteId == vote.Id)
.Select(k => k.MediaUrl.ToString()),
// Same here
Options = vote.VoteOptions.Where(m => m.VoteId == vote.Id).Select( ques => ... );
}
How can I make this lesser than or equal work in my .Where() clause? I am getting an error.
var filteredProducts = Products.Where(p => p.State.Contains("Bruikbaar"))
.Where(p => p.Privilege <= ui.GetPrivilegeNumber())
.ToList();
Error:
LINQ to Entities does not recognize the method 'Int32 GetPrivilegeNumber()' method, and this method cannot be translated into a store expression.
I hope this question is never asked before. Googled couldn't find it either or I am using the wrong words to express my problem.
ui.GetPrivilegeNumber() is not a recognized method.
Use this:
var uiPrivilege = ui.GetPrivilegeNumber();
var filteredProducts = Products.Where(p => p.State.Contains("Bruikbaar"))
.Where(p => p.Privilege <= uiPrivilege)
.ToList();
And as other users mentionted, you can optimize your Where.
EF does not execute method calls which you use in predicates. It stores them as expression (i.e. syntax tree) and then analyzes this tree to build SQL query by translating C# code to SQL code. It cannot translate GetPrivilegeNumber() method call into SQL, because there is no appropriate SQL code for that. So all you need is move this method call out of expression and pass only result of method call instead:
var privilegeNumber = ui.GetPrivilegeNumber();
var filteredProducts = Products.Where(p => p.State.Contains("Bruikbaar"))
.Where(p => p.Privilege <= privilegeNumber)
.ToList();
Now privilegeNumber is just an integer variable which is translated into SQL parameter
SELECT * FROM Products p
WHERE p.State LIKE '%Bruikbaar%' AND p.Privilege <= #privilegeNumber
You need to move ui.GetPrivilegeNumber() outside of the query. You can also merge those Where queries into a single one:
var privilegeNumber = ui.GetPrivilegeNumber();
var filteredProducts = Products.Where(p =>
p.State.Contains("Bruikbaar")
&& p => p.Privilege <= privilegeNumber)
.ToList();
You can use other evaluation method inside LinQ. To simplified the code, you can use it in little old way of writing LinQ.
var uiPrivilege = ui.GetPrivilegeNumber();
var filteredProducts =(from p in Products
where p.State.Contains("Bruikbaar") && p.Privilege <= uiPrivilege
select p).ToList();
The above query generate same output but easy to understood.
So I am converting a old project with ordinary SQL queries to a ORM using the Entity Framework. So I have created database model like this:
So I had this old query which I want to translate to a linq expression
SELECT UGLINK.USERNAME
FROM GMLINK
INNER JOIN UGLINK
ON GMLINK.GROUPID = UGLINK.GROUPID
WHERE (((GMLINK.MODULEID)=%ID%))
And the problem I have is that I can't figure out how to do a join query using the objects.
Instead I have to go though the properties like this (which seems to be working):
// So this is one of the module objects that is located in a listView in the GUI
Module m = ModuleList.selectedItem as Module;
/* Now I want to fetch all the User objects that,
* via a group, is connected to a certain module */
var query = context.gmLink
.Join(context.ugLink,
gmlink => gmlink.GroupId,
uglink => uglink.GroupId,
(gmlink, uglink) => new { gmLink = gmlink, ugLink = uglink })
.Where(gmlink => gmlink.gmLink.ModuleId == m.ModuleId)
.Select(x => x.ugLink.User);
So as I said this works, but as you see I kind of have to connect the modules via the link tables properties .GroupId and .ModuleId and so on. Instead I would like to go through the objects created by EF.
I wanted to write a question a bit like this, but can't figure out how to do it, is it at all possible?
var query = context.User
.Select(u => u.ugLink
.Select(uglink => uglink.Group.gmLink
.Where(gmLink => gmLink.Module == m)));
This should be working:
var query = context.gmLink
.Where(gmlink => gmlink.ModuleId == m.ModuleId)
.SelectMany(gmlink => gmlink.Group.ugLink)
.Select(uglink => uglink.User);
It's impossible to filter gmLinks using .Where(gmlink => gmlink.Module == m) in EF, so this comparison needs to be done using identifiers. Another option is .Where(gmlink => gmlink.Module.ModuleId == m.ModuleId)
If you have lazy loading enabled, you do not need to apply specific join notation (you can access the navigation properties directly) - but the queries that are ran against SQL are inefficient (generally the results are returned in a number of different select statements).
My preference is to disable lazy loading on the context, and use .Include() notation to join tables together manually, resulting in generally more efficient queries. .Include() is used to explicitly join entities in Entity Framework.
Join() is misleading, and not appropriate for joining tables in EF.
So, to replicate this statement:
SELECT UGLINK.USERNAME
FROM GMLINK
INNER JOIN UGLINK
ON GMLINK.GROUPID = UGLINK.GROUPID
WHERE (((GMLINK.MODULEID)=%ID%))
You would use the following:
var query = context.gmLink
.Include(x => x.Group.gmLink)
.Where(x => x.ModuleId == myIdVariable)
.Select(x => new {
UserName = x.Group.ugLink.UserName
});
Assuming that your navigation properties are correctly set up. I have not tested this, so I'm not 100% on the syntax.
You should really run SQL profiler while you write and run LINQ to Entity queries against your database, so you can understand what's actually being generated and run against your database. A lot of the time, an EF query may be functioning correctly, but you may experience performance issues when deployed to a production system.
This whitepaper might help you out.
I haven't tested it, but something like this:
var users = context.User
.Where(x => x.ugLink
.Any(y => context.gmLink
.Where(z => z.ModuleId == m)
.Select(z => z.GroupId)
.Contains(y.GroupId)
)
)
.ToList();