We Keep Coding

Entity Framework - slow query after adding group by

I have a following query which runs very fast:
var query =
(from art in ctx.Articles
join phot in ctx.ArticlePhotos on art.Id equals phot.ArticleId
join artCat in ctx.ArticleCategories on art.Id equals artCat.ArticleId
join cat in ctx.Categories on artCat.CategoryId equals cat.Id
where art.Active && art.ArticleCategories.Any(c => c.Category.MaterializedPath.StartsWith(categoryPath))
orderby art.PublishDate descending
select new ArticleSmallResponse
{
Id = art.Id,
Title = art.Title,
Active = art.Active,
PublishDate = art.PublishDate ?? art.CreateDate,
MainImage = phot.RelativePath,
RootCategory = art.Category.Name,
Summary = art.Summary
})
.AsNoTracking().Take(request.Take);
However, if I add group by and change query to following statement, it runs much much slower.
var query =
(from art in ctx.Articles
join phot in ctx.ArticlePhotos on art.Id equals phot.ArticleId
join artCat in ctx.ArticleCategories on art.Id equals artCat.ArticleId
join cat in ctx.Categories on artCat.CategoryId equals cat.Id
where art.Active && art.ArticleCategories.Any(c => c.Category.MaterializedPath.StartsWith(categoryPath))
orderby art.PublishDate descending
select new ArticleSmallResponse
{
Id = art.Id,
Title = art.Title,
Active = art.Active,
PublishDate = art.PublishDate ?? art.CreateDate,
MainImage = phot.RelativePath,
RootCategory = art.Category.Name,
Summary = art.Summary
})
.GroupBy(m => m.Id)
.Select(m => m.FirstOrDefault())
.AsNoTracking().Take(request.Take);
Homepage calls query 9 times for each category. With the first version of query, without caching turned on and connecting to SQL remotely, page load is around 1.5 seconds, which makes it almost instant when application is on server, but second way makes homepage load around 39 seconds when SQL is remotely.
Can it be fixed without rewriting the entire query in to the view or stored procedure?

Grouping is an expensive operation on the database end. Without knowing what your database looks like and what indexes you've setup, it will be difficult to determine. Why not just group on the client side after the data has arrived (assuming its not an overwhelming amount).
This question explains how.
Group by in LINQ

SQL Query to Linq to Entities - C#

I have been trying to convert this SQL statement into a linq as i am trying to move the functionality into a program.
Here is the SQL statement
SELECT cust.sg_group_name AS customer,
(SELECT Sum(du.used_space)
FROM sg_groups AS clnt
LEFT JOIN client_disk_usage AS du
ON clnt.sg_group_id = du.sg_group_id
AND clnt.group_role_id = 3
WHERE clnt.parent_group_id = cust.sg_group_id
AND du.day_of_month = 15
AND du.month_of_year = 05
AND du.used_space_year = 2016) AS disk_usage
FROM sg_groups AS cust
WHERE cust.group_role_id = 2
ORDER BY cust.sg_group_name
Essentially the output is just a list with two columns
customer disk_usage
Customer1 136401537652
Customer2 42208008210
If possible i just want to convert this to a linq statement. I have tried putting the query into LinqPad, but it doesn't seem to want to convert from SQL to Linq (just comes up with a blank white page). I have had a crack at the query myself, but i either get something that doesn't work altogether, or an incorrect number of results.
If anyone has any suggestions that would be great!

disk_usage(Sub Query) is a bit Complicated Part. Converted over here. Try this out
var CoreList = (from clnt in EntityName.sg_groups
join du in EntityName.client_disk_usage
on new { GrpId = clnt.sg_group_id, RoleId = clnt.group_role_id } equals new { GrpId = du.sg_group_id, RoleId = 3 } into LJ
from RT in LJ.DefaultIfEmpty()
where du.day_of_month == 15 && du.month_of_year == 05 && du.used_space_year == 2016
select new {clnt, du, RT}
).ToList();
var CoreListSet = CoreList.Select(i=> new YourEntityClass
{
//Fetch the ParentGroupId & UsedSpace
}).ToList();
var CoreListComplete = (from cl in CoreListSet
join cust in EntityName.sg_groups
on cust.sg_group_id equals cl.parent_group_id).ToList();
Now get the sum of CoreListComplete & just implement the base Select Query in Linq!

Apologies for the delayed response. I've marked #Anil answer up as this is the one that helped me find the answer. You solution did work #Sathish but it can be accomplished in a single command. Here is my final solution. Many thanks for your help!
storeGridUsage = (
from cust in db.sg_groups
from client in db.sg_groups
join du in db.client_disk_usage on client.SG_GROUP_ID equals du.SG_GROUP_ID
where client.GROUP_ROLE_ID == 3
where client.PARENT_GROUP_ID == cust.SG_GROUP_ID && du.DAY_OF_MONTH == day && du.MONTH_OF_YEAR == month && du.USED_SPACE_YEAR == year
where cust.GROUP_ROLE_ID == 2
orderby cust.SG_GROUP_NAME
group new {cust, du} by cust.SG_GROUP_NAME
into g
select new StoreGridUsage
{
CustomerName = g.Key,
DiskUsageInBytes = g.Sum(o => o.du.USED_SPACE)
}).ToList();

How to retrieve all columns from table1 and matching columns from table2(Left outer join) using Linq

I have to retrieve all the columns from table1 and matching columns from table2. I have a stored procedure as :
alter Procedure [dbo].[usp_Property]
#UserId bigint =null
As
Begin
select P.PID, P.PropertyName, P.SBUArea, P.ListedOn,
P.Availability, P.Price, F.UserID, F.PID as FavProjId
from dbo.Property P left outer join dbo.Favorite F
on (F.PID=P.PID And F.UserID=#UserId)
I want to get Linq query for the same. So far I tried with something like
//User Id comes from session..
//var userId
var result=(from p in Properties
join f in Favorites
on p.PID equals f.PID into r
from r1 in r.DefaultIfEmpty()
where r1.UserID==userId
select new
{
p.PID,
p.PropertyName,
p.SBUArea, p.ListedOn,
r1.UserId
});
Can anyone please correct me. I want to use left outer join or any other alternate thing here.

If I beautify your SP's code, I get this:
DECLARE #UserId int
SET #UserId = 12435
SELECT
P.PID
,P.PropertyName
,P.SBUArea
,P.ListedOn
,P.Availability
,P.Price
,F.UserID
,F.PID AS FavProjId
FROM Property AS P
LEFT JOIN Favorite AS F
ON (F.PID=P.PID AND F.UserID = #UserId)
Now I wonder if you need that UserId in the WHERE clause of the SQL, or really in the join.
But anyway, here the LINQ-equivalent of exactly that SQL:
System.Int64 __UserId = 12435;
var query = (
from P in Repo.Property
from F in Repo.Favorite
.Where(fav=> fav.PID == P.PID && fav.UserID == __UserId)
.DefaultIfEmpty() // <== makes join left join
select new
{
PID = P.PID
,PropertyName = P.PropertyName
,SBUArea = P.SBUArea
,ListenOn = P.ListedOn
,Availabiity = P.Availability
,Price = P.Price
,UserId = F.UserID
,FavProjId = F.PID
}
);
var data = (query).ToList();

Use anonymous objects in your selection
var result = from t in table1
join x in table2
on t.id equals x.id
select new { id = t.id, col1 = t.col1, col2 = x.col2 }

If you will put the where clause after join you may get null reference exception because DefaultIfEmpty returns default value for non matching rows. You can filter the records before joining itself like this:-
var result=(from p in Properties
join f in Favorites.Where(x => x.UserID == userId)
on p.PID equals f.PID into r
from r1 in r.DefaultIfEmpty()
select new
{
p.PID,
p.PropertyName,
p.SBUArea,
p.ListedOn,
r1.UserId
});
Please note you need to access properties of Favorites using r1.
Update:
As far as I have understood you need all records from Property table and only matching rows from Favorite table. But you have a filter on your Favorite table so the ultimate data source will differ. Let me make my point clear by this example:-
Suppose you have following data in Property table:-
PID PropertyName Availability Price
1 aaa true 20
2 bbb false 10
3 ccc true 50
4 ddd false 80
5 eee true 55
6 fff false 70
and Favorite table like this:-
FID PID UserId
1 4 1001
2 2 1005
3 5 1007
And let's say you want all records for UserId 1005, then the result should contain all the property Id's from 1 till 6 even if UserId 1005 doesn't match for property Id's 4 & 2 right? So the query above is as per this understanding. Check this Fiddle with same example and output.

LINQ SQL Trying to merge 3 recordsets into one

I have three sets of data representing a counted value, grouped by country code.
select distinct m.CountryCode, count(m.MetricId) as 'Impressions'
from Metrics m
inner join impressions i on m.MetricId = i.MetricId
where ...
group by m.CountryCode
select distinct m.CountryCode, count(m.MetricId) as 'Conversions'
from Metrics m
inner join Conversions c on m.MetricId = c.MetricId
where ...
group by m.CountryCode
..and there's a third one that joins with a table called "Leads"
So each of these give me a nice set of distinct country codes and a corresponding number.
CountryCode Impressions
AU 25
DE 34
US 264
CountryCode Conversions
AU 11
US 140
something like that. so my goal is to get all three recordsets merged to one that looks like this:
CountryCode Impressions Conversions Leads
US 264 140 98
I'd like to learn how to do this with LINQ and without doing three queries. There's gotta be a more straightforward approach but I've been working on it too long and my eyes aren't seeing it. Would appreciate a nudge in the proper direction, thanks

var qry1 = (from m in Db.Metrics
join i in Db.Impressions on m.MetricId equals i.MetricId
//where
group m by m.CountryCode into grp
select new
{
CountryCode = grp.Key,
Impressions = grp.Count()
});
var qry2 = (from m in Db.Metrics
join c in Db.Conversions on m.MetricId equals c.MetricId
//where
group m by m.CountryCode into grp
select new
{
CountryCode = grp.Key,
Conversions = grp.Count()
});
var result = (from x in qry1
join y in qry2 on x.CountryCode equals y.CountryCode
select new
{
CountryCode = x.CountryCode,
Impressions = x.Impressions,
Conversions = y.Conversions
});
var lst = result.ToList();
The first 2 queries are lazy, they will not yet execute. The result-variable just joins them together and the last part executes the final query and materializes the objects.
Splitting these in their separate queries can be helpfull in keeping it simpler.

Get the "latest" datetime from a large linq query that currently returns every record that has a datetime

I have a fairly long linq query and everything works as it should.. but in a final join i am doing an innerjoin on a table that has a log, the log returns more than 50 records, i just want the latest record..
Here is an example
var tst = from w in context.storage
join p in context.products on w.id equals p.wid
join l in context.logger on p.id equals l.pid
select new
{
storageid = w.id,
productid = p.id
productname = p.name
bought = l.when
};
So a quick explanation of what happens, each product is stored in a storage center and there is a log when that product was bought, if it was bought 100 times then there is 100 records in the logger.
So currently it returns 50 records for productid = 5 ... why .. because it was bought 50 times but i only want 1 record, hence i only want the latest date time for from the logger.
Can anyone help? I am a little stuck.

Use result.Distinct(x => x.Prop) to get unique entries only
Use result.Max(x => x.Prop) to get latest date, and Min() to get earliest.

This is a case where you want to restrict to collection of records on which to join, which you can do by coding the join manually (sort of):
from w in context.storage
join p in context.products on w.id equals p.wid
// "manual" join:
from l in context.logger.Where(l => l.pid == p.id).OrderByDescencing(l => l.when).Take(1)
select new
{
storageid = w.id,
productid = p.id
productname = p.name
bought = l.when
};
In fluent linq syntax this is a SelectMany with a result selector.