We Keep Coding

LINQ query to retrieve pivoted data taking too long

I am working on a LINQ query which includes some pivot data as below
var q = data.GroupBy(x => new
{
x.Med.Name,
x.Med.GenericName,
}).ToList().Select(g =>
new SummaryDto
{
Name= g.Key.Name,
GenericName = g.Key.GenericName,
Data2012 = g.Where(z => z.ProcessDate.Year == 2012).Count(),
Data2013 = g.Where(z => z.ProcessDate.Year == 2013).Count(),
Data2014 = g.Where(z => z.ProcessDate.Year == 2014).Count(),
Data2015 = g.Where(z => z.ProcessDate.Year == 2015).Count(),
Data2016 = g.Where(z => z.ProcessDate.Year == 2016).Count(),
Data2017 = g.Where(z => z.ProcessDate.Year == 2017).Count(),
TotalCount = g.Count(),
}).AsQueryable();
return q;
The above LINQ takes too long as it queries grp q.Count()*6 times. If there are 10000 records, then it queries 60000 times
Is there a better way to make this faster?

Add year to the group key, then group again, and harvest per-group counts:
return data.GroupBy(x => new {
x.Med.Name
, x.Med.GenericName
, x.ProcessDate.Year
}).Select(g => new {
g.Key.Name
, g.Key.GenericName
, g.Key.Year
, Count = g.Count()
}).GroupBy(g => new {
g.Name
, g.GenericName
}).Select(g => new SummaryDto {
Name = g.Key.Name
, GenericName = g.Key.GenericName
, Data2012 = g.SingleOrDefault(x => x.Year == 2012)?.Count ?? 0
, Data2013 = g.SingleOrDefault(x => x.Year == 2013)?.Count ?? 0
, Data2014 = g.SingleOrDefault(x => x.Year == 2014)?.Count ?? 0
, Data2015 = g.SingleOrDefault(x => x.Year == 2015)?.Count ?? 0
, Data2016 = g.SingleOrDefault(x => x.Year == 2016)?.Count ?? 0
, Data2017 = g.SingleOrDefault(x => x.Year == 2017)?.Count ?? 0
, TotalCount = g.Sum(x => x.Count)
}).AsQueryable();
Note: This approach is problematic, because year is hard-coded in the SummaryDto class. You would be better off passing your DTO constructor an IDictionary<int,int> with counts for each year. If you make this change, the final Select(...) would look like this:
.Select(g => new SummaryDto {
Name = g.Key.Name
, GenericName = g.Key.GenericName
, TotalCount = g.Sum(x => x.Count)
, DataByYear = g.ToDictionary(i => i.Year, i => i.Count)
}).AsQueryable();

I suggest grouping inside the group by year and then converting to a dictionary to access the counts. Whether it is faster to group with year first and then count in-memory depends on the distribution of the initial grouping, but with the database it may depend on how efficiently it can group by year, so I would test to determine which seems fastest.
In any case grouping by year after the initial grouping is about 33% faster than your query in-memory, but again it is vastly dependent on the distribution. As the number of initial groups increase, the grouping by Year queries slow down to match the original query. Note that the original query without any year counts is about 1/3 the time.
Here is grouping after the database grouping:
var q = data.GroupBy(x => new {
x.Med.Name,
x.Med.GenericName,
}).ToList().Select(g => {
var gg = g.GroupBy(d => d.ProcessDate.Year).ToDictionary(d => d.Key, d => d.Count());
return new SummaryDto {
Name = g.Key.Name,
GenericName = g.Key.GenericName,
Data2012 = gg.GetValueOrDefault(2012),
Data2013 = gg.GetValueOrDefault(2013),
Data2014 = gg.GetValueOrDefault(2014),
Data2015 = gg.GetValueOrDefault(2015),
Data2016 = gg.GetValueOrDefault(2016),
Data2017 = gg.GetValueOrDefault(2017),
TotalCount = g.Count(),
};
}).AsQueryable();

LINQ left self join

I have LINQ query as below:
lst_direct_managers = context.sf_guard_user_profile
.Join(context.sf_guard_user_profile, up => up.user_id, dm => dm.direct_manager_id,
(up, dm) => new { up, dm })
.Where(m => m.up.is_gvo == 1)
.Select(m => new DirectManagerModel
{
user_id = m.up.direct_manager_id == null ? 0 : m.up.direct_manager_id,
dm_full_name = (m.up.first_name + " " + m.up.last_name == null ? "No Direct Manager" : m.up.first_name + " " + m.up.last_name)
})
.Distinct()
.OrderBy(m => m.dm_full_name).ToList();
Problem is that it does not return default value in case of nulls "No Direct Manager". Can you please help me?

Append index number to duplicated string value in a list - by using Lambda

I have a IList<string>() which holds some string values, and there could be duplicated items in the list. What I want is to append a index number to end of the string to eliminate the duplication.
For example, I have these values in my list: StringA, StringB, StringC, StringA, StringA, StringB. And I want the result looks like: StringA1, StringB1, StringC, StringA2, StringA3, StringB2. I need to retain the original order in list.
Is there a way I can just use one Lambda expression?

You are looking for something like this:
yourList.GroupBy(x => x)
.SelectMany(g => g.Select((x,idx) => g.Count() == 1 ? x : x + idx))
.ToList();
Edit: If the element order matters, here is another solution:
var counts = yourList.GroupBy(x => x).ToDictionary(x => x.Key, x => x.Count());
var values = counts.ToDictionary(x => x.Key, x => 0);
var list = yourList.Select(x => counts[x] > 1 ? x + ++values[x] : x).ToList();

You can do:
List<string> list = new List<string> { "StringA", "StringB", "StringC", "StringA", "StringA", "StringB" };
var newList =
list.Select((r, i) => new { Value = r, Index = i })
.GroupBy(r => r.Value)
.Select(grp => grp.Count() > 1 ?
grp.Select((subItem, i) => new
{
Value = subItem.Value + (i + 1),
OriginalIndex = subItem.Index
})
: grp.Select(subItem => new
{
Value = subItem.Value,
OriginalIndex = subItem.Index
}))
.SelectMany(r => r)
.OrderBy(r => r.OriginalIndex)
.Select(r => r.Value)
.ToList();
and you will get:
StringA1,StringB1,StringC,StringA2,StringA3,StringB2
If you don't want to preserve order then you can do:
var newList = list.GroupBy(r => r)
.Select(grp => grp.Count() > 1 ?
grp.Select((subItem, i) => subItem + (i + 1))
: grp.Select(subItem => subItem))
.SelectMany(r => r)
.ToList();

This uses some lambda expressions and linq to do it, maintaining the order but I'd suggested a function with a foreach loop and yield return would be better.
var result = list.Aggregate(
new List<KeyValuePair<string, int>>(),
(cache, s) =>
{
var last = cache.Reverse().FirstOrDefault(p => p.Key == s);
if (last == null)
{
cache.Add(new KeyValuePair<string, int>(s, 0));
}
else
{
if (last.Value = 0)
{
last.Value = 1;
}
cache.Add(new KeyValuePair<string, int>(s, last.Value + 1));
}
return cache;
},
cache => cache.Select(p => p.Value == 0 ?
p.Key :
p.Key + p.Value.ToString()));

C# - LINQ to Entity query attaching the value of the properties to one variable

I have this query:
var sole = SoleService.All().Where(c => c.Status != 250)
.Select(x => new {
ID = x.ID, Code = new {CodeName = x.Code + x.Country},
Name = x.Name
})
.Distinct()
.OrderBy(s => s.Code);
Like this it gives me an error. What I want is to combine Code and Country like concat strings so I can use the new variable for my datasource. Something like - 001France
P.S
What I am using and is working right now is this :
var sole = SoleService.All().Where(c => c.Status != 250)
.Select(x => new {
ID = x.ID, Code = x.Code, Name = x.Name, Country = x.Country })
.Distinct()
.OrderBy(s => s.Code);
So what I need is to modify this query so I can use Code +Country as one variable. Above is just my try that I thought would work.

Sound like:
var sole = SoleService.All().Where(c => c.Status != 250)
.AsEnumerable()
.Select(x => new {
ID = x.ID,
Code = x.Code + x.Country,
Name = x.Name
})
.Distinct()
.OrderBy(s => s.Code);
You don't need inner anonymous type at all. If you are working on EF, sine string + is not supported, call AsEnumerable before doing select.

You can't sort by s.Code because it's an instance of an anonymous type. I'd go with
var sole = SoleService.All().Where(c => c.Status != 250)
.Select(x => new { ID = x.ID, Code = new {CodeName = x.Code + x.Country}, Name = x.Name })
.Distinct()
.OrderBy(s => s.Code.CodeName);

Try to add .toString() because the problem should be the + operator believe you try to add numeric properties...
Like that :
var sole = SoleService.All().Where(c => c.Status != 250)
.Select(x => new { ID = x.ID, Code = new {CodeName = x.Code.ToString() + x.Country.ToString()}, Name = x.Name })
.Distinct()
.OrderBy(s => s.Code);

Lambda expression Group by in C#

I would like to group my LINQ query by ItemNumber and return the whole table with the total for Quantity.
Example:
ItemNumber - ItemName - Quantity
100 Item1 1
150 Item2 2
100 Item1 2
200 Item3 1
150 Item2 2
Should be:
ItemNumber - ItemName - Quantity
100 Item1 3
150 Item2 4
200 Item3 1
This is the query I am trying to group:
public IQueryable<WebsiteOrderStatus> GetOrderStatusByAccountNumberWithoutDeleted
(string accountNumber)
{
return db.WebsiteOrderStatus
.Where(x => x.AccountNumber == accountNumber && x.LastUpdatedStatus != 1);
}
And my best result so far(this can't compile though):
public IQueryable<IGrouping<Int32?, WebsiteOrderStatus>> lol(string accountNumber)
{
db.WebsiteOrderStatus
.Where(x => x.AccountNumber == accountNumber && x.LastUpdatedStatus != 1)
.GroupBy(g => g.ItemNumber)
.Select(g => new
{
g.Key.ItemNumber,
Column1 = (Int32?)g.Sum(p => p.Quantity)
});
}
EDIT:
Thanks for the replies everyone, I must face it. Theese anonymous types are pretty hard to work with in my opinion, so I found another solution.
I made another method, which sums the quantity of the users items and grouped the first one.
public IQueryable<WebsiteOrderStatus> GetOrderStatusByAccountNumberWithoutDeleted(string accountNumber)
{
return db.WebsiteOrderStatus.Where(x => x.AccountNumber == accountNumber && x.LastUpdatedStatus != 1).GroupBy(x => x.ItemNumber).Select(grp => grp.First());
}
public int GetQuantityOfUsersItem(string accountNumber, string itemNumber)
{
return db.WebsiteOrderStatus.Where(x => x.ItemNumber == itemNumber && x.AccountNumber == accountNumber).Sum(x => x.Quantity);
}
At the page where I have my gridview I did:
var query = websiteOrderStatusRep.GetOrderStatusByAccountNumberWithoutDeleted(AppSession.CurrentLoginTicket.AccountNumber).Select(x => new { x.ItemName, x.ItemNumber, x.FormatName, x.Price, x.Status, x.Levering, Quantity = websiteOrderStatusRep.GetQuantityOfUsersItem(x.AccountNumber, x.ItemNumber)});

public IQueryable<IGrouping<Int32?, WebsiteOrderStatus>> lol(string accountNumber)
{
db.WebsiteOrderStatus
.Where(x => x.AccountNumber == accountNumber && x.LastUpdatedStatus != 1)
.GroupBy(g => g.ItemNumber)
.Select(g => new
{
ItemNumber = g.Key,
ItemName = g.First().ItemName,
Count = g.Sum(item => item.Quantity)
});
}

public IQueryable<OrderStatus > lol(string accountNumber)
{
return db.WebsiteOrderStatus
.Where(x => x.AccountNumber == accountNumber && x.LastUpdatedStatus != 1)
.GroupBy(g => g.ItemNumber)
.Select(g =>
new OrderStatus //This is your custom class, for binding only
{
ItemNumber = g.Key,
ItemName = g.First().ItemName,
Quantity = g.Sum(g => g.Quantity)
}
);
}

I think the Select should be:
.Select(g => new
{
ItemNumber = g.Key,
Column1 = (Int32?)g.Sum(p => p.Quantity)
});
Note the change in the first line of the anonymous type. The key of the grouping is already the item number.

The only problems I see with your query are
Missing return statement as per comments
The select statement should be:
-
.Select(g => new {
ItemNumber = g.Key,
Total = g.Sum(p => p.Quantity)
});
EDIT: If you want to get, lets say ItemNumber and ItemName , in the resulting object, you must also group on those fields
db.WebsiteOrderStatus
.Where(x => x.AccountNumber == accountNumber && x.LastUpdatedStatus != 1)
.GroupBy(g => new { g.ItemNumber, g.ItemName })
.Select(g => new
{
ItemNumber = g.Key.ItemNumber,
ItemName = g.Key.ItemName,
Count = g.Sum(item => item.Quantity)
});

You cannot use anonymous type for return value type. So you will never compile the code.
Also your linq expression has IQueryable< [anonymous type] > result type.
I believe that you can do something like this:
public IQueryable<OrderStatus> lol(string accountNumber)
{
db.WebsiteOrderStatus
.Where(order => order.AccountNumber == accountNumber && order.LastUpdatedStatus != 1)
.GroupBy(order => order.ItemNumber)
.Select(grouping => new OrderStatus //This is your custom class, for binding only
{
ItemNumber = grouping.Key,
ItemName = grouping.First().ItemName,
Quantity = grouping.Sum(order => order.Quantity)
});
}
I`ve fixed my answer too :)