We Keep Coding

Linq not grouping

Given the data below, I am trying to write a LINQ statement that will group by ParentProductId, and select the maximum EndDate if there is more than one item in the group.
Since the data listed below has two items with the same ParentProductId, I would expect three records to be returned (and for the "2020" date to be used for the ParentProductId = 1 group, not "2019"). However, the LINQ statement that I have is still returning all four records. What am I doing wrong?
Data:
Subscription.Add(new Subscription() { CustomerId = 555, ParentProductId = 37 , EndDate= null});
Subscription.Add(new Subscription() { CustomerId = 555, ParentProductId = 38 , EndDate = null });
Subscription.Add(new Subscription() { CustomerId = 555, ParentProductId = 1 , EndDate = new DateTime(2019, 11, 28) });
Subscription.Add(new Subscription() { CustomerId = 555, ParentProductId = 1, EndDate = new DateTime(2020, 1, 28) });
LINQ Statement:
var IndividualSubscription = (from s in db.Subscriptions
join ptp in db.ProductToProducts on s.ProductToProductId equals ptp.ProductToProductId
join p in db.Products on ptp.ParentProductId equals p.ProductId
where SubscriptionIds.Contains(s.OriginalSubscriptionId)
&& s.CustomerId == CustomerId
group new {ptp.ParentProductId, s.EndDate }
by new
{
s.CustomerId,
ptp.ParentProductId,
p.Name,
s.EndDate,
} into grp
select new NCCN.Model.IndividualGroupSubscription
{
CustomerId = grp.Key.CustomerId,
ParentProductId = grp.Key.ParentProductId,
ParentProductName = grp.Key.Name,
EndDate = grp.Max(p => p.EndDate),
}).ToList();

You group for {ptp.ParentProductId, s.EndDate }.
Group only for ParentProductId, if you need 3 rows result.
var subscription = new List<Subscription>(){new Subscription() { CustomerId = 555, ParentProductId = 37 , EndDate= null}
,new Subscription() { CustomerId = 555, ParentProductId = 38 , EndDate = null }
,new Subscription() { CustomerId = 555, ParentProductId = 1 , EndDate = new DateTime(2019, 11, 28) }
,new Subscription() { CustomerId = 555, ParentProductId = 1, EndDate = new DateTime(2020, 1, 28) }
};
var query = subscription
.GroupBy(x=> x.ParentProductId)
.Select(x=> new {x.Key, EndDate = x.Max(s=>s.EndDate)});

Note sure why your linq is that overkill but following your written requirement and not your code what you need is a million times shorter than what you have wrote
// get grouped by product id
var result = Subscription.GroupBy(s => s.ParentProductId)
// select to output something for each group. What you want is
// order descending all elements and pick the first which is the highest
.Select(grp => grp.OrderByDescending(o => o.EndDate).FirstOrDefault())
// returned as a list
.ToList();

How can I do a "one hit" Linq query with nullable values?

I have the following class:
public class MyTrelloCard
{
...
public DateTime? completed { get; set; }
public Decimal? estHours { get; set; }
public bool complete { get; set; }
}
...and I want to query the number of estHours on each date in the completed date. These have to be nullable values as not every card is completed, and not every card has estimated hours added to it.
At the moment I have to run two queries. The first one:
List<burnDownData> bData = (from c in cards
where c.complete
group c by new
{
date = Convert.ToDateTime(c.completed).Date
} into g
select new burnDownData
{
date = g.Key.date,
completedHours = g.Sum(x=>x.estHours) ?? 0
}).ToList();
returns all the cards that have been completed.
The second one is used to iterate through all dates between a start and end period and cumulatively add up the completed hours:
for (DateTime d = start; d.Date <= end; d = d.AddDays(1))
{
if ((d.DayOfWeek >= DayOfWeek.Monday) && (d.DayOfWeek <= DayOfWeek.Friday))
{
List<C> values = new List<C>();
Decimal xx = (from b in bData
where b.date.Date == d
select b.completedHours).SingleOrDefault();
total = total + xx;
values.Add(new C { v = d.ToString("d MMM") });
values.Add(new C { v = total.ToString() });
myRows.Add(new Row { c = values });
}
}
This seems inefficient though. Is it possible to do this directly? i.e. to replace this part of the loop:
Decimal xx = (from b in bData
where b.date.Date == d
select b.completedHours).SingleOrDefault();
With something that queries the cards data directly?

Interpreting your question:
I want to query the number of estHours on each date in the completed
date. These have to be nullable values as not every card is completed,
and not every card has estimated hours added to it
...as "I'm trying to get a sum of estimated hours for each completed date instance, and want to allow for null completed and estHours values".
You could include the check for null in your Where clause:
var x = cards.Where(c => c.completed != null )
.GroupBy(c => c.completed, (key, group) => new
{
dateComp = key.Value,
totEstHrs = group.Sum(i => i.estHours)
});
x.ToList().ForEach(
item => Debug.Print("{0:MM/dd/yyyy} {1}", item.dateComp, item.totEstHrs));
For example:
List<MyTrelloCard> cards = new List<MyTrelloCard>() {
new MyTrelloCard() { completed = new DateTime(2014, 4, 28), estHours = 5, complete = true },
new MyTrelloCard() { completed = null, estHours = 5, complete = false },
new MyTrelloCard() { completed = null, estHours = null, complete = true },
new MyTrelloCard() { completed = new DateTime(2014, 4, 28), estHours = 7, complete = false },
new MyTrelloCard() { completed = new DateTime(2014, 4, 29), estHours = null, complete = false },
new MyTrelloCard() { completed = new DateTime(2014, 4, 29), estHours = 3, complete = false },
};
Produces:
04/28/2014 12
04/29/2014 3
Edit:
To group on just the date portion (omitting the timestamp) you can group on the ToShortDateString() value:
var x = cards.Where(c => c.completed != null)
.GroupBy(c =>
((DateTime)c.completed).ToShortDateString(),
(key, group) => new
{
dateComp = key,
totEstHrs = group.Sum(i => i.estHours)
});

Ah - so you want an entry for every weekday in your range, even if no data exists for that day. In which case, build an array of eligible days, and groupjoin your cards.
Here's how:
var start = DateTime.Today;
var end = start.AddDays(14);
var cards = new[]{new {complete = true, completed = (DateTime?)DateTime.Now, estHours = new decimal?(3)} };
var days = Enumerable.Range(0, end.Subtract(start).Days)
.Select(x => start.AddDays(x))
.Where(x => !new []{DayOfWeek.Saturday,DayOfWeek.Sunday}.Contains(x.DayOfWeek));
var results = from d in days
join c in cards on d equals c.completed.GetValueOrDefault().Date
into cGrp
select new {d, completedHours = cGrp.Sum(x => x.estHours)};
of course you don't need the var cards = line, as you already have your own source for that!
Edit - All timestamps for a day are grouped into the 1 day.

get only latest items for each id type

I have a collection of objects and i need to retreive the ones that were last created for the same type(similar ids). I tried the idea below from an other post but i do not get the expected result.
Any other suggestions?
Thanks in advance
void Main()
{
List<data_item> itemlist1= new List<data_item>();
itemlist1.Add(new UserQuery.data_item
{
id=1,
Creationdate = new DateTime(2001, 6, 1),
Size=28373
})
;
itemlist1.Add(new UserQuery.data_item
{
id=2,
Creationdate = new DateTime(1992, 6, 1),
Size=2000
})
;
itemlist1.Add(new UserQuery.data_item
{
id=2,
Creationdate = new DateTime(2002, 6, 1),
Size=300
})
;
itemlist1.Add(new UserQuery.data_item
{
id=1,
Creationdate = new DateTime(2005, 6, 1),
Size =400
})
;
itemlist1.Add(new UserQuery.data_item
{
id=3,
Creationdate = new DateTime(1999, 6, 1),
Size =10000
})
;
itemlist1.Add(new UserQuery.data_item
{
id=3,
Creationdate = new DateTime(2003, 6, 1),
Size =5000
})
;
itemlist1.Add(new UserQuery.data_item
{
id=3,
Creationdate = new DateTime(1999, 4, 1),
Size=3000
})
;
itemlist1.Add(new UserQuery.data_item
{
id=5,
Creationdate = new DateTime(1999, 6, 5),
Size=230
})
;
itemlist1.Add(new UserQuery.data_item
{
id=4,
Creationdate = new DateTime(1999, 6, 1),
Size =8700
})
;
itemlist1.Add(new UserQuery.data_item
{
id=5,
Creationdate = new DateTime(1991, 6, 1),
Size =6000
})
;
itemlist1.Dump();
var result = from t in itemlist1 group t by new UserQuery.data_item
{ id=t.id,
Creationdate=t.Creationdate
} into gt
select gt.OrderByDescending(d=>d.Creationdate).First();
result.Distinct().Dump();
}
// Define other methods and classes here
public class data_item
{
public int id {get;set;}
public DateTime Creationdate {get;set;}
public int? Size {get;set;}
}

Is this what you need?
var group = itemlist1.GroupBy(x=>x.id); // grouped by id
var newList =
group.Select(g => g.OrderByDescending(y => y.Creationdate).First());
it will get most recent items for all ids, by grouping them by ID and then taking only most recent ones from that list and selecting to new list
OrderByDescending is ordering by date descending so First is taking most recent date. Select is selecting expression result to new list, so there should be a list of most recent items for all id's groups

Looking for a more elegant LINQ solution

I am trying to figure out an efficient way to retrieve the data I am after. I need to get a list of all of the most recent children by ParentId coupled with all parent entries that do NOT have children. I have created a visual guide to illustrate what the response should be.
The query needs to remain as IQueryable until ALL sorting and paging is completed.
Last and LastOrDefault are not supported by LINQ to Entities (as stated by the error messages I have received while using them).
Using First or FirstOrDefault will return the error "This method or operation is not implemented"
Original Data:
-------------------------------
- Id - ParentId - CreatedDate -
-------------------------------
- 1 - - 07/01/2013 -
- 2 - - 07/01/2013 -
- 3 - - 07/01/2013 -
- 4 - 1 - 07/02/2013 -
- 5 - 2 - 07/03/2013 -
- 6 - 2 - 07/04/2013 -
- 7 - 1 - 07/05/2013 -
-------------------------------
Data returned by query
-------------------------------
- Id - ParentId - CreatedDate -
-------------------------------
- 3 - - 07/01/2013 -
- 6 - 2 - 07/04/2013 -
- 7 - 1 - 07/05/2013 -
-------------------------------
Currently, my LINQ query looks like this:
// Retrieves parent records with NO children.
var q1 = myTable
.Where(x => x.ParentId == null)
.Except(myTable
.Where(x => myTable
.Any(c => (c.ParentId == x.Id))));
// Retrieves most recent child records for each parentId
var q2 =
(from a in myTable
join b in
(myTable.Where(a => a.ParentId != null)
.GroupBy(a => a.ParentId)
.Select(b => new { ParentId = b.Key, CreatedDate = b.Max(t => t.CreatedDate) }))
on a.ParentId equals b.ParentId
where a.CreatedDate == b.CreatedDate
select a);
q1 = q1.Union(q2);
The back-end is using Npgsql2 with PostgreSQL. I am looking for a more elegant solution for this query. I am very new to LINQ and would like to optimize this.
Sorting code (sloppy, but jTable returns these strings):
if (string.IsNullOrEmpty(sorting) || sorting.Equals("Name ASC")) {
q1 = q1.OrderBy(p => p.Customer.Name);
} else if (sorting.Equals("Name DESC")) {
q1 = q1.OrderByDescending(p => p.Customer.Name);
} else if (sorting.Equals("Date ASC")) {
q1 = q1.OrderBy(p => p.CreatedDate);
} else if (sorting.Equals("Date DESC")) {
q1 = q1.OrderByDescending(p => p.CreatedDate);
}
Paging code:
var result = pageSize > 0
? q1.Skip(startIndex).Take(pageSize).ToList()
: q1.ToList();

Use grouping:
Mock data:
public class Entry {
public int Id { get; set; }
public int? ParentId { get; set; }
public DateTime Date { get; set; }
};
var list = new List<Entry> {
new Entry{ Id = 1, ParentId = null, Date = new DateTime(2013, 7, 1) },
new Entry{ Id = 2, ParentId = null, Date = new DateTime(2013, 7, 1) },
new Entry{ Id = 3, ParentId = null, Date = new DateTime(2013, 7, 1) },
new Entry{ Id = 4, ParentId = 1, Date = new DateTime(2013, 7, 2) },
new Entry{ Id = 5, ParentId = 2, Date = new DateTime(2013, 7, 3) },
new Entry{ Id = 6, ParentId = 2, Date = new DateTime(2013, 7, 4) },
new Entry{ Id = 7, ParentId = 1, Date = new DateTime(2013, 7, 5) }
};
Query:
var query = from l in list
group l by l.ParentId into g
select new {
Items = g.OrderBy(x => x.Date).Last()
};
var res = query.OrderBy(x => x.Items.Id).Select(x => x.Items).ToList();
LinqPad result:
Id ParentId Date
3 null 01.07.2013 0:00:00
6 2 04.07.2013 0:00:00
7 1 05.07.2013 0:00:00

I can propose a different query, still in two phases
var firstQuery = myTable.Select(p => new { p.ID, ParentID = p.ParentID ?? p.ID, p.CreatedDate })
.GroupBy( p => p.ParentID).Select( q => new
{
el = q.OrderByDescending( k => k.CreatedDate).Take(1)
}).SelectMany(t => t.el);
var result = dc.TabellaId_ParentId.Where(p => test.Select(q => q.ID).Contains(p.ID));

How to re-write this inner join subquery from SQL to Lambda

SELECT ulcch.ID, ulcch.UserLoginHistoryID, ulcch.StatusID,
ulcch.ClientModuleID, ulcch.DeviceState, ulcch.UpdatedAt, ulcch.CreatedAt
FROM UserLoginClientConnectionHistory AS ulcch INNER JOIN
(SELECT MAX(CreatedAt) AS maxCreatedAt
FROM UserLoginClientConnectionHistory AS ulcch1
GROUP BY UserLoginHistoryID) AS m ON m.maxCreatedAt = ulcch.CreatedAt
There can be many updates of 'device state' per day audited into this login table. This query returns the last unique one for each day.
I would like this re-written as a Lambda statement. This is how far I got, I don't know if i'm on the right track, and my Max() is throwing a type error, probably because the group by is making another list or something...
Hope you can work it out from my object examples.... :S
userLogin.UserLoginClientConnectionHistories.Where(x => x.CreatedAt ==
userLoginClientConnectionHistoryRepository.GetAll(
GenericStatus.Active).GroupBy(y => y.UserLoginHistoryID).Max(y => y.CreatedAt));

I think this does what you want:
var result = userLogin.UserLoginClientConnectionHistories
.GroupBy(y => new { Id = y.UserLoginHistoryID, Day = y.CreatedAt.Date })
.Select(x => new
{
Id = x.Key.Id,
Day = x.Key.Day,
MostRecent = x.Max(y => y.CreatedAt)
});
Here is a testbed for it:
public class Program
{
class LoginEntry
{
public int UserLoginHistoryID { get; set; }
public DateTime CreatedAt { get; set; }
}
class UserLogin
{
public List<LoginEntry> UserLoginClientConnectionHistories = new List<LoginEntry>();
}
public static void Main(string[] args)
{
UserLogin userLogin = new UserLogin();
userLogin.UserLoginClientConnectionHistories = new List<LoginEntry> {
new LoginEntry {UserLoginHistoryID = 1, CreatedAt = new DateTime(2009, 1, 1, 3, 0 ,0)},
new LoginEntry {UserLoginHistoryID = 1, CreatedAt = new DateTime(2009, 1, 1, 15, 0 ,0)},
new LoginEntry {UserLoginHistoryID = 1, CreatedAt = new DateTime(2009, 1, 3, 11, 0 ,0)},
new LoginEntry {UserLoginHistoryID = 1, CreatedAt = new DateTime(2009, 1, 1, 10, 0 ,0)},
new LoginEntry {UserLoginHistoryID = 2, CreatedAt = new DateTime(2009, 1, 3, 4, 0 ,0)},
new LoginEntry {UserLoginHistoryID = 2, CreatedAt = new DateTime(2009, 1, 3, 5, 0 ,0)},
};
var result = userLogin.UserLoginClientConnectionHistories
.GroupBy(y => new { Id = y.UserLoginHistoryID, Day = y.CreatedAt.Date })
.Select(x => new
{
Id = x.Key.Id,
Day = x.Key.Day,
MostRecent = x.Max(y => y.CreatedAt)
});
foreach (var item in result)
{
Console.WriteLine("User {0}, day {1}, most recent {2}",
item.Id,
item.Day,
item.MostRecent);
}
}
}
Output:
User 1, day 01-01-2009 00:00:00, most recent 01-01-2009 15:00:00
User 1, day 03-01-2009 00:00:00, most recent 03-01-2009 11:00:00
User 2, day 03-01-2009 00:00:00, most recent 03-01-2009 05:00:00

Here is the inner join portion as a lambda. I assumed CreatedAt was a dateTime.
UserLoginClientConnectionHistory
.GroupBy (ulcch1 =>
new
{
Name = ulcch1.Name
})
.Select (g =>
new
{
maxCreatedAt = (DateTime?)(g.Max (p => p.CreatedAt))
})

I think you want to group by CreatedAt rather than UserLoginHistoryID:
var q = userLogin.UserLoginClientConnectionHistories
.GroupBy(h => h.CreatedAt)
.OrderByDescending(g => g.Key) // Sort by CreatedAt
.First()
.Select(h => new { h.Id, h.UserLoginHistoryID, ... });
This will return the set of UserLoginClientConnectionHistory entries that share the most recent CreatedAt value.

Thanks for all your help guys, i've voted you all up, but you wouldn't believe it but a few hours later I searched for a program to convert SQL to LINQ, and to my surprise found one called "Linqer". Sounds crazy and didn't expect to get far, but it worked perfectly.. definitely worth checking out that app if anyone else gets stuck in the same boat...
Check the mammoth query it returned! After analysing it, don't think it's got extra bloat? Anyone have any optimisation tips or spot any unnecessary code?
moduleDeviceStates = from ulh in user.UserLoginHistories
join ulcch in userLogin.UserLoginClientConnectionHistories on new { ID = ulh.ID } equals new { ID = ulcch.UserLoginHistoryID }
join cm in clientModuleRepository.GetAll(GenericStatus.Active) on new { ClientModuleID = ulcch.ClientModuleID } equals new { ClientModuleID = cm.ID }
join mo in moduleRepository.GetAll(GenericStatus.Active) on new { ModuleID = cm.ModuleID } equals new { ModuleID = mo.ID }
join m in
(
(from ulcch1 in userLogin.UserLoginClientConnectionHistories
group ulcch1 by new
{
ulcch1.UserLoginHistoryID
} into g
select new
{
maxCreatedAt = g.Max(p => p.CreatedAt)
})) on new { maxCreatedAt = ulcch.CreatedAt } equals new { maxCreatedAt = m.maxCreatedAt }
select new ModuleDeviceState()
{
ModuleID = mo.ID,
Name = mo.Name,
DeviceState = (State.DeviceState)ulcch.DeviceState,
CreatedAt = ulcch.CreatedAt
};
Cheers for your help dahlbyk, but I did want to group on UserLoginHistoryID, I had my query confirmed in SQL before delving into a lambda equivalent :) thanks.
#Mark Thanks for taking the time to reply, yes I do what the [last] entries per user (userloginhistory.. which in turn contains a userID) for each day, and exporting my sql into the linq query did produce what I wanted (which can be seen in the query result below; this is what I want. The reason you see double entries for each day is because there are also attached ClientModule's.. so I really want all client module, per login entry per day - so hard to get a programming requirement across over a discussion forum argh!) Perhaps yours does exactly the same thing (it appears to if I am reading your output correctly) just a lot more streamlined.
See I didn't know too much about the anon casting you've done there with GroupBy and Select, but now I see it, it makes sense. I might give yours a go. Hopefully I can give it a tweak to include distinct ClientModule's per day too. So anyway.. here is the query result from my SQL, and effectively what I got through my own lambda:
ID UserLoginHistoryID StatusID ClientModuleID DeviceState UpdatedAt CreatedAt
277 62 1 1 4 NULL 2009-10-31 13:28:59.003
278 62 1 16 4 NULL 2009-10-31 13:28:59.003
331 65 1 1 4 NULL 2009-10-31 17:13:28.333
332 65 1 16 4 NULL 2009-10-31 17:13:28.333
Update Mark: Hi again, well after a couple of tweaks on your query, I could produce the same object graph in .NET between both lambda statements. This is the one I will use now, derived from yours as it's more streamlined and easier to understand than the auto-gen'd one and I will award you the points :)
I added a few more entries to the Group By as I need that for my new ModuleDeviceState class.
moduleDeviceStates = userLogin.UserLoginClientConnectionHistories
.GroupBy(y => new { Id = y.UserLoginHistoryID,
CreatedAt = y.CreatedAt.Date,
ModuleID = y.ClientModule.ModuleID,
ModuleName = y.ClientModule.Module.Name,
DeviceState = y.DeviceState })
.Select(x => new ModuleDeviceState()
{
ModuleID = x.Key.ModuleID,
Name = x.Key.ModuleName,
DeviceState = (State.DeviceState)x.Key.DeviceState,
CreatedAt = x.Max(y => y.CreatedAt)
});