We Keep Coding

Select TOP 1 for each FK in list using Entity Framework

I have a large table where I'm trying to select the top 1 row for each FK in a list.
My table is laid out as:
ChangeId | AssetId | Timestamp
1 1 123
2 2 999
3 1 3478
4 3 344
5 2 1092
Where ChangeId is my PK, AssetId is my FK and Timestamp is the value I'm trying to select.
If I try the following:
var results =
from Asset in _context.Asset
join change in _context.Change on Asset.AssetId equals change.AssetId into potentialChange
from actualChange in potentialChange.OrderByDescending(y => y.ChangeId).Take(1)
select
{
AssetId,
Timestamp
}
Where my expected result would be:
[
{
AssetId: 1,
Timestamp: 3478
},
{
AssetId: 2,
Timestamp: 1092
},
{
AssetId: 3,
Timestamp: 344
}
]
This query flags up the The LINQ expression could not be translated and will be evaluated locally. which is not suitable for a production rollout.
Running a foreach loop and selecting each item out 1 by 1 works, not it's not a performant solution.
Is there a suitable way to achieve the above?

Try to group it by AssetId and take max from each group
var results =
from Asset in _context.Asset
join change in _context.Change on Asset.AssetId equals change.AssetId into potentialChange
group potentialChange by potentialCharge.AssetId into g
select
{
g.Key,
g.Max().Timestamp
}

Use Group By as follows:
List<MyTable> data = new List<MyTable>()
{
new MyTable(){ChangeId = 1, AssetId = 1, Timestamp = 123},
new MyTable(){ChangeId = 2, AssetId = 2, Timestamp = 999},
new MyTable(){ChangeId = 3, AssetId = 1, Timestamp = 123},
new MyTable(){ChangeId = 5, AssetId = 3, Timestamp = 123},
new MyTable(){ChangeId = 5, AssetId = 2, Timestamp = 123},
};
var expectedData = data.OrderByDescending(d => d.Timestamp).GroupBy(d => d.AssetId).Select(g => new
{
AssetId = g.Key,
TimeStamp = g.First().Timestamp
}).ToList();
This will give your expected result.

Try using .First() instead of .Take(1)
LINQ How to take one record and skip rest c#

LINQ LEFT JOIN not working on NULL values

I have two tables Student and Marks.
Student table have the following fields:
StudentID,Name,MarkID(Nullable).
Marks table have the following fields:
MarkID,Mark
Student table
StudentID Name MarkID
1 Mark 1
2 Mike NULL
3 John NULL
4 Paul 2
Mark table
MarkID Mark
1 80
2 100
If I use the left join then i getting only mark and paul records.
I want all the records in the left table(Student)
My Query is:
var query = (from s in Students
join m in Marks on s.MarkID equals m.MarkID
into mar from subMark in mar.DefaultIfEmpty()
where(m.Mark > 80)
Select s.Name)
.ToList()
Note: It is an Example only.
While joining two tables using left join and applying where condition on the second table ,If joined column value is null in first table,it won't bring the record from first table.

NULL comparisons are always false. That's the way SQL's three-valued logic works. If you want to match rows where the values are both null you should use a statement that checks both of them for null.
In a SQL statement you would write:
ON S.MARKID=M.MARKID OR (S.MARKID IS NULL AND M.MARKID IS NULL)
In C# you can use the comparison operator and your LINQ provider will convert this to IS NULL, eg:
on s.MarkID == m.MarkID || (s.MarkID == null && m.MarkID==null)

The problem is we use the where clause in Left join.So it will discard the null value records.
var sampleQuery= (from f in food
join j in juice on f.ID equals j.ID into juiceDetails from juice in juiceDetails.DefaultIfEmpty()
where(!j.deleted)
join fr in fruit on f.ID equals fr.ID into fruitDetails from fruit in fruitDetails.DefaultIfEmpty()
where(!fr.deleted)
select new
{
// codes
});
Instead of this we have to check the where clause in table itself.Like this
var sampleQuery= (from f in food
join j in juice.Table().where(x=>!x.deleted) on f.ID equals j.ID into juiceDetails from juice in juiceDetails.DefaultIfEmpty()
join fr in fruit.Table().where(x=>!x.deleted) on f.ID equals fr.ID into fruitDetails from fruit in fruitDetails.DefaultIfEmpty()
select new
{
// codes
});
It will work fine.
Thank you.

/EDIT: My first answer was using a FULL OUTER JOIN. this was way over the top and probably wrong or not compleltly correct.
The new answer uses a LEFT OUTER JOIN. I have created some sample data using LinqPad to get a working example. Ignore the .Dump() method if you are not using LinqPad.
var Students = new List<Student>() {
new Student() {StudentId = 1, Name ="John", MarkId = 1},
new Student() {StudentId = 1, Name ="Paul", MarkId = 1},
new Student() {StudentId = 1, Name ="Steve", MarkId = 1},
new Student() {StudentId = 1, Name ="John", MarkId = 2},
new Student() {StudentId = 1, Name ="Paul", MarkId = 3},
new Student() {StudentId = 1, Name ="Steve", MarkId = 1},
new Student() {StudentId = 1, Name ="Paul", MarkId = 3},
new Student() {StudentId = 1, Name ="John" },
new Student() {StudentId = 1, Name ="Steve" },
new Student() {StudentId = 1, Name ="John", MarkId = 1}
};
var Marks = new List<Mark>() {
new Mark() {MarkId = 1, Value = 60},
new Mark() {MarkId = 2, Value = 80},
new Mark() {MarkId = 3, Value = 100}
};
var StudentMarks = Students
.GroupJoin(
Marks,
st => st.MarkId,
mk => mk.MarkId,
(x,y) => new {
StudentId = x.StudentId,
Name = x.Name,
Mark = y.Select (z => z.Value).SingleOrDefault()
}
)
.Dump();
}
public class Student
{
public int StudentId { get; set; }
public string Name { get; set; }
public int MarkId { get; set; }
}
public class Mark
{
public int MarkId { get; set; }
public int Value { get; set; }
}
Output:
As you cann see in my Students list, there a 2 students without a MarkId. Those 2 get the default value assigned due to .SingleOrDefault(). I think this will solve your problem and gives you a good basis for further fiddeling.
references:
How do you perform a left outer join using linq extension methods

In your query you have written From in your Join statement while joining it.
Instead you should use in::
from s in Students
join m in Marks on s.MarkID equals m.ID into mar
from subMark in mar.DefaultIfEmpty()
Select s.Name).ToList()

I had the same problem. This solution only works if you have at least one row in subMark. The rows' ID doesn't matter.
var query = (from s in Students
join m in Marks on s.MarkID equals m.MarkID into fullM
into mar from subMark in mar.DefaultIfEmpty()
where(m.Mark > 80)
Select s.Name)
.ToList()
the keyword into does the magic. Adding it shows all rows, also those, which have NULL-Values in mar.

Looking for a more elegant LINQ solution

I am trying to figure out an efficient way to retrieve the data I am after. I need to get a list of all of the most recent children by ParentId coupled with all parent entries that do NOT have children. I have created a visual guide to illustrate what the response should be.
The query needs to remain as IQueryable until ALL sorting and paging is completed.
Last and LastOrDefault are not supported by LINQ to Entities (as stated by the error messages I have received while using them).
Using First or FirstOrDefault will return the error "This method or operation is not implemented"
Original Data:
-------------------------------
- Id - ParentId - CreatedDate -
-------------------------------
- 1 - - 07/01/2013 -
- 2 - - 07/01/2013 -
- 3 - - 07/01/2013 -
- 4 - 1 - 07/02/2013 -
- 5 - 2 - 07/03/2013 -
- 6 - 2 - 07/04/2013 -
- 7 - 1 - 07/05/2013 -
-------------------------------
Data returned by query
-------------------------------
- Id - ParentId - CreatedDate -
-------------------------------
- 3 - - 07/01/2013 -
- 6 - 2 - 07/04/2013 -
- 7 - 1 - 07/05/2013 -
-------------------------------
Currently, my LINQ query looks like this:
// Retrieves parent records with NO children.
var q1 = myTable
.Where(x => x.ParentId == null)
.Except(myTable
.Where(x => myTable
.Any(c => (c.ParentId == x.Id))));
// Retrieves most recent child records for each parentId
var q2 =
(from a in myTable
join b in
(myTable.Where(a => a.ParentId != null)
.GroupBy(a => a.ParentId)
.Select(b => new { ParentId = b.Key, CreatedDate = b.Max(t => t.CreatedDate) }))
on a.ParentId equals b.ParentId
where a.CreatedDate == b.CreatedDate
select a);
q1 = q1.Union(q2);
The back-end is using Npgsql2 with PostgreSQL. I am looking for a more elegant solution for this query. I am very new to LINQ and would like to optimize this.
Sorting code (sloppy, but jTable returns these strings):
if (string.IsNullOrEmpty(sorting) || sorting.Equals("Name ASC")) {
q1 = q1.OrderBy(p => p.Customer.Name);
} else if (sorting.Equals("Name DESC")) {
q1 = q1.OrderByDescending(p => p.Customer.Name);
} else if (sorting.Equals("Date ASC")) {
q1 = q1.OrderBy(p => p.CreatedDate);
} else if (sorting.Equals("Date DESC")) {
q1 = q1.OrderByDescending(p => p.CreatedDate);
}
Paging code:
var result = pageSize > 0
? q1.Skip(startIndex).Take(pageSize).ToList()
: q1.ToList();

Use grouping:
Mock data:
public class Entry {
public int Id { get; set; }
public int? ParentId { get; set; }
public DateTime Date { get; set; }
};
var list = new List<Entry> {
new Entry{ Id = 1, ParentId = null, Date = new DateTime(2013, 7, 1) },
new Entry{ Id = 2, ParentId = null, Date = new DateTime(2013, 7, 1) },
new Entry{ Id = 3, ParentId = null, Date = new DateTime(2013, 7, 1) },
new Entry{ Id = 4, ParentId = 1, Date = new DateTime(2013, 7, 2) },
new Entry{ Id = 5, ParentId = 2, Date = new DateTime(2013, 7, 3) },
new Entry{ Id = 6, ParentId = 2, Date = new DateTime(2013, 7, 4) },
new Entry{ Id = 7, ParentId = 1, Date = new DateTime(2013, 7, 5) }
};
Query:
var query = from l in list
group l by l.ParentId into g
select new {
Items = g.OrderBy(x => x.Date).Last()
};
var res = query.OrderBy(x => x.Items.Id).Select(x => x.Items).ToList();
LinqPad result:
Id ParentId Date
3 null 01.07.2013 0:00:00
6 2 04.07.2013 0:00:00
7 1 05.07.2013 0:00:00

I can propose a different query, still in two phases
var firstQuery = myTable.Select(p => new { p.ID, ParentID = p.ParentID ?? p.ID, p.CreatedDate })
.GroupBy( p => p.ParentID).Select( q => new
{
el = q.OrderByDescending( k => k.CreatedDate).Take(1)
}).SelectMany(t => t.el);
var result = dc.TabellaId_ParentId.Where(p => test.Select(q => q.ID).Contains(p.ID));

Select multiple fields group by and sum

I want to do a query with linq (list of objects) and I really don't know how to do it, I can do the group and the sum but can't select rest of the fields.
Example:
ID Value Name Category
1 5 Name1 Category1
1 7 Name1 Category1
2 1 Name2 Category2
3 6 Name3 Category3
3 2 Name3 Category3
I want to group by ID, SUM by Value and return all fields like this.
ID Value Name Category
1 12 Name1 Category1
2 1 Name2 Category2
3 8 Name3 Category3

Updated :
If you're trying to avoid grouping for all the fields, you can group just by Id:
data.GroupBy(d => d.Id)
.Select(
g => new
{
Key = g.Key,
Value = g.Sum(s => s.Value),
Name = g.First().Name,
Category = g.First().Category
});
But this code assumes that for each Id, the same Name and Category apply. If so, you should consider normalizing as #Aron suggests. It would imply keeping Id and Value in one class and moving Name, Category (and whichever other fields would be the same for the same Id) to another class, while also having the Id for reference. The normalization process reduces data redundancy and dependency.

void Main()
{
//Me being lazy in init
var foos = new []
{
new Foo { Id = 1, Value = 5},
new Foo { Id = 1, Value = 7},
new Foo { Id = 2, Value = 1},
new Foo { Id = 3, Value = 6},
new Foo { Id = 3, Value = 2},
};
foreach(var x in foos)
{
x.Name = "Name" + x.Id;
x.Category = "Category" + x.Id;
}
//end init.
var result = from x in foos
group x.Value by new { x.Id, x.Name, x.Category}
into g
select new { g.Key.Id, g.Key.Name, g.Key.Category, Value = g.Sum()};
Console.WriteLine(result);
}
// Define other methods and classes here
public class Foo
{
public int Id {get;set;}
public int Value {get;set;}
public string Name {get;set;}
public string Category {get;set;}
}

If your class is really long and you don't want to copy all the stuff, you can try something like this:
l.GroupBy(x => x.id).
Select(x => {
var ret = x.First();
ret.value = x.Sum(xt => xt.value);
return ret;
}).ToList();
With great power great responsibility comes. You need to be careful. Line ret.value = x.Sum(xt => xt.value) will change your original collection, as you are passing reference, not new object. If you want to avoid it, you need to add some Clone method into your class like MemberwiseClone (but again, this will create shallow copy, so be careful). Afer that just replace the line with: var ret = x.First().Clone();

try this:
var objList = new List<SampleObject>();
objList.Add(new SampleObject() { ID = 1, Value = 5, Name = "Name1", Category = "Catergory1"});
objList.Add(new SampleObject() { ID = 1, Value = 7, Name = "Name1", Category = "Catergory1"});
objList.Add(new SampleObject() { ID = 2, Value = 1, Name = "Name2", Category = "Catergory2"});
objList.Add(new SampleObject() { ID = 3, Value = 6, Name = "Name3", Category = "Catergory3"});
objList.Add(new SampleObject() { ID = 3, Value = 2, Name = "Name3", Category = "Catergory3"});
var newList = from val in objList
group val by new { val.ID, val.Name, val.Category } into grouped
select new SampleObject() { ID = grouped.ID, Value = grouped.Sum(), Name = grouped.Name, Category = grouped.Category };
to check with LINQPad:
newList.Dump();

How to re-write this inner join subquery from SQL to Lambda

SELECT ulcch.ID, ulcch.UserLoginHistoryID, ulcch.StatusID,
ulcch.ClientModuleID, ulcch.DeviceState, ulcch.UpdatedAt, ulcch.CreatedAt
FROM UserLoginClientConnectionHistory AS ulcch INNER JOIN
(SELECT MAX(CreatedAt) AS maxCreatedAt
FROM UserLoginClientConnectionHistory AS ulcch1
GROUP BY UserLoginHistoryID) AS m ON m.maxCreatedAt = ulcch.CreatedAt
There can be many updates of 'device state' per day audited into this login table. This query returns the last unique one for each day.
I would like this re-written as a Lambda statement. This is how far I got, I don't know if i'm on the right track, and my Max() is throwing a type error, probably because the group by is making another list or something...
Hope you can work it out from my object examples.... :S
userLogin.UserLoginClientConnectionHistories.Where(x => x.CreatedAt ==
userLoginClientConnectionHistoryRepository.GetAll(
GenericStatus.Active).GroupBy(y => y.UserLoginHistoryID).Max(y => y.CreatedAt));

I think this does what you want:
var result = userLogin.UserLoginClientConnectionHistories
.GroupBy(y => new { Id = y.UserLoginHistoryID, Day = y.CreatedAt.Date })
.Select(x => new
{
Id = x.Key.Id,
Day = x.Key.Day,
MostRecent = x.Max(y => y.CreatedAt)
});
Here is a testbed for it:
public class Program
{
class LoginEntry
{
public int UserLoginHistoryID { get; set; }
public DateTime CreatedAt { get; set; }
}
class UserLogin
{
public List<LoginEntry> UserLoginClientConnectionHistories = new List<LoginEntry>();
}
public static void Main(string[] args)
{
UserLogin userLogin = new UserLogin();
userLogin.UserLoginClientConnectionHistories = new List<LoginEntry> {
new LoginEntry {UserLoginHistoryID = 1, CreatedAt = new DateTime(2009, 1, 1, 3, 0 ,0)},
new LoginEntry {UserLoginHistoryID = 1, CreatedAt = new DateTime(2009, 1, 1, 15, 0 ,0)},
new LoginEntry {UserLoginHistoryID = 1, CreatedAt = new DateTime(2009, 1, 3, 11, 0 ,0)},
new LoginEntry {UserLoginHistoryID = 1, CreatedAt = new DateTime(2009, 1, 1, 10, 0 ,0)},
new LoginEntry {UserLoginHistoryID = 2, CreatedAt = new DateTime(2009, 1, 3, 4, 0 ,0)},
new LoginEntry {UserLoginHistoryID = 2, CreatedAt = new DateTime(2009, 1, 3, 5, 0 ,0)},
};
var result = userLogin.UserLoginClientConnectionHistories
.GroupBy(y => new { Id = y.UserLoginHistoryID, Day = y.CreatedAt.Date })
.Select(x => new
{
Id = x.Key.Id,
Day = x.Key.Day,
MostRecent = x.Max(y => y.CreatedAt)
});
foreach (var item in result)
{
Console.WriteLine("User {0}, day {1}, most recent {2}",
item.Id,
item.Day,
item.MostRecent);
}
}
}
Output:
User 1, day 01-01-2009 00:00:00, most recent 01-01-2009 15:00:00
User 1, day 03-01-2009 00:00:00, most recent 03-01-2009 11:00:00
User 2, day 03-01-2009 00:00:00, most recent 03-01-2009 05:00:00

Here is the inner join portion as a lambda. I assumed CreatedAt was a dateTime.
UserLoginClientConnectionHistory
.GroupBy (ulcch1 =>
new
{
Name = ulcch1.Name
})
.Select (g =>
new
{
maxCreatedAt = (DateTime?)(g.Max (p => p.CreatedAt))
})

I think you want to group by CreatedAt rather than UserLoginHistoryID:
var q = userLogin.UserLoginClientConnectionHistories
.GroupBy(h => h.CreatedAt)
.OrderByDescending(g => g.Key) // Sort by CreatedAt
.First()
.Select(h => new { h.Id, h.UserLoginHistoryID, ... });
This will return the set of UserLoginClientConnectionHistory entries that share the most recent CreatedAt value.

Thanks for all your help guys, i've voted you all up, but you wouldn't believe it but a few hours later I searched for a program to convert SQL to LINQ, and to my surprise found one called "Linqer". Sounds crazy and didn't expect to get far, but it worked perfectly.. definitely worth checking out that app if anyone else gets stuck in the same boat...
Check the mammoth query it returned! After analysing it, don't think it's got extra bloat? Anyone have any optimisation tips or spot any unnecessary code?
moduleDeviceStates = from ulh in user.UserLoginHistories
join ulcch in userLogin.UserLoginClientConnectionHistories on new { ID = ulh.ID } equals new { ID = ulcch.UserLoginHistoryID }
join cm in clientModuleRepository.GetAll(GenericStatus.Active) on new { ClientModuleID = ulcch.ClientModuleID } equals new { ClientModuleID = cm.ID }
join mo in moduleRepository.GetAll(GenericStatus.Active) on new { ModuleID = cm.ModuleID } equals new { ModuleID = mo.ID }
join m in
(
(from ulcch1 in userLogin.UserLoginClientConnectionHistories
group ulcch1 by new
{
ulcch1.UserLoginHistoryID
} into g
select new
{
maxCreatedAt = g.Max(p => p.CreatedAt)
})) on new { maxCreatedAt = ulcch.CreatedAt } equals new { maxCreatedAt = m.maxCreatedAt }
select new ModuleDeviceState()
{
ModuleID = mo.ID,
Name = mo.Name,
DeviceState = (State.DeviceState)ulcch.DeviceState,
CreatedAt = ulcch.CreatedAt
};
Cheers for your help dahlbyk, but I did want to group on UserLoginHistoryID, I had my query confirmed in SQL before delving into a lambda equivalent :) thanks.
#Mark Thanks for taking the time to reply, yes I do what the [last] entries per user (userloginhistory.. which in turn contains a userID) for each day, and exporting my sql into the linq query did produce what I wanted (which can be seen in the query result below; this is what I want. The reason you see double entries for each day is because there are also attached ClientModule's.. so I really want all client module, per login entry per day - so hard to get a programming requirement across over a discussion forum argh!) Perhaps yours does exactly the same thing (it appears to if I am reading your output correctly) just a lot more streamlined.
See I didn't know too much about the anon casting you've done there with GroupBy and Select, but now I see it, it makes sense. I might give yours a go. Hopefully I can give it a tweak to include distinct ClientModule's per day too. So anyway.. here is the query result from my SQL, and effectively what I got through my own lambda:
ID UserLoginHistoryID StatusID ClientModuleID DeviceState UpdatedAt CreatedAt
277 62 1 1 4 NULL 2009-10-31 13:28:59.003
278 62 1 16 4 NULL 2009-10-31 13:28:59.003
331 65 1 1 4 NULL 2009-10-31 17:13:28.333
332 65 1 16 4 NULL 2009-10-31 17:13:28.333
Update Mark: Hi again, well after a couple of tweaks on your query, I could produce the same object graph in .NET between both lambda statements. This is the one I will use now, derived from yours as it's more streamlined and easier to understand than the auto-gen'd one and I will award you the points :)
I added a few more entries to the Group By as I need that for my new ModuleDeviceState class.
moduleDeviceStates = userLogin.UserLoginClientConnectionHistories
.GroupBy(y => new { Id = y.UserLoginHistoryID,
CreatedAt = y.CreatedAt.Date,
ModuleID = y.ClientModule.ModuleID,
ModuleName = y.ClientModule.Module.Name,
DeviceState = y.DeviceState })
.Select(x => new ModuleDeviceState()
{
ModuleID = x.Key.ModuleID,
Name = x.Key.ModuleName,
DeviceState = (State.DeviceState)x.Key.DeviceState,
CreatedAt = x.Max(y => y.CreatedAt)
});