We Keep Coding

Query with Linq and count matches in the Where clause and project it

Need to query many columns for possible matches and then see how many did match and add that to a column in the Select projection. I could have two column of the four match but how many did? Then want to sort the result by how many matches. Do I need to group them first? Do a separate aggregation? Kind of stumped as which way to go because the real one would have a lot more tests of fields in production. Could possibly match as many as 8 tests in the where clause.
var results = _RList
.Where(d => d.RMI15Min == RMI.ConfirmedBottom || d.RMI15Min == RMI.InPlaceBottomConfirmed
|| d.RMI30Min == RMI.ConfirmedBottom || d.RMI30Min == RMI.InPlaceBottomConfirmed)
.OrderBy(d => d.Instrument)
.Select(d => new
{
d.Instrument,
d.Description,
d.RMI15Min,
d.RMI30Min,
NewColumn with the total of the matches in the .Where clause above.
}).ToList();

Assuming _RList does not tie back to a database table
var confirmedList = new List<int> { RMI.ConfirmedBottom, RMI.InPlaceBottomConfirmed };
var results = _RList
.OrderBy(d => d.Instrument)
.Select(d => new
{
d.Instrument,
d.Description,
d.RMI15Min,
d.RMI30Min,
Count = (new List<int> { d.RMI15Min, d.RMI30Min }).Count(c => confirmedList.Contains(c))
})
.Where(d => d.Count > 0)
.ToList();
If it ties back to a database table, it depends on whether your library can convert the above LINQ statement.

I would consider using ternary operators to add 1/0 to the total;
Count = (d.RMI15Min == RMI.ConfirmedBottom ? 1 : 0)
+ (d.RMI15Min == RMI.InPlaceBottomConfirmed ? 1 : 0)
+ (d.RMI30Min == RMI.ConfirmedBottom ? 1 : 0)
+ (d.RMI30Min == RMI.InPlaceBottomConfirmed ? 1 : 0)
Then filter the list after calculating the count.

Linq is very slow with predicate and order by clause

We have List holding 20K objects which contains dates into it. We want to find most recent date from that list considering one condition. Using code as below.
listObject.Where(r => r.Date <= asOfDate).OrderByDescending(r => r.Date).FirstOrDefault();
This is taking too long than expected.
Can you please help what would be the better way to do it?
Thank You!

You can do (based on #Barns comment)
var maxDate = listObject.Where(r => r.Date <= asOfDate).Max(r => r.date);
var item = listObject.FirstOrDefault(r => r.date == maxDate);
this will only loop over your list twice instead of sorting it.

Try using Aggregate:
listObject
.Where(r => r.Date <= asOfDate)
.Aggregate((acc, curr) => curr.Date > acc.Date ? curr : acc)
Performance-wise it can be improved moving filtering logic inside the Aggregate and introducing null accumulator with null handling inside, but if performance is a big concern just switch to for loop.

You currently have three operations:
.Where(r => r.Date <= asOfDate) - time complexity O(n)
.OrderByDescending(r => r.Date)- time complexity (I imagine) O(n log(n))
.FirstOrDefault();- time complexity O(0)
You could do the following an get the same result:
var maxDate= listObject.Where(r => r.Date <= asOfDate).Max(r => r.date); - time complexity O(n)
var result = listObject.FirstOrDefault(r => r.Date == maxDate); - time complexity O(n)

Why not combine Where and Max operations like this:
var maxDate = listObject.Max(r => r.Date <= asOfDate ? r.Date : DateTime.MinValue);
var item = listObject.FirstOrDefault(r => r.date == maxDate);
This will run over the list only twice.

Have you tried sorting the collection first?
listObject
.OrderByDescending(ordr => ordr.Date)
.Where(obj => obj.Date <= asOfDate)
.FirstOrDefault();

Getting the count of most repeated records in Linq

I am working on an application in which I have to store play history of a song in the data table. I have a table named PlayHistory which has four columns.
Id | SoundRecordingId(FK) | UserId(FK) | DateTime
Now i have to implement a query that will return the songs that are in trending phase i.e. being mostly played. I have written the following query in sql server that returns me data somehow closer to what I want.
select COUNT(*) as High,SoundRecordingId
from PlayHistory
where DateTime >= GETDATE()-30
group by SoundRecordingId
Having COUNT(*) > 1
order by SoundRecordingId desc
It returned me following data:
High SoundRecordingId
2 5
2 3
Which means Song with Ids 5 and 3 were played the most number of times i.e.2
How can I implement this through Linq in c#.
I have done this so far:
DateTime d = DateTime.Now;
var monthBefore = d.AddMonths(-1);
var list =
_db.PlayHistories
.OrderByDescending(x=>x.SoundRecordingId)
.Where(t => t.DateTime >= monthBefore)
.GroupBy(x=>x.SoundRecordingId)
.Take(20)
.ToList();
It returns me list of whole table with the count of SoundRecording objects but i want just count of the most repeated records.
Thanks

There is an overload of the .GroupBy method which will solve your problem.
DateTime d = DateTime.Now;
var monthBefore = d.AddMonths(-1);
var list =
_db.PlayHistories
.OrderByDescending(x=>x.SoundRecordingId)
.Where(t => t.DateTime >= monthBefore)
.GroupBy(x=>x.SoundRecordingId, (key,values) => new {SoundRecordingID=key, High=values.count()})
.Take(20)
.ToList();
I have simply added the result selector to the GroupBy method call here which does the same transformation you have written in your SQL.
The method overload in question is documented here
To go further into your problem, you will probably want to do another OrderByDescending to get your results in popularity order. To match the SQL statement you also have to filter for only counts > 1.
DateTime d = DateTime.Now;
var monthBefore = d.AddMonths(-1);
var list =
_db.PlayHistories
.Where(t => t.DateTime >= monthBefore)
.GroupBy(x=>x.SoundRecordingId, (key,values) => new {SoundRecordingID=key, High=values.count()})
.Where(x=>x.High>1)
.OrderByDescending(x=>x.High)
.ToList();

I like the 'linq' syntax it's similar to SQL
var query = from history in _db.PlayHistories
where history.DateTime >= monthBefore
group history by history.SoundRecordingId into historyGroup
where historyGroup.Count() > 1
orderby historyGroup.Key
select new { High = historyGroup.Count(), SoundRecordingId = historyGroup.Key };
var data = query.Take(20).ToList();

You´re allmost done. Just order your list by the count and take the first:
var max =
_db.PlayHistories
.OrderByDescending(x=>x.SoundRecordingId)
.Where(t => t.DateTime >= monthBefore)
.GroupBy(x=>x.SoundRecordingId)
.OrderByDescending(x => x.Count())
.First();
This gives you a single key-value-pair where the Key is your SoundRecordingId and the value is the number of its occurences in your input-list.
EDIT: To get all records with that amount chose this instead:
var grouped =
_db.PlayHistories
.OrderByDescending(x => x.SoundRecordingId)
.Where(t => t.DateTime >= monthBefore)
.GroupBy(x => x.SoundRecordingId)
.Select(x => new { Id = x.Key, Count = x.Count() }
.OrderByDescending(x => x.Count)
.ToList();
var maxCount = grouped.First().Count;
var result = grouped.Where(x => x.Count == maxCount);

This solves the problem by giving you what you asked for. Your query in LINQ, returning just the play counts.
var list = _db.PlayHistories.Where(x => x.DateTimeProp > (DateTime.Now).AddMonths(-1))
.OrderByDescending(y => y.SoundRecordingId.Count())
.ThenBy(z => z.SoundRecordingId)
.Select(xx => xx.SoundRecordingId).Take(20).ToList();

'int' does not contain a definition for 'ToList'

So I'm trying to run a Linq query that's analogous to the SQL query:
SELECT COUNT(*)
FROM table
WHERE ww >= [wwStartSelected]
AND ww <= [wwEndSelected]
AND manager='[managerName]'
AND status='Done'
GROUP BY ww;
To get a count of the number of rows (tasks) within the given ww range that are marked as done under a particular manager and grouped by ww. I've tried to create a LINQ query that would return something similar (wwStartSelected && wwEndSelected are global vars):
protected List<int> getManagerDoneCount(string managerName)
{
using (var context = new InfoDBContext())
{
List<int> managerDoneCount = context.InfoSet.Where(x => x.ww >= wwStartSelected && x.ww <= wwEndSelected && x.manager == managerName && x.status == "Done").GroupBy(x => x.ww).Count().ToList();
return managerDoneCount;
}
}
This query would then feed into a chart:
var testChart = new Chart(width: 600, height: 400)
.AddTitle("Test")
.AddSeries(
name: "Done",
chartType: "StackedColumn100",
xValue: new[] { WWList },
yValues: new[] { getManagerDoneCount("someManager") })
However I'm running into an issue with my Linq line and it says:
'int' does not contain a definition for 'ToList' and no extension method 'ToList'
accepting a first argument of type 'int' could be found (are you
missing a using directive or an assembly reference?)
Is there way to get fix this easily or do I have to convert to string, then convert back to int for the chart series (the latter of which seems a bit silly [but if so, best way to do so])?

.Count().ToList()
You're asking it to count the items in the list (which results in a number) and then convert that single number into a list, which makes no sense.
Either return a list and count it later (omit the .Count()) or, change the method to return an int not a List<int> and omit the .ToList()
protected int getManagerDoneCount(string managerName)
{
using (var context = new InfoDBContext())
{
int managerDoneCount = context.InfoSet
.Where(x => x.ww >= wwStartSelected &&
x.ww <= wwEndSelected &&
x.manager == managerName &&
x.status == "Done")
.GroupBy(x => x.ww)
.Count();
return managerDoneCount;
}
}
As an aside, to save you writing hundreds of these methods, you can pass the Where clause in as a parameter...
using System.Linq.Expressions;
protected int getManagerCount(string managerName, Expression<Info> predicate)
{
using (var context = new InfoDBContext())
{
int managerDoneCount = context.InfoSet
.Where(predicate)
.GroupBy(x => x.ww)
.Count();
return managerDoneCount;
}
}
Then call it like this...
var count = getManagerCount("...", x => x.ww >= wwStartSelected &&
x.ww <= wwEndSelected &&
x.manager == managerName &&
x.status == "Done");
Edit Re: Comments
To return a count of each group, List<int> is a bad idea as you aren't ordering the groups so the counts will be in an undefined order. The ideal solution is to have a class that has an appropriate Key and Count property, but to simplify the example, I'll use a Tuple.
//I'm assuming `ww` is an `int`, change the first type parameter of the tuple as appropriate
List<Tuple<int, int>> counts = context.InfoSet
.Where(x => x.ww >= wwStartSelected &&
x.ww <= wwEndSelected &&
x.manager == managerName &&
x.status == "Done")
.GroupBy(x => x.ww)
.Select(x => new Tuple<int, int>(x.Key, x.Count())).ToList();
Note that after you've done the group, the next Select is against the group, which has a Key property for the thing you've grouped on and a lot of aggregate methods for counting, summing, etc..
If you really just want a list of ints, change the last Select to be...
.Select(x => x.Count())
If you weren't passing it out of the method, I'd just use an anonymous class...
.Select(x => new {Ww = x.Key, Count = x.Count()})
But that's no use in a method signature. If you created a CountResult class with Ww and Count properties...
.Select(x => new CountResult{Ww = x.Key, Count = x.Count()})
Edit Re: Comments #2
Linq-To-Entities builds an expression tree which is executed against SQL server, whereas Linq-To-Objects runs in-memory on the client and has more features (as it doesn't need to work out equivalent SQL). In this case, when it gets results from SQL it creates a special proxy class that looks/behaves the same as your entities but handle additional things like tracking which properties have changed. Because of this, you can only use classes which can be constructed (with a parameterless constructor) and then have their properties set (and tracked).
(Although you didn't specify Linq-To-Entities, it's obvious from your question so I should've caught this).
LINQ doesn't deal in lists, but IQueryables, which support lazy evaluation.
Eg If you do...
var a = dbcontext.SomeSet.Where(x => true); //You could omit the Where entirely, just for illustration purposes
var b = a.Where(x => x.Id < 100);
var c = b.ToList();
The query is only executed on the last line and at most 100 records will be returned by the database. a and b are both IQueryable<SomeSet> and "just" contain the expression tree (basically a hierarchy representing the constrains/operations applied so far).
So, to be able to use parameterised constructors / other Linq-To-Object features, we can force the evaluation ...
List<Tuple<int, int>> counts = context.InfoSet
.Where(x => x.ww >= wwStartSelected &&
x.ww <= wwEndSelected &&
x.manager == managerName &&
x.status == "Done")
.GroupBy(x => x.ww)
.ToList() // <<<< Force execution of SQL query
.Select(x => new Tuple<int, int>(x.Key, x.Count())).ToList();
Which should allow you to use constructors, should you wish.
That said, getting a zero count is difficult - the same as it would be getting it from a database (if you group by a field, it doesn't show any 0 counts). There are a number of ways to approach this and which one you use depends on how much data you're playing with. All of them require some way of defining all possible values. I'll use a List<string> as it works well with LINQ
You could, for example get a list of all values and run a different count for each. This is easy but requires multiple queries. If there are lots of groups, it might be expensive...
var groups = new List<string> {"ww1", "ww2", ...};
var counts = groups.Select(g => context.InfoSet.Where(x => x.ww == g &&
x.manager == managerName &&
x.status == "Done").Count());
(This will only return counts, but in the same order as your groups list. As before, you can Select anything you like, including a CountResult...)
var counts = groups.Select(g => new CountResult {
Ww = g,
Count = context.InfoSet.Where(x => x.ww == g &&
x.manager == managerName &&
x.status == "Done").Count();
});
Alternatively, you can run the query you were doing previously and add the missing groups with a count of 0. This has the benefit of running a single SQL query and letting the database do all the heavy lifting (Ok, handling a few counts isn't too expensive but it's worth bearing in mind for other problems - you don't want to get the whole DB table in memory and do the processing there!).
var groups = new List<string> {"ww1", "ww2", ...};
var previousQuery = ... //(I'll assume a List<CountResult> but tuple would work the same)
var finalList = previousQuery.Concat(
groups.Where(g => ! previousQuery.Exists(p => p.Ww == g))
.Select(g => new CountResult {Ww=g, Count=0})
);
In short, take the previous results set, and concatenate (join) it with the result of; (take a list of all groups, remove those already in set 1, for the remainder create a new object with the appropriate ww and a count of 0)

LINQ - How to query a range of effective dates that only has start dates

I'm using C# 3.5 and EntityFramework. I have a list of items in the database that contain interest rates. Unfortunately this list only contains the Effective Start Date. I need to query this list for all items within a range.
However, I can't see a way to do this without querying the database twice. (Although I'm wondering if delayed execution with EntityFramework is making only one call.) Regardless, I'm wondering if I can do this without using my context twice.
internal IQueryable<Interest> GetInterests(DateTime startDate, DateTime endDate) {
var FirstDate = Context.All().Where(x => x.START_DATE < startDate).Max(x => x.START_DATE);
IQueryable<Interest> listOfItems = Context.All().Where(x => x.START_DATE >= FirstDate && x.START_DATE <= endDate);
return listOfItems;
}

If you could use a LINQ query, you can use let to do this:
(from c in dbContext.Table
let firstdate = dbContext.Table.Max(i => c.StartDate < startDate)
where c.StartDate >= firstdate
and c.StartDate <= enddate
select c)
I'm not sure if the max will work this way, so you may need to alternatively do:
(from c in dbContext.Table
let firstdate = dbContext.Table.Select(i => i.StartDate).Max(i => c.StartDate < i)
where c.StartDate >= firstdate
and c.StartDate <= enddate
select c)
Something like that.

I haven't tried this on EF but on Linq to objects it works fine:
var result = source
.OrderBy(x => x.start)
.GroupBy(x => x.start < startDate)
.SelectMany((x, i) => i == 0 ? new[] {new { value = x.Last().value, start = x.Last().start }} : x.Where(y => y.start < endDate));
The issue is that C# LINQ is missing an operator which gives you access to the previous item in a sequence. F# apparently can handle this. Workarounds involved either a GroupBy or an Aggregate operation. In this case, GroupBy can handle it.
It's not pretty and I wouldn't recommend using it over the two phase approach.