We Keep Coding

Using Linq to look for gap in dates

I have a single queryable that is ordered by id and then by date in descending order (most recent will appear first), tracking when the item (id) was online. If a gap is detected (more than 1 day) I want to filter out all of the dates that come prior to the gap, as to only get the most recent range of online days.
Currently I am looping through with for loops, however the data set is very large so I would like to improve performance using linq.
Are there any ways to compare the records by id, then date, and remove elements of that id after a gap is detected ( current.date - next.date != 1)?
Id
Date
1
2022/01/01
1
2021/12/31
1
2021/12/25
2
2021/12/20
2
2021/12/19
2
2021/12/18
2
2021/12/15
would return:
Id
Date
1
2022/01/01
1
2021/12/31
2
2021/12/20
2
2021/12/19
2
2021/12/18

var result = queryable
.GroupBy(entry => entry.Id)
.AsEnumerable()
.Select(entryGroup => entryGroup
.OrderByDescending(entry => entry.Date)
.Aggregate((EntryGroup: new List<Entry>(), GapDetected: false), (accumulated, current) =>
{
if (accumulated.GapDetected) return accumulated;
var previous = accumulated.EntryGroup.LastOrDefault();
if (previous == null || (previous.Date - current.Date).Days < 2) accumulated.EntryGroup.Add(current);
else accumulated.GapDetected = true;
return accumulated;
}))
.SelectMany(entryGroup => entryGroup.EntryGroup)
.ToList();
Note that only the GroupBy portion of the code is actually executed as an SQL query and the rest of the query is done locally since it can not be translated to SQL. I couldn't come up with a solution where the entire query could be translated to SQL but I wanted to show how this can be done with LinQ.

Ordering Object and Selecting based on custom priority

I have a custom priority array.
int[] priorities = { 866, 663, 855, 853, 854};
And I have this code to currently to sort through my list and pick the first based on the criteria.
var target = (from people in GetNearbyPeopleList()
where people.DistanceToTravel > 0 && people.ReachedDestination == false
orderby //not sure what to do here
select people).FirstOrDefault();
So I want to order by my custom priorities. Where people.currentlocation is ordered by my priorities array.
Do I do - ?
orderby priorities.Contains(people.currentlocation)
That's all I can think of, but it does not properly order by the order of my custom priorities. I want it to follow this order exactly:
int[] priorities = { 866, 663, 855, 853, 854};
So if location is 866, pick that one. I only want to select one and I want to select the first based on that priority list. If currentlocation == 866 does not exist, then pick 663 and so and so on.

If your values are fixed at compile-time... avoid the array and write:
orderby
people.currentLocation == 866 ? 1 :
people.currentLocation == 663 ? 2 :
people.currentLocation == 855 ? 3 :
people.currentLocation == 853 ? 4 :
people.currentLocation == 854 ? 5 :
6
If your values change at run-time, but the number of values has some fixed maximum, then write:
Person FindPriorityPerson(IQueryable<Person> query,
int p1 = 0, int p2 = 0, int p3 = 0, int p4 = 0,
int p5 = 0, int p6 = 0, int p7 = 0, int p8 = 0)
{
return query.OrderBy(person =>
person.currentLocation == p1 ? 1 :
person.currentLocation == p2 ? 2 :
person.currentLocation == p3 ? 3 :
person.currentLocation == p4 ? 4 :
person.currentLocation == p5 ? 5 :
person.currentLocation == p6 ? 6 :
person.currentLocation == p7 ? 7 :
person.currentLocation == p8 ? 8 :
9).FirstOrDefault();
}

I think this will do it. This gives you sorting based on array order.
Had to add logic so -1 (not found) is at the end
orderby Array.IndexOf(priorities, people.currentLocation) == -1 ? Integer.MaxValue : Array.IndexOf(priorities, people.currentLocation);

Ctznkan525's answer gave me the idea to use "Select with index" to sort your array in the order or your priorities.
So you want all people with priority is first in the list or Priorities. 2nd are the people with priority that is mentioned 2nd in the list, 3rd are those with priority 3rd in the list etc.
Did you notice that I mentioned the index to identify the order? So if I add the index to your priorities list, I can order by this index.
This works if GetNearByPeopleList returns an IEnumerable.
var indexedPriorities = priorities.Select( (priority, index) => new
{
Priority = priority,
OrderIndex = index,
});
var result = GetNearbyPeopleList()
.Where(...) // take only people you want
.Join(indexedPriorities, // join with indexedPriorities
people => people.CurrentLocation, // from each people take the CurrentLocation
indexedPrio => indexedPrio.Priority, // from each indexedPriority take the priority
(people, prio) => new // when they match, make a new object
{
Index = prio.Index, // containing the index of the matching priority
People = people, // and the matching data
})
.OrderBy(item => item.Index) // order by ascending index
.Select(item => item.People); // keep only the People
This code leaves out all people with CurrentLocation that are not in your Priority list. If you want them, you should Concat them where you want them, probably at the end.
Alas, linq-to-entities does not support "select with index". You could try to first add the index, make your list AsQueryable and then do the join:
var indexedPriorities = = priorities.Select( (priority, index) => new
{
Priority = priority,
OrderIndex = index,
})
.AsQueryable();
var result = GetNearbyPeopleList()
.Where(...)
.Join(indexedPriorities,
people => people.CurrentLocation, CurrentLocation
indexedPrio => indexedPrio.Priority,
...
Alas this doesn't work, because you can only transfer simple types AsQueryable to be used in a join.
However, if the data is remote, and you really need this, you could transfer your priorities and indexes as a decimal:
var indexedPriorities = = priorities.Select( (x, i) => (decimal)x + (decimal)i/1000M)
.AsQueryable();
The number before the decimal point is the priority, after the decimal point is the ordering index: (.001 comes before .002 etc
IQueryable<People> nearbyPeople = GetNearbyPeopleList()
.Where(...);
var result = nearbyPeople.Join(indexedPriorities,
people => people.CurrentLocation,
prio => (int)Math.Floor(prio),
(people, prio) => new
{
OrderIndex =(prio - Math.Truncate(prio),
People = people,
})
.OrderBy(item => item.OrderIndex)
.Select(item => item.People);
First I change the original priority list: { 866, 663, 855, 853, 854} into {866.000, 663.001, 855.002, 853.003, etc}. Assuming your priority list does not have 1000 elements.
Then I do a join with the Math.Floor of 866.000, 663.001 etc. When matching I keep the part after the decimal point as OrderIndex: .000, .001, .002 etc. (Not needed, but if desired: multiply by 1000)
After Ordering I get rid of the OrderIndex.

This can not be done this way.* Linq can not work with your own array the way you need. But there are several other ways to do that I can think of
Redesign you application so you have the priorities in table on the server.
Use table valued parameter (TVP). That is the right way to send your array to the server to be used in the query, it scales to thousands of items without issue. It needs server that supports that (like Microsoft SQL server) and the query has to be done outside the linq. Linq does not support that.
Use Contains and query all items, pick the first on the client. Contains can use your own array, however be aware it converts it into parameters, one value means one parameter. Do not use it if you have too many items (>100) in the array or the query would be slow and would crash for more than approx 2000 items (depends on server type).
Loop the array and query one item a time. You can combine it with Contains and query 10 items a time for example to speed it up.
*Amy B found a way, my bad. But beware of the limits. It uses not one, but two parameters per every value, making it even more constrained. And it would probably lead to table scan which may be much more expensive, depends on table size and array size. On 20M table it needs about 25000x more database resources than just querying all 5 rows with Contains. You should probably combine it with Contains filter to avoid the table scan, but that means three parameters per item...

Group list and perform a function on the resulting matching items

I want to retrieve data from a database by a grouping query and then calculate their ratio. Take for instance positions in a warehouse where you retrieve stock values for 2 days and you want to know the change ratio.
E.g.:
var query = from o in dbContext.Orders
where (o.Date == firstDate) || (o.Date == secondDate)
group o by o.Date into g
select g;
How can i now (inside or outside of the query) caluclate the change ratio of the matching order items? (The ratio being defined as (newOrder.Stock / oldOrder.Stock) -1) I know how to do it by a simple somewhat verbose way, but i was hoping that there is a more elegant solution in linq.
Edit: An example of the data queried and the desired result.
ID Date InStock ItemID
1 15.01 5000 1
2 16.01 7000 1
3 15.01 9000 2
4 16.01 2000 2
This would now show an 40% increase for item 1 and an -78% decrease for item 2.
I already did achieve this by separating the groups into two lists and then checking each list for the corresponding items in the other list. This way you can easily calculate the ratios but you create some new variables and nested foreach loops which seem unnecessary. I'm simply searching for a more elegant solution.

var query = from o in orders
where (o.Date == firstDate || o.Date == secondDate)
group o by o.ItemID into g
select new
{
ItemID = g.Key,
DateStart = g.ElementAt(0).Date,
DateEnd = g.ElementAt(1).Date,
Ratio = g.ElementAt(1).InStock / (float)g.ElementAt(0).InStock
};
We use the fact that we know that each grouping will only contain two items (one for each date) to simply select the result of the division in a new anonymously typed item.
My example used int as the type for InStock, so feel free to remove the (float) cast if it's not needed.

RavenDB Select Distinct on a single property

I have an object stored in RavenDB with three properties: ID, Score, Date.
I want to create an index for retrieving the top 5 scores within a given date range. However, I only want to retrieve one record per ID. If a single ID shows up more than once in the top scores, I only want to retrieve the highest score for that ID, then move on to the next ID.
example scores:
Score____ID____
1000 1
950 1
900 1
850 2
800 2
750 3
700 4
650 5
600 6
550 7
desired query results:
Score____ID____
1000 1
850 2
750 3
700 4
650 5
I have created an explicit index similar to this (adjusted for simplicity):
Map = docs => from doc in docs
orderby doc.Score descending
select new
{
Score = doc.Score,
ID = doc.ID,
Date = doc.Date
};
I call my query with code similar to this (adjusted for simplicity):
HighScores = RavenSession.Query<Score, Scores_ByDate>()
.Customize(x => x.WaitForNonStaleResultsAsOfNow())
.Where(x => x.Date > StartDate)
.Where(x => x.Date < EndDate)
.OrderByDescending(x => x.Score)
.Take(5)
.ToList();
I don't know how to say "only give me the results from each ID one time in the list."

So a few pointers:
Don't order in the Map function. Maps are designed to just dump documents out.
Use the Reduce to do grouping, as this is the way they work by design
Add a hint to RavenDB that a particular column will be sorted in code, and what type of field it is.
By default, the map/reduce assumes the sorting is for text, even if it is a number - (I learned this the hard way and got help for it.)
So..
Just define the Map/Reduce index as normal, and add a sort condition at the end, like this:
public class Score_TopScoringIndex : AbstractIndexCreationTask<Score, Score>
{
public Score_TopScoringIndex()
{
Map = docs => from doc in docs
select new
{
Score = doc.Score,
ID = doc.ID,
Date = doc.Date
};
Reduce = results => from result in results
group result by result.ID into g
select new
{
Score = g.First().Score,
ID = g.Key,
Date = g.First().Date
};
Sort(x=>x.Score, SortOptions.Int);
}
}
Make sure the index is in the DB by using at the startup of your application:
IndexCreation.CreateIndexes(typeof(Score_TopScoringIndex).Assembly, documentStore);
Now, when you query, the OrderByDescending, it will be very fast.
using(var session = store.OpenSession())
{
var highScores = session.Query<Score>("Scores/TopScoringIndex")
.OrderByDescending(x=>x.Score)
.Take(5);
}

You can try using morelinq library
https://code.google.com/p/morelinq/
which has a DistintBy extension.

Converting SQL to LINQ needing to group records created per second

I am looking for a way in C# LINQ using lambda format to group records per second. in my search i have yet to find a good way to do this.
the SQL query is as follows.
select count(cct_id) as 'cnt'
,Year(cct_date_created)
,Month(cct_date_created)
,datepart(dd,cct_date_created)
,datepart(hh,cct_date_created)
,datepart(mi,cct_date_created)
,datepart(ss,cct_date_created)
from ams_transactions with (nolock)
where cct_date_created between dateadd(dd,-1,getdate()) and getdate()
group by
Year(cct_date_created)
,Month(cct_date_created)
,datepart(dd,cct_date_created)
,datepart(hh,cct_date_created)
,datepart(mi,cct_date_created)
,datepart(ss,cct_date_created)
now the closest i was able to come was the following but it is not giving me the right results.
var groupedResult = MyTable.Where(t => t.cct_date_created > start
&& t.t.cct_date_created < end)
.GroupBy(t => new { t.cct_date_created.Month,
t.cct_date_created.Day,
t.cct_date_created.Hour,
t.cct_date_created.Minute,
t.cct_date_created.Second })
.Select(group => new {
TPS = group.Key.Second
});
this appears to be grouping by seconds but not considering it as per individual minute in the date range and instead that second of every minute in the date range. To get Transactions per second i need it to consider each minute of the month, hour, day, minute separately.
The goal will be to pull a Max and Average then from this grouped list. Any help would be greatly appreciated :)

Currently you're selecting the second, rather than the count - why? (You're also using an anonymous type for no obvious reason - whenever you have a single property, consider just selecting that property instead of wrapping it in an anonymous type.)
So change your Select to:
.Select(group => new { Key = group.Key,
Transactions = group.Count() });
Or to have all of the key properties separately:
.Select(group => new { group.Month,
group.Day,
group.Hour,
group.Minute,
group.Second,
Transactions = group.Count() });
(As an aside, do you definitely not need the year part? It's in your SQL...)