We Keep Coding

LINQ: Group by index and value [duplicate]

This question already has answers here:
linq group by contiguous blocks
(5 answers)
Closed 4 years ago.
Lets say I have an list of strings with the following values:
["a","a","b","a","a","a","c","c"]
I want to execute a linq query that will group into 4 groups:
Group 1: ["a","a"] Group 2: ["b"] Group 3: ["a","a","a"] Group 4:
["c","c"]
Basically I want to create 2 different groups for the value "a" because they are not coming from the same "index sequence".
Anyone has a LINQ solution for this?

You just need key other than items of array
var x = new string[] { "a", "a", "a", "b", "a", "a", "c" };
int groupId = -1;
var result = x.Select((s, i) => new
{
value = s,
groupId = (i > 0 && x[i - 1] == s) ? groupId : ++groupId
}).GroupBy(u => new { groupId });
foreach (var item in result)
{
Console.WriteLine(item.Key);
foreach (var inner in item)
{
Console.WriteLine(" => " + inner.value);
}
}
Here is the result: Link

Calculate the "index sequence" first, then do your group.
private class IndexedData
{
public int Sequence;
public string Text;
}
string[] data = [ "a", "a", "b" ... ]
// Calculate "index sequence" for each data element.
List<IndexedData> indexes = new List<IndexedData>();
foreach (string s in data)
{
IndexedData last = indexes.LastOrDefault() ?? new IndexedData();
indexes.Add(new IndexedData
{
Text = s,
Sequence = (last.Text == s
? last.Sequence
: last.Sequence + 1)
});
}
// Group by "index sequence"
var grouped = indexes.GroupBy(i => i.Sequence)
.Select(g => g.Select(i => i.Text));

This is a naive foreach implementation where whole dataset ends up in memory (probably not an issue for you since you do GroupBy):
public static IEnumerable<List<string>> Split(IEnumerable<string> values)
{
var result = new List<List<string>>();
foreach (var value in values)
{
var currentGroup = result.LastOrDefault();
if (currentGroup?.FirstOrDefault()?.Equals(value) == true)
{
currentGroup.Add(value);
}
else
{
result.Add(new List<string> { value });
}
}
return result;
}
Here comes a slightly complicated implementation with foreach and yield return enumerator state machine which keeps only current group in memory - this is probably how this would be implemented on framework level:
EDIT: This is apparently also the way MoreLINQ does it.
public static IEnumerable<List<string>> Split(IEnumerable<string> values)
{
var currentValue = default(string);
var group = (List<string>)null;
foreach (var value in values)
{
if (group == null)
{
currentValue = value;
group = new List<string> { value };
}
else if (currentValue.Equals(value))
{
group.Add(value);
}
else
{
yield return group;
currentValue = value;
group = new List<string> { value };
}
}
if (group != null)
{
yield return group;
}
}
And this is a joke version using LINQ only, it is basically the same as the first one but is slightly harder to understand (especially since Aggregate is not the most frequently used LINQ method):
public static IEnumerable<List<string>> Split(IEnumerable<string> values)
{
return values.Aggregate(
new List<List<string>>(),
(lists, str) =>
{
var currentGroup = lists.LastOrDefault();
if (currentGroup?.FirstOrDefault()?.Equals(str) == true)
{
currentGroup.Add(str);
}
else
{
lists.Add(new List<string> { str });
}
return lists;
},
lists => lists);
}

Using an extension method based on the APL scan operator, that is like Aggregate but returns intermediate results paired with source values:
public static IEnumerable<KeyValuePair<TKey, T>> ScanPair<T, TKey>(this IEnumerable<T> src, TKey seedKey, Func<KeyValuePair<TKey, T>, T, TKey> combine) {
using (var srce = src.GetEnumerator()) {
if (srce.MoveNext()) {
var prevkv = new KeyValuePair<TKey, T>(seedKey, srce.Current);
while (srce.MoveNext()) {
yield return prevkv;
prevkv = new KeyValuePair<TKey, T>(combine(prevkv, srce.Current), srce.Current);
}
yield return prevkv;
}
}
}
You can create extension methods for grouping by consistent runs:
public static IEnumerable<IGrouping<int, TResult>> GroupByRuns<TElement, TKey, TResult>(this IEnumerable<TElement> src, Func<TElement, TKey> key, Func<TElement, TResult> result, IEqualityComparer<TKey> cmp = null) {
cmp = cmp ?? EqualityComparer<TKey>.Default;
return src.ScanPair(0,
(kvp, cur) => cmp.Equals(key(kvp.Value), key(cur)) ? kvp.Key : kvp.Key + 1)
.GroupBy(kvp => kvp.Key, kvp => result(kvp.Value));
}
public static IEnumerable<IGrouping<int, TElement>> GroupByRuns<TElement, TKey>(this IEnumerable<TElement> src, Func<TElement, TKey> key) => src.GroupByRuns(key, e => e);
public static IEnumerable<IGrouping<int, TElement>> GroupByRuns<TElement>(this IEnumerable<TElement> src) => src.GroupByRuns(e => e, e => e);
public static IEnumerable<IEnumerable<TResult>> Runs<TElement, TKey, TResult>(this IEnumerable<TElement> src, Func<TElement, TKey> key, Func<TElement, TResult> result, IEqualityComparer<TKey> cmp = null) =>
src.GroupByRuns(key, result).Select(g => g.Select(s => s));
public static IEnumerable<IEnumerable<TElement>> Runs<TElement, TKey>(this IEnumerable<TElement> src, Func<TElement, TKey> key) => src.Runs(key, e => e);
public static IEnumerable<IEnumerable<TElement>> Runs<TElement>(this IEnumerable<TElement> src) => src.Runs(e => e, e => e);
And using the simplest version, you can get either an IEnumerable<IGrouping>>:
var ans1 = src.GroupByRuns();
Or a version that dumps the IGrouping (and its Key) for an IEnumerable:
var ans2 = src.Runs();

Create lambda expression for OrderBy based on string

The method MyMethod as a string parameter. Based on the value of this parameter, I'd like get back an expression to use with an OrderBy. I don't find the right syntax for Expression<Func<>> to use with the dictionary (as TValue type)
public void MyMethod(string orderBy)
{
var dico = new Dictionary<string, string>
{
{ "property1", x => x.Name},
{ "property2", x => x.Age},
};
dico.TryGetValue("property1", out string myOrder);
myList.OrderBy(myOrder)......
}
Update :
var dico = new Dictionary<string, Expression<Func<Person, xxxxx>>>
{
{ "property1", x => x.Name},
{ "property2", x => x.Age},
};
Thanks,

I think you may get hints from this:
public void MyMethod(string orderBy)
{
// Assuming Product has 'Name' and 'Age' property ?
var dico = new Dictionary<string, Expression<Func<Product,object>>>
{
{ "property1", x => x.Name},
{ "property2", x => x.Age},
};
Expression<Func<Product,object>> myorder;
dico.TryGetValue(orderBy, out myOrder);
_context.Products.OrderBy(myOrder);
}

Write similar logic using generics

I have the following method which determines which cars I need to delete from the DB.
private List<CarDTO> BuildCarsToDelete(IList<CarDTO> newCars, IList<CarDTO> existingCars)
{
var missingCars = new List<CarDTO>();
var cars = newCars.Select(c => c.CarId);
var newCarIds = new HashSet<int>(cars);
foreach (var car in existingCars)
{
//If there are no new cars then it had some and they have been removed
if (newCars.Count() == 0)
{
missingCars.Add(car);
}
else
{
if (!newCarIds.Contains(car.CarId))
{
missingCars.Add(car);
}
}
}
return missingCars;
}
This works as I want - but if I want to achieve the same functionality for Customers or Apartments of other DTOs I will be copying a pasting the code but only changing the variable names and the Type of DTO around - is there a nicer way possible using generics which would keep the algorithm and logic as it is but allow me to use on any DTO?

If all the ids are of type int then you can do that by passing in a Func to determine the id.
private List<T> BuildToDelete<T>(
IList<T> newItems,
IList<T> existingItems,
Func<T, int> getId)
{
var missingItems = new List<T>();
var items = newItems.Select(getId);
var newItemIds = new HashSet<int>(items);
foreach (var item in existingItems)
{
if (newItems.Count() == 0)
{
missingItems.Add(item);
}
else
{
if (!newItemIds.Contains(getId(item)))
{
missingItems.Add(item);
}
}
}
return missingItems;
}
Then call as shown below:
var results = BuildToDelete(newCars, existingCars, c => c.CarId);

Assuming you use the interface approach mentioned in comments, a generic version could look something like this:
private List<TEntity> BuildEntitiesToDelete(IList<TEntity> newEntities, IList<TEntity> existingEntities) where TEntity : IEntityWithId
{
var missingEntities = new List<TEntity>();
var entities = newEntities.Select(e => e.Id);
var newEntityIds = new HashSet<int>(entities);
foreach (var entity in existingEntities)
{
if (entities.Count() == 0)
{
missingEntities.Add(entity);
}
else
{
if (!newEntityIds.Contains(entity.Id))
{
missingEntities.Add(entity);
}
}
}
return missingEntities;
}
IEntityWithId is probably a poor name for the interface, but I'll leave picking a better name up to you.

Try something cleaner:
1) create flexible equality comparer (need to add some null checking etc.)
public class FuncEqualityComparer<T> : IEqualityComparer<T>
{
Func<T, T, bool> comparer;
Func<T, int> hash;
public FuncEqualityComparer (Func<T, T, bool> comparer, Func<T, int> hash)
{
this.comparer = comparer;
this.hash = hash;
}
public bool Equals (T x, T y) => comparer (x, y);
public int GetHashCode (T obj) => hash (obj);
}
2) and now, just simply:
var carComparerByID = new FuncEqualityComparer<CarDTO> ((a, b) => a.CarId == b.CarId, x => x.CarId.GetHashCode ());
var result = existingCars.Except (newCars, carComparerByID).ToList ();

Linq group by except column

I have a class with large amount of properties that I need to group by almost all columns.
class Sample {
public string S1 { get; set; }
public string S2 { get; set; }
public string S3 { get; set; }
public string S4 { get; set; }
// ... all the way to this:
public string S99 { get; set; }
public decimal? N1 { get; set; }
public decimal? N2 { get; set; }
public decimal? N3 { get; set; }
public decimal? N4 { get; set; }
// ... all the way to this:
public decimal? N99 { get; set; }
}
From time to time I need to group by all columns except one or two decimal columns and return some result based on this (namely object with all the fields, but with some decimal value as a sum or max).
Is there are any extension method that would allow me to do something like this:
sampleCollection.GroupByExcept(x => x.N2, x => x.N5).Select(....);
instead of specifying all columns in object?

You won't find anything builtin that handles such a case. You'd have to create one yourself. Depending on how robust you need this to be, you could take a number of approaches.
The main hurdle you'll come across is how you'll generate the key type. In an ideal situation, the new keys that are generated would have their own distinct type. But it would have to be dynamically generated.
Alternatively, you could use another type that could hold multiple distinct values and still could be suitably used as the key. Problem here is that it will still have to be dynamically generated, but you will be using existing types.
A different approach you could take that doesn't involve generating new types, would be to use the existing source type, but reset the excluded properties to their default values (or not set them at all). Then they would have no effect on the grouping. This assumes you can create instances of this type and modify its values.
public static class Extensions
{
public static IQueryable<IGrouping<TSource, TSource>> GroupByExcept<TSource, TXKey>(this IQueryable<TSource> source, Expression<Func<TSource, TXKey>> exceptKeySelector) =>
GroupByExcept(source, exceptKeySelector, s => s);
public static IQueryable<IGrouping<TSource, TElement>> GroupByExcept<TSource, TXKey, TElement>(this IQueryable<TSource> source, Expression<Func<TSource, TXKey>> exceptKeySelector, Expression<Func<TSource, TElement>> elementSelector)
{
return source.GroupBy(BuildKeySelector(), elementSelector);
Expression<Func<TSource, TSource>> BuildKeySelector()
{
var exclude = typeof(TXKey).GetProperties()
.Select(p => (p.PropertyType, p.Name))
.ToHashSet();
var itemExpr = Expression.Parameter(typeof(TSource));
var keyExpr = Expression.MemberInit(
Expression.New(typeof(TSource).GetConstructor(Type.EmptyTypes)),
from p in typeof(TSource).GetProperties()
where !exclude.Contains((p.PropertyType, p.Name))
select Expression.Bind(p, Expression.Property(itemExpr, p))
);
return Expression.Lambda<Func<TSource, TSource>>(keyExpr, itemExpr);
}
}
}
Then to use it you would do this:
sampleCollection.GroupByExcept(x => new { x.N2, x.N5 })...
But alas, this approach won't work under normal circumstances. You won't be able to create new instances of the type within a query (unless you're using Linq to Objects).
If you're using Roslyn, you could generate that type as needed, then use that object as your key. Though that'll mean you'll need to generate the type asynchronously. So you probably will want to separate this from your query all together and just generate the key selector.
public static async Task<Expression<Func<TSource, object>>> BuildExceptKeySelectorAsync<TSource, TXKey>(Expression<Func<TSource, TXKey>> exceptKeySelector)
{
var exclude = typeof(TXKey).GetProperties()
.Select(p => (p.PropertyType, p.Name))
.ToHashSet();
var properties =
(from p in typeof(TSource).GetProperties()
where !exclude.Contains((p.PropertyType, p.Name))
select p).ToList();
var targetType = await CreateTypeWithPropertiesAsync(
properties.Select(p => (p.PropertyType, p.Name))
);
var itemExpr = Expression.Parameter(typeof(TSource));
var keyExpr = Expression.New(
targetType.GetConstructors().Single(),
properties.Select(p => Expression.Property(itemExpr, p)),
targetType.GetProperties()
);
return Expression.Lambda<Func<TSource, object>>(keyExpr, itemExpr);
async Task<Type> CreateTypeWithPropertiesAsync(IEnumerable<(Type type, string name)> properties) =>
(await CSharpScript.EvaluateAsync<object>(
AnonymousObjectCreationExpression(
SeparatedList(
properties.Select(p =>
AnonymousObjectMemberDeclarator(
NameEquals(p.name),
DefaultExpression(ParseTypeName(p.type.FullName))
)
)
)
).ToFullString()
)).GetType();
}
To use this:
sampleCollection.GroupBy(
await BuildExceptKeySelector((CollectionType x) => new { x.N2, x.N5 })
).Select(....);

Borrowing from this answer here:
Create a class EqualityComparer
public class EqualityComparer<T> : IEqualityComparer<T>
{
public bool Equals(T x, T y)
{
IDictionary<string, object> xP = x as IDictionary<string, object>;
IDictionary<string, object> yP = y as IDictionary<string, object>;
if (xP.Count != yP.Count)
return false;
if (xP.Keys.Except(yP.Keys).Any())
return false;
if (yP.Keys.Except(xP.Keys).Any())
return false;
foreach (var pair in xP)
if (pair.Value.Equals( yP[pair.Key])==false)
return false;
return true;
}
public int GetHashCode(T obj)
{
return obj.ToString().GetHashCode();
}
}
Then create your GroupContent method:
private void GroupContent<T>(List<T> dataList, string[] columns, string[] columnsToExclude)
{
string[] columnsToGroup = columns.Except(columnsToExclude).ToArray();
EqualityComparer<IDictionary<string, object>> equalityComparer = new EqualityComparer<IDictionary<string, object>>();
var groupedList = dataList.GroupBy(x =>
{
var groupByColumns = new System.Dynamic.ExpandoObject();
((IDictionary<string, object>)groupByColumns).Clear();
foreach (string column in columnsToGroup)
((IDictionary<string, object>)groupByColumns).Add(column, GetPropertyValue(x, column));
return groupByColumns;
}, equalityComparer);
foreach (var item in groupedList)
{
Console.WriteLine("Group : " + string.Join(",", item.Key));
foreach (object obj in item)
Console.WriteLine("Item : " + obj);
Console.WriteLine();
}
}
private static object GetPropertyValue(object obj, string propertyName)
{
return obj.GetType().GetProperty(propertyName).GetValue(obj, null);
}

I extended the code above borrowing another answer.
public static class IEnumerableExt {
public static IEnumerable<T> GroupBye<T, C>(this IEnumerable<T> query, Func<IGrouping<IDictionary<string, object>, T>, C> grouping) where T : class
{
var cProps = typeof(C).GetProperties().Select(prop => prop.Name).ToArray();
var columnsToGroup = typeof(T).GetProperties().Select(prop => prop.Name).Except(cProps).ToArray();
var equalityComparer = new EqualityComparer<IDictionary<string, object>>();
return query
.GroupBy(x => ExpandoGroupBy(x, columnsToGroup), equalityComparer)
.Select(x => MergeIntoNew(x, grouping, cProps));
}
private static IDictionary<string, object> ExpandoGroupBy<T>(T x, string[] columnsToGroup) where T : class
{
var groupByColumns = new System.Dynamic.ExpandoObject() as IDictionary<string, object>;
groupByColumns.Clear();
foreach (string column in columnsToGroup)
groupByColumns.Add(column, typeof(T).GetProperty(column).GetValue(x, null));
return groupByColumns;
}
private static T MergeIntoNew<T, C>(IGrouping<IDictionary<string, object>, T> x, Func<IGrouping<IDictionary<string, object>, T>, C> grouping, string[] cProps) where T : class
{
var tCtor = typeof(T).GetConstructors().Single();
var tCtorParams = tCtor.GetParameters().Select(param => param.Name).ToArray();
//Calling grouping lambda function
var grouped = grouping(x);
var paramsValues = tCtorParams.Select(p => cProps.Contains(p) ? typeof(C).GetProperty(p).GetValue(grouped, null) : x.Key[p]).ToArray();
return (T)tCtor.Invoke(paramsValues);
}
private class EqualityComparer<T> : IEqualityComparer<T>
{
public bool Equals(T x, T y)
{
var xDict = x as IDictionary<string, object>;
var yDict = y as IDictionary<string, object>;
if (xDict.Count != yDict.Count)
return false;
if (xDict.Keys.Except(yDict.Keys).Any())
return false;
if (yDict.Keys.Except(xDict.Keys).Any())
return false;
foreach (var pair in xDict)
if (pair.Value == null && yDict[pair.Key] == null)
continue;
else if (pair.Value == null || !pair.Value.Equals(yDict[pair.Key]))
return false;
return true;
}
public int GetHashCode(T obj)
{
return obj.ToString().GetHashCode();
}
}
}
Which can be used in the following way:
var list = enumerable.GroupBye(grp => new
{
Value = grp.Sum(val => val.Value)
});
The result will like grouping all other columns but Value, which will be valued to the sum of grouped elements' value

Linq handling variable number of OrderBy

I need to support a variable number of Orderby terms in a Linq (to Entity) statement. That is, my function will accept a list of properties on which the data should be order. The properties can have both ascending or descending sorts. What is the best way to handle constructing the Linq query?
Thanks!

You should be able to do something along these lines:
public IEnumerable<MyType> DoSomething(params Expression<Func<MyType,object>>[] properties)
{
var query = // create LINQ query that returns IQueryable<MyType>
query = query.OrderBy(properties.First());
foreach (var property in properties.Skip(1))
{
query = query.ThenBy(property);
}
}
…
var results = DoSomething(() => x.Age, () => x.Height, () => x.LastName);
You'd need to handle the case where fewer than 2 properties are specified.

Following on from Jay's answer, this can be made into a nice extension method:
public static class EnumerableExtensions
{
public static IEnumerable<T> OrderByMany<T>(this IEnumerable<T> enumerable,
params Expression<Func<T, object>>[] expressions)
{
if (expressions.Length == 1)
return enumerable.OrderBy(expressions[0].Compile());
var query = enumerable.OrderBy(expressions[0].Compile());
for (int i = 1; i < expressions.Length;i++)
{
query = query.ThenBy(expressions[i].Compile());
}
return query;
}
}
Usage becomes quite simple, given a test object:
public class Person
{
public string Name { get; set; }
public int Age { get; set; }
}
This is then possible:
var people = new Person[]
{
new Person() {Name = "John", Age = 40},
new Person() {Name = "John", Age = 20},
new Person() {Name = "Agnes", Age = 11}
};
foreach(var per in people.OrderByMany(x => x.Name, x => x.Age))
{
Console.WriteLine("{0} Age={1}",per.Name,per.Age);
}
Output:
Agnes Age=11
John Age=20
John Age=40
UPDATE
You could add another overload of the OrderByMany method to support SortOrder as well, although it gets clunky rather quickly. Personally I'd just go for the syntax
var query = from p
in people
order by Name, Age descending;
However, for the record, in C#4 at least, I would accomplish the overload using an enum & tuple.
public enum SortOrder
{
Ascending,
Descending
}
and the extra overload:
public static IEnumerable<T> OrderByMany<T>(this IEnumerable<T> enumerable,
params Tuple<Expression<Func<T, object>>,SortOrder>[] expressions)
{
var query = (expressions[0].Item2 == SortOrder.Ascending)
? enumerable.OrderBy(expressions[0].Item1.Compile())
: enumerable.OrderByDescending(expressions[0].Item1.Compile());
for (int i = 1; i < expressions.Length; i++)
{
query = expressions[i].Item2 == SortOrder.Ascending
? query.ThenBy(expressions[i].Item1.Compile())
: query.ThenByDescending(expressions[i].Item1.Compile());
}
return query;
}
Usage becomes clumsy and hard to read:
foreach (var per in people.OrderByMany(
new Tuple<Expression<Func<Person, object>>, SortOrder>(x => x.Age, SortOrder.Descending),
new Tuple<Expression<Func<Person, object>>, SortOrder>(x => x.Name, SortOrder.Ascending)))
{
Console.WriteLine("{0} Age={1}", per.Name, per.Age);
}

To sort by an arbitrary property, you need to build an expression tree to pass to OrderBy.
To sort by an arbitrary number of properties, you need to call ThenBy in a loop.

I like Jamiec's idea but I hate using Tuples because the syntax is ugly. Therefore I built a small class that encapsulates the Tuple and exposes getters for the Item1 and Item2 properties with better variable names.
Also notice that I used a default sort order of ascending so you only need to specify a SortOrder if you want to sort in descending order.
public class SortExpression<T>
{
private Tuple<Expression<Func<T, object>>, SortOrder> tuple;
public SortExpression( Expression<Func<T, object>> expression, SortOrder order =SortOrder.Ascending )
{
tuple = new Tuple<Expression<Func<T,object>>, SortOrder>(expression, order);
}
public Expression<Func<T, object>> Expression {
get { return tuple.Item1; }
}
public SortOrder Order {
get { return tuple.Item2; }
}
}
In my specific application, I have a repository base class which takes an IQueryable and converts it to a ObservableCollection. In that method I use the SortExpression class:
public ObservableCollection<T> GetCollection(params SortExpression<T>[] sortExpressions) {
var list = new ObservableCollection<T>();
var query = FindAll();
if (!sortExpressions.Any()) {
query.ToList().ForEach(list.Add);
return list;
}
var ordered = (sortExpressions[0].Order == SortOrder.Ascending)
? query.OrderBy(sortExpressions[0].Expression.Compile())
: query.OrderByDescending(sortExpressions[0].Expression.Compile());
for (var i = 1; i < sortExpressions.Length; i++) {
ordered = sortExpressions[i].Order == SortOrder.Ascending
? ordered.ThenBy(sortExpressions[i].Expression.Compile())
: ordered.ThenByDescending(sortExpressions[i].Expression.Compile());
}
ordered.ToList().ForEach(list.Add);
return list;
}
Here is the method in use:
var repository = new ContactRepository(UnitOfWork);
return repository.GetCollection(
new SortExpression<Contact>(x => x.FirstName),
new SortExpression<Contact>(x => x.LastName));