Related
I am playing with LINQ to learn about it, but I can't figure out how to use Distinct when I do not have a simple list (a simple list of integers is pretty easy to do, this is not the question). What I if want to use Distinct on a List<TElement> on one or more properties of the TElement?
Example: If an object is Person, with property Id. How can I get all Person and use Distinct on them with the property Id of the object?
Person1: Id=1, Name="Test1"
Person2: Id=1, Name="Test1"
Person3: Id=2, Name="Test2"
How can I get just Person1 and Person3? Is that possible?
If it's not possible with LINQ, what would be the best way to have a list of Person depending on some of its properties?
What if I want to obtain a distinct list based on one or more properties?
Simple! You want to group them and pick a winner out of the group.
List<Person> distinctPeople = allPeople
.GroupBy(p => p.PersonId)
.Select(g => g.First())
.ToList();
If you want to define groups on multiple properties, here's how:
List<Person> distinctPeople = allPeople
.GroupBy(p => new {p.PersonId, p.FavoriteColor} )
.Select(g => g.First())
.ToList();
Note: Certain query providers are unable to resolve that each group must have at least one element, and that First is the appropriate method to call in that situation. If you find yourself working with such a query provider, FirstOrDefault may help get your query through the query provider.
Note2: Consider this answer for an EF Core (prior to EF Core 6) compatible approach. https://stackoverflow.com/a/66529949/8155
EDIT: This is now part of MoreLINQ.
What you need is a "distinct-by" effectively. I don't believe it's part of LINQ as it stands, although it's fairly easy to write:
public static IEnumerable<TSource> DistinctBy<TSource, TKey>
(this IEnumerable<TSource> source, Func<TSource, TKey> keySelector)
{
HashSet<TKey> seenKeys = new HashSet<TKey>();
foreach (TSource element in source)
{
if (seenKeys.Add(keySelector(element)))
{
yield return element;
}
}
}
So to find the distinct values using just the Id property, you could use:
var query = people.DistinctBy(p => p.Id);
And to use multiple properties, you can use anonymous types, which implement equality appropriately:
var query = people.DistinctBy(p => new { p.Id, p.Name });
Untested, but it should work (and it now at least compiles).
It assumes the default comparer for the keys though - if you want to pass in an equality comparer, just pass it on to the HashSet constructor.
Use:
List<Person> pList = new List<Person>();
/* Fill list */
var result = pList.Where(p => p.Name != null).GroupBy(p => p.Id)
.Select(grp => grp.FirstOrDefault());
The where helps you filter the entries (could be more complex) and the groupby and select perform the distinct function.
You could also use query syntax if you want it to look all LINQ-like:
var uniquePeople = from p in people
group p by new {p.ID} //or group by new {p.ID, p.Name, p.Whatever}
into mygroup
select mygroup.FirstOrDefault();
I think it is enough:
list.Select(s => s.MyField).Distinct();
Solution first group by your fields then select FirstOrDefault item.
List<Person> distinctPeople = allPeople
.GroupBy(p => p.PersonId)
.Select(g => g.FirstOrDefault())
.ToList();
Starting with .NET 6, there is new solution using the new DistinctBy() extension in Linq, so we can do:
var distinctPersonsById = personList.DistinctBy(x => x.Id);
The signature of the DistinctBy method:
// Returns distinct elements from a sequence according to a specified
// key selector function.
public static IEnumerable<TSource> DistinctBy<TSource, TKey> (
this IEnumerable<TSource> source,
Func<TSource, TKey> keySelector);
You can do this with the standard Linq.ToLookup(). This will create a collection of values for each unique key. Just select the first item in the collection
Persons.ToLookup(p => p.Id).Select(coll => coll.First());
The following code is functionally equivalent to Jon Skeet's answer.
Tested on .NET 4.5, should work on any earlier version of LINQ.
public static IEnumerable<TSource> DistinctBy<TSource, TKey>(
this IEnumerable<TSource> source, Func<TSource, TKey> keySelector)
{
HashSet<TKey> seenKeys = new HashSet<TKey>();
return source.Where(element => seenKeys.Add(keySelector(element)));
}
Incidentially, check out Jon Skeet's latest version of DistinctBy.cs on Google Code.
Update 2022-04-03
Based on an comment by Andrew McClement, best to take John Skeet's answer over this one.
I've written an article that explains how to extend the Distinct function so that you can do as follows:
var people = new List<Person>();
people.Add(new Person(1, "a", "b"));
people.Add(new Person(2, "c", "d"));
people.Add(new Person(1, "a", "b"));
foreach (var person in people.Distinct(p => p.ID))
// Do stuff with unique list here.
Here's the article (now in the Web Archive): Extending LINQ - Specifying a Property in the Distinct Function
Personally I use the following class:
public class LambdaEqualityComparer<TSource, TDest> :
IEqualityComparer<TSource>
{
private Func<TSource, TDest> _selector;
public LambdaEqualityComparer(Func<TSource, TDest> selector)
{
_selector = selector;
}
public bool Equals(TSource obj, TSource other)
{
return _selector(obj).Equals(_selector(other));
}
public int GetHashCode(TSource obj)
{
return _selector(obj).GetHashCode();
}
}
Then, an extension method:
public static IEnumerable<TSource> Distinct<TSource, TCompare>(
this IEnumerable<TSource> source, Func<TSource, TCompare> selector)
{
return source.Distinct(new LambdaEqualityComparer<TSource, TCompare>(selector));
}
Finally, the intended usage:
var dates = new List<DateTime>() { /* ... */ }
var distinctYears = dates.Distinct(date => date.Year);
The advantage I found using this approach is the re-usage of LambdaEqualityComparer class for other methods that accept an IEqualityComparer. (Oh, and I leave the yield stuff to the original LINQ implementation...)
You can use DistinctBy() for getting Distinct records by an object property. Just add the following statement before using it:
using Microsoft.Ajax.Utilities;
and then use it like following:
var listToReturn = responseList.DistinctBy(x => x.Index).ToList();
where 'Index' is the property on which i want the data to be distinct.
You can do it (albeit not lightning-quickly) like so:
people.Where(p => !people.Any(q => (p != q && p.Id == q.Id)));
That is, "select all people where there isn't another different person in the list with the same ID."
Mind you, in your example, that would just select person 3. I'm not sure how to tell which you want, out of the previous two.
In case you need a Distinct method on multiple properties, you can check out my PowerfulExtensions library. Currently it's in a very young stage, but already you can use methods like Distinct, Union, Intersect, Except on any number of properties;
This is how you use it:
using PowerfulExtensions.Linq;
...
var distinct = myArray.Distinct(x => x.A, x => x.B);
When we faced such a task in our project we defined a small API to compose comparators.
So, the use case was like this:
var wordComparer = KeyEqualityComparer.Null<Word>().
ThenBy(item => item.Text).
ThenBy(item => item.LangID);
...
source.Select(...).Distinct(wordComparer);
And API itself looks like this:
using System;
using System.Collections;
using System.Collections.Generic;
public static class KeyEqualityComparer
{
public static IEqualityComparer<T> Null<T>()
{
return null;
}
public static IEqualityComparer<T> EqualityComparerBy<T, K>(
this IEnumerable<T> source,
Func<T, K> keyFunc)
{
return new KeyEqualityComparer<T, K>(keyFunc);
}
public static KeyEqualityComparer<T, K> ThenBy<T, K>(
this IEqualityComparer<T> equalityComparer,
Func<T, K> keyFunc)
{
return new KeyEqualityComparer<T, K>(keyFunc, equalityComparer);
}
}
public struct KeyEqualityComparer<T, K>: IEqualityComparer<T>
{
public KeyEqualityComparer(
Func<T, K> keyFunc,
IEqualityComparer<T> equalityComparer = null)
{
KeyFunc = keyFunc;
EqualityComparer = equalityComparer;
}
public bool Equals(T x, T y)
{
return ((EqualityComparer == null) || EqualityComparer.Equals(x, y)) &&
EqualityComparer<K>.Default.Equals(KeyFunc(x), KeyFunc(y));
}
public int GetHashCode(T obj)
{
var hash = EqualityComparer<K>.Default.GetHashCode(KeyFunc(obj));
if (EqualityComparer != null)
{
var hash2 = EqualityComparer.GetHashCode(obj);
hash ^= (hash2 << 5) + hash2;
}
return hash;
}
public readonly Func<T, K> KeyFunc;
public readonly IEqualityComparer<T> EqualityComparer;
}
More details is on our site: IEqualityComparer in LINQ.
If you don't want to add the MoreLinq library to your project just to get the DistinctBy functionality then you can get the same end result using the overload of Linq's Distinct method that takes in an IEqualityComparer argument.
You begin by creating a generic custom equality comparer class that uses lambda syntax to perform custom comparison of two instances of a generic class:
public class CustomEqualityComparer<T> : IEqualityComparer<T>
{
Func<T, T, bool> _comparison;
Func<T, int> _hashCodeFactory;
public CustomEqualityComparer(Func<T, T, bool> comparison, Func<T, int> hashCodeFactory)
{
_comparison = comparison;
_hashCodeFactory = hashCodeFactory;
}
public bool Equals(T x, T y)
{
return _comparison(x, y);
}
public int GetHashCode(T obj)
{
return _hashCodeFactory(obj);
}
}
Then in your main code you use it like so:
Func<Person, Person, bool> areEqual = (p1, p2) => int.Equals(p1.Id, p2.Id);
Func<Person, int> getHashCode = (p) => p.Id.GetHashCode();
var query = people.Distinct(new CustomEqualityComparer<Person>(areEqual, getHashCode));
Voila! :)
The above assumes the following:
Property Person.Id is of type int
The people collection does not contain any null elements
If the collection could contain nulls then simply rewrite the lambdas to check for null, e.g.:
Func<Person, Person, bool> areEqual = (p1, p2) =>
{
return (p1 != null && p2 != null) ? int.Equals(p1.Id, p2.Id) : false;
};
EDIT
This approach is similar to the one in Vladimir Nesterovsky's answer but simpler.
It is also similar to the one in Joel's answer but allows for complex comparison logic involving multiple properties.
However, if your objects can only ever differ by Id then another user gave the correct answer that all you need to do is override the default implementations of GetHashCode() and Equals() in your Person class and then just use the out-of-the-box Distinct() method of Linq to filter out any duplicates.
Override Equals(object obj) and GetHashCode() methods:
class Person
{
public int Id { get; set; }
public int Name { get; set; }
public override bool Equals(object obj)
{
return ((Person)obj).Id == Id;
// or:
// var o = (Person)obj;
// return o.Id == Id && o.Name == Name;
}
public override int GetHashCode()
{
return Id.GetHashCode();
}
}
and then just call:
List<Person> distinctList = new[] { person1, person2, person3 }.Distinct().ToList();
The best way to do this that will be compatible with other .NET versions is to override Equals and GetHash to handle this (see Stack Overflow question This code returns distinct values. However, what I want is to return a strongly typed collection as opposed to an anonymous type), but if you need something that is generic throughout your code, the solutions in this article are great.
List<Person>lst=new List<Person>
var result1 = lst.OrderByDescending(a => a.ID).Select(a =>new Player {ID=a.ID,Name=a.Name} ).Distinct();
You should be able to override Equals on person to actually do Equals on Person.id. This ought to result in the behavior you're after.
If you use old .NET version, where the extension method is not built-in, then you may define your own extension method:
public static class EnumerableExtensions
{
public static IEnumerable<T> DistinctBy<T, TKey>(this IEnumerable<T> enumerable, Func<T, TKey> keySelector)
{
return enumerable.GroupBy(keySelector).Select(grp => grp.First());
}
}
Example of usage:
var personsDist = persons.DistinctBy(item => item.Name);
Definitely not the most efficient but for those, who are looking for a short and simple answer:
list.Select(x => x.Id).Distinct().Select(x => list.First(y => x == y.Id)).ToList();
Please give a try with below code.
var Item = GetAll().GroupBy(x => x .Id).ToList();
Assume I have a collection of the form:
List<Member> myMembers= new List<Member>()
{
new Member(){ID=1,FirstName="John",LastName="Doe"},
new Member(){ID=3,FirstName="Allan",LastName="Jones"},
new Member(){ID=2,FirstName="Martin",LastName="Moe"},
new Member(){ID=4,FirstName="Ludwig",LastName="Issac"}
};
I can sort this list via FirstName by using:
myMembers.Sort((x, y) => x.FirstName.CompareTo(y.FirstName));
I would like to do this inside of a function, so that I can pass the desired search parameter. Something like:
public void sortCollection( parameter SearchTerm, List<Member> myCollection )
{
myCollection ((x, y) => x.SearchTerm.CompareTo(y.FirstName));
}
Obviously here, I cannot pass in the desired search field this way, but is it possible to do what I am asking?
You can create a generic extension method for that and pass Func<T, TResult> delegate to it, which is used as key selector for built-in Sort method
public static void SortExt<T, TValue>(this List<T> collection, Func<T, TValue> selector)
{
var comparer = Comparer<TValue>.Default;
collection.Sort((x, y) => comparer.Compare(selector(x), selector(y)));
}
Keep in mind that you should compare the same fields of compared objects.
Example of the usage
myMembers.SortExt(member => member.FirstName);
If you want to compare the myMembers collection only, the declaration can be simplified
public static void SortExt<TValue>(this List<Member> members, Func<Member, TValue> selector)
{
var comparer = Comparer<TValue>.Default;
members.Sort((x, y) => comparer.Compare(selector(x), selector(y)));
}
Another option is to introduce a Comparison<T> delegate instance to store the logic of field comparison and pass it to the Sort method. Here you can specify any custom comparison that you want
Comparison<Member> comparison = (x, y) => x.FirstName.CompareTo(y.FirstName);
myMembers.Sort(comparison);
Demo on dotnet fiddle
It seems to me that you can use Func<> to be able to dynamic order by.
var result = sortCollection(p => p.ID, myMembers);
public static IEnumerable<Member> sortCollection(Func<Member, object> keySelector, List<Member> myCollection)
{
return myCollection.OrderBy(keySelector);
}
Read the following post to have a better understanding
Dynamic Order By in Linq
Instead of writing one more wrapper on Linq .OrderBy() or .Sort() method and calling that where ever you want to sort, use Linq OrderBy() or .Sort() method
Use Sort() like
//If you want to sort collection by Id then
myMembers.Sort(x => x.Id);
//If you want to sort collection by FirstName then
myMembers.Sort(x => x.FirstName);
...
To decide between OrderBy()/Sort() read below Q&A thread
C# Sort and OrderBy comparison
Why not simply use the built in function?
myMembers.OrderBy(m => m.FirstName)
If you really want to, you can write your own wrapper around this function, but the call would look more or less identical and would kind of be reinventing the wheel.
something like this extension method:
public static IEnumerable<T> OrderByWrapper<T, TKey>(this IEnumerable<T> source, Func<T,TKey> keySelector)
{
return source.OrderBy(keySelector);
}
you would call it with:
myMembers.OrderByWrapper(m => m.FirstName)
I am trying to create an extension method that "extends" on an IEnumerable of a type, accepts an expression as a parameter and returns an IEnumerable of the same type.
public static IEnumerable<T> CustomExtension<T>(this IEnumerable<T> cities, Expression<Func<T, bool>> predicate)
{
return Enumerable.Where(cities, predicate);
//the line above is wrong, doesn't compile, but it explains my intentions...
}
Then call it like so:
var bigCities = cities.CustomExtension(c => c.occupants >= 1000000);
OR
var coldCities = cities.CustomExtension(c => c.avgTemp <= 20);
NOTE: It isn't just for "city" objects, the plan is to keep it generic so I can use the same method on similar types
Thanks in advance...
public static IEnumerable<T> CustomExtension<T>(this IEnumerable<T> cities, Func<T, bool> predicate)
{
return cities.Where(x => predicate(x));
}
I have attempted to create an extension method that looks like this...
public static IEnumerable<T> Distinct<T>(this IEnumerable<T> value, IEnumerable<T> compareTo, Func<T, object> compareFieldPredicate)
{
return value.Where(o => !compareTo.Exists(p => compareFieldPredicate.Invoke(p) == compareFieldPredicate.Invoke(o)));
}
The idea is that I would be able to do something like this...
IEnumerable<MyCollection> distinctValues = MyCollection.Distinct(MyOtherCollection, o => o.ID); //Note that o.ID is a guid
Now at this point I would have expected to have only my distinct items returned to me but what I found is that this was never the case.
Upon further research breaking down this method using the following code.
Guid guid1 = Guid.NewGuid();
Guid guid2 = new Guid(guid1.ToString());
Func<MyObject, object> myFunction = o => o.ID;
Func<MyObject, object> myFunction1 = o => o.ID;
bool result = myFunction(MyObject) == myFunction1(MyObject);
//result = false
I have found that infact even if the Guids are the same the comparison will always return false.
What is the cause of this?
Your problem is that you're boxing the Guids into Objects before you compare them. Consider this code:
Guid g1 = Guid.NewGuid();
var g2 = g1;
Console.WriteLine(g1 == g2);
object o1 = g1;
object o2 = g2;
Console.WriteLine(o1 == o2);
That actually outputs:
true
false
Since "o1" and "o2", while equal to the same Guid, are not the same object.
If you truly want your "Distinct" extension method to not be tied to a specific type (like Guid), you can do this:
public static IEnumerable<TItem> Distinct<TItem, TProp>(this IEnumerable<TItem> value, IEnumerable<TItem> compareTo, Func<TItem, TProp> compareFieldPredicate)
where TProp : IEquatable<TProp>
{
return value.Where(o => !compareTo.Any(p => compareFieldPredicate(p).Equals(compareFieldPredicate(o))));
}
bool result = (guid1==guid2); //result -> true
You can try to change return type Object to GUID in myfunction and myfunction1
Func<MyObject, Guid> myFunction = o => o.ID;
Func<MyObject, Guid> myFunction1 = o => o.ID;
Otherwise, the return value (true) is boxed to Object, and Reference equality is checked, which is false.
Change to use
Func<MyObject, Guid> myFunction = o => o.ID;
Func<MyObject, Guid> myFunction1 = o => o.ID;
It's because your function was defined as
Func<MyObject, object>
The Guid returned by myFunction and myFunction1 will be boxed in two different obejcts. See here for boxing and unboxing feature in .NET
Therefore, when the comparison was done, two different objects are compared.
The default implementation of Equals in object is doing reference equality check. It's not checking the boxed values. See here for more details on how object.Equals implemented.
As others have said, your compareFieldPredicate returns an object and its operator == uses object.ReferenceEquals, rather than object.Equals, so your code always checks for object identity rather than equality.
One solution to this would be to use object.Equals method instead of operator ==:
public static IEnumerable<T> Distinct<T>(
this IEnumerable<T> value,
IEnumerable<T> compareTo,
Func<T, object> compareFieldPredicate
)
{
return value.Where(o => !compareTo.Exists(
p => object.Equals(compareFieldPredicate(p), compareFieldPredicate(o))
));
}
A better solution uses the default comparer for the actual key type, eliminating boxing if the type implements IEquatable interface for itself:
public static IEnumerable<T> Distinct<T, TKey>(
this IEnumerable<T> value,
IEnumerable<T> compareTo,
Func<T, TKey> compareFieldPredicate
)
{
return value.Where(o => !compareTo.Exists(
p => EqualityComparer<TKey>.Default.Equals(compareFieldPredicate(p), compareFieldPredicate(o))
));
}
However, most of the functionality of your Distinct method is already implemented by
Enumerable.Except LINQ method.
You can rewrite your implementation in terms of Enumerable.Except by providing an implementation of IEqualityComparer:
private class KeyEqualityComparer<T, TKey> : IEqualityComparer<T>
{
private readonly Func<T, TKey> _keySelector;
public KeyEqualityComparer(Func<T, TKey> keySelector)
{ _keySelector = keySelector; }
public int GetHashCode(T item)
{ return _keySelector(item).GetHashCode(); }
public bool Equals(T x, T y)
{ return EqualityComparer<TKey>.Default.Equals(_keySelector(x), _keySelector(y)); }
}
public static IEnumerable<T> ExceptBy<T, TKey>(
this IEnumerable<T> first,
IEnumerable<T> second,
Func<T, TKey> keySelector
)
{
return first.Except(second, new KeyEqualityComparer<T, TKey>(keySelector));
}
If you change the lambda to return a Guid, then it works:
Func<MyObject, Guid> myFunction = o => o.ID;
Func<MyObject, Guid> myFunction1 = o => o.ID;
It is possible to create a Func object what references a generic method? like the LINQ OrderBy:
public static IOrderedEnumerable<TSource> OrderBy<TSource, TKey>(
this IEnumerable<TSource> source,
Func<TSource, TKey> keySelector
)
If I understand you correctly, you're asking if you can reference a generic method from within an anonymous method.
The answer is yes.
For example, suppose you want some Func that returns the elements of an IEnumerable<int> object in sorted order (precisely like OrderBy<int, int>). You could do this:
Func<IEnumerable<int>, Func<int, int>, IOrderedEnumerable<int>> orderByFunc =
System.Linq.Enumerable.OrderBy<int, int>;
Then you could use this Func just like any other:
int[] ints = new int[] { 1, 3, 5, 4, 7, 2, 6, 9, 8 };
// here you're really calling OrderBy<int, int> --
// you've just stored its address in a variable of type Func<...>
foreach (int i in orderByFunc(ints, x => x))
Console.WriteLine(i);
Output:
1
2
3
4
5
6
7
8
9
On the other hand, if you're asking whether it's possible to create a "generic anonymous method," like this:
Func<T> getDefault<T> = () => default(T);
Then it depends on your context. This can be done from within a context where T is already declared as a generic type parameter -- namely, within a generic class or generic method. (See Freddy Rios's answer.) Outside of such a context, unfortunately, it is illegal.
Yes, but it depends on the context - if you are already working with generics, just use the T in the context / if not, then you already know the specific type. In the later, if you need to reuse a bit of logic on a method, u probably already would benefit of moving that into a method, so just do like my second example below.
2 samples:
public T Something<T>() {
Func<T> someFunc = () => { return default(T); };
return someFunc();
}
public Func<T> GetDefaultCreator<T>() {
return () => { return default(T); };
}
Something like this?
Func<Nullable<int>, string> myFunc = c => c.HasValue ? c.ToString() : "null";
That successfully compiles, and you could assign any function to that that takes in a Nullable and returns a string.
I have done something like this:
public static class Helper{
public static IEnumerable<KeyValuePair<string, string>> ToPairs(this NameValueCollection Form)
{
return Form.AllKeys.Cast<string>()
.Select(key => new KeyValuePair<string, string>(key, Form[key]));
}
}
Where this method has become an extension method to the request.form in C# web development.
I think I get it: Given the function static TResult DoSomeStuff<T, TResult>(T obj), can you create a Func<T, TResult> such that it will reference the function above, with no type parameters given at the creation of the reference to it.
I think this could work (You're welcome to test it, I have no C# near me at the moment):
class UselessClass<T, TResult>
{
// If it's a static method, this is fine:
public Func<T, TResult> DaFunc = RelevantClass.DoSomeStuff<T, TResult>;
// If not, something like this is needed:
public UselessClass(SomeClassWhereTheFunctionIs from)
{
DaFunc = from.DoSomeStuff<T, TResult>;
}
}
Also, in OrderBy, it's not actually a generic delegate. It's a declaration of a variable. When the function is given to it, the types are inferred from it.
Yes it's possible but you'll need to specify the type argument(s)
func<int> f = myClass.returnsT<int>;
where
class myClass
{
T returnsT<T>()
{...}
}
it Will not work without the type arguments