We Keep Coding

How to find duplicate values in List<Dictionary<string, object>>?

I have a List of Dictionary<string, object>.
How to find duplicate values in all dictionaries by value?

You can find duplicate values with their occurrences using LINQ.
It gives you duplicate values and its occurrences (index in list and key in dictionary).
var duplicates = dicList
.SelectMany((x, i) => x.Select(p => new { Index = i, Key = p.Key, Value = p.Value }))
.GroupBy(x => x.Value)
.Where(x => x.Count() > 1)
.Select(x => new
{
Value = x.First().Value,
Occurrences = x.Select(o => new { Index = o.Index, Key = o.Key })
});
If you just want duplicate values then use simplified version
var duplicates = listOfDic
.SelectMany(x => x.Values)
.GroupBy(x => x)
.Where(x => x.Count() > 1);

Old classic loop
var uniqueValues = new HashSet<object>();
var duplicateValues = new List<object>();
foreach (var value in yourDictionaries.SelectMany(dict => dict.Values))
{
if (uniqueValues.Add(value) == false)
{
duplicateValues.Add(value);
}
}
SelectMany is a key method for getting all values of all dictionaries.
If you are fan of LINQ you can convert it to the LINQ expression for example by using Aggregate or GroupBy

Use linq for compact code:
List<Dictionary<string, object>> list = new List<Dictionary<string, object>>();
list.SelectMany(dictionary => dictionary.Values).GroupBy(d => d).Where(x => x.Count() >1);

How so I take the Top N (percentage) values in a dictionary?

I have a dictionary with a string key and integer value. The value represents the number of occurrences of the key.
How do I create a new dictionary with the keys and values representing the top 25% of values? The sum of the values should be equal to or greater than the sum of all values. For example, if my dictionary contains 5 items with values (5, 3, 2, 1, 1) and I want the top 50%, the new dictionary would contain values (5, 3) because their sum is 8 and that is >= 50% of 12. This dictionary needs to be sorted descending by value and then the top N taken such that their sum meets the specified percentage.
This code gives me the top N but is based on a known count. How do I take into account the desired percentage?
var topItemsCount = dictionary.OrderByDescending(entry => entry.Value)
.Take(topN)
.ToDictionary(pair => pair.Key, pair => pair.Value);

Something like:
var topItemsCount = dictionary.OrderByDescending(entry => entry.Value)
.Take(Math.Floor(dictionary.Count * 0.25))
.ToDictionary(pair => pair.Key, pair => pair.Value);
Running .Count on a dictionary returns the number of key-value pairs in the collection. Taking Math.Floor rounds it down to the nearest int.
Edited to reflect comments
I would probably just use a simple non-linq solution to achieve what you want. Maybe more verbose, but it's pretty clear to anyone what it does:
var total = dictionary.Sum(e => e.Value);
var cutoff = total * 0.5;
var sum = 0;
var pairs = new List<KeyValuePair<string, int>>();
foreach (var pair in dictionary.OrderByDescending(e => e.Value))
{
sum += pair.Value;
pairs.Add(pair);
if (sum > cutoff)
break;
}
dictionary = pairs.ToDictionary(pair => pair.Key, pair => pair.Value);
One more edit
If you really want more linq, you could try holding an accumulated class level variable.
private static int sum = 0;
static void Main(string[] args)
{
var dictionary = new Dictionary<string, int>()
{
{"1",5},
{"2",3},
{"3",2},
{"4",1},
{"5",1},
};
var total = dictionary.Sum(e => e.Value);
var cutoff = total * 0.5;
var filtered = dictionary.OrderByDescending(e => e.Value)
.TakeWhile(e => Add(e.Value).Item1 < cutoff)
.ToDictionary(pair => pair.Key, pair => pair.Value);
}
private static Tuple<int, int> Add(int x)
{
return Tuple.Create(sum, sum += x);
}
It's a bit convoluted with the add function returning a tuple because you are including the first value that breaches the cut off in the result (i.e. even if 5 + 3 = 8 is greater than the cut off 6, you still include 3).

Rephrasing the question, into two parts:
Given a list of strings and values, find a value representing the Nth percentage
Given a list of string and values, and a value representing the Nth percentage, return a new list of string and values having values greater than or equal to the given number.
Question 1 would look like
double percent = inputValue;
double n = dictionary.Values.Sum() * percent;
Question 2 would look like:
Dictionary<string, int> newValues = dictionary.OrderByDescending(_ => _.Value)
.Aggregate(
new {sum = 0.0, values = new Dictionary<string, int>()},
(sumValues, kv) =>
{
if (sumValues.sum <= n)
sumValues.values.Add(kv.Key, kv.Value);
return new {sum = sumValues.sum + kv.Value, values = sumValues.values};
},
sumValues => sumValues.values);
You could also use a for loop and a running sum, but for running totals with limited scope, I like the compactness of the Aggregate function. The downside to this is that the entire source Dictionary is still iterated. A custom iterator method would get around this. For example:
public static class Extensions
{
public static IEnumerable<TThis> TakeGreaterThan<TThis>(this IEnumerable<TThis> source, Func<TThis, double> valueFunc, double compareTo)
{
double sum = 0.0;
IEnumerable<TThis> orderedSource = source.OrderByDescending(valueFunc);
var enumerator = orderedSource.GetEnumerator();
while (sum <= compareTo && enumerator.MoveNext())
{
yield return enumerator.Current;
sum += valueFunc(enumerator.Current);
}
}
}
Used as
Dictionary<string, int> newValues = dictionary.TakeGreaterThan(_ => _.Value, n).ToDictionary(_ => _.Key, _ => _.Value);

May be this?
var dictionary = new Dictionary<string, int>()
{
{"1",5},
{"2",3},
{"3",2},
{"4",1},
{"5",1},
};
var max = dictionary.Values.Max();
int percent = 50;
int percentageValue = max*percent /100;
var topItems = dictionary.OrderByDescending(entry => entry.Value)
.TakeWhile(x => x.Value > percentageValue)
.ToDictionary(pair => pair.Key, pair => pair.Value);
foreach (var item in topItems)
{
Console.WriteLine(item.Value);
}
Outputs:
5
3

Convert a delimited string to a dictionary<string,string>

I have a string in the format
=key1=value1=key2=value2=key3=value3
I need to convert it to a Dictionary<string,string> for the above mentioned key value pairs.
What would be the best way to go about this?
I've tried this:
var input = "key1=value1=key2=value2=key3=value3";
var dict = Regex.Matches(input , #"\s*(.*?)\s*=\s*(.*?)\s*(=|$)")
.OfType<Match>()
.ToDictionary(m => m.Groups[1].Value, m => m.Groups[2].Value);

This can be accomplished with a little Linq:
"=key1=value1=key2=value2=key3=value3"
.Split('=') // Split into an array of strings
.Skip(1) // Skip the first (empty) value
.Select((v, i) => new { v, i }) // Get value and index
.GroupBy(x => x.i / 2) // Group every pair together
.ToDictionary(g => g.First().v, // First item in group is the key
g => g.Last().v) // Last item in group is the value

var dict = new Dictionary<string,string>();
var input = str.Split(new [] { '=' },StringSplitOptions.RemoveEmptyEntries);
for(int i=0; i<input.Length; i+=2)
{
dict.Add(input[i], input[i+1]);
}

How to get values AND indices of duplicate items in a list?

I have a list of file names (targetFileList), some of which are duplicates (ex. I have two files called m4.txt). The following statement finds the duplicated filenames and adds them to another list (currentTargetFiles):
currentTargetFiles = targetFileList.FindAll(item => item == baselineFilename);
As is, this line is returning a list of strings (filenames), which is good, but I also need their index value. Is there some way to modify it so that it also returns the indices of the files?

Well, here is my answer to "find the duplicate names and their indices". It might not fit the presented problem exactly, as there is no baselineFilename considered - but that is covered by other answers. YMMV.
var names = new [] {"a", "a", "c", "b", "a", "b"};
var duplicatesWithIndices = names
// Associate each name/value with an index
.Select((Name, Index) => new { Name, Index })
// Group according to name
.GroupBy(x => x.Name)
// Only care about Name -> {Index1, Index2, ..}
.Select(xg => new {
Name = xg.Key,
Indices = xg.Select(x => x.Index)
})
// And groups with more than one index represent a duplicate key
.Where(x => x.Indices.Count() > 1);
// Now, duplicatesWithIndices is typed like:
// IEnumerable<{Name:string,Indices:IEnumerable<int>}>
// Let's say we print out the duplicates (the ToArray is for .NET 3.5):
foreach (var g in duplicatesWithIndices) {
Console.WriteLine("Have duplicate " + g.Name + " with indices " +
string.Join(",", g.Indices.ToArray()));
}
// The output for the above input is:
// > Have duplicate a with indices 0,1,4
// > Have duplicate b with indices 3,5
Of course, the provided results must be used correctly - and this depends on what must ultimately be done.

You can select all the items, with their indexes, with:
tempList = targetFileList.Select((item, index) =>
new { Value = item, Index = index }).Where(x => x.Value == baselineFilename);
Now, you can create lists of the names and corresponding indexes with:
var indexes = tempList.Select(x => x.Index).ToList();
And the values:
currentTargetFiles = tempList.Select(x => x.Value).ToList();
Then, indexes[0] will hold the list index of currentTargetFiles[0].

int i = -1;
var currentTargetFiles = targetFileList.Select(x => new
{
Value = x,
Index = i++
})
.Where(x => x.Value == baselineFilename);

Is linq a requirement?
A traditional for loop and a dictionary would do fine:
Dictionary<int, string> currentTargetFiles = new Dictionary<int, string>();
for (int i = 0; i < targetFileList.Count; ++i)
if(targetFileList[i] == baselineFilename)
currentTargetFiles.Add(i, targetFileList[i]);
P.S.:
Just realized that you comparing an exact string (item == baselineFilename).
If this is the case you don't even need to keep each value for each index (since all values are the same).
List<int> currentTargetFilesIndices = new List<int>();
for (int i = 0; i < targetFileList.Count; ++i)
if(targetFileList[i] == baselineFilename)
currentTargetFiles.Add(i);

Checking a dictionary if there are same values for a key

I have a dictionary object like this:
CustomKeys<int, string>
eg;
1000, F1
1001, F2
1002, F1
1003, F4
1004, F2
I want to know if I have more than 1 of same values in this dictionary. I would also want to keep a note of which keys(unique id) has duplicates.
Is that possible?

It is possible using GroupBy and than Count() > 1 to keep track of which values that have duplicates.
var q = dic.GroupBy(x => x.Value)
.Select (x => new { Item = x, HasDuplicates = x.Count() > 1 });

You can find all key values they had the same values like this;
Dictionary<int, string> d = new Dictionary<int, string>();
d.Add(1000, "F1");
d.Add(1001, "F2");
d.Add(1002, "F1");
d.Add(1003, "F4");
d.Add(1004, "F2");
var dublicate = d.ToLookup(x => x.Value, x => x.Key).Where(x => x.Count() > 1);
foreach (var i in dublicate)
{
Console.WriteLine(i.Key);
}
Here is a DEMO.
But if you want to get a boolean value since your item's has a same value, look at Magnus's answer which is great.

I'm not sure by what you mean by "keeping note of which has duplicate values". If you mean keeping note of the keys, you could do this:
var keys = new Dictionary<int, string>();
keys.Add(1000, "F1");
keys.Add(1001, "F2");
keys.Add(1002, "F1");
keys.Add(1003, "F4");
keys.Add(1004, "F2");
var duplicates = keys.GroupBy(i => i.Value).Select(i => new
{
keys = i.Select(x => x.Key),
value = i.Key,
count = i.Count()
});
foreach (var duplicate in duplicates)
{
Console.WriteLine("Value: {0} Count: {1}", duplicate.value, duplicate.count);
foreach (var key in duplicate.keys)
{
Console.WriteLine(" - {0}", key);
}
}
If you mean keeping track of the duplicate values only, see Sonor's answer.

Another solution could be:
var duplicates = dictionary.GroupBy( g => g.Value )
.Where( x => x.Count( ) > 1 )
.Select( x => new { Item = x.First( ), Count = x.Count( ) } )
.ToList( );