We Keep Coding

Fastest way to count number of distinct elements in array

I have a class of object like this:
public class Individual
{
public double[] Number { get; set; } = new double[2]{ 0.0, 0.0 };
}
I stock these class in a list of dictionary and give values for Individual.Number:
selection = List<Dictionary<int, Individual>>
Now, I have to count the number of distinct values of Individual.Number (in the whole list). What I've done so far is:
selection.Values.SelectMany(list => list.Number).Distinct().Count();
I wonder if this is the fastest way to count? How can I improve the performance?
Thanks,

Internally the Distinct() method creates a new Set<T> without specifiying the size.
If you have a vague idea of the number of elements, this can prevent a number of allocations (and memory moves).
And since you only want the Count() You can include that directly (Credits #TimSchmelter).
public static int OptimizedDistinctAndCount<TSource>(this IEnumerable<TSource> source, int numberOfElements) {
if (source == null) throw Error.ArgumentNull("source");
var set = new HashSet<TSource>(numberOfElements);
foreach (TSource element in source) {
set.Add(element);
}
return set.Count;
}
You could then use:
selection.Values.SelectMany(list => list.Number).OptimizedDistinctAndCount(123);

What do you think about this?
public class Individual
{
public double[] Numbers { get; set; }
public Individual()
{
Numbers = new double[0];
}
public Individual(double[] values)
{
Numbers = values/*.ToArray() if a copy must be done*/;
}
}
class Program
{
static void Main()
{
// Populate data
var selection = new List<Dictionary<int, Individual>>();
var dico1 = new Dictionary<int, Individual>();
var dico2 = new Dictionary<int, Individual>();
selection.Add(dico1);
selection.Add(dico2);
dico1.Add(1, new Individual(new double[] { 1.2, 1.3, 4.0, 10, 40 }));
dico1.Add(2, new Individual(new double[] { 1.2, 1.5, 4.0, 20, 40 }));
dico2.Add(3, new Individual(new double[] { 1.7, 1.6, 5.0, 30, 60 }));
// Count distinct
var found = new List<double>();
foreach ( var dico in selection )
foreach ( var item in dico )
foreach ( var value in item.Value.Numbers )
if ( !found.Contains(value) )
found.Add(value);
// Must show 12
Console.WriteLine("Distinct values of the data pool = " + found.Count);
Console.ReadKey();
}
}
This approach eliminates some calling methods timings.
A further optimization would use for loops instead of foreach, and perhaps using a chained list instead of List (faster but requires more memory).

creating array of bad names to check and replace in c#

I'm looking to create a method that loops through an list and replaces with matched values with a new value. I have something working below but it really doesnt follow the DRY principal and looks ugly.
How could I create a dictionary of value pairs that would hold my data of values to match and replace?
var match = acreData.data;
foreach(var i in match)
{
if (i.county_name == "DE KALB")
{
i.county_name = "DEKALB";
}
if (i.county_name == "DU PAGE")
{
i.county_name = "DUPAGE";
}
}

In your question, you can try to use linq and Replace to make it.
var match = acreData.data.ToList();
match.ForEach(x =>
x.county_name = x.county_name.Replace(" ", "")
);
or you can try to create a mapper table to let your data mapper with your value. as #user2864740 say.
Dictionary<string, string> dict = new Dictionary<string, string>();
dict.Add("DE KALB", "DEKALB");
dict.Add("DU PAGE", "DUPAGE");
var match = acreData.data;
string val = string.Empty;
foreach (var i in match)
{
if (dict.TryGetValue(i.county_name, out val))
i.county_name = val;
}

If this were my problem and it is possible a county could have more than one common misspelling I would create a class to hold the correct name and the common misspellings. The you could easily determine if the misspelling exists and correct if. Something like this:
public class County
{
public string CountyName { get; set; }
public List<string> CommonMisspellings { get; set; }
public County()
{
CommonMisspellings = new List<string>();
}
}
Usage:
//most likely populate from db
var counties = new List<County>();
var dekalb = new County { CountyName = "DEKALB" };
dekalb.CommonMisspellings.Add("DE KALB");
dekalb.CommonMisspellings.Add("DE_KALB");
var test = "DE KALB";
if (counties.Any(c => c.CommonMisspellings.Contains(test)))
{
test = counties.First(c => c.CommonMisspellings.Contains(test)).CountyName;
}

If you are simply replacing all words in a list containing space without space, then can use below:
var newList = match.ConvertAll(word => word.Replace(" ", ""));
ConvertAll returns a new list.
Also, I suggest not to use variable names like i, j, k etc..but use temp etc.
Sample code below:
var oldList = new List<string> {"DE KALB", "DE PAGE"};
var newList = oldList.ConvertAll(word => word.Replace(" ", ""));

We can try removing all the characters but letters and apostroph (Cote d'Ivoire has it)
...
i.country_name = String.Concat(i.country_name
.Where(c => char.IsLetter(c) || c == '\''));
...

I made a comment under answer of #Kevin and it seems it needs further explanation. Sequential searching in list does not scale well and unfortunately for Kevin, that is not my opinion, asymptotic computational complexity is math. While searching in dictionary is more or less O(1), searching in list is O(n). To show a practical impact for solution with 100 countries with 100 misspellings each, lets make a test
public class Country
{
public string CountryName { get; set; }
public List<string> CommonMisspellings { get; set; }
public Country()
{
CommonMisspellings = new List<string>();
}
}
static void Main()
{
var counties = new List<Country>();
Dictionary<string, string> dict = new Dictionary<string, string>();
Random rnd = new Random();
List<string> allCountryNames = new List<string>();
List<string> allMissNames = new List<string>();
for (int state = 0; state < 100; ++state)
{
string countryName = state.ToString() + rnd.NextDouble();
allCountryNames.Add(countryName);
var country = new Country { CountryName = countryName };
counties.Add(country);
for (int miss = 0; miss < 100; ++miss)
{
string missname = countryName + miss;
allMissNames.Add(missname);
country.CommonMisspellings.Add(missname);
dict.Add(missname, countryName);
}
}
List<string> testNames = new List<string>();
for (int i = 0; i < 100000; ++i)
{
if (rnd.Next(20) == 1)
{
testNames.Add(allMissNames[rnd.Next(allMissNames.Count)]);
}
else
{
testNames.Add(allCountryNames[rnd.Next(allCountryNames.Count)]);
}
}
System.Diagnostics.Stopwatch st = new System.Diagnostics.Stopwatch();
st.Start();
List<string> repairs = new List<string>();
foreach (var test in testNames)
{
if (counties.Any(c => c.CommonMisspellings.Contains(test)))
{
repairs.Add(counties.First(c => c.CommonMisspellings.Contains(test)).CountryName);
}
}
st.Stop();
Console.WriteLine("List approach: " + st.ElapsedMilliseconds.ToString() + "ms");
st = new System.Diagnostics.Stopwatch();
st.Start();
List<string> repairsDict = new List<string>();
foreach (var test in testNames)
{
if (dict.TryGetValue(test, out var val))
{
repairsDict.Add(val);
}
}
st.Stop();
Console.WriteLine("Dict approach: " + st.ElapsedMilliseconds.ToString() + "ms");
Console.WriteLine("Repaired count: " + repairs.Count
+ ", check " + (repairs.SequenceEqual(repairsDict) ? "OK" : "ERROR"));
Console.ReadLine();
}
And the result is
List approach: 7264ms
Dict approach: 4ms
Repaired count: 4968, check OK
List approach is about 1800x slower, actually more the thousand times slower in this case. The results are as expected. If that is a problem is another question, it depends on concrete usage pattern in concrete application and is out of scope of this post.

Pick Random string from List<string> with exclusions and non-repetitive pick

im trying to write a program that would let a user:
Load a set of string.
Loop through the set, and pick another string from the same set.
Avoid a picked string from being picked again.
Have specific strings not be able to pick specified strings.
Example is table below:
And below table is a sample scenario:
How am i supposed to do this the easy way?
i have below code, but it is taking like forever to generate a valid set since it restarts everything if there are nothing left to pick, while not eliminating the possibility.
private List<Participant> _participants;
AllOverAgain:
var pickedParticipants = new List<Participant>();
var participantPicks = new List<ParticipantPick>();
foreach(var participant in _participants)
{
var pickedParticipantNames = from rp in participantPicks select rp.PickedParticipant;
var picks = (from p in _participants where p.Name != participant.Name & !Utilities.IsInList(p.Name, pickedParticipantNames) select p).ToList();
var pick = picks[new Random().Next(0, picks.Count())];
if(pick == null)
{
UpdateStatus($"No Available Picks left for {participant.Name}, Restarting...");
goto AllOverAgain;
}
var exclusions = participant.Exclusions.Split(',').Select(p => p.Trim()).ToList();
if(exclusions.Contains(pick.Name))
{
UpdateStatus($"No Available Picks left for {participant.Name}, Restarting...");
goto AllOverAgain;
}
participantPicks.Add(new ParticipantPick(participant.Name, pick.Name, participant.Number));
}
return participantPicks; // Returns the final output result
The Participant Class consists of these Properties:
public string Name { get; set; }
public string Number { get; set; }
public string Exclusions { get; set; }
The ParticipantPick Class consists of these Properties:
public string Participant { get; set; }
public string PickedParticipant { get; set; }
public string Number { get; set; }

One way you can solve this is by using a dictionary, using a composite key of a tuple and the matching value of a datatype bool.
Dictionary<Tuple<string, string>, bool>
The composite key Tuple<sring,string> will contain every permutation of participants and match them to their appropriate bool value.
For example, the dictionary filled with values such as:
Dictionary<Tuple<"Judith","James">, true>
...would be indicating that Judith picking James is valid.
So lets create a dictionary with every single possible combination of participants, and set the value of them to true for them being valid at the start of the program.
This can be accomplished by a cartesian join using an array with itself.
Dictionary<Tuple<string, string>, bool> dictionary = participants.SelectMany(left => participants, (left, right) => new Tuple<string, string>(left, right)).ToDictionary(item=> item, item=>true);
After getting every permutation of possible picks and setting them to true, we can go through the "not allowed to pick" lists and change the dictionary value for that composite key to false.
dictionary[new Tuple<string, string>(personNotAllowing, notAllowedPerson)] = false;
You can remove a participant from picking itself by using a loop in the following way:
for(int abc=0;abc<participants.Length;abc++)
{
//remove clone set
Tuple<string, string> clonePair = Tuple.Create(participants[abc], participants[abc]);
dictionary.Remove(clonePair);
}
Or by simply changing the value of the clone pair to false.
for(int abc=0;abc<participants.Length;abc++)
{
dictionary[Tuple.Create(participants[abc],participants[abc])] = false;
}
In this example program, I create a string[] of participants, and a string[] for the respective list of people they do not allow. I then perform a cartesian join, the participants array with itself. This leads to every permutation, with an initial true boolean value.
I change the dictionary where the participants are not allowed to false, and display the example dictionary.
Afterward, I create 10 instances of random participants who are picking other random participants and test if it would be valid.
Every time a participant picks another participant, I check that composite key to see if it has a value of true.
If it does result in a valid pick, then every combination of the resulting participant who was picked gets set to false.
for(int j=0; j<participants.Length;j++)
{
//Make the partner never be able to be picked again
Tuple<string, string> currentPair2 = Tuple.Create(partner, participants[j]);
try
{
dictionary[currentPair2] = false;
}
catch
{
}
}
This concept is better illustrated with running the code.
The demo:
static void Main(string[] args)
{
//Create participants set
string[] participants = {"James","John","Tyrone","Rebecca","Tiffany","Judith"};
//Create not allowed lists
string[] jamesNotAllowedList = {"Tiffany", "Tyrone"};
string[] johnNotAllowedList = {};
string[] tyroneNotAllowedList = {};
string[] rebeccaNotAllowedList ={"James", "Tiffany"};
string[] judithNotAllowedList = {};
//Create list of not allowed lists
string[][] notAllowedLists = { jamesNotAllowedList, johnNotAllowedList, tyroneNotAllowedList, rebeccaNotAllowedList, judithNotAllowedList};
//Create dictionary<Tuple<string,string>, bool> from participants array by using cartesian join on itself
Dictionary<Tuple<string, string>, bool> dictionary = participants.SelectMany(left => participants, (left, right) => new Tuple<string, string>(left, right)).ToDictionary(item=> item, item=>true);
//Loop through each person who owns a notAllowedList
for (int list = 0; list < notAllowedLists.Length; list++)
{
//Loop through each name on the not allowed list
for (int person = 0; person<notAllowedLists[list].Length; person++)
{
string personNotAllowing = participants[list];
string notAllowedPerson = notAllowedLists[list][person];
//Change the boolean value matched to the composite key
dictionary[new Tuple<string, string>(personNotAllowing, notAllowedPerson)] = false;
Console.WriteLine(personNotAllowing + " did not allow " + notAllowedPerson);
}
}
//Then since a participant cant pick itself
for(int abc=0;abc<participants.Length;abc++)
{
//remove clone set
Tuple<string, string> clonePair = Tuple.Create(participants[abc], participants[abc]);
dictionary.Remove(clonePair);
}
//Display whats going on with this Dictionary<Tuple<string,string>, bool>
Console.WriteLine("--------Allowed?--Dictionary------------\n");
Console.WriteLine(string.Join(" \n", dictionary));
Console.WriteLine("----------------------------------------\n\n");
//Create Random Object
Random rand = new Random();
//Now that the data is organized in a dictionary..
//..Let's have random participants pick random participants
//For this demonstration lets try it 10 times
for (int i=0;i<20;i++)
{
//Create a new random participant
int rNum = rand.Next(participants.Length);
string randomParticipant = participants[rNum];
//Random participant picks a random participant
string partner = participants[rand.Next(participants.Length)];
//Create composite key for the current pair
Tuple<string, string> currentPair = Tuple.Create(partner,randomParticipant);
//Check if it's a valid choice
try
{
if (dictionary[currentPair])
{
Console.WriteLine(randomParticipant + " tries to pick " + partner);
Console.WriteLine("Valid.\n");
//add to dictionary
for(int j=0; j<participants.Length;j++)
{
//Make the partner never be able to be picked again
Tuple<string, string> currentPair2 = Tuple.Create(partner, participants[j]);
try
{
dictionary[currentPair2] = false;
}
catch
{
}
}
}
else
{
Console.WriteLine(randomParticipant + " tries to pick " + partner);
Console.WriteLine(">>>>>>>>Invalid.\n");
}
}
catch
{
//otherwise exception happens because the random participant
//And its partner participant are the same person
//You can also handle the random participant picking itself differently
//In this catch block
//Make sure the loop continues as many times as necessary
//by acting like this instance never existed
i = i - 1;
}
}
Console.ReadLine();
}

This code will always give you output that adheres to your criteria:
public static class Program
{
public static void Main(string[] args)
{
var gathering = new Gathering();
gathering.MakeSelections();
foreach (var item in gathering.participants)
{
Console.WriteLine(item.name + ":" + item.selectedParticipant);
}
}
public class Participant
{
public string name;
public List<string> exclusions;
public string selectedParticipant;
}
public class Gathering
{
public List<Participant> participants;
public List<string> availableParticipants;
public List<string> usedNames;
public Dictionary<string, string> result;
public Gathering()
{
//initialize participants
participants = new List<Participant>();
participants.Add(new Participant
{
name = "James",
exclusions = new List<string> { "Tiffany", "Tyrone" }
});
participants.Add(new Participant
{
name = "John",
exclusions = new List<string> { }
});
participants.Add(new Participant
{
name = "Judith",
exclusions = new List<string> { }
});
participants.Add(new Participant
{
name = "Rebecca",
exclusions = new List<string> { "James", "Tiffany" }
});
participants.Add(new Participant
{
name = "Tiffany",
exclusions = new List<string> { }
});
participants.Add(new Participant
{
name = "Tyrone",
exclusions = new List<string> { }
});
//prevent participants from selecting themselves
foreach (Participant p in participants)
{
p.exclusions.Add(p.name);
}
//create list of all the names (all available participants at the beginning)
availableParticipants = participants.Select(x => x.name).ToList();
}
public void MakeSelections()
{
Participant currentParticipant;
Random randy = new Random();
//Sort Participants by the length of their exclusion lists, in descending order.
participants.Sort((p1, p2) => p2.exclusions.Count.CompareTo(p1.exclusions.Count));
//Get the first participant in the list which hasn't selected someone yet
currentParticipant = participants.FirstOrDefault(p => p.selectedParticipant == null);
while (currentParticipant != null)
{
//of the available participants, create a list to choose from for the current participant
List<string> listToChooseFrom = availableParticipants.Where(x => !currentParticipant.exclusions.Contains(x)).ToList();
//select a random participant from the list of eligible ones to be matched with the current participant
string assignee = listToChooseFrom[randy.Next(listToChooseFrom.Count)];
currentParticipant.selectedParticipant = assignee;
//remove the selected participant from the list of available participants
availableParticipants.RemoveAt(availableParticipants.IndexOf(assignee));
//remove the selected participant from everyone's exclusion lists
foreach (Participant p in participants)
if (p.exclusions.Contains(assignee))
p.exclusions.RemoveAt(p.exclusions.IndexOf(assignee));
//Resort Participants by the length of their exclusion lists, in descending order.
participants.Sort((p1, p2) => p2.exclusions.Count.CompareTo(p1.exclusions.Count));
//Get the first participant in the list which hasn't selected someone yet
currentParticipant = participants.FirstOrDefault(p => p.selectedParticipant == null);
}
//finally, sort by alphabetical order
participants.Sort((p1, p2) => p1.name.CompareTo(p2.name));
}
}
}

In the simpler version, the items can just be shuffled:
string[] source = { "A", "B", "C", "D", "E", "F" };
string[] picked = source.ToArray(); // copy
var rand = new Random();
for (int i = source.Length - 1, r; i > 0; --i)
{
var pick = picked[r = rand.Next(i)]; // pick random item less than the current one
picked[r] = picked[i]; // and swap with the current one
picked[i] = pick;
Console.WriteLine(i + " swapped with " + r);
}
Console.WriteLine("\nsource: " + string.Join(", ", source) +
"\npicked: " + string.Join(", ", picked));
sample result:
5 swapped with 4
4 swapped with 2
3 swapped with 0
2 swapped with 1
1 swapped with 0
source: A, B, C, D, E, F
picked: F, D, B, A, C, E
or, the source can be optionally shuffled, and each person can pick the person that is next in the list.

Pass by reference issue using a delegate inside a loop

I have the following code which creates and returns a Dictionary<int, List<string>>
private static readonly Random rnd = new Random();
private static Dictionary<int, List<string>> GenerateValues(string entityType, int rows)
{
var values = new Dictionary<int, List<string>>();
var baseValues = new List<string> { "99 Hellesdon Road","Home","Norfolk","NR6 5EG","Norwich" };
Action<int> rowLoop = null;
switch (entityType)
{
case "Employer":
rowLoop = (row) =>
{
var employer = new List<string>
{
$"Employer {rnd.Next(10000, 99999)}",
$"{rnd.Next(100000000, 999999999)}",
"TRUE"
};
var rowValue = baseValues;
rowValue.AddRange(employer);
values.Add(row, rowValue);
};
break;
case "AssessmentCentre":
rowLoop = (row) =>
{
var rowValue = baseValues;
rowValue.Add($"Assessment Centre {rnd.Next(10000, 99999)}");
values.Add(row, rowValue);
};
break;
}
for (int i = 1; i <= rows; i++)
{
rowLoop(i);
}
return values;
}
And then to call it with...
var spreadsheetValues = GenerateValues("Employer", 5);
Then intention for the code is to have some base values stored in a List<string>, then depending on what entityType is passed into the method, add some additional values to those base values. Then loop round a certain number of rows to create the Dictionary<int, List<string>>, where int is the row number.
The issue I'm having is that baseValues is being updated during each iteration of my for loop. I didn't expect this when using var rowValue = baseValues;
This is a debug screenshot of the last item in the collection....
I am expecting each List<string> in Dictionary<int, List<string>> to contain 8 items, but instead it is growing with each iteration.
Why is this occurring and how might I correct it?

It seems like you are already on track with the solution to your problem but here is a way to solve it.
Where you write
var rowValue = baseValues;
you are really just making "rowValue" a pointer to "baseValues". To make a copy, you could do something like this:
var rowValue = baseValues.ToList();

C# How to work-around identical keys in a Dictionary

This program add's values with their other values into a dictionary, all is fine until there are identical keys (var eat(.8) and var extra(.8) with different values. How do i ensure that i can use the right key every time even though they are similar? For example, var example = gigantDictionary[.8] (but i want var edmg value instead of '500' in the code?
var wqat = 1.1; //| index 0
var rat = .2; //| index 1
var eat = .8; //| index 2
var baat = 1.2; //| index 3
var extra = .8; //| index 4
var wqdmg = 120; //| index 0
var rdmg = 60; //| index 1
var edmg = 50; //| index 2
var badmg = 40; //| index 3
var extradmg = 500; //| index 4
List<double> theOneList = new List<double>();
List<double> damageList = new List<double>();
theOneList.Add(wqat);
theOneList.Add(rat);
theOneList.Add(eat);
theOneList.Add(baat);
damageList.Add(wqdmg);
damageList.Add(edmg);
damageList.Add(rdmg);
damageList.Add(badmg);
Dictionary<double, double> gigantDictionary = new Dictionary<double, double>();
for (int i = 0; i < theOneList.Count; i++)
{
gigantDictionary.Add(theOneList[i], damageList[i]);
gigantDictionary.Add(extra, 500); //this is the malignant similar key
}
theOneList.Sort((c, p) => -c.CompareTo(p)); //orders the list
List<double> finalList = new List<double>();
for (int i = 0; i < theOneList.Count; i++)
{
finalList.Add(gigantDictionary[theOneList[i]]); //grabs damage values and add's it to 'finalList'
Console.WriteLine(finalList[i]);
}
So ultimately, i want to order 'theOneList' by descent, in doing so i can get the damages from 'gigantDictionary' and put those into 'finalList', now i have an ordered damage list that i need, but since 2 keys are similar... this is holding me back.
*Edit: Could identical indexes be the key to this? be the bridge? for example, in index 0 i get 1.1 and 120, maybe the answer lies with the identical indexes, i want to get '120' damage from '1.1', notice both have index 0, this might work

They keys aren't "similar" they're "identical". If the keys were just "similar" then, as far as the dictionary is concerned, it's no different than being "completely different". From the point of view of a dictionary items are either equal, or not equal.
For example,
var example = gigantDictionary[.8]
(but i want var edmg value instead of '500' in the code?)
But how should the dictionary possibly know that? How would it know if you actually wanted to get 500?
Do you want to prevent the duplicate keys from being added, and instead always use the first value paired with every key? If so, just check if a key exists before adding a new one.
Do you want to just get all values associated with a key, if there are duplicates? Then have a dictionary where the value is a collection, and add all values associated with that one key to the collection.
Is there actually some way to distinguish the keys so that they're not actually identical? Then do that. With just a double (which is a very bad type to use as a key for a dictionary in the first place, as it's easy for floating point rounding errors to result in similar but different doubles that you consider equivalent) there's no good way to do this, but if your actual key could be different in such a way that distinguishes the two keys, then each could point to a unique value.

Right now you have two separate list for values that must go together. A better approach is to create a structure with the two values and keep a single list.
public class Thing
{
public string Name { get; set; }
public double TheOne { get; set; }
public double Dmg { get; set; }
}
class Program
{
static void Main(string[] args)
{
List<Thing> list=new List<Thing>() {
new Thing() { Name = "wq", TheOne = 1.1, Dmg=120 },
new Thing() { Name = "r", TheOne = 0.2, Dmg=60 },
new Thing() { Name = "e", TheOne = 0.8, Dmg=50 },
new Thing() { Name = "ba", TheOne = 1.2, Dmg=40 },
new Thing() { Name = "extra", TheOne = 0.8, Dmg=500 },
};
list.Sort((t1, t2) => t1.TheOne.CompareTo(t2.TheOne));
double[] dmg_list=list.Select((t) => t.Dmg).ToArray();
}
}
Edit 1
A constructor to Thing can be used to assign values with one operation.
public class Thing
{
// Constructor sets all the values
public Thing(string name, double theone, double dmg)
{
this.Name=name;
this.TheOne=theone;
this.Dmg=dmg;
}
public string Name { get; private set; }
public double TheOne { get; private set; }
public double Dmg { get; private set; }
}
class Program
{
static void Main(string[] args)
{
List<Thing> list=new List<Thing>();
list.Add(new Thing("wq", 1.1, 120));
list.Add(new Thing("r", 0.2, 60));
list.Add(new Thing("e", 0.8, 50));
list.Add(new Thing("ba", 1.2, 40));
list.Add(new Thing("extra", 0.8, 500));
list.Sort((t1, t2) => t1.TheOne.CompareTo(t2.TheOne));
double[] dmg_list=list.Select((t) => t.Dmg).ToArray();
}
}