We Keep Coding

algorithm for selecting N random elements from a List<T> in C# [duplicate]

This question already has answers here:
Randomize a List<T>
(28 answers)
Closed 6 years ago.
I need a quick algorithm to select 4 random elements from a generic list. For example, I'd like to get 4 random elements from a List and then based on some calculations if elements found not valid then it should again select next 4 random elements from the list.

You could do it like this
public static class Extensions
{
public static Dictionary<int, T> GetRandomElements<T>(this IList<T> list, int quantity)
{
var result = new Dictionary<int, T>();
if (list == null)
return result;
Random rnd = new Random(DateTime.Now.Millisecond);
for (int i = 0; i < quantity; i++)
{
int idx = rnd.Next(0, list.Count);
result.Add(idx, list[idx]);
}
return result;
}
}
Then use the extension method like this:
List<string> list = new List<string>() { "a", "b", "c", "d", "e", "f", "g", "h" };
Dictionary<int, string> randomElements = list.GetRandomElements(3);
foreach (KeyValuePair<int, string> elem in randomElements)
{
Console.WriteLine($"index in original list: {elem.Key} value: {elem.Value}");
}

something like that:
using System;
using System.Collections.Generic;
public class Program
{
public static void Main()
{
var list = new List<int>();
list.Add(1);
list.Add(2);
list.Add(3);
list.Add(4);
list.Add(5);
int n = 4;
var rand = new Random();
var randomObjects = new List<int>();
for (int i = 0; i<n; i++)
{
var index = rand.Next(list.Count);
randomObjects.Add(list[index]);
}
}
}

You can store indexes in some list to get non-repeated indexes:
List<T> GetRandomElements<T>(List<T> allElements, int randomCount = 4)
{
if (allElements.Count < randomCount)
{
return allElements;
}
List<int> indexes = new List<int>();
// use HashSet if performance is very critical and you need a lot of indexes
//HashSet<int> indexes = new HashSet<int>();
List<T> elements = new List<T>();
Random random = new Random();
while (indexes.Count < randomCount)
{
int index = random.Next(allElements.Count);
if (!indexes.Contains(index))
{
indexes.Add(index);
elements.Add(allElements[index]);
}
}
return elements;
}
Then you can do some calculation and call this method:
void Main(String[] args)
{
do
{
List<int> elements = GetRandomelements(yourElements);
//do some calculations
} while (some condition); // while result is not right
}

Suppose that the length of the List is N. Now suppose that you will put these 4 numbers in another List called out. Then you can loop through the List and the probability of the element you are on being chosen is
(4 - (out.Count)) / (N - currentIndex)

funcion (list)
(
loop i=0 i < 4
index = (int) length(list)*random(0 -> 1)
element[i] = list[index]
return element
)
while(check == false)
(
elements = funcion (list)
Do some calculation which returns check == false /true
)
This is the pseudo code, but i think you should of come up with this yourself.
Hope it helps:)

All the answers up to now have one fundamental flaw; you are asking for an algorithm that will generate a random combination of n elements and this combination, following some logic rules, will be valid or not. If its not, a new combination should be produced. Obviously, this new combination should be one that has never been produced before. All the proposed algorithms do not enforce this. If for example out of 1000000 possible combinations, only one is valid, you might waste a whole lot of resources until that particular unique combination is produced.
So, how to solve this? Well, the answer is simple, create all possible unique solutions, and then simply produce them in a random order. Caveat: I will suppose that the input stream has no repeating elements, if it does, then some combinations will not be unique.
First of all, lets write ourselves a handy immutable stack:
class ImmutableStack<T> : IEnumerable<T>
{
public static readonly ImmutableStack<T> Empty = new ImmutableStack<T>();
private readonly T head;
private readonly ImmutableStack<T> tail;
public int Count { get; }
private ImmutableStack()
{
Count = 0;
}
private ImmutableStack(T head, ImmutableStack<T> tail)
{
this.head = head;
this.tail = tail;
Count = tail.Count + 1;
}
public T Peek()
{
if (this == Empty)
throw new InvalidOperationException("Can not peek a empty stack.");
return head;
}
public ImmutableStack<T> Pop()
{
if (this == Empty)
throw new InvalidOperationException("Can not pop a empty stack.");
return tail;
}
public ImmutableStack<T> Push(T item) => new ImmutableStack<T>(item, this);
public IEnumerator<T> GetEnumerator()
{
var current = this;
while (current != Empty)
{
yield return current.head;
current = current.tail;
}
}
IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}
This will make our life easier while producing all combinations by recursion. Next, let's get the signature of our main method right:
public static IEnumerable<IEnumerable<T>> GetAllPossibleCombinationsInRandomOrder<T>(
IEnumerable<T> data, int combinationLength)
Ok, that looks about right. Now let's implement this thing:
var allCombinations = GetAllPossibleCombinations(data, combinationLength).ToArray();
var rnd = new Random();
var producedIndexes = new HashSet<int>();
while (producedIndexes.Count < allCombinations.Length)
{
while (true)
{
var index = rnd.Next(allCombinations.Length);
if (!producedIndexes.Contains(index))
{
producedIndexes.Add(index);
yield return allCombinations[index];
break;
}
}
}
Ok, all we are doing here is producing random indexees, checking we haven't produced it yet (we use a HashSet<int> for this), and returning the combination at that index.
Simple, now we only need to take care of GetAllPossibleCombinations(data, combinationLength).
Thats easy, we'll use recursion. Our bail out condition is when our current combination is the specified length. Another caveat: I'm omitting argument validation throughout the whole code, things like checking for null or if the specified length is not bigger than the input length, etc. should be taken care of.
Just for the fun, I'll be using some minor C#7 syntax here: nested functions.
public static IEnumerable<IEnumerable<T>> GetAllPossibleCombinations<T>(
IEnumerable<T> stream, int length)
{
return getAllCombinations(stream, ImmutableStack<T>.Empty);
IEnumerable<IEnumerable<T>> getAllCombinations<T>(IEnumerable<T> currentData, ImmutableStack<T> combination)
{
if (combination.Count == length)
yield return combination;
foreach (var d in currentData)
{
var newCombination = combination.Push(d);
foreach (var c in getAllCombinations(currentData.Except(new[] { d }), newCombination))
{
yield return c;
}
}
}
}
And there we go, now we can use this:
var data = "abc";
var random = GetAllPossibleCombinationsInRandomOrder(data, 2);
foreach (var r in random)
{
Console.WriteLine(string.Join("", r));
}
And sure enough, the output is:
bc
cb
ab
ac
ba
ca

Convert a for loop nested in a ForEach loop to LINQ

I'm getting a compilation error "not all code paths return a value" on the following code, How come?!
public class SomeEntity
{
public int m_i;
public SomeEntity(int i)
{
m_i = i;
}
public override string ToString()
{
return m_i.ToString();
}
public static int someFunction(int i) { return i + 100; }
public static IEnumerable GetEntities()
{
int [] arr = {1,2,3};
foreach (int i in arr)
{
// for (int i = 0; i < someArray.Count();i++)
// yield return new SomeEntity(someFunction(i));
// *** Equivalent linq function ***
return Enumerable.Range(0, 7).Select(a => new SomeEntity(someFunction(a)));
}
}
}
I can't seem to figure this out.....
I tried converting the outer foreach loop to a linq expression
public static IEnumerable GetEntities()
{
int [] arr = {1,2,3};
return arr.Select(Xenv =>
Enumerable.Range(0, 7).Select(a => new SomeEntity(someFunction(a)))
);
}
but then I just got an error :/

Because it is possible that arr is empty and you'll not return inside the foreach loop. Put a return after the foreach loop.
public static IEnumerable GetEntities()
{
int[] arr = { 1, 2, 3 };
foreach (int i in arr)
{
// for (int i = 0; i < someArray.Count();i++)
// yield return new SomeEntity(someFunction(i));
// *** Equivalent linq function ***
return Enumerable.Range(0, 7).Select(a => new SomeEntity(someFunction(a)));
}
return Enumerable.Empty<int>(); // <<<< this is what you need
}

The yield code you are replacing returned IEnumrable<SomeEntity>
while the new code returns IEnumarable<IEnumrable<SomeEntity>>.
you can use SelectMany
public static IEnumerable GetEntities()
{
int [] arr = {1,2,3};
return arr.SelectMany(Xenv =>
Enumerable.Range(0, 7).Select(a => new SomeEntity(someFunction(a)))
);
}
on side note, you use the old non-generic IEnumerable which prevent .Net compiler from doing type consistency checks. always use the generic one IEnumerable<> with the specific type.

The problem is with the 'foreach' loop: the program cannot assume that the loop will always iterate it through it at least once. If the array being iterated through was a length 0 the code within the loop would not be called; and therefore the return statement would not be triggered; resulting in the error that not all code paths return a value. In order to fix this you need to put a return statement outside of the loop:
public static IEnumerable GetEntities()
{
int [] arr = {1,2,3};
foreach (int i in arr)
{
// for (int i = 0; i < someArray.Count();i++)
// yield return new SomeEntity(someFunction(i));
// *** Equivalent linq function ***
return Enumerable.Range(0, 7).Select(a => new SomeEntity(someFunction(a)));
}
//INSERT RETURN STATEMENT HERE
}

If you are using any return type on your method you have to return anything with type of method in before the final braces of method like below.you no need to return anything if your using return type as void
public static IEnumerable GetEntities()
{
int [] arr = {1,2,3};
foreach (int i in arr)
{
// for (int i = 0; i < someArray.Count();i++)
// yield return new SomeEntity(someFunction(i));
// *** Equivalent linq function ***
var numcol = Enumerable.Range(0, 7).Select(a => new SomeEntity(someFunction(a)));
}
return numcol;
}

Find values that appear in all lists (or arrays or collections)

Given the following:
List<List<int>> lists = new List<List<int>>();
lists.Add(new List<int>() { 1,2,3,4,5,6,7 });
lists.Add(new List<int>() { 1,2 });
lists.Add(new List<int>() { 1,2,3,4 });
lists.Add(new List<int>() { 1,2,5,6,7 });
What is the best/fastest way of identifying which numbers appear in all lists?

You can use the .net 3.5 .Intersect() extension method:-
List<int> a = new List<int>() { 1, 2, 3, 4, 5 };
List<int> b = new List<int>() { 0, 4, 8, 12 };
List<int> common = a.Intersect(b).ToList();

To do it for two lists one would use x.Intersect(y).
To do it for several we would want to do something like:
var intersection = lists.Aggregate((x, y) => x.Intersect(y));
But this won't work because the result of the lambda isn't List<int> and so it can't be fed back in. This might tempt us to try:
var intersection = lists.Aggregate((x, y) => x.Intersect(y).ToList());
But then this makes n-1 needless calls to ToList() which is relatively expensive. We can get around this with:
var intersection = lists.Aggregate(
(IEnumerable<int> x, IEnumerable<int> y) => x.Intersect(y));
Which applies the same logic, but in using explicit types in the lambda, we can feed the result of Intersect() back in without wasting time and memory creating a list each time, and so gives faster results.
If this came up a lot we can get further (slight) performance improvements by rolling our own rather than using Linq:
public static IEnumerable<T> IntersectAll<T>(this IEnumerable<IEnumerable<T>> source)
{
using(var en = source.GetEnumerator())
{
if(!en.MoveNext()) return Enumerable.Empty<T>();
var set = new HashSet<T>(en.Current);
while(en.MoveNext())
{
var newSet = new HashSet<T>();
foreach(T item in en.Current)
if(set.Remove(item))
newSet.Add(item);
set = newSet;
}
return set;
}
}
This assumes its for internal use only. If it could be called from another assembly it should have error checks, and perhaps should be defined so as to only perform the intersect operations on the first MoveNext() of the calling code:
public static IEnumerable<T> IntersectAll<T>(this IEnumerable<IEnumerable<T>> source)
{
if(source == null)
throw new ArgumentNullException("source");
return IntersectAllIterator(source);
}
public static IEnumerable<T> IntersectAllIterator<T>(IEnumerable<IEnumerable<T>> source)
{
using(var en = source.GetEnumerator())
{
if(en.MoveNext())
{
var set = new HashSet<T>(en.Current);
while(en.MoveNext())
{
var newSet = new HashSet<T>();
foreach(T item in en.Current)
if(set.Remove(item))
newSet.Add(item);
set = newSet;
}
foreach(T item in set)
yield return item;
}
}
}
(In these final two versions there's an opportunity to short-circuit if we end up emptying the set, but it only pays off if this happens relatively often, otherwise it's a nett loss).
Conversely, if these aren't concerns, and if we know that we're only ever going to want to do this with lists, we can optimise a bit further with the use of Count and indices:
public static IEnumerable<T> IntersectAll<T>(this List<List<T>> source)
{
if (source.Count == 0) return Enumerable.Empty<T>();
if (source.Count == 1) return source[0];
var set = new HashSet<T>(source[0]);
for(int i = 1; i != source.Count; ++i)
{
var newSet = new HashSet<T>();
var list = source[i];
for(int j = 0; j != list.Count; ++j)
{
T item = list[j];
if(set.Remove(item))
newSet.Add(item);
}
set = newSet;
}
return set;
}
And further if we know we're always going to want the results in a list, and we know that either we won't mutate the list, or it won't matter if the input list got mutated, we can optimise for the case of there being zero or one lists (but this costs more if we might ever not need the output in a list):
public static List<T> IntersectAll<T>(this List<List<T>> source)
{
if (source.Count == 0) return new List<T>(0);
if (source.Count == 1) return source[0];
var set = new HashSet<T>(source[0]);
for(int i = 1; i != source.Count; ++i)
{
var newSet = new HashSet<T>();
var list = source[i];
for(int j = 0; j != list.Count; ++j)
{
T item = list[j];
if(set.Remove(item))
newSet.Add(item);
}
set = newSet;
}
return new List<T>(set);
}
Again though, as well as making the method less widely-applicable, this has risks in terms of how it could be used, so is only appropriate for internal code were you can know either that you won't change either the input or the output after the fact, or that this won't matter.

Linq already offers Intersect and you can exploit Aggregate as well:
var result = lists.Aggregate((a, b) => a.Intersect(b).ToList());

If you don't trust the Intersect method or you just prefer to see what's going on, here's a snippet of code that should do the trick:
// Output goes here
List<int> output = new List<int>();
// Make sure lists are sorted
for (int i = 0; i < lists.Count; ++i) lists[i].Sort();
// Maintain array of indices so we can step through all the lists in parallel
int[] index = new int[lists.Count];
while(index[0] < lists[0].Count)
{
// Search for each value in the first list
int value = lists[0][index[0]];
// No. lists that value appears in, we want this to equal lists.Count
int count = 1;
// Search all the other lists for the value
for (int i = 1; i < lists.Count; ++i)
{
while (index[i] < lists[i].Count)
{
// Stop if we've passed the spot where value would have been
if (lists[i][index[i]] > value) break;
// Stop if we find value
if (lists[i][index[i]] == value)
{
++count;
break;
}
++index[i];
}
// If we reach the end of any list there can't be any more matches so end the search now
if (index[i] >= lists[i].Count) goto done;
}
// Store the value if we found it in all the lists
if (count == lists.Count) output.Add(value);
// Skip multiple occurrances of the same value
while (index[0] < lists[0].Count && lists[0][index[0]] == value) ++index[0];
}
done:
Edit:
I got bored and did some benchmarks on this vs. Jon Hanna's version. His is consistently faster, typically by around 50%. Mine wins by about the same margin if you happen to have presorted lists, though. Also you can gain a further 20% or so with unsafe optimisations. Just thought I'd share that.

You can also get it with SelectMany and Distinct:
List<int> result = lists
.SelectMany(x => x.Where(e => lists.All(l => l.Contains(e))))
.Distinct().ToList();
Edit:
List<int> result2 = lists.First().Where(e => lists.Skip(1).All(l => l.Contains(e)))
.ToList();
Edit 2:
List<int> result3 = lists
.Select(l => l.OrderBy(n => n).Take(lists.Min(x => x.Count()))).First()
.TakeWhile((n, index) => lists.Select(l => l.OrderBy(x => x)).Skip(1).All(l => l.ElementAt(index) == n))
.ToList();

Is there a way to yield, emit or otherwise materialize a previously empty array in C# / LINQ?

I have an object with a dynamic array of strings which I've implemented as follows:
public class MyThing {
public int NumberOfThings { get; set; }
public string _BaseName { get; set; }
public string[] DynamicStringArray {
get {
List<string> dsa = new List<string>();
for (int i = 1; i <= this.NumberOfThings; i++) {
dsa.Add(string.Format(this._BaseName, i));
}
return dsa.ToArray();
}
}
}
I was trying to be a little cooler earlier and implement something that autocreated the formatted list of arrays in LINQ but I've managed to fail.
As an example of the thing I was trying:
int i = 1;
// create a list with a capacity of NumberOfThings
return new List<string>(this.NumberOfThings)
// create each of the things in the array dynamically
.Select(x => string.Format(this._BaseName, i++))
.ToArray();
It's really not terribly important in this case, and performance-wise it might actually be worse, but I was wondering if there was a cool way to build or emit an array in LINQ extensions.

Will Range help?
return Enumerable
.Range(1, this.NumberOfThings)
.Select(x => string.Format(this._BaseName, x))
.ToArray();

Your property could return an IEnumerable and you could then invoke the ToArray() extension on that, if you needed to.
public string[] DynamicStringArray
{
get
{
for (int i=1; i <= this.NumberOfThings; i++)
yield return string.Format(this._BaseName, i);
}
}
However, yields are inherently slow because of the context switching that goes on. You're better off doing this:
public string[] DynamicStringArray
{
get
{
string[] result = new string[this.NumberOfThings];
for (int i = 0; i < this.NumberOfThings; i++)
{
result[i] = string.Format(this._BaseName, i + 1));
}
return result;
}
}
Those Linq extension methods are nice for when you're feeling lazy. But if you need it to perform well you should avoid them.

I'd rather redesign a bit the current solution:
public class MyThing {
...
// Note IEnumerable<String> instead of String[]
public IEnumerable<String> DynamicString(int numberOfThings) {
if (numberOfThings < 0)
throw new ArgumentOutOfRangeException("numberOfThings");
for (int i = 0; i < numberOfThings; ++i)
yield return string.Format(this._BaseName, i + 1);
}
}
whenever you want, say, an array you can easily obtain it:
MyThing thing = ...;
// Add .ToArray() to have an array
String[] myArray = thing.DynamicString(18).ToArray();
but whenever all you want is just loop there's no need to create an array or list (materialize a result)
// no materialization: no array of 1000000 items
foreach (String item in thing.DynamicString(1000000)) {
...
}

IComparer for string that checks if x starts with y

I've got an array of strings and I need to get all strings, that start with some 'prefix'. I wanna use Array.BinarySearch(). Is it possible? And how should I write a comparer if so?

No, you cannot use BinarySearch in this case. You could use Enumerable.Where instead:
Dim query = From str In array Where str.StartsWith("prefix")
or with (ugly in VB.NET) method synatx:
query = array.Where(Function(str) str.StartsWith("prefix"))
Edit: whoops, C#
var query = array.Where(s => s.StartsWith("prefix"));
Use ToArray if you want to create a new filtered array.

It's easy to create your own StartsWithComparer:
class StartsWithComparer : IComparer<string>
{
public int Compare(string a, string b) {
if(a.StartsWith(b)) {
return 0;
}
return a.CompareTo(b);
}
}
As others pointed out, this will only return one index. You can have a couple of helpers to return all items:
IEnumerable<string> GetBefore(IList<string> sorted, int foundIndex, string prefix) {
for(var i = foundIndex - 1; i >= 0; i--) {
if(sorted[i].StartsWith(prefix)) {
yield return sorted[i];
}
}
}
IEnumerable<string> GetCurrentAndAfter(IList<string> sorted, int foundIndex, string prefix) {
for(var i = foundIndex; i < sorted.Count; i++) {
if(sorted[i].StartsWith(prefix)) {
yield return sorted[i];
}
}
}
Then to use it:
var index = sorted.BinarySearch("asdf", new StartsWithComparer());
var previous = GetBefore(sorted, index, "asdf");
var currentAndAfter = GetCurrentAndAfter(sorted, index, "asdf");
You can wrap the whole thing in your own class, with a single method that returns all items that start with your prefix.