We Keep Coding

Linq performance: should I first use `where` or `select`

I have a large List in memory, from a class that has about 20 properties.
I'd like to filter this list based on just one property, for a particular task I only need a list of that property. So my query is something like:
data.Select(x => x.field).Where(x => x == "desired value").ToList()
Which one gives me a better performance, using Select first, or using Where?
data.Where(x => x.field == "desired value").Select(x => x.field).ToList()
Please let me know if this is related to the data type I'm keeping the data in memory, or field's type. Please note that I need these objects for other tasks too, so I can't filter them in the first place and before loading them into memory.

Which one gives me a better performance, using Select first, or using Where.
Where first approach is more performant, since it filters your collection first, and then executes Select for filtered values only.
Mathematically speaking, Where-first approach takes N + N' operations, where N' is the number of collection items which fall under your Where condition.
So, it takes N + 0 = N operations at minimum (if no items pass this Where condition) and N + N = 2 * N operations at maximum (if all items pass the condition).
At the same time, Select first approach will always take exactly 2 * N operations, since it iterates through all objects to acquire the property, and then iterates through all objects to filter them.
Benchmark proof
I have completed the benchmark to prove my answer.
Results:
Condition value: 50
Where -> Select: 88 ms, 10500319 hits
Select -> Where: 137 ms, 20000000 hits
Condition value: 500
Where -> Select: 187 ms, 14999212 hits
Select -> Where: 238 ms, 20000000 hits
Condition value: 950
Where -> Select: 186 ms, 19500126 hits
Select -> Where: 402 ms, 20000000 hits
If you run the benchmark many times, then you will see that Where -> Select approach hits change from time to time, while Select -> Where approach always takes 2N operations.
IDEOne demonstration:
https://ideone.com/jwZJLt
Code:
class Point
{
public int X { get; set; }
public int Y { get; set; }
}
class Program
{
static void Main()
{
var random = new Random();
List<Point> points = Enumerable.Range(0, 10000000).Select(x => new Point { X = random.Next(1000), Y = random.Next(1000) }).ToList();
int conditionValue = 250;
Console.WriteLine($"Condition value: {conditionValue}");
Stopwatch sw = new Stopwatch();
sw.Start();
int hitCount1 = 0;
var points1 = points.Where(x =>
{
hitCount1++;
return x.X < conditionValue;
}).Select(x =>
{
hitCount1++;
return x.Y;
}).ToArray();
sw.Stop();
Console.WriteLine($"Where -> Select: {sw.ElapsedMilliseconds} ms, {hitCount1} hits");
sw.Restart();
int hitCount2 = 0;
var points2 = points.Select(x =>
{
hitCount2++;
return x.Y;
}).Where(x =>
{
hitCount2++;
return x < conditionValue;
}).ToArray();
sw.Stop();
Console.WriteLine($"Select -> Where: {sw.ElapsedMilliseconds} ms, {hitCount2} hits");
Console.ReadLine();
}
}
Related questions
These questions can also be interesting to you. They are not related to Select and Where, but they are about LINQ order performance:
Does the order of LINQ functions matter?
Order of LINQ extension methods does not affect performance?

The answer will depend on the state of your collection.
If most entities will pass the Where test, apply Select first;
If fewer entities will pass the Where test, apply Where first.
Update:
#YeldarKurmangaliyev wrote the answer with a concrete example and benchmarking. I ran similar code to verify his claim and our results are exactly opposite and that is because I ran the same test as his but with an object not as simple as the Point type he used to run his tests.
The code very much looks like his code, except that I changed the name of class from Point to EnumerableClass.
Given below the classes I used to constitute the EnumerableClass class:
public class EnumerableClass
{
public int X { get; set; }
public int Y { get; set; }
public String A { get; set; }
public String B { get; set; }
public String C { get; set; }
public String D { get; set; }
public String E { get; set; }
public Frame F { get; set; }
public Gatorade Gatorade { get; set; }
public Home Home { get; set; }
}
public class Home
{
private Home(int rooms, double bathrooms, Stove stove, InternetConnection internetConnection)
{
Rooms = rooms;
Bathrooms = (decimal) bathrooms;
StoveType = stove;
Internet = internetConnection;
}
public int Rooms { get; set; }
public decimal Bathrooms { get; set; }
public Stove StoveType { get; set; }
public InternetConnection Internet { get; set; }
public static Home GetUnitOfHome()
{
return new Home(5, 2.5, Stove.Gas, InternetConnection.Att);
}
}
public enum InternetConnection
{
Comcast = 0,
Verizon = 1,
Att = 2,
Google = 3
}
public enum Stove
{
Gas = 0,
Electric = 1,
Induction = 2
}
public class Gatorade
{
private Gatorade(int volume, Color liquidColor, int bottleSize)
{
Volume = volume;
LiquidColor = liquidColor;
BottleSize = bottleSize;
}
public int Volume { get; set; }
public Color LiquidColor { get; set; }
public int BottleSize { get; set; }
public static Gatorade GetGatoradeBottle()
{
return new Gatorade(100, Color.Orange, 150);
}
}
public class Frame
{
public int X { get; set; }
public int Y { get; set; }
private Frame(int x, int y)
{
X = x;
Y = y;
}
public static Frame GetFrame()
{
return new Frame(5, 10);
}
}
The classes Frame, Gatorade and Home have a static method each to return an instance of their type.
Below is the main program:
public static class Program
{
const string Chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
private static readonly Random Random = new Random();
private static string RandomString(int length)
{
return new string(Enumerable.Repeat(Chars, length)
.Select(s => s[Random.Next(s.Length)]).ToArray());
}
private static void Main()
{
var random = new Random();
var largeCollection =
Enumerable.Range(0, 1000000)
.Select(
x =>
new EnumerableClass
{
A = RandomString(500),
B = RandomString(1000),
C = RandomString(100),
D = RandomString(256),
E = RandomString(1024),
F = Frame.GetFrame(),
Gatorade = Gatorade.GetGatoradeBottle(),
Home = Home.GetUnitOfHome(),
X = random.Next(1000),
Y = random.Next(1000)
})
.ToList();
const int conditionValue = 250;
Console.WriteLine(#"Condition value: {0}", conditionValue);
var sw = new Stopwatch();
sw.Start();
var firstWhere = largeCollection
.Where(x => x.Y < conditionValue)
.Select(x => x.Y)
.ToArray();
sw.Stop();
Console.WriteLine(#"Where -> Select: {0} ms", sw.ElapsedMilliseconds);
sw.Restart();
var firstSelect = largeCollection
.Select(x => x.Y)
.Where(y => y < conditionValue)
.ToArray();
sw.Stop();
Console.WriteLine(#"Select -> Where: {0} ms", sw.ElapsedMilliseconds);
Console.ReadLine();
Console.WriteLine();
Console.WriteLine(#"First Where's first item: {0}", firstWhere.FirstOrDefault());
Console.WriteLine(#"First Select's first item: {0}", firstSelect.FirstOrDefault());
Console.WriteLine();
Console.ReadLine();
}
}
Results:
I ran the tests multiple times and found that
.Select().Where() performed better than .Where().Select().
when collection size is 1000000.
Here is the first test result where I forced every EnumerableClass object's Y value to be 5, so every item passed Where:
Condition value: 250
Where -> Select: 149 ms
Select -> Where: 115 ms
First Where's first item: 5
First Select's first item: 5
Here is the second test result where I forced every EnumerableClass object's Y value to be 251, so no item passed Where:
Condition value: 250
Where -> Select: 110 ms
Select -> Where: 100 ms
First Where's first item: 0
First Select's first item: 0
Clearly, the result is so dependent on the state of the collection that:
In #YeldarKurmangaliyev's tests .Where().Select() performed better; and,
In my tests .Select().Where() performed better.
The state of the collection, which I am mentioning over and over includes:
the size of each item;
the total number of items in the collection; and,
the number of items likely to pass the Where clause.
Response to comments on the answer:
Further, #Enigmativity said that knowing ahead of time the result of Where in order to know whether to put Where first or Select first is a Catch-22. Ideally and theoretically, he is correct and not surprisingly, this situation is seen in another domain of Computer Science - Scheduling.
The best scheduling algorithm is Shortest Job First where we schedule that job first that will execute for the least time. But, how would anyone know how much time will a particular job take to complete? Well, the answer is that:
Shortest job next is used in specialized environments where accurate estimates of running time are available.
Therefore, as I said right at the top (which was also the first, shorter version of my answer), the correct answer to this question will depend on the current state of the collection.
In general,
if your objects are within a reasonable size range; and,
you are Selecting a very small chunk out of each object; and,
your collection size is also not just in thousands,
then the guideline mentioned right at the top of this answer will be useful for you.

How to clear a list of type class

i have a List which get's and set's data from/to my class "Type" after a specific condition is fulfilled i want to clear all the variables value's that have been gathered here's my code
List<Type> Win = new List<Type>();
void Check(int a)
{
if (a>10)
{
Win.Add(new Type() { Power = 10 + a * 100, Current = 1 });
}
if(a<10)
{
Win.Add(new Type() { Power = 10 + a * 100, Current = 1 });
}
}
My class:
public class Type
{
public int Power { get; set; }
public int Current { get; set; }
}
And like this it will always enter the 2 if's at least once because I'm giving him a parameter "a" and a will change let's say 5 times and it will get a lot of different values.How can i clear the entire "Type.Power" and "Type.Current" variables or just clear the entire list "Win" ?

To empty the entire list, you can use .Clear().
Win.Clear();
To reset all instances of `Type.Power' and 'Type.Current' per instance in your list, you would need to iterate the list and update the members to their default value 0.
Win.ForEach(x =>
{
x.Power = 0;
x.Current = 0;
});

Get top 5 scores/names from list

I am adding a function to keep track of scores for a small game I made.
I want to get the top 5 scores (including name for that score) from a file that contains the scores.
The format of the saved scores is:
[name]-[score]
The scores and names are stored in 2 lists, which I parse this way:
string scores = File.ReadAllText(Environment.GetEnvironmentVariable("TEMP") + "/scores");
string[] splitscores = scores.Split('\n');
foreach (string entry in splitscores)
{
string replace = entry.Replace("[", "").Replace("]", "");
string[] splitentry = replace.Split('-');
if (splitentry.Count() > 1)
{
scorenames.Add(splitentry[0]);
scorelist.Add(Int32.Parse(splitentry[1]));
}
}
Then I retrieve #1 player by using:
int indexMax
= !scorelist.Any() ? -1 :
scorelist
.Select((value, index) => new { Value = value, Index = index })
.Aggregate((a, b) => (a.Value > b.Value) ? a : b)
.Index;
lblhighscore1.Text = "#1: " + scorelist[indexMax] + " by " + scorenames[indexMax];
How can I set the remaining 4 players assuming this is my scorelist:
[broodplank]-[12240]
[flo]-[10944]
[bill]-[11456]
[tony]-[9900]
[benji]-[7562]
I've figured I could do a descending sort of the score list, but that wouldn't cover the changes in the indexes of the usernames list, what is the best approach for this?

Best approach? Don't use parallel collections anti-pattern.
Instead of having 2 lists, create a class that can hold both the name and the score together
class Score
{
public string Name { get; private set; }
public int Score { get; private set; }
public Score(string name, int score)
{
Name = name;
Score = score;
}
}
and have just one list
List<Score> scores = new List<Score>();
foreach (string entry in splitscores)
{
string replace = entry.Replace("[", "").Replace("]", "");
string[] splitentry = replace.Split('-');
if (splitentry.Count() > 1)
{
scores.Add(new Score(splitentry[0], Int32.Parse(splitentry[1]))
}
}
You can easily order by one property and because the whole object will be reordered you'll keep the names in the right order without any additional code:
topScores = scores.OrderByDescending(x => x.Score).Take(5);

In addition to MarcinJuraszeks useful answer, some small things that I came across using his solution which I decided to share.
First problem was with the class which threw me the following error
'Score': member names cannot be the same as their enclosing type
Changing the case of "s" fixed it
class Score
{
public string Name { get; private set; }
public int score { get; private set; }
public Score(string name, int Score)
{
Name = name;
score = Score;
}
}
And calling the individual values can be done with Linq
string numberOne = topScores.Skip(0).First().score
string numberTwo = topScores.Skip(1).First().score
and so on

Variable not returning actual values

I want correctly return some variables (arrays)
kazkas.Ads[n]; (n = how many ads are)
kazkas.Ads[n].id;
kazkas.Ads[n].Days[m].Stats.Clicks; // every day have his own clicks
kazkas.Ads[n].Days[m].Stats.Impresons; // every day have his own impresions
from this method and use these variables in other class.
public static void GetAdsStats(string Ticket, DateTime start, DateTime end, int CamId)
{
var client = new CampaignStatsServiceClient();
var id = new CampaignIdFilter();
id.CampaignId = CamId;
var statsdata = new GetAdStatsData();
var kazkas = new Campaign();
kazkas = client.GetAdStats(Ticket, new GetAdStatsData
{
IdFilter = id,
StartDate = start,
EndDate = end
});
long AllClicks = 0;
long AllImpresions = 0;
int reklamos = kazkas.Ads.Length;
long[] statistikaClikai = new long[reklamos];
long[] statistikaImpresions = new long[reklamos];
for (int i = 0; i < reklamos; i++)
{
int dienos = kazkas.Ads[i].Days.Length;
for (int lop = 0; lop < dienos; lop++)
{
AllClicks = AllClicks + kazkas.Ads[i].Days[lop].Stats.Clicks;
AllImpresions = AllImpresions + kazkas.Ads[i].Days[lop].Stats.Impressions;
}
statistikaClikai[i] = AllClicks;
statistikaImpresions[i] = AllImpresions;
}
}
I know that void type can't return anything, but this how I know that my method works ( from debugging). Like you see I was trying do that with for loop. Here i have 9 Ads and every ad have one day.
Like I says I want return every Ads id[in array], and every days.stats.impresions and days.stats.click
how can I do that ? Ore how return more variables/arrays from method to other class, I am using webservises, so i cant use database ore something like that.

As can be seen by the downvotes of the question, you need to design the return value and then code against it.
Your query almost does it (now):
kazkas.Ads[n]; (n = how many ads are)
kazkas.Ads[n].id;
kazkas.Ads[n].Days[m].Stats.Clicks; // every day have his own clicks
kazkas.Ads[n].Days[m].Stats.Impressions; // every day have his own impressions
Your existing code show this should be expanded to include:
kazkas.Ads[n].Total.Clicks;
kazkas.Ads[n].Total.Impressions;
So now you're ready to design. First you want a Stat Class that just contains CLicks and Impressions:
public class Stat
{
public long Impressions { get; set; }
public long Clicks { get; set; }
}
An optimisation here may be to use a struct, but I won't go into that.
As you currently have defined it each Day has just a Stats property:
public class DayStat
{
public Stat Stats { get; set; }
}
Now finally we can define the top level AdStat:
public class AdStat
{
public int id { get; set; }
public DayStat Day[];
public Stat Total { get; set; }
}
Etc... There's further issues here, such as ensuring arrays are created and Stat instances are never null (which is why making some of these classes structs is an option). But I'm really a VB programmer so I'll stop here before I get caught typing crap into the SO IDE :-)

Create a class or struct with members you need
public class Stat
{
public int Id { get; set; }
public long Clicks { get; set; }
...
}
Change the signature of your method from void GetAdsStats to IEnumberable<Stat> GetAdsStats and either return a collection of stats or use yield keyword to return the stat object.
Also if you do not want your method to return anything (return type void) do not use a name starting with Get.
Example:
public static IEnumerable<Stat> GetAdsStats(...)
{
...
var statList = new List<Stat>();
for (int i = 0; i < reklamos; i++)
{
var stat = new Stat();
statList.Add(stat);
int dienos = kazkas.Ads[i].Days.Length;
for (int lop = 0; lop < dienos; lop++)
{
AllClicks = AllClicks + kazkas.Ads[i].Days[lop].Stats.Clicks;
AllImpresions = AllImpresions + kazkas.Ads[i].Days[lop].Stats.Impressions;
}
stat.Clicks = AllClicks;
stat.Impression = AllImpresions;
}
return statList;
}

Change your void to the type you want to return, say Campaign, and return the appropriate variable. The variables you define in your method, only live in your method and are not accessible from another method or class.

How to shuffle multiple related arrays?

I have some unusual I need to do. I am wondering if anyone can think of an easy
way to make the change that I need. What I have is a
public class Report
{
public string[] Text { get; set; }
public string[] Image { get; set; }
public string[] Explanation { get; set; }
}
The report class can have any number of Texts, Images and Explanations and the size of each array is always the consistent but maybe be different for each report instance.
What I need to do is to be able to sort the array elements in a random order. So for example I might have
Report.Text[0] = "text0";
Report.Text[1] = "text1";
Report.Text[2] = "text2";
Report.Image[0] = "img0";
Report.Image[1] = "img1";
Report.Image[2] = "img2";
Report.Explanation[0] = "exp0";
Report.Explanation[1] = "exp1";
Report.Explanation[2] = "exp2";
then after sorting
Report.Text[0] = "text2";
Report.Text[1] = "text0";
Report.Text[2] = "text1";
Report.Image[0] = "img2";
Report.Image[1] = "img0";
Report.Image[2] = "img1";
Report.Explanation[0] = "exp2";
Report.Explanation[1] = "exp0";
Report.Explanation[2] = "exp1";
Can anyone think of a simple way to do this? All I can think of is that I need to create a
new temporary object of the same size and do some kind of swapping. But I am not sure how
to randomize. The reason I am asking is just in case someone has had this need in the past.

I would strongly recommend that you refactor this to create a single class to encapsulate the { Text, Image, Explanation } tuple. At that point, the code will be cleaner and it'll be trivial to reorder the values. Heck, you may not even need a Report type at that point... you may just be able to have a List<ReportItem> or whatever. You'd only need a separate Report type if you wanted to add extra behaviour or data to tie things together.
(As an aside, I hope you don't really have public fields for these to start with...)
If you then have a question around shuffling a single collection, a modified Fisher-Yates shuffle is probably the easiest approach. You could do this with the multiple arrays as well, but it wouldn't be nice - and would have to be specific to Report... whereas you could easily write a generic Fisher-Yates implementation based on IList<T>. If you search on Stack Overflow, you should easily be able to find a few existing implementations :)

If you choose to change your class to the following:
public class Report
{
public string Text { get; set; }
public string Image { get; set; }
public string Explanation { get; set; }
}
You could then do this using an extension method:
(See answer on this SO question)
Then call it this way:
List<Report> reports = new List<Report> { /* create list of reports */ }
Random rnd = new Random();
foreach (Report r in reports.Shuffle(rnd)) {
/* do something with each report */
}

Why don't you create a class
public class Report
{
public string Text { get; set; }
public string Image { get; set; }
public string Explanation { get; set; }
}
and then create a List of those objects and manage it through the list properties:
IList<Report> yourList = new List<Report>()

Here is my solution
class StringWrapper
{
public int Index;
public string Str;
}
public string[] MixArray(string[] array)
{
Random random = new Random();
StringWrapper[] wrappedArray = WrapArray(array);
for (int i = 0; i < wrappedArray.Length; i++)
{
int randomIndex = random.Next(0, wrappedArray.Length - 1);
wrappedArray[i].Index = randomIndex;
}
Array.Sort(wrappedArray, (str1, str2) => str1.Index.CompareTo(str2.Index));
return wrappedArray.Select(wrappedStr => wrappedStr.Str).ToArray();
}
private StringWrapper[] WrapArray(string[] array)
{
int i = 0;
return array.Select(str => new StringWrapper {Index = ++i, Str = str}).ToArray();
}
Then you can call MixArray for each Report object for each property you wand to randomize.

I am not sure I am fond of this direction, but ...
To do exactly what you ask (the law, not the spirit of the law), you will have to add additional arrays and pull items over. In addition, for each array, you will need a List or similar to store the items you have already randomly pulled over. After that, things are simple. Use the Random class to create random numbers, check if the item has already been moved (using the List), if not store the result in the new array/list, add the value to your List to make sure you do not move the same item twice. Once everything is moved, set this new array to the old array.
Now, what is the business reason for randomizing? That might affect whether or not this is a good idea.
ADDED:
After examination of skeet's response, here is a way to solve this if you can use the following type of class:
public class Report {
public string Text { get; set; }
public string Image { get; set; }
public string Explanation { get; set; }
}
Here is one "down and dirty" type of sort:
private static SortedList<int, Report> SortRandomly(List<Report> reports)
{
Random rnd = new Random((int)DateTime.Now.Ticks);
List<int> usedNumbers = new List<int>();
SortedList<int, Report> sortedReports = new SortedList<int, Report>();
int maxValue = reports.Count;
foreach(Report report in reports)
{
bool finished = false;
int randomNumber = 0;
//Get unique random (refactor out?)
while(!finished)
{
randomNumber = rnd.Next(0, maxValue);
if(!usedNumbers.Contains(randomNumber))
{
finished = true;
usedNumbers.Add(randomNumber);
}
}
sortedReports.Add(randomNumber, report);
}
return sortedReports;
}
Note, you can also work to keep the sort in order and randomly picking from the original list, which means you can, in theory, keep it as a list.
private static List<Report> SortRandomly(List<Report> reports)
{
Random rnd = new Random((int)DateTime.Now.Ticks);
List<Report> outputList = new List<Report>();
List<int> usedNumbers = new List<int>();
int maxValue = reports.Count-1;
while(outputList.Count < reports.Count)
{
int randomNumber = rnd.Next(0, maxValue);
if(!usedNumbers.Contains(randomNumber))
{
outputList.Add(reports[randomNumber]);
}
}
return outputList;
}
Even better, consider sorting the list of numbers first and then grabbing the reports, in an order manner. Once again, the above are down and dirty implementations and using the specific requirements will certainly refine the algorithms.