I have this huge json file neatly formated starting with the characters "[\r\n" and ending with "]". I have this piece of code:
foreach (var line in File.ReadLines(#"d:\wikipedia\wikipedia.json").Skip(1))
{
if (line[0] == ']') break;
// Do stuff
}
I'm wondering, what would be best performance-wise, what machine code would be the most optimal in regards to how many clock cycles and memory is consumed if I were to compare the above code to one where I have replaced "break" with "continue", or would both of those pieces of code compile to the same MSIL and machine code? If you know the answer, please explain exactly how you reached your conclusion? I'd really like to know.
EDIT: Before you close this as nonsensical, consider that this code is equivalent to the above code and consider that the c# compiler optimizes when the code path is flat and does not fork in a lot of ways, would all of the following examples generate the same amount of work for the CPU?
IEnumerable<char> text = new[] {'[', 'a', 'b', 'c', ']'};
foreach (var c in text.Skip(1))
{
if (c == ']') break;
// Do stuff
}
foreach (var c in text.Skip(1))
{
if (c == ']') continue;
// Do stuff
}
foreach (var c in text.Skip(1))
{
if (c != ']')
{
// Do stuff
}
}
foreach (var c in text.Skip(1))
{
if (c != ']')
{
// Do stuff
}
}
foreach (var c in text.Skip(1))
{
if (c != ']')
{
// Do stuff
}
else
{
break;
}
}
EDIT2: Here's another way of putting it: what's the prettiest way to skip the first and last item in an IEnumerable while still deferring the executing until //Do stuff?
Q: Different MSIL for break or continue in loop?
Yes, that's because it works like this:
foreach (var item in foo)
{
// more code...
if (...) { continue; } // jump to #1
if (...) { break; } // jump to #2
// more code...
// #1 -- just before the '}'
}
// #2 -- after the exit of the loop.
Q: What will give you the most performance?
Branches are branches for the compiler. If you have a goto, a continue or a break, it will eventually be compiled as a branch (opcode br), which will be analyzes as such. In other words: it doesn't make a difference.
What does make a difference is having predictable patterns of both data and code flow in the code. Branching breaks code flow, so if you want performance, you should avoid irregular branches.
In other words, prefer:
for (int i=0; i<10 && someCondition; ++i)
to:
for (int i=0; i<10; ++i)
{
// some code
if (someCondition) { ... }
// some code
}
As always with performance, the best thing to do is to run benchmarks. There's no surrogate.
Q: What will give you the most performance? (#2)
You're doing a lot with IEnumerable's. If you want raw performance and have the option, it's best to use an array or a string. There's no better alternative in terms of raw performance for sequential access of elements.
If an array isn't an option (for example because it doesn't match the access pattern), it's best to use a data structure that best suits the access pattern. Learn about the characteristics of hash tables (Dictionary), red black trees (SortedDictionary) and how List works. Knowledge about how stuff really works is the thing you need. If unsure, test, test and test again.
Q: What will give you the most performance? (#3)
I'd also try JSON libraries if your intent is to parse that. These people probably already invented the wheel for you - if not, it'll give you a baseline "to beat".
Q: [...] what's the prettiest way to skip the first and last item [...]
If the underlying data structure is a string, List or array, I'd simply do this:
for (int i=1; i<str.Length-1; ++i)
{ ... }
To be frank, other data structures don't really make sense here IMO. That said, people somethings like to put Linq code everywhere, so...
Using an enumerator
You can easily make a method that returns all but the first and last element. In my book, enumerators always are accessed in code through things like foreach to ensure that IDisposable is called correctly.
public static IEnumerable<T> GetAllButFirstAndLast<T>(IEnumerable<T> myEnum)
{
T jtem = default(T);
bool first = true;
foreach (T item in myEnum.Skip(1))
{
if (first) { first = false; } else { yield return jtem; }
jtem = item;
}
}
Note that this has little to do with "getting the best performance out of your code". One look at the IL tells you all you need to know.
This example is for a method called "WriteLines", which takes an array of strings and adds them to an asynchronous file writer. It works, but I am curious if there is an interesting way to support -any- collection of strings, rather than relying on the programmer to convert to an array.
I came up with something like:
public void AddLines(IEnumerable<string> lines)
{
// grab the queue
lock (_queue)
{
// loop through the collection and enqueue each line
for (int i = 0, count = lines.Count(); i < count; i++)
{
_queue.Enqueue(lines.ElementAt(i));
}
}
// notify the thread it has work to do.
_hasNewItems.Set();
}
This appears to work but I have no idea of any performance implications it has, or any logic implications either (What happens to the order? I assume this will allow even unordered collections to work, e.g. HashSet).
Is there a more accepted way to achieve this?
You've been passed an IEnumerable<string> - that means you can iterate over it. Heck, there's even a language feature specifically for it - foreach:
foreach (string line in lines)
{
_queue.Enqueue(line);
}
Unlike your existing approach, this will only iterate over the sequence once. Your current code will behave differently based on the underlying implementation - in some cases Count() and ElementAt are optimized, but in some cases they aren't. You can see this really easily if you use an iterator block and log:
public IEnumerable<string> GetItems()
{
Console.WriteLine("yielding a");
yield return "a";
Console.WriteLine("yielding b");
yield return "b";
Console.WriteLine("yielding c");
yield return "c";
}
Try calling AddLines(GetItems()) with your current implementation, and look at the console...
Adding this answer as well since you are using threads, use a ConcurrentQueue instead, like so:
// the provider method
// _queue = new BlockingCollection<string>()
public void AddLines(IEnumerable<string> lines)
{
foreach (var line in lines)
{
_queue.Add(line);
}
}
No locks required, and allows for multiple consumers and providers since we flag for each element added.
The consumer basically only has to do var workitem = _queue.Take();
My question is, that when I loop through a list with for loop, and add elements to it during this, does it count the elements added while looping?
Simple code example:
for (int i = 0; i < listOfIds.Count(); i++) // Does loop counts the items added below?
{
foreach (var post in this.Collection)
{
if (post.ResponsePostID == listOfIds.ElementAt(i))
{
listOfIds.Add(post.PostId); // I add new item to list in here
}
}
}
I hope my explanation is good enough for you to understand what my question is.
Yes, it usually does. But changing a collection at the same time you're iterating over it can lead to weird behavior and hard-to-find bugs. It isn't recommended at all.
If you want this loop run only for preAdded item count then do this
int nLstCount = listOfIds.Count();
for (int i = 0; i < nLstCount ; i++)
{
foreach (var post in this.Collection)
{
if (post.ResponsePostID == listOfIds.ElementAt(i))
{
listOfIds.Add(post.PostId);
}
}
}
Yes it surely will.The inner foreach loop will execute and add the elements the outer collection and thus will increament the count of the elements.
listOfIds.Count=2 //iteration 1
listOfIds.Add(//element)
when it come to the for loop again
listOfIds.Count=3 //iteration 2
As a slightly abridged explanation of the for loop. You're essentially defining the following:
for (initializer; condition; iterator)
body
Your initializer will will establish your initial conditions, and will only happen once (effectively outside the loop).
Your condition will be evaluated every time to determine whether your loop should run again, or simply exit.
Your iterator defines an action that will occur after each iteration in your loop.
So in your case, your loop will reevaluate listOfIds.Count(); each time, to decide if it should execute; that may or may not be your desired behaviour.
As Dennis points out, you can let yourself get into a bit of a mess (youy loop may run infinitely) if you aren't careful.
A much more detailed/better written explanation can be found on msdn: http://msdn.microsoft.com/en-us/library/ch45axte.aspx
This question already has answers here:
What is the best way to modify a list in a 'foreach' loop?
(11 answers)
Closed 9 years ago.
I want to do a foreach loop while taking out members of that foreach loop, but that's throwing errors. My only idea is to create another list inside of this loop to find which Slices to remove, and loop through the new list to remove items from Pizza.
foreach(var Slice in Pizza)
{
if(Slice.Flavor == "Sausage")
{
Me.Eat(Slice); //This removes an item from the list: "Pizza"
}
}
You can do this, by far the simplest way I have found (like to think I invented it, sure that's not true though ;))
foreach (var Slice in Pizza.ToArray())
{
if (Slice.Flavor == "Sausage") // each to their own.. would have gone for BBQ
{
Me.Eat(Slice);
}
}
..because it's iterating over a fixed copy of the loop. It will iterate all items, even if they are removed.
Handy isn't it!
(By the way guys, this is a handy way of iterating through a copy of a collection, with thread safety, and removing the time an object is locked: Lock, get the ToArray() copy, release the lock, then iterate)
Hope that helps!
If you have to iterate through a list and need to remove items, iterate backwards using a for loop:
// taken from Preet Sangha's answer and modified
for(int i = Pizza.Count-1; i >= 0, i--)
{
var Slice = Pizza[i];
if(Slice.Flavor == "Sausage")
{
Me.Eat(Slice); //This removes an item from the list: "Pizza"
}
}
The reason to iterate backwards is so that when you remove Elements you don't run into an IndexOutOfRangeException that's caused by accessing Pizza[5] on a Pizza that has only 5 Elements because we removed the sixth one.
The reason to use a for loop is because the iterator variable i has no relation to the Pizza, so you can modify the Pizza without the enumerator "breaking"
use a for loop not a foreach
for(int i = 0; i < in Pizza.Count(), ++i)
{
var Slice = Pizza[i];
if(Slice.Flavor == "Sausage")
{
Me.Eat(Slice); //This removes an item from the list: "Pizza"
}
}
Proably the clearest way to approach this would be to build a list of slices to eat, then to process it, avoiding altering the original enumeration in-loop. I've never been a fan of using indexed loops for this, as it can be error-prone.
List<Slice> slicesToEat=new List<Slice>();
foreach(var Slice in Pizza)
{
if(Slice.Flavor == "Sausage")
{
slicesToEat.Add(Slice);
}
}
foreach(var slice in slicesToEat)
{
Me.Eat(slice);
}
Perhaps change your Me.Eat() signature to take an IEnumerable<Slice>
Me.Eat(Pizza.Where(s=>s.Flavor=="Sausage").ToList());
This lets you perform the task in 1 line of code.
Then your Eat() could be like:
public void Eat(IEnumerable<Slice> remove)
{
foreach (Slice r in remove)
{
Pizza.Remove(r);
}
}
The VB6-styled "Collection" object allows for modification during enumeration, and seems to work sensibly when such modifications occur. Too bad it has other limitations (key type is limited to case-insensitive strings) and doesn't support generics, since none of the other collection types allow modification.
Frankly, I'm not clear why Microsoft's iEnumerable contract requires that an exception be thrown if a collection is modified. I would understand a requirement that an exception be thrown if a changes to a collection would make it impossible for an enumeration to continue without wackiness (skipping or duplicating values that did not change during enumeration, crashing, etc.) but see no reason not to allow a collection that could enumerate sensibly to do so.
Where can you order pizza where slices have separate toppings? Anyway...
Using Linq:
// Was "Me.Eat()" supposed to be "this.Eat()"?
Pizza
.Where(slice => slice.Flavor == "Sausage")
.Foreach(sausageSlice => { Me.Eat(sausageSlice); });
The first two lines create a new list with only sausage slices. The third will take that new subset and pass each slice to Me.Eat(). The { and ;} may be superfluous. This is not the most efficient method because it first makes a copy (as do many other approaches that have been given), but it is certainly clean and readable.
BTW, This is only for posterity as the best answer was already given - iterate backward by index.
What kind of collection is Pizza? If it's a List<T> then you can call the RemoveAll method:
Pizza.RemoveAll(slice => string.Equals(slice.Flavor, "Sausage"));
This question already has answers here:
Foreach loop, determine which is the last iteration of the loop
(24 answers)
Closed 10 days ago.
I want to do something different with the last loop iteration when performing 'foreach' on an object. I'm using Ruby but the same goes for C#, Java etc.
list = ['A','B','C']
list.each{|i|
puts "Looping: "+i # if not last loop iteration
puts "Last one: "+i # if last loop iteration
}
The output desired is equivalent to:
Looping: 'A'
Looping: 'B'
Last one: 'C'
The obvious workaround is to migrate the code to a for loop using 'for i in 1..list.length', but the for each solution feels more graceful. What is the most graceful way to code a special case during a loop? Can it be done with foreach?
I see a lot of complex, hardly readable code here... why not keep it simple:
var count = list.Length;
foreach(var item in list)
if (--count > 0)
Console.WriteLine("Looping: " + item);
else
Console.Writeline("Lastone: " + item);
It's only one extra statement!
Another common situation is that you want to do something extra or less with the last item, like putting a separator between the items:
var count = list.Length;
foreach(var item in list)
{
Console.Write(item);
if (--count > 0)
Console.Write(",");
}
The foreach construct (in Java definitely, probably also in other languages) is intended to represent the most general kind if iteration, which includes iteration over collections that have no meaningful iteration order. For example, a hash-based set does not have an ordering, and therefore there is no "last element". The last iteration may yield a different element each time you iterate.
Basically: no, the foreach construct is not meant to be used that way.
How about obtaining a reference to the last item first and then use it for comparison inside the foreach loop? I am not say that you should do this as I myself would use the index based loop as mentioned by KlauseMeier. And sorry I don't know Ruby so the following sample is in C#! Hope u dont mind :-)
string lastItem = list[list.Count - 1];
foreach (string item in list) {
if (item != lastItem)
Console.WriteLine("Looping: " + item);
else Console.Writeline("Lastone: " + item);
}
I revised the following code to compare by reference not value (can only use reference types not value types). the following code should support multiple objects containing same string (but not same string object) since MattChurcy's example did not specify that the strings must be distinct and I used LINQ Last method instead of calculating the index.
string lastItem = list.Last();
foreach (string item in list) {
if (!object.ReferenceEquals(item, lastItem))
Console.WriteLine("Looping: " + item);
else Console.WriteLine("Lastone: " + item);
}
Limitations of the above code. (1) It can only work for strings or reference types not value types. (2) Same object can only appear once in the list. You can have different objects containing the same content. Literal strings cannot be used repeatedly since C# does not create a unique object for strings that have the same content.
And i no stupid. I know an index based loop is the one to use. I already said so when i first posted the initial answer. I provided the best answer I can in the context of the question. I am too tired to keep explaining this so can you all just vote to delete my answer. I'll be so happy if this one goes away. thanks
Is this elegant enough? It assumes a non-empty list.
list[0,list.length-1].each{|i|
puts "Looping:"+i # if not last loop iteration
}
puts "Last one:" + list[list.length-1]
In Ruby I'd use each_with_index in this situation
list = ['A','B','C']
last = list.length-1
list.each_with_index{|i,index|
if index == last
puts "Last one: "+i
else
puts "Looping: "+i # if not last loop iteration
end
}
You can define an eachwithlast method in your class to do the same as each on all elements but the last, but something else for the last:
class MyColl
def eachwithlast
for i in 0...(size-1)
yield(self[i], false)
end
yield(self[size-1], true)
end
end
Then you could call it like this (foo being an instance of MyColl or a subclass thereof):
foo.eachwithlast do |value, last|
if last
puts "Last one: "+value
else
puts "Looping: "+value
end
end
Edit: Following molf's suggestion:
class MyColl
def eachwithlast (defaultAction, lastAction)
for i in 0...(size-1)
defaultAction.call(self[i])
end
lastAction.call(self[size-1])
end
end
foo.eachwithlast(
lambda { |x| puts "looping "+x },
lambda { |x| puts "last "+x } )
C# 3.0 or newer
Firstly, I would write an extension method:
public static void ForEachEx<T>(this IEnumerable<T> s, Action<T, bool> act)
{
IEnumerator<T> curr = s.GetEnumerator();
if (curr.MoveNext())
{
bool last;
while (true)
{
T item = curr.Current;
last = !curr.MoveNext();
act(item, last);
if (last)
break;
}
}
}
Then using the new foreach is very simple:
int[] lData = new int[] { 1, 2, 3, 5, -1};
void Run()
{
lData.ForEachEx((el, last) =>
{
if (last)
Console.Write("last one: ");
Console.WriteLine(el);
});
}
You should use foreach only if you handle each one same. Use index based interation instead. Else you must add a different structure around the items, which you can use to differentiate the normal from last one in the foreach call (look at good Papers about the map reduced from google for the background: http://labs.google.com/papers/mapreduce.html, map == foreach, reduced == e.g. sum or filter).
Map has no knowledge about the structure (esp. which position a item is), it only transforms one item by item (no knowledge from one item can be used to transform an other!), but reduce can use a memory to for example count the position and handle the last item.
A common trick is to reverse the list and handle the first (which has now a known index = 0), and later apply reverse again. (Which is elegant but not fast ;) )
Foreach is elegant in that it has no concern for the number of items in a list and treats each element equally, I think your only solution will be using a for loop that either stops at itemcount-1 and then you present your last item outside of the loop or a conditional within the loop that handles that specific condition, i.e. if (i==itemcount) { ... } else { ... }
You could do something like that (C#) :
string previous = null;
foreach(string item in list)
{
if (previous != null)
Console.WriteLine("Looping : {0}", previous);
previous = item;
}
if (previous != null)
Console.WriteLine("Last one : {0}", previous);
Ruby also has each_index method:
list = ['A','B','C']
list.each_index{|i|
if i < list.size - 1
puts "Looping:"+list[i]
else
puts "Last one:"+list[i]
}
EDIT:
Or using each (corrected TomatoGG and Kirschstein solution):
list = ['A', 'B', 'C', 'A']
list.each { |i|
if (i.object_id != list.last.object_id)
puts "Looping:#{i}"
else
puts "Last one:#{i}"
end
}
Looping:A
Looping:B
Looping:C
Last one:A
Or
list = ['A', 'B', 'C', 'A']
list.each {|i|
i.object_id != list.last.object_id ? puts "Looping:#{i}" : puts "Last one:#{i}"
}
What you are trying to do seems just a little too advanced for the foreach-loop. However, you can use Iterators explicitly. For example, in Java, I would write this:
Collection<String> ss = Arrays.asList("A","B","C");
Iterator<String> it = ss.iterator();
while (it.hasNext()) {
String s = it.next();
if(it.hasNext())
System.out.println("Looping: " + s);
else
System.out.println("Last one: " + s);
}
If you're using a collection that exposes a Count property - an assumption made by many of the other answers, so I'll make it too - then you can do something like this using C# and LINQ:
foreach (var item in list.Select((x, i) => new { Val = x, Pos = i }))
{
Console.Write(item.Pos == (list.Count - 1) ? "Last one: " : "Looping: ");
Console.WriteLine(item.Val);
}
If we additionally assume that the items in the collection can be accessed directly by index - the currently accepted answer assumes this - then a plain for loop will be more elegant/readable than a foreach:
for (int i = 0; i < list.Count; i++)
{
Console.Write(i == (list.Count - 1) ? "Last one: " : "Looping: ");
Console.WriteLine(list[i]);
}
If the collection doesn't expose a Count property and can't be accessed by index then there isn't really any elegant way to do this, at least not in C#. A bug-fixed variation of Thomas Levesque's answer is probably as close as you'll get.
Here's the bug-fixed version of Thomas's answer:
string previous = null;
bool isFirst = true;
foreach (var item in list)
{
if (!isFirst)
{
Console.WriteLine("Looping: " + previous);
}
previous = item;
isFirst = false;
}
if (!isFirst)
{
Console.WriteLine("Last one: " + previous);
}
And here's how I would do it in C# if the collection doesn't expose a Count property and the items aren't directly accessible by index. (Notice that there's no foreach and the code isn't particularly succinct, but it will give decent performance over pretty much any enumerable collection.)
// i'm assuming the non-generic IEnumerable in this code
// wrap the enumerator in a "using" block if dealing with IEnumerable<T>
var e = list.GetEnumerator();
if (e.MoveNext())
{
var item = e.Current;
while (e.MoveNext())
{
Console.WriteLine("Looping: " + item);
item = e.Current;
}
Console.WriteLine("Last one: " + item);
}
At least in C# that's not possible without a regular for loop.
The enumerator of the collection decides whether a next elements exists (MoveNext method), the loop doesn't know about this.
I think I prefer kgiannakakis's solution, however you could always do something like this;
list = ['A','B','C']
list.each { |i|
if (i != list.last)
puts "Looping:#{i}"
else
puts "Last one:#{i}"
end
}
I notice a number of suggestions assume that you can find the last item in the list before beginning the loop, and then compare every item to this item. If you can do this efficiently, then the underlying data structure is likely a simple array. If that's the case, why bother with the foreach at all? Just write:
for (int x=0;x<list.size()-1;++x)
{
System.out.println("Looping: "+list.get(x));
}
System.out.println("Last one: "+list.get(list.size()-1));
If you cannot retrieve an item from an arbitrary position efficiently -- like it the underlying structure is a linked list -- then getting the last item probably involved a sequential search of the entire list. Depending on the size of the list, that may be a performance issue. If this is a frequently-executed function, you might want to consider using an array or ArrayList or comparable structure so you can do it this way.
Sounds to me like you're asking, "What's the best way to put a screw in using a hammer?", when of course the better question to ask is, "What's the correct tool to use to put in a screw?"
Would it be a viable solution for your case to just take the first/last elements out of your array before doing the "general" each run?
Like this:
list = ['A','B','C','D']
first = list.shift
last = list.pop
puts "First one: #{first}"
list.each{|i|
puts "Looping: "+i
}
puts "Last one: #{last}"
This problem can be solved in an elegant way using pattern matching in a functional programming language such as F#:
let rec printList (ls:string list) =
match ls with
| [last] -> "Last " + last
| head::rest -> "Looping " + head + "\n" + printList (rest)
| [] -> ""
I don't know how for-each loops works in other languages but java.
In java for-each uses the Iterable interface that is used by the for-each to get an Iterator and loop with it. Iterator has a method hasNext that you could use if you could see the iterator within the loop.
You can actually do the trick by enclosing an already obtained Iterator in an Iterable object so the for loop got what it needs and you can get a hasNext method inside the loop.
List<X> list = ...
final Iterator<X> it = list.iterator();
Iterable<X> itw = new Iterable<X>(){
public Iterator<X> iterator () {
return it;
}
}
for (X x: itw) {
doSomething(x);
if (!it.hasNext()) {
doSomethingElse(x);
}
}
You can create a class that wraps all this iterable and iterator stuff so the code looks like this:
IterableIterator<X> itt = new IterableIterator<X>(list);
for (X x: itit) {
doSomething(x);
if (!itit.hasNext()) {
doSomethingElse(x);
}
}
Similar to kgiannakakis's answer:
list.first(list.size - 1).each { |i| puts "Looping: " + i }
puts "Last one: " + list.last
How about this one? just learnt a little Ruby. hehehe
list.collect {|x|(x!=list.last ? "Looping:"+x:"Lastone:"+x) }.each{|i|puts i}
Remove the last one from the list and retain its avlue.
Spec spec = specs.Find(s=>s.Value == 'C');
if (spec != null)
{
specs.Remove(spec);
}
foreach(Spec spec in specs)
{
}
Another pattern that works, without having to rewrite the foreach loop:
var delayed = null;
foreach (var X in collection)
{
if (delayed != null)
{
puts("Looping");
// Use delayed
}
delayed = X;
}
puts("Last one");
// Use delayed
This way the compiler keeps the loop straight, iterators (including those without counts) work as expected, and the last one is separated out from the others.
I also use this pattern when I want something to happen in between iterations, but not after the last one. In that case, X is used normally, delayed refers to something else, and the usage of delayed is only at the loop beginning and nothing needs to be done after the loop ends.
Use join whenever possible.
Most often the delimiter is the same between all elements, just join them together with the corresponding function in your language.
Ruby example,
puts array.join(", ")
This should cover 99% of all cases, and if not split the array into head and tail.
Ruby example,
*head, tail = array
head.each { |each| put "looping: " << each }
puts "last element: " << tail
<hello> what are we thinking here?
public static void main(String[] args) {
// TODO Auto-generated method stub
String str = readIndex();
String comp[] = str.split("}");
StringBuffer sb = new StringBuffer();
for (String s : comp) {
sb.append(s);
sb.append("}\n");
}
System.out.println (sb.toString());
}
As a modeling notation, the influence of the OMT notation dominates (e. g., using rectangles for classes and objects). Though the Booch "cloud" notation was dropped, the Booch capability to specify lower-level design detail was embraced. The use case notation from Objectory and the component notation from Booch were integrated with the rest of the notation, but the semantic integration was relatively weak in UML 1.1, and was not really fixed until the UML 2.0 major revision.