I've recently been implementing a recursive directory search implementation and I'm using a Stack to track the path elements. When I used string.Join() to join the path elements, I found that they were reversed. When I debugged the method, I looked into the stack and found that the elements themselves were reversed in the Stack's internal array, ie the most recently Push()ed element was at the beginning of the internal array, and the least recently Push()ed element was at the end of the internal array. This seems ass-backward and very counter-intuitive. Can somebody please tell me why Microsoft would implement a stack in such a manner ?
I think you're mistaken.
It isn't that Stack<T>.Push internally inserts an item at the start of its internal array (it doesn't). Rather, it enumerates from the top to the bottom, as this is the manner in which one would intuitively enumerate through a stack (think of a stack of pancakes: you start at the top and work your way down).
If you look at the contents of a collection from within Visual Studio's debugger, I think it will display them to you in the order they're enumerated -- not the order they're stored internally*.
Take a look at the Stack<T>.Push method in Reflector and you'll see that the code is basically exactly what you'd expect:
// (code to check array size)
this._array[this._size++] = item;
// (code to update internal version number)
So the stack internally adds new elements onto the end of its internal array. It's the Stack<T>.Enumerator class that's got you confused, not the Stack<T> class itself.
*I don't know whether this is true in general, but it's true for Stack<T>; see Hans Passant's excellent answer for the reason why.
You had me going there for a bit, that indeed looks completely bass-ackwards. There is however something else going on. The Stack<> class has a debugger visualizer, named System_StackDebugView<>. It is an internal class, you'd have to look with Reflector to see it.
That visualizer has an Items property, that's what you look at when you expand the node in the debugger. That Items property uses Stack<>.ToArray(). Which looks like this:
public T[] ToArray()
{
T[] localArray = new T[this._size];
for (int i = 0; i < this._size; i++)
{
localArray[i] = this._array[(this._size - i) - 1];
}
return localArray;
}
Yup, backwards.
What you described is the correct implementation, as a stack is a LIFO (Last in First out) structure. Imagine it like a stack of plates, the most recently element placed into the stack is the first one removed. Have you run into a stack elsewhere that is FIFO?
FIFO would be a Queue.
Here is how the stack's push and pops methods are implemented. Notice that it is using the last index in the array, rather than the first. So there must be some other problem going on to get yours backwards.
public virtual void Push(object obj)
{
if (this._size == this._array.Length)
{
object[] destinationArray = new object[2 * this._array.Length];
Array.Copy(this._array, 0, destinationArray, 0, this._size);
this._array = destinationArray;
}
this._array[this._size++] = obj;
this._version++;
}
public virtual object Pop()
{
if (this._size == 0)
{
throw new InvalidOperationException(Environment.GetResourceString("InvalidOperation_EmptyStack"));
}
this._version++;
object obj2 = this._array[--this._size];
this._array[this._size] = null;
return obj2;
}
To add to the other answers, if in the debugger you scroll down to the bottom of your Stack<>'s elements and open up Raw View->Non-Public members->_array you can see the contents of the actual internal array used to hold the items and verify that they are in the expected order.
I don't see what it matters which end they consider the top of the stack, as long as you now know which end it is. It actually makes more sense that when you 'push' something on to the stack, you're pushing it on the top (beginning) and moving the other items down...
Related
The below functions results in hundred of levels of recursion. I'm wondering if anybody has a suggestion on how to switch this to a loop function instead. I believe the order of execution does matter in this case, but I may need to do more of an investigation. I was hoping I could directly convert it to an iterative function instead of recursion.
As you can hopefully see, the only parameter passed to each level of recursion is "after". So the recursive function call itself is relatively simple, but the loop surrounding the call is what throws me off.
I've considered doing a queue, but the condition of "changed" seems to imply depth first checks. I'd have to perform part of the shifting operation prior to adding it to the queue, but in the current code the next level of recursion would start immediately after, so I can't just build up a queue of items to process and execute them in order.
I've considered a stack, but I'm not sure how I'd go about implementing that to replace the recursion.
I decided to simplify the code, as it probably was a bit confusing. Here's a skeleton (that you could actually run if you initialize the variables!) that's probably more "psuedo" like
private void DataChangedRecursive(LinkedNode node)
{
InitializeVariables();
try
{
foreach (LinkedNode after in node.After)
{
var afterDetails = after.Before;
bool changed = CheckData(afterDetails);
if (changed)
{
DataChangedRecursive(afterDetails);
}
}
}
catch
{
// Assume relavant error handling at this level in the stack. This probably isn't important to maintain, but it'd be interested if we could.
throw;
}
}
public object InitializeVariables()
{
// Assume relavant work happens here.
return new object();
}
public bool CheckData(LinkedNode dr)
{
// Logic is that something changes, so it needs to save. This does a bunch of comparisons on the current item.
return dr.DataChanged;
}
public class LinkedNode
{
public LinkedNode Before {get;set;}
public bool DataChanged {get;set;}
public List<LinkedNode> After {get;set;}
}
Finally figured it out, had an epiphany about it. Turns out the easiest way to deal with recursion and a loop is to take advantage of IEnumerator. It requires that you manually deal with the iteration (no for loops), but the order of the recursion and loops will be identical.
I split the function into two parts, the entry function and the function to do the iteration along with the "recursion". Essentially, this will guarantee all of the children are fully complete first, just like the recursion, and return to the correct iteration point in the parent. This should theoretically work for any recursive function with a loop inside it.
private void DataChangedRecursive(LinkedNode node)
{
try
{
DataChanged(node);
}
catch
{
throw;
}
}
private void DataChanged(LinkedNode node)
{
var state = new Stack<IEnumerator<LinkedNode>>();
state.Push(node.After.GetEnumerator());
while (state.Count > 0)
{
InitializeVariables();
while (state.Peek().MoveNext())
{
ItemWithPredPostcessor after = state.Peek().Current;
ItemWithPredPostcessor afterDetails = after.Before;
bool dataChanged = StartShift(afterDetails);
if (dataChanged)
{
Save(afterDetails);
state.Push(afterDetails.After.GetEnumerator());
}
}
state.Pop(); // Remove current from list, as we've completed.
}
}
There is a section in memory, called the stack. Function calls are stored in the stack. Considering that your f function calls a g function, the g function is pushed to the stack. When g is executed, it will be popped from the stack and its result - if any - will be returned to f at the location where f has been interrupted to call and execute g.
This behavior is automatic, but if you understand it - and I advise you to read some articles about it - then you will realize what behavior you will need to emulate. Yes, your thought that you need a stack is true. Each item in your stack will have to store the state, that is, you will need to make sure that whenever you push a new item to your stack you do not forget the state of the interrupted item you were at before pushing a new item to the stack. Also, when you pop an item from a stack, you will need to ensure that its result (in this case you have void, so you will not care about the result, but I'm speaking in general terms) is given back correctly to the new top of the stack after the pop. You will also need to accurately get back to the correct state after the pop. So you will need to carefully handle your variables.
Now, let's see how should it work. For each level you have a queue of potential new levels. In our case you will need a cycle which will handle a step in each iteration. A step can be an initialization, a queue item handling, a pop. The cycle will run until the stack gets empty, so make sure that your stack will get empty eventually to avoid some frustrations. To know what you need to do you will need to have a state for each level of your stack, so your program will always know whether a given item was already initialized, its queue is being iterated or not or is it to be finalized. The queue is the set of possible sub-items. A very simplified pseudocode to illustrate the idea:
root.state <- preinitialized
stack.push(root);
while (not stack.empty()) do
item <- stack.top()
if (item.state = preinitialized) then
item.initializeQueue()
item.queueIterator.initialize()
item.state <- queue
else if (item.state = queue) then
if (item.queueIterator.hasNext()) then
next <- item.queueIterator.next()
if (someCondition) then
next.state <- preinitialized
stack.push(next)
end if
else
item.state <- finished
result <- stack.pop(item)
if (stack.top()) then
stack.top.apply(result)
else
finalResult <- result
end
end if
end if
end while
In C#, is there a way to push one Stack onto another Stack without iterating through the stack elements? If not, is there a better data structure I should be using? In Java you can do:
stack1.addAll(stack2)
I was hoping to find the C# analogue...
0. Safe Solution - Extension Method
public static class Util {
public static void AddAll<T>(this Stack<T> stack1, Stack<T> stack2) {
T[] arr = new T[stack2.Count];
stack2.CopyTo(arr, 0);
for (int i = arr.Length - 1; i >= 0; i--) {
stack1.Push(arr[i]);
}
}
}
Probably the best is to create an extension method. Note that I am putting the first stack "on top" of the other stack so to speak by looping from arr.Length-1 to 0. So this code:
Stack<int> x = new Stack<int>();
x.Push(1);
x.Push(2);
Stack<int> y = new Stack<int>();
y.Push(3);
y.Push(4);
x.AddAll(y);
Will result in x being: 4,3,2,1. Which is what you would expect if you push 1,2,3,4. Of course, if you were to loop through your second stack and actually pop elements and then push those to the first stack, you would end up with 1,2,4,3. Again, modify the for loop as you see fit. Or you could add another parameter to specify which behavior you would like. I don't have Java handy, so I don't know what they do.
Having said that, you could do this, but I don't make any guarantees that it will continue to work. MS could always change the default behavior of how stack works when calling ToList. But, this is shorter, and on my machine with .NET 4.5 works the same as the extension method above:
1 Line Linq solution:
y.Reverse().ToList().ForEach(item => x.Push(item));
In your question, wanting to do this "without iterating through the stack elements" basically means a LinkedList-based stack where you would just join the first and last elements to combine stacks in constant time.
However, unless you've a very specific reason for using LinkedList, it's likely a better idea to just iterate over an array-based (List-based) stack elements.
As far as a specific implementation goes, you should probably clarify whether you want the second stack to be added to the first in the same stack order or to be reversed into the first stack by being popped out.
An addAll would just be a convenience method for a foreach loop that adds all of the items. There really isn't much you can do besides that:
foreach(var item in stack2)
stack1.Push(item);
If you do it particularly frequently you can add an extension method for it, for your own convenience.
This isn't meant to be done with the current .NET Stack implementation.
In order for the content of a Stack to be "grafted" onto the end of another Stack without iterating though its elements internal implementation details how the Stack class stores them in memory has to be known. Based on the principle of encapsulation this information is "officially" only know inside the Stack class itself. .NET's Stack does not expose methods to do this, so without using reflection there is no way to do it as the OP requested.
Conceivably you could use reflection to append to the internal array of one Stack the content of another Stack and also update the field that stores the stack length but this would be highly dependent on the implementation of the Stack class which could be changed in future versions of the framework without warning.
If you really need a Stack that can do this you could write your own Stack class from scratch or simply use another collection like ArrayList or LinkedList which have the method you want and add Push and Pop extension methods to them.
I have a simple methode like this:
public int method(int a)
{
if(// something)
{
methode(a);
}
else return 0;
}
When the depth of calling increases, Visual Studio throws a stackoverflow exeption.
How can i solve this problem?
Is there any way to save return address and local data manually and implement a customized stack??
I should note that I don't want to change my method to non recursive type.
Yes, in C# (and probably in Java etc) there is a Stack<T> class. For your method you can simply create a Stack and store the argument there. Iterate over the Stack until it's empty.
While this introduces an iterative loop, your algorithm is still recursive. (That is, depth first instead of breadth first)
Of course you need to make sure that your algorithm terminates eventually. This is just a way to increase stack space above what the operating system gives you. Windows allocates a certain amount of stack space for each process that suffices for most applications. If you need more, you can implement your own stack-like data structure on the heap. The heap is generally limited by the available RAM and the "bitness" of your application.
How can i solve this problem? Is there any way to save return address
and local data manually and implement a customized stack??
Recursive method should have a termination point, where from result is returned.
You need to decrease the depth of recursion.
public int method(int a ){
//do stuff
var something = ... //you have to make sure something turns false at some point
if(something){
method(a);
}
}
Like Dunken says - "you have to make sure something turns false at some point".
#define MAX 1000
#include <iostream>
using namespace std;
int methode(int a ){
if(a>0 && a<MAX) return methode(a);
return 0 ;
}// end
int main(void){
methode(1);
cout<<" \nPress any key to continue\n";
cin.ignore();
cin.get();
return 0;
}
Basically I am wondering if it's ok to use an enumeration more than once in code subsequently. Whether you break early or not, would the enumeration always reset in every foreach case below, so giving us consistent enumeration from start to end of the effects collection?
var effects = this.EffectsRecursive;
foreach (Effect effect in effects)
{
...
}
foreach (Effect effect in effects)
{
if(effect.Name = "Pixelate")
break;
}
foreach (Effect effect in effects)
{
...
}
EDIT: The implementation of EffectsRecursive is this:
public IEnumerable<Effect> Effects
{
get
{
for ( int i = 0; i < this.IEffect.NumberOfChildren; ++i )
{
IEffect effect = this.IEffect.GetChildEffect ( i );
if ( effect != null )
yield return new Effect ( effect );
}
}
}
public IEnumerable<Effect> EffectsRecursive
{
get
{
foreach ( Effect effect in this.Effects )
{
yield return effect;
foreach ( Effect seffect in effect.ChildrenRecursive )
yield return seffect;
}
}
}
Yes this is legal to do. The IEnumerable<T> pattern is meant to support multiple enumerations on a single source. Collections which can only be enumerated once should expose IEnumerator instead.
The code that consumes the sequence is fine. As spender points out, the code that produces the enumeration might have performance problems if the tree is deep.
Suppose at the deepest point your tree is four deep; think about what happens on the nodes that are four deep. To get that node, you iterate the root, which calls an iterator, which calls an iterator, which calls an iterator, which passes the node back to code that passes the node back to code that passes the node back... Instead of just handing the node to the caller, you've made a little bucket brigade with four guys in it, and they're tramping the data around from object to object before it finally gets to the loop that wanted it.
If the tree is only four deep, no big deal probably. But suppose the tree is ten thousand elements, and has a thousand nodes forming a linked list at the top and the remaining nine thousand nodes on the bottom. Now when you iterate those nine thousand nodes each one has to pass through a thousand iterators, for a total of nine million copies to fetch nine thousand nodes. (Of course, you've probably gotten a stack overflow error and crashed the process as well.)
The way to deal with this problem if you have it is to manage the stack yourself rather than pushing new iterators on the stack.
public IEnumerable<Effect> EffectsNotRecursive()
{
var stack = new Stack<Effect>();
stack.Push(this);
while(stack.Count != 0)
{
var current = stack.Pop();
yield return current;
foreach(var child in current.Effects)
stack.Push(child);
}
}
The original implementation has a time complexity of O(nd) where n is the number of nodes and d is the average depth of the tree; since d can in the worst case be O(n), and in the best case be O(lg n), that means that the algorithm is between O(n lg n) and O(n^2) in time. It is O(d) in heap space (for all the iterators) and O(d) in stack space (for all the recursive calls.)
The new implementation has a time complexity of O(n), and is O(d) in heap space, and O(1) in stack space.
One down side of this is that the order is different; the tree is traversed from top to bottom and right to left in the new algorithm, instead of top to bottom and left to right. If that bothers you then you can just say
foreach(var child in current.Effects.Reverse())
instead.
For more analysis of this problem, see my colleague Wes Dyer's article on the subject:
http://blogs.msdn.com/b/wesdyer/archive/2007/03/23/all-about-iterators.aspx
Legal, yes. Whether it will function as you expect depends on:
The implementation of the IEnumerable returned by EffectsRecursive and whether it always returns the same set;
Whether you want to enumerate the same set both times
If it returns an IEnumerable that requires some intensive work, and it doesn't cache the results internally, then you may need to .ToList() it yourself. If it does cache the results, then ToList() would be slightly redundant but probably no harm.
Also, if GetEnumerator() is implemented in a typical/proper (*) way, then you can safely enumerate any number of times - each foreach will be a new call to GetEnumerator() which returns a new instance of the IEnumerator. But it could be that in some situations it returns the same IEnumerator instance that's already been partially or fully enumerated, so it all really just depends on the specific intended usage of that particular IEnumerable.
*I'm pretty sure returning the same enumerator multiple times is actually a violation of the implied contract for the pattern, but I have seen some implementations that do it anyway.
Most probably, yes. Most implementations of IEnumerable return a fresh IEnumerator which starts at the beginning of the list.
It all depends on the implementation of the type of EffectsRecursive.
So I've been trying to implement a quicksort myself, just to learn something from it, but it also generates a stackoverflowexception, but I can't seem to find what the cause is.
Can someone give me a clue?
public void Partition(List<int> valuelist, out List<int> greater, out List<int> lesser)
{
lesser = new List<int>(); // <-- Stackoverflow exception here!
greater = new List<int>();
if (valuelist.Count <= 1)
return;
pivot = valuelist.First();
foreach (int Element in valuelist)
{
if (Element <= pivot)
lesser.Add(Element);
else
greater.Add(Element);
}
}
public List<int> DoQuickSort(List<int> list)
{
List<int> great;
List<int> less;
Partition(list, out great, out less);
DoQuickSort(great);
DoQuickSort(less);
list.Clear();
list = (List<int>)less.Concat(great);
return list;
}
you are doing an infinite loop right there
DoQuickSort(great);
you need a way to get out of that loop with a flag or a condition
Edit
I will add that in debugging mode, with default setting, you can only reach between 10,000 and 16,000 recursive call before an exception is thrown and between 50,000 and 80,000 when in release mode, all depend on the actual code executed.
If you play with a huge number of values, you might need to manage yourself that recursive call by using a Stack object.
sample code to see how much call before it crash;
(debug; 14,210 call, release 80,071 call)
static int s = 1;
static void Main(string[] args)
{
o();
}
static void o()
{
s++;
Console.WriteLine(s.ToString());
o();
}
You're not putting any conditions on your recursive calls to DoQuicksort, so it'll never stop recursing, leading to a stack overflow. You should only be calling DoQuicksort on a list if it contains more than one element.
Edit: As Will said in his comment, this is a very slow approach to "Quicksort". You should look at in-place partitioning algorithms, as mentioned on Wikipedia's Quicksort article.
I think one of the problems in your code that you keep the pivot value when partitioning the list. This means that you will run into a situation where all values partition into either greater or less, and the partitioning will stop working. This will effectively not letting you split one of the lists anylonger, so the exit condition in the Partition method is never satisfied.
You should select a pivot value, remove the pivot element from the list (this part is missing in your code), parition it in greater and less lists, sort those (recursively), and then concatenate the less list, the pivot element (this is also, naturally, missing in your code) and the greater list.
I can post an updated, working, code sample, but since you are in "learning mode", I will keep it to myself until you ask for it :)