We Keep Coding

Poor use of dictionary?

I've read on here that iterating though a dictionary is generally considered abusing the data structure and to use something else.
However, I'm having trouble coming up with a better way to accomplish what I'm trying to do.
When a tag is scanned I use its ID as the key and the value is a list of zones it was seen in. About every second I check to see if a tag in my dictionary has been seen in two or more zones and if it has, queue it up for some calculations.
for (int i = 0; i < TagReads.Count; i++)
{
var tag = TagReads.ElementAt(i).Value;
if (tag.ZoneReads.Count > 1)
{
Report.Tags.Enqueue(tag);
Boolean success = false;
do
{
success = TagReads.TryRemove(tag.Epc, out TagInfo outTag);
} while (!success);
}
}
I feel like a dictionary is the correct choice here because there can be many tags to look up but something about this code nags me as being poor.
As far as efficiency goes. The speed is fine for now in our small scale test environment but I don't have a good way to find out how it will work on a massive scale until it is put to use, hence my concern.

I believe that there's an alternative approach which doesn't involve iterating a big dictionary.
First of all, you need to create a HashSet<T> of tags on which you'll store those tags that have been detected in more than 2 zones. We'll call it tagsDetectedInMoreThanTwoZones.
And you may refactor your code flow as follows:
A. Whenever you detect a tag in one zone...
Add the tag and the zone to the main dictionary.
Create an exclusive lock against tagsDetectedInMoreThanTwoZones to avoid undesired behaviors in B..
Check if the key has more than one zone. If this is true, add it to tagsDetectedInMoreThanTwoZones.
Release the lock against tagsDetectedInMoreThanTwoZones.
B. Whenever you need to process a tag which has been detected in more than one zone...
Create an exclusive lock against tagsDetectedInMoreThanTwoZones to avoid more than a thread trying to process them.
Iterate tagsDetectedInTwoOrMoreZones.
Use each tag in tagsDetectedInMoreThanTwoZones to get the zones in your current dictionary.
Clear tagsDetectedInMoreThanTwoZones.
Release the exclusive lock against tagsDetectedInMoreThanTwoZones.
Now you'll iterate those tags that you already know that have been detected in more than a zone!
In the long run, you can even make per-region partitions so you never get a tagsDetectedInMoreThanTwoZones set with too many items to iterate, and each set could be consumed by a dedicated thread!

If you are going to do a lot of lookup in your code and only sometimes iterate through the whole thing, then I think the dictionary use is ok. I would like to point out thought that your use of ElementAt is more alarming. ElementAt performs very poorly when used on objects that do not implement IList<T> and the dictionary does not. For IEnumerables<T> that do not implement IList the way the nth element is found is through iteration, so your for-loop will iterate the dictionary once for each element. You are better off with a standard foreach.

I feel like this is a good use for a dictionary, giving you good access speed when you want to check if an ID is already in the collection.

Pattern for handling execution order of multiple callbacks

I'm developing a role-playing game in C# (Unity) with a Lua scripting front-end for game logic and modding. I have a design question I've been thinking about and can't seem to find the answer. I'm implementing an Effect class, which provides a common interface to define and handle effects that affect champions or creatures, whether due to a spell, an enchanted item, or a condition (paralyzed, afraid...). The objective is to be as flexible as possible and decouple effects code from the actual champion components/classes.
I want the effect to have access to callbacks, so that it can alter what happens to the entity. If the character health changes for example, active effects can kick in and change that change before it's applied. Here are two examples in Lua, the API should be self-explanatory:
Ring of Health Loss Halving:
onHealthAdjustment = function(entity, val)
if val < 0 then val = math.floor(val / 2); end
return val;
end
Mana Shield spell:
onHealthAdjustment = function(entity, val)
if val < 0 then
championProperties = entity.championProperties;
if championProperties then
championProperties:adjustMana(val);
end
return 0;
else
return val;
end
end
That's fine, but how to handle execution order of callbacks?
Let's say the champion loses 10 health. If the ring gets processed first, it lowers that to 5, then the spell reduces health loss to 0 and removes 5 mana instead.
If the spell gets processed first, it reduces health loss to 0, removes 10 mana, and then the ring callback gets a 0 and does nothing.
I can add an effect priority variable, but there would always end up some with the same value. Process in last-applied first order, or process last-applied only leads to stupid exploits with for example picking and clicking back items in the inventory to make sure what order the effects are processed... I don't see a way to call callbacks in parallel instead of sequentially...
I'm either not seeing an obvious way to fix the current pattern, or I need to change to another design pattern. I've read about Strategy and Observer patterns but can't seem to find a clear answer. How are these cases usually handled?
Thanks!

there would always end up some with the same value
So? If you get a collision, fix it. The order in which the effects are applied is not arbitrary, it's part of your design
Presumably in your code you have a list of event handlers per event type which you iterate through when the event happens. Just make sure that list is in the right order (e.g. by controlling the order they are registered) and you're done.
Side note. In case you didn't know, this:
onHealthAdjustment = function(entity, val) end
Can be written like this:
function onHealthAdjustment(entity, val) end

Asynchronously updating a graph?

I am playing around with an idea in C#, and would like some advice on the best way to go about asynchronously updating a large number of nodes in a graph. I haven't read anything about how to do things like that, everything I've seen in textbooks / examples use graphs whose nodes don't really change.
Suppose I have a graph of some large number of nodes (thousands). Each node has some internal state that depends on some public properties of each of its neighbors, as well as potentially some external input.
So schematically a node is simply:
class Node
{
State internalState;
public State exposedState;
Input input;
List<Node> neigbors;
void Update()
{
while (true)
{
DoCalculations(input, internalState, neighbors);
exposedState = ExposedState(internalState);
}
}
State ExposedState(State state) { ... }
void DoCalculations() { ... }
}
The difficulty is that I would like nodes to be updated as soon as either their their input state changes (by subscribing to an event or polling) or their neighbor's state changes. If I try to do this synchronously in the naive way, I have the obvious problem:
Node A updates when input changes
Its neighbor B sees A has changed, updates.
Node A sees its neighbor B has changed, updates
B updates
A updates
....
Stack overflows
If I update by instead, enumerating through all nodes and calling their update methods, nodes may be inconsistently updated (e.g.: A's input changes, B updates and doesn't see A's change, A updates and changes exposed state).
I could update by trying to maintain a stack of nodes who want to be updated first, but then their neighbors need to be updated next, and theirs next, etc, which means each update cycle I would need to carefully walk the graph and determine the right update order, which could be very slow...
The naive asynchronous way is to have each node in its own thread (or more simply, an initial asynchronous method call happens to each node's update method, which updates indefinitely in a while(true){...}). The problem with his is that having thousands of threads does not seem like a good idea!
It seems like this should have a simple solution; this isn't too different from cellular automata, but any synchronous solution I come up with either has to update a large number of times compared to the number of nodes to get a message from one end to the other, or solving some kind of complicated graph-walking problem with multiple starting points.
The async method seems trivially simple, if only I could have thousands of threads...
So what is the best way to go about doing something like this?

I would think that Rx (The Reactive Extensions) would be a good starting point.
Each piece of state that other nodes might need to depend on is exposed as an IObserable<TState> and other nodes can then subscribe to those observables:
otherNode.PieceOfState.SubScribe(v => { UpdateMyState(v) });
Rx provides lots of filtering and processing functions for observables: these can be used to filter duplicate events (but you'll need to define "duplicate" of course).
Here's an introductory article: http://weblogs.asp.net/podwysocki/archive/2009/10/14/introducing-the-reactive-framework-part-i.aspx

First you need to make sure your updates converge. This has nothing to do with how you perform them (synchronously, asynchronously, serially or in parallel).
Suppose you have two nodes A and B, that are connection. A changes, triggering a recalculation of B. B then changes, triggering a recalculation of A. If the recalculation of A changes A's value, it will trigger a recalculation of B and so on. You need this sequence of triggers to stop at one point - you need your changes to converge. If they don't, no technique you use can fix it.
Once you are sure the calculations converge and you don't get into endless recalculations you should start with the simple single-threaded synchronous calculation and see if it performs well. If it's fast enough, stop there. If not, you can try to parallelize it.
I wouldn't create a thread per calculation, it doesn't scale at all. Instead use a queue of the calculations that need to be performed, and each time you change the value of node A, put all its neighbors in the queue. You can have a few threads processing the queue, making it a much more scalable architecture.
If this still isn't fast enough, you'll need to optimize what you put in the queue and how you handle it. I think it's way too early to worry about that now.

How to handle many updating objects efficiently in C#?

I'm developing a 2D overhead shooter game using C# and XNA. I have a class that I'll call "bullet" and need to update many of these instances every fraction of a second.
My first way to do this was to have a generic List of bullets and simply remove and add new bullets as needed. But in doing so the GC kicks in often and my game had some periodic jerky lag. (Alot of code cut out, but just wanted to show a simple snippet)
if (triggerButton)
{
bullets.Add(new bullet());
}
if (bulletDestroyed)
{
bullets.Remove(bullet);
}
My second and current attempt is to have a separate generic Stack of bullets that I push to when I'm done with a bullet, and pop off a bullet when I need a new one if there's anything in the stack. If there's nothing in the stack then I add a new bullet to the list. It seems to cut the jerky lag but then again, sometimes there's still some jerky lag springing up (though I don't know if it's related).
if (triggerButton)
{
if (bulletStack.Count > 0)
{
bullet temp = bulletStack.Pop();
temp.resetPosition();
bullets.Add(temp);
}
else
{
bullets.Add(new bullet());
}
}
if (bulletDestroyed)
{
bulletStack.Push(bullet);
bullets.Remove(bullet);
}
So, I know premature optimization is the root of all evil, but this was very noticeable inefficiency that I could catch early (and this was before even having to worry about enemy bullets filling the screen). So my questions are: Will pushing unused objects to a stack invoke the garbage collection? Will the references by kept alive or are objects still being destroyed? Is there a better way to handle updating many different objects? For instance, am I getting too fancy? Would it be fine to just iterate through the list and find an unused bullet that way?

There are a lot of issues here, and it's tricky to tell.
First off, is bullet a struct or a class? If bullet is a class, any time you construct one, then unroot it (let it go out of scope or set it to null), you're going to be adding something the GC needs to collection.
If you're going to be making many of these, and updating them every frame, you may want to consider using a List<bullet> with bullet being a struct, and the List being pre-allocated (generate it with a size large enough to hold all of your bullets, so it's not being recreated as you call List.Add). This will help dramatically with the GC pressure.
Also, just because I need to rant:
So, I know premature optimization is the root of all evil, but this was very noticeable inefficiency
Never, ever, be afraid to optimize a routine that you know is causing problems. If you're seeing a performance issue (ie: your lags), this is no longer premature optimization. Yes, you don't want to be optimizing every line of code, but you do need to optimize code, especially when you see a real performance issue. Optimizing it as soon as you see it's a problem is much easier than trying to optimize it later, as any design changes required will be much more easily implemented before you've added a lot of other code that uses your bullet class.

You may find the flyweight design pattern useful. There need be only one bullet object, but multiple flyweights may specify different positions and velocities for it. The flyweights can be stored in a preallocated array (say, 100) and flagged as active or not.
That should eliminate garbage-collection completely, and may reduce the space necessary for tracking each bullet's malleable properties.

I will admit that I don't have any experience in this per se, but I would consider using a traditional array. Initialize the array to a size that is more then you need, and would be the theoretical maximum number of bullets, say 100. Then starting at 0 assign the bullets at the beginning of the array, leaving the last element as a null. So if you had four active bullets your array would look like:
0 B
1 B
2 B
3 B
4 null
...
99 null
The benefit is that the array would always be allocated and therefore you are not dealing with the overhead of a more complex data structure. This is actually fairly similar to how strings work, since they are actually char[] with a null terminator.
Might be worth a shot. One downside, you'll have to do some manual manipulation when removing a bullet, probably move everything after that bullet up a slot. But you are just moving pointers at that point, so I don't think it would have a high penalty like allocating memory or a GC.

You are correct in assuming that by keeping the unused Bullets in a Stack prevents them from being Garbage Collected.
As for the cause of the Lag, have you tried any profiling tools? Just to find where the problem is.

Your stack based solution is pretty close to a class I wrote to generically do this sort of resource pooling:
http://codecube.net/2010/01/xna-resource-pool/
You mentioned that this makes the problem mostly go away, but it still crops up here and there. What's happening is that with this stack/queue based method of pooling, the system will reach a point of stability once you are no longer requesting more new objects than the pool can supply. But if the requests go higher than your previous max # of requested items, it will cause you to have to create a new instance to service the request (thus invoking GC from time to time).
One way you can side-step this is to go through and pre-allocate as many instances as you think you might need at the peak. That way, you won't have any new allocations (at least from the pooled objects), and the GC won't be triggered :-)

List actually has a built in capacity to prevent allocation for every add/remove. Once you exceed the capacity, it adds more ( I think I doubles every time ). The problem may be more on remove than add. Add will just drop on at the first open spot which is tracked by size. To remove, the list has to be condensed to fill in the now empty slot. If you are always removing for the front of the list, then every element needs to slide down.
A Stack still uses an array as its internal storage mechanism. So you are still bound by the add/remove properties of an array.
To make the array work, you need to create all the bullets up from with an Active property for each. When you need a new one, filled the Active flag to true and set all of the new bullets properties. Once complete, flip the Active flag false.
If you wanted to try to eliminate the need to iterate the list ( which could be very large depending on what you are going to allow ) for each repaint, you could try to implement a double linked list within the array. When a new bullet is needed, asked the array for the first available free entry. Go to the last active bullet ( a variable ) and add the new bullet array position into its next active bullet property. When it is time to remove it, go to the previous bullet and change its active bullet property to the removed next active.
//I am using public fields for demonstration. You will want to make them properties
public class Bullet {
public bool Active;
public int thisPosition;
public int PrevBullet = -1;
public int NextBullet = -1;
public List<Bullet> list;
public void Activate(Bullet lastBullet) {
this.Active = true;
this.PrevBullet = lastBullet.thisPosition;
list[this.PrevBullet].NextBullet = this.thisPosition;
}
public void Deactivate() {
this.Active = false;
list[PrevBullet].NextBullet = this.NextBullet;
list[NextBullet].PrevBullet= this.PrevBullet;
}
}
That way, you have a prebuilt array with all the needed bullets but the paint only hits the bullets that are active regardless of their position within the array. You just need to maintain a link to the first active bullet to start the paint and the last active bullet to know where list starts anew.
Now you are just worried about the memory to hold the entire list instead of when the GC is going to clean up.

C#: Avoid infinite recursion when traversing object graph

I have an object graph wherein each child object contains a property that refers back to its parent. Are there any good strategies for ignoring the parent references in order to avoid infinite recursion? I have considered adding a special [Parent] attribute to these properties or using a special naming convention, but perhaps there is a better way.

If the loops can be generalised (you can have any number of elements making up the loop), you can keep track of objects you've seen already in a HashSet and stop if the object is already in the set when you visit it. Or add a flag to the objects which you set when you visit it (but you then have to go back & unset all the flags when you're done, and the graph can only be traversed by a single thread at a time).
Alternatively, if the loops will only be back to the parent, you can keep a reference to the parent and not loop on properties that refer back to it.
For simplicity, if you know the parent reference will have a certain name, you could just not loop on that property :)

What a coincidence; this is the topic of my blog this coming Monday. See it for more details. Until then, here's some code to give you an idea of how to do this:
static IEnumerable<T> Traversal<T>(
T item,
Func<T, IEnumerable<T>> children)
{
var seen = new HashSet<T>();
var stack = new Stack<T>();
seen.Add(item);
stack.Push(item);
yield return item;
while(stack.Count > 0)
{
T current = stack.Pop();
foreach(T newItem in children(current))
{
if (!seen.Contains(newItem))
{
seen.Add(newItem);
stack.Push(newItem);
yield return newItem;
}
}
}
}
The method takes two things: an item, and a relation that produces the set of everything that is adjacent to the item. It produces a depth-first traversal of the transitive and reflexive closure of the adjacency relation on the item. Let the number of items in the graph be n, and the maximum depth be 1 <= d <= n, assuming the branching factor is not bounded. This algorithm uses an explicit stack rather than recursion because (1) recursion in this case turns what should be an O(n) algorithm into O(nd), which is then something between O(n) and O(n^2), and (2) excessive recursion can blow the stack if the d is more than a few hundred nodes.
Note that the peak memory usage of this algorithm is of course O(n + d) = O(n).
So, for example:
foreach(Node node in Traversal(myGraph.Root, n => n.Children))
Console.WriteLine(node.Name);
Make sense?

If you're doing a graph traversal, you can have a "visited" flag on each node. This ensures that you don't revisit a node and possibly get stuck in an infinite loop. I believe this is the standard way of performing a graph traversal.

This is a common problem, but the best approach depends on the scenario. An additional problem is that in many cases it isn't a problem visiting the same object twice - that doesn't imply recursion - for example, consider the tree:
A
=> B
=> C
=> D
=> C
This may be valid (think XmlSerializer, which would simply write the C instance out twice), so it is often necessary to push/pop objects on a stack to check for true recursion. The last time I implemented a "visitor", I kept a "depth" counter, and only enabled the stack checking beyond a certain threshold - that means that most trees simply end up doing some ++/--, but nothing more expensive. You can see the approach I took here.

I'm not exactly sure what you are trying to do here but you could just maintain a hashtable with all previously visited nodes when you are doing your breadth first search of depth first search.

I published a post explaining in detail with code examples how to do object traversal by recursive reflection and also detect and avoid recursive references to prevent a stack over flow exception: https://doguarslan.wordpress.com/2016/10/03/object-graph-traversal-by-recursive-reflection/
In that example I did a depth first traversal using recursive reflection and I maintained a HashSet of visited nodes for reference types. One thing to be careful is to initialize your HashSet with your custom equality comparer which uses the object reference for hash calculation, basically the GetHashCode() method implemented by the base object class itself and not any overloaded versions of GetHashCode() because if the types of properties you traverse overload GetHashCode method, you may detect false hash collisions and think that you detected a recursive reference which in reality could be that the overloaded version of GetHashCode producing the same hash value via some heuristics and confusing the HashSet, all you need to detect is to check if there is any parent child in anywhere in the object tree pointing to the same location in memory.