I followed the directions given in this question (the answer by Jason) in order to write my PriorityQueue<T> using a SortedList. I understand that the count field within this class is used to ensure unique priorities and to preserve the enqueue order among the same priority.
However, when count reaches its maximum value and I sum 1 to it, the latter will starts again from 0, so the priority of the subsequent items would be higher than the priority of the previous items. Using this approach I could need for a way to "securely" reset the counter count... In fact, suppose to have the following queue state (in the format priority | count | item):
0 | 123 | A
0 | 345 | B
1 | 234 | C
2 | 200 | D
Now suppose the counter limit has reached, so I have to reset it to 0: as a consequence, the next inserted item will have counter 0: for example, if I insert an element with priority equal to 1, it will be wrongly inserted before 1 | 234 | D
0 | 123 | A
0 | 345 | B
1 | 000 | new element
1 | 234 | C
2 | 200 | D
The problem of the priority can be solved by implementing an heap: I created an Heap class, then I used Heap<KeyValuePair<TPriority, TElement> and a custom PriorityComparer in order to sort elements by TPriority.
Given TPriority as an int and TElement as a string, the PriorityComparer is as follows:
public class MyComparer : IComparer<KeyValuePair<int, string>>
{
public int Compare(KeyValuePair<int, string> x, KeyValuePair<int, string> y)
{
return x.Key.CompareTo(y.Key);
}
}
...
int capacity = 10;
Heap<KeyValuePair<int, string>> queue;
queue = new Heap<KeyValuePair<int, string>>(capacity, new PriorityComparer());
...
UPDATE
In this way (using the PriorityComparer), I have succeeded to implement a priority queue.
Now I'd like to add support to modify its behavior at runtime, ie switch from FIFO to priority sorting and vice-versa. Since my implementation of priority queue has an IComparer field, I think it is sufficient to add a Comparer property to edit this field, like as follows:
public IComparer
{
set
{
this._comparer = value;
}
}
In the meantime I thought I'd take a different approach: instead of using a binary heap to manage priorities, I could wrap different queues (each queue refers to a given priority) as follows.
public class PriorityQueue<T, int>
{
private Queue<T> _defaultQueue;
private bool _priority;
private SortedList<int, Queue<T>> _priorityQueues;
public PriorityQueue(int capacity)
{
this._defaultQueue = new Queue<T>(capacity);
this._priority = false;
this._priorityQueues = new SortedList<int, Queue<T>>(0);
}
public void PriorityEnable()
{
this._priority = true;
}
public void PriorityDisable()
{
this._priority = false;
}
public void Enqueue(T item)
{
if (this._priority)
{
// enqueue to one of the queues
// with associated priority
// ...
}
else this._defaultQueue.Enqueue(item);
}
public T Dequeue()
{
if (this._priority)
{
// dequeue from one of the queues
// with associated priority and
// return
// ...
}
return this._defaultQueue.Dequeue();
}
}
How to manage the transition from FIFO mode to priority mode when there are still elements in the default queue? I could copy them in the priority queues based on the item priority... Other better solutions?
How to manage the transition from priority mode to FIFO mode? In this case, I would have several priority queues, which may contain elements, but no longer have to manage them according to priority and not even know the original order of arrival...
How can I manage the capacity of the different queues?
What about the performances of the above two solutions? Which does use more memory?
You could "cheat" and use BigInteger so you never "run out of numbers". This of course leads to gradual deterioration of performance over time, but probably not significant enough to matter.
Combine that with a heap-based priority queue and you are set!
Don't try to "switch from FIFO to priority sorting and vice-versa" - simply put elements in both data structures appropriate for the task (Queue and priority queue).
Using both Queue and Priority Queue is what I would do.
But if you must...
Instead of one key use 2 keys for an element.
The first key priority will be the priority.
The second key time will be a counter that will be like a timestamp.
For the regular behavior use the priority key.
When the heap is full, HEAPIFY it by the time key.
Then remove n needed elements.
Now HEAPIFY it again with the prioritykey to return to the regular behavior.
EDIT: You have kind of changed what you are asking with your edits. You went from asking one question to doing a new approach and asking a new question. Should probably open a new question for your new approach, as this one is now confusing as to what answer/response is to what question/comment. I believe your original question about sorting equal priorities has been answered.
You could use a long to allow for more values. You will always reach an end eventually, so you would need to use a new pattern for unique values or 'recount' the items when the max is reached (loop through each and reset the unique count value).
Maybe use a GUID for each item instead?
Guid.NewGuid()
EDIT:
To add after your edit: If you want the new 1 to be placed after the existing, In the Compare override, return a greater than result (1) when the values are equal. That way the following will happen:
1 > 0, return greater (1), continue
1 > 0, return greater (1), continue
1 == 1, return greater (1), continue
1 < 2, return less than (-1), insert
EDIT 2:
If the second parameter is only meant to be a unique value, you could always use a string and set the value as numeric strings instead. That way you will never reach a cap, would just have to parse the string accordingly. You can use leading alpha values that represent a new set.
I have not tested this code, just an idea as to what you could do.
static string leadingStr = "";
static char currentChar = 'a';
static Int32 currentId = Int32.MinValue;
static string getNextId()
{
if (currentId >= Int32.MaxValue)
{
currentId = Int32.MinValue;
if (currentChar >= 'z')
{
currentChar = 'a';
leadingStr = leadingStr.Insert(0, "X");
}
else
currentChar++;
}
else
currentId++;
return String.Format("{0}{1}-{2}", leadingStr, currentChar, currentId);
}
EDIT 3: Reset Values
static Int64 currentValue = Int64.MinValue;
static void AddItem(object item)
{
if (currentValue == Int64.MaxValue)
RecountItems();
item.counter = currentValue++;
SortedList.Add(item);
}
static void RecountItems()
{
currentValue = 0;
foreach (var item in SortedList)
{
item.counter = currentValue++;
}
}
Edit 4: For your second question:
You could use a FIFO stack as you normally would, but also have a priority List that only stores the unique ID of the items. However you would then need to remove the item from the list every time you remove from the FIFO stack.
static Object RemoveNextFIFO()
{
if (fifoList.Count > 0)
{
var removedItem = fifoList[0];
fifoList.RemoveAt(0);
RemoveItemFromPriority(removedItem);
return removedItem;
}
}
static void RemoveItemFromPriority(Object itemToRemove)
{
foreach (var counter in priorityQueue)
{
if (counter == itemToRemove.counter)
{
priorityQueue.remove(item);
break;
}
}
}
static Object RemoveFromFIFO(int itemCounter)
{
foreach (var item in fifoList)
{
if (item.counter == itemCounter)
{
fifoList.Remove(item);
return item;
}
}
}
static Object RemoveNextPriority()
{
if (priorityQueue.Count > 0)
{
var counter = priorityQueue.Pop();
return RemoveFromFIFO(counter);
}
}
Related
I have 3 threads in 3 classes running in parallel. Each of them, increase Pos or Neg of the Fourthclass by "1". After 3 threads are done, if Fourclass.Pos > Fourclass.Neg, it will run Terminal4.
Q: How can i run the Terminal4 only 1 time. Because putting Fourthclass.Terminal4(); in each Terminal1-2-3 will run the Terminal4 3 times.
Here is what i have done:
public class Firstclass
{
static int Pos = 1;
static int Neg = 0;
public static void Terminal1()
{
if (Pos > Neg)
{
Fourthclass.Pos += 1;
// Fourthclass.Terminal4();
}
}
}
public class Secondclass
{
static int Pos = 1;
static int Neg = 0;
public static void Terminal2()
{
if (Pos > Neg)
{
Fourthclass.Pos += 1;
// Fourthclass.Terminal4();
}
}
}
public class Thirdclass
{
static int Pos = 1;
static int Neg = 0;
public static void Terminal3()
{
if (Pos > Neg)
{
Fourthclass.Neg += 1;
// Fourthclass.Terminal4();
}
}
}
public static class Fourthclass
{
public static int Pos = 0;
public static int Neg = 0;
public static void Terminal4()
{
if (Pos > Neg)
{
Console.WriteLine("Pos = {0} - Neg = {1}", Pos, Neg);
Console.WriteLine();
}
else { Console.WriteLine("fail"); }
}
}
class Program
{
static void Main(string[] args)
{
Thread obj1 = new Thread(new ThreadStart(Firstclass.Terminal1));
Thread obj2 = new Thread(new ThreadStart(Secondclass.Terminal2));
Thread obj3 = new Thread(new ThreadStart(Thirdclass.Terminal3));
obj1.Start();
obj2.Start();
obj3.Start();
}
}
Original Answer
By the by... these increments are not thread safe, they may suffer the ABA problem and that is ignoring thread visibility problems.
For that problem, please use Interloked. Interlocked.Increment and Interlocked.Decrement will take care of it.
Now, for making a code block run only once, keep an int variable that will be 1 if it did run, and 0 if it did not. Then use Interlocked.CompareExchange:
int didrun;
// ...
if (Interlocked.CompareExchange(ref didrun, 1, 0) == 0)
{
// code here will only run once, until you put didrun back to 0
}
There are other ways, of course. This one is just very versatile.
Addendum: Ok, what it does...
Interlocked.CompareExchange will look at the value of the passed variable (didrun) compare it to the second parameter (0) and if it matches, it will change the variable to the value of first parameter (1) [Since it may change the variable, you have to pass it by ref]. The return value is what it found in the variable.
Thus, if it returns 0, you know it found 0, which means that it did update the value to 1. Now, the next time this piece of code is called, the value of the variable is 1, so Interlocked.CompareExchange returns 1 and the thread does not enter the block.
Ok, why do not use a bool instead? Because of thread visibility. A thread may change the value of the variable, but this update could happen in CPU cache only, and not be visible to other threads... Interlocked gets around of that problem. Just use Interlocked. And MSDN is your friend.
That will work regardless if you are using ThreadPool, Task or async/await or just plain old Threads as you do. I mention that because I would like to suggest using those...
Sneaky link to Threading in C#.
Extended Answer
In comment, you ask about a different behavior:
The Terminal4 has cooldown until the next run
Well, if there is a cool down (which I understand as a period) you do not only need to store whatever or not the code did run, but also when was the last time it did.
Now, the conditional cannot be just "run only if it has not run yet" but instead "run if it has not run yet or if the period from the last time it ran to now is greater than the cool down".
We have to check multiple things, that is a problem. Now the check will no longer be atomic (from the Latin atomus which means indivisible, from a- "not" + tomos "a cutting, slice, volume, section").
That is relevant because if the check is not atomic, we are back to the ABA problem.
I will use this case to explain the ABA problem. If we encode the following:
1. Check if the operation has not run (if it has not go to 4)
2. Get the last time it ran
3. Compute the difference from the last run to now (exit if less than cool down)
4. Update the last run time to now
5. Run code
Two threads may do the following:
|
t Thread1: Check if the operation has not run (it has)
i Thread2: Check if the operation has not run (it has)
m Thread2: Get the last time it ran
e Thread1: Get the last time it ran
| Thread1: Compute the difference from the last run to now (more than cool down)
v Thread2: Compute the difference from the last run to now (more than cool down)
Thread2: Update the last run time to now
Thread2: Run code
Thread1: Update the last run time to now
Thread1: Run code
As you see, they both Run code.
What we need is a way to check and update in a single atomic operation, that way the act of checking will alter the result of the other thread. That is what we get with Interlocked.
How Interlocked manages to do that is beyond the scope of the question. Suffice to say that there are some special CPU instructions for that.
The new pattern I suggest is the following (pseudocode):
bool canRun = false;
DateTime lastRunCopy;
DateTime now = DateTime.Now;
if (try to set lastRun to now if lastRun is not set, copy lastRun to lastRunCopy)
{
// We set lastRun
canRun = true;
}
else
{
if ((now - lastRunCopy) < cooldown)
{
if (try to set lastRun to now if lastRun = lastRunCopy, copy lastRun to lastRunCopy)
{
// we updated it
canRun = true;
}
}
else
{
// Another thread got in
}
}
if (canRun)
{
// code here will only run once per cool down
}
Notice I have expressed the operations in terms of "try to set X to Y if X is Z, copy X to W" which is how Interlocked.CompareExchange works.
There is a reason I left that in pseudo code, and that is that DateTime is not an atomic type.
In order to make the code work we will have to use DateTime.Ticks. For an unset value we will use 0 (00:00:00.0000000 UTC, January 1, 0001), which is something you have to worry about for a cool down greater than a couple of millennia.
In addition, of course, we will use the overload of Interlocked.CompareExchange that takes long because DateTime.Ticks is of that type.
Note: Ah, we will use TimeSpan.Ticks for the cool down.
The code is as follows:
long lastRun = 0;
long cooldown = TimeSpan.FromSeconds(1).Ticks; // Or whatever, I do not know.
// ...
bool canRun = false;
long lastRunCopy = 0;
long now = DateTime.Now.Ticks;
lastRunCopy = Interlocked.CompareExchange(ref lastRun, now, 0);
if (lastRunCopy == 0)
{
// We set lastRun
canRun = true;
}
else
{
if ((now - lastRunCopy) < cooldown)
{
if (Interlocked.CompareExchange(ref lastRun, now, lastRunCopy) == lastRunCopy)
{
// we updated it
canRun = true;
}
else
{
// Another thread got in
}
}
}
if (canRun)
{
// code here will only run once per cooldown
}
Alternatively, if you want to boil it down to a single conditional:
long lastRun = 0;
long cooldown = TimeSpan.FromSeconds(1).Ticks; // Or whatever, I do not know.
// ...
long lastRunCopy;
var now = DateTime.Now.Ticks;
if
(
(lastRunCopy = Interlocked.CompareExchange(ref lastRun, now, 0)) == 0
|| now - lastRunCopy < cooldown
&& Interlocked.CompareExchange(ref lastRun, now, lastRunCopy) == lastRunCopy
)
{
// code here will only run once per cooldown
}
As I said, Interlocked.CompareExchange is versatile. Although, as you can see, you still need to think around the requirements.
I'm wondering if there's an inbuilt collection (or any way to make a custom one) in C# that can be used to emit numbers in a rotating/cyclic fashion (see example below), and is thread-safe (so each thread gets the next number in the collection).
Collection with 5 sequential numbers:
Thread 1 read: return value 1
Thread 2 read: return value 2
Thread 3 read: return value 3
Thread 4 read: return value 4
Thread 5 read: return value 5
Thread 6 read: return value 1
Thread 7 read: return value 2
Thread 8 read: return value 3
and so on.
Basically, the next number emitted by the collection (when read by a thread) should be one after the previous one, and it should restart from the beginning at the end of the number set.
You can greate a method, that will return infinate enumerable:
private object _lock = new object();
private int i = 0;
private int max = 6;
public IEnumerable<int> GetNumbers()
{
while (true)
{
lock (_lock)
{
i++;
if (i == max)
i = 1;
yield return i;
}
}
}
And get them:
var numbers = GetNumbers().Take(1000).ToArray();
I tried to use recursion for the problem at hand as follows,
int newlevelgen()
{
int exampleno = Random.Range (1,4);
if (exampleno != lastlevelno)
{
lastlevelno = exampleno;
return exampleno;
}
else
{
newlevelgen();
}
return exampleno;
}
This is my code above, what I want to do is generate new number without repeating the previous one, but this simply does not work. Help!
The idea is that you get a value between a and b. If that value is greater of equal to previous value, then you increase that and return it. Other case, you return as is.
Think of it, it does work.
public int GetUniqueLevelInclusiveOrdinal(int a , int b, int previous){
// given the ordinal numbers from a to b INCLUSIVE,
// so including a and b,
// (*** NOTE, no random int calls in Unity or any system
// work inclusively so take care ***)
// choose one of the numbers from a to b inclusive
// but do not choose "previous"
// which top index to use with Random.Range which is exclusive?
int top = b+1;
// everyday algorithm for excluding one item from a random run
int topExcludeOne = top-1;
int value = Random.Range(a, topExcludeOne);
if (value >= previous) return value+1;
else return value;
}
This is an extremely well-known, basic, pattern in programming...
int result = UnityEngine.Random.Range(0,highest-1);
if (result>=exclude)
return result+1;
else
return result;
In Unity you must use extensions:
public static int RandomIndexButNotThis(this int highest, int exclude)
{
if (highest<2) Debug.Break();
int result = UnityEngine.Random.Range(0,highest-1);
if (result>=exclude)
return result+1;
else
return result;
}
To get a random index 0 to 99
Random.Range(0,100)
To get a random index 0 to 99, but excluding 61
100.RandomIndexButNotThis(61)
To get a random index 0 to 9
Random.Range(0,10)
To get a random index 0 to 9, but excluding 8
10.RandomIndexButNotThis(8)
If new to Unity, intro to extensions
Let me preface this by saying that the following is an answer to the problem with the original method that was posted. It is far from that best way to get the desired result but that's not the focus of this answer. Yes there is a chance that the recursion will go on for a few calls. 33% chance to need another recursive call, though that does not mean a 33% for an infinite loop as the likelihood of needing X recursive calls is 0.33^X so the likelihood of reaching 3 recursive calls is only 0.33^3 ~= 0.036.
Still, a different method is advised. See Joe Blow's answer for example.
There's a problem with your recursion. You aren't using the new value that the recursive call to newlevelgen() would give you in case exampleno is the same as lastlevelno and always returning the (possibly duplicate) exampleno value. Change it to:
int newlevelgen()
{
int exampleno = Random.Range (1,4);
if (exampleno != lastlevelno)
{
lastlevelno = exampleno;
return exampleno;
}
else
{
return newlevelgen();
}
}
I created utility to easely handle random without repetitions, flat distributed random, and weighted lists (universal property drawer included!). You can find it on GtiHub and use freely:
https://github.com/khadzhynov/RandomUtils
I have a very specific and long-winded question for you all. This question is both about programming and game-theory. I recently added spawnable ore to my Turn Based Strategy Game: http://imgur.com/gallery/0F5D5Ij (For those of you that look please forgive the development textures).
Now, onto the enigma that I have been contemplating. In my game, ore is generated each time a new map is created. 0-8 ore nodes are generated per level-creation. I already have this working; except it only generates "Emeraldite" at this point, which brings me to my question.
How would I, the programmer, make it so nodes have specific rarity? Consider this short mockup which is not actually game data:
(Pseudo Chances that a node will be one of the following)
Bloodstone 1 in 100
Default(Empty Node) 1 in 10
Copper 1 in 15
Emeraldite 1 in 35
Gold 1 in 50
Heronite 1 in 60
Platinum 1 in 60
Shadownite 1 in 75
Silver 1 in 35
Soranite 1 in 1000
Umbrarite 1 in 1000
Cobalt 1 in 75
Iron 1 in 15
I want to make it so that a generated node could be, theoretically, any of the above, however, with the odds also considered. I hope that question is clear enough. I have been trying to wrap my head around this, and even tried to write out a few if statements with randoms, however, I keep coming up empty handed.
Basically, I just want you guys to see my issue, and hopefully provide me with some insight on how I could approach this in a dynamic kind of way.
If any clarification is needed, please ask; sorry again if this was convoluted.
(I am adding C# as a tag only because that is the language I am using for this project)
I'd first represent the probability of each loot type as a simple number.
A probability in pure mathematics is conventionally expressed as a floating point number in the range 0 to 1, but for efficiency, you can use integers in any (large enough) range (each value is the 0-1 value multiplied by the maximum (which I'm calling MaxProbability here)).
e.g. Bloodstone (1 in 100) is 1/100 = 0.01, or MaxProbability * (1/100).
Copper (1 in 15) is 1/15 = 0.06667, or MaxProbability * (1/15).
I'm assuming that 'Default (Empty Node)' means the probability of none of the others.
In this case, the simplest way is not to define it - you get it if none of the others are chosen.
If 'Default' was included, the sum of all these probabilities would be 1 (i.e. 100%) (or MaxProbability, if using integers).
The 1/10 probability of 'Default' in your example is actually a contradiction because the total of all those probabilities is not 1 (it's 0.38247619 - the sum of the probability as calculated in my examples above).
Then you would choose a random number in the range 0 to 1 (or MaxProbability if using integers), and the chosen loot type is the first one in the list such that the sum of the probabilities of it and all previous ones ("cumulative probability") is greater than the random number.
e.g.
MaxProbability = 1000 (I'm using this to make it easy to read).
(For accurate probabilities, you could use 0x7FFFFFFF).
Type Probability Cumulative
---- ----------- ----------
Bloodstone 10 10 (0..9 yield Bloodstone)
Copper 67 77 (10+67) (10..76 yield Copper)
Emeraldite 29 105 (77+29)
Gold 20 125 etc.
Heronite 17 142
Platinum 17 159
Shadownite 13 172
Silver 29 200
Soranite 1 201
Umbrarite 1 202
Cobalt 13 216
Iron 67 282
Default (Empty Node) 7175 1000 (anything else)
e.g. If your random number in the range 0 to 999 (inclusive) was 184 (or anything in the range 172 to 199), you would choose "Silver" (the first one with cumulative probability greater than this).
You could hold the cumulative probabilities in an array and loop through it until you find one higher than the random number, or reach the end.
The order of the list does not matter.
You chose a random number only once per instance.
Including 'Default (Empty Node)' in the list means that the last cumulative probability will always be MaxProbability and the loop that searches it would never go past the end. (Alternatively, 'Default' can be omitted, and you choose it if the loop reaches the end of the list.)
Note that choosing a random number for each one in turn, e.g. a 1/10 chance of 'Bloodstone', then a 1/15 chance of Copper if not Bloodstone, skews the probabilities towards the earlier items:
The actual probability of Copper would be (1/15) * (1 - (1/10)) - 10% less than 1/15.
Here's code to do it (the actual choosing is 5 statements - in the method Choose ).
using System;
namespace ConsoleApplication1
{
class LootChooser
{
/// <summary>
/// Choose a random loot type.
/// </summary>
public LootType Choose()
{
LootType lootType = 0; // start at first one
int randomValue = _rnd.Next(MaxProbability);
while (_lootProbabilites[(int)lootType] <= randomValue)
{
lootType++; // next loot type
}
return lootType;
}
/// <summary>
/// The loot types.
/// </summary>
public enum LootType
{
Bloodstone, Copper, Emeraldite, Gold, Heronite, Platinum,
Shadownite, Silver, Soranite, Umbrarite, Cobalt, Iron, Default
};
/// <summary>
/// Cumulative probabilities - each entry corresponds to the member of LootType in the corresponding position.
/// </summary>
protected int[] _lootProbabilites = new int[]
{
10, 77, 105, 125, 142, 159, 172, 200, 201, 202, 216, 282, // (from the table in the answer - I used a spreadsheet to generate these)
MaxProbability
};
/// <summary>
/// The range of the probability values (dividing a value in _lootProbabilites by this would give a probability in the range 0..1).
/// </summary>
protected const int MaxProbability = 1000;
protected Random _rnd = new Random((int)(DateTime.Now.Ticks & 0x7FFFFFFF));
/// <summary>
/// Simple 'main' to demonstrate.
/// </summary>
/// <param name="args"></param>
static void Main(string[] args)
{
var chooser = new LootChooser();
for(int n=0; n < 100; n++)
Console.Out.WriteLine(chooser.Choose());
}
}
}
You could rewrite all chances so that they use the same divisor (e.g. 1000), your chances then become
Bloodstone 10 in 1000
Default(Empty Node) 100 in 1000
Gold 20 in 1000
Next, create an array of a 1000 elements, and fill it with
10 Bloodstone elements,
100 Empty elements,
20 Gold elements,
etc.
Finally, generate a random number between 0 and 1000, and use that as the index into the element array, this will give
you your random element.
You might have to play with the chances a bit, since you'll probably want all 1000 array elements to be filled, but this is the general idea.
edit its not the most efficient implementation (at least in terms of memory usage, its running time should be good), but I chose this since it allows for a concise explanation that doesn't require a whole lot of math.
First of all, specifying the the default-empty node's probability is unnecessary. The other probabilities should be defined in such a way, that the empty node is created if no other type is created.
How to do this and ensure the generation probabiltiies are equal to those you specified? In short:
convert the probabilities to a floating point (it's a value with a common divisor of 1)
sum all probabilities and check if they are < 1
write a class which will store the all the probabilities
write a function which will get a random node based on those probabilities
For your example:
Bloodstone 1 in 100 = 0.01
Copper 1 in 15 ~= 0.07
Emeraldite 1 in 35 ~= 0.03
Gold 1 in 50 = 0.02
Default = 0.87
Now the class can be implemented in at least two ways. My option consumes much memory, does the computations once, but it also rounds the probability values which may introduce some error. Note, that the error depends on the arrSize variable - the larger it is, the smaller the error.
The other option is as in Bogusz's answer. It is more precise, but required more operations per each generated element.
Option suggested by Thomas requires a lot of repeatable code for each option hence is not versatile. Shellshock's answer will have invalid effective probabilities.
Astrotrain's idea to force yourself to use the same divisor is virtually the same as my own, though the implementation would be slightly different.
Here's a sample implementation of my idea (in java, but should be ported very easily):
public class NodeEntry {
String name;
double probability;
public NodeEntry(String name, double probability) {
super();
this.name = name;
this.probability = probability;
}
public NodeEntry(String name, int howMany, int inHowMany) {
this.name = name;
this.probability = 1.0 * howMany / inHowMany;
}
public final String getName() {
return name;
}
public final void setName(String name) {
this.name = name;
}
public final double getProbability() {
return probability;
}
public final void setProbability(double probability) {
this.probability = probability;
}
#Override
public String toString() {
return name+"("+probability+")";
}
static final NodeEntry defaultNode = new NodeEntry("default", 0);
public static final NodeEntry getDefaultNode() {
return defaultNode;
}
}
public class NodeGen {
List<NodeEntry> nodeDefinitions = new LinkedList<NodeEntry>();
public NodeGen() {
}
public boolean addNode(NodeEntry e) {
return nodeDefinitions.add(e);
}
public boolean addAllNodes(Collection<? extends NodeEntry> c) {
return nodeDefinitions.addAll(c);
}
static final int arrSize = 10000;
NodeEntry randSource[] = new NodeEntry[arrSize];
public void compile() {
checkProbSum();
int offset = 0;
for (NodeEntry ne: nodeDefinitions) {
int amount = (int) (ne.getProbability() * arrSize);
for (int a=0; a<amount;a++) {
randSource[a+offset] = ne;
}
offset+=amount;
}
while (offset<arrSize) {
randSource[offset] = NodeEntry.getDefaultNode();
offset++;
}
}
Random gen = new Random();
public NodeEntry getRandomNode() {
return randSource[gen.nextInt(arrSize)];
}
private void checkProbSum() {
double sum = 0;
for (NodeEntry ne: nodeDefinitions) {
sum+=ne.getProbability();
}
if (sum >1) {
throw new RuntimeException("nodes probability > 1");
}
}
public static void main(String[] args) {
NodeGen ng = new NodeGen();
ng.addNode(new NodeEntry("Test 1", 0.1));
ng.addNode(new NodeEntry("Test 2", 0.2));
ng.addNode(new NodeEntry("Test 3", 0.2));
ng.compile();
Map<NodeEntry, Integer> resCount = new HashMap<NodeEntry, Integer>();
int generations = 10000;
for (int a=0; a<generations; a++) {
NodeEntry node = ng.getRandomNode();
Integer val = resCount.get(node);
if (val == null) {
resCount.put(node, new Integer(1));
} else {
resCount.put(node, new Integer(val+1));
}
}
for (Map.Entry<NodeEntry, Integer> entry: resCount.entrySet()) {
System.out.println(entry.getKey()+": "+entry.getValue()+" ("+(100.0*entry.getValue()/generations)+"%)");
}
}
}
This makes sure the probabilities are actually uniform. If you checked for the first node spawn, then the other, then the other - you would get wrong results: nodes checked first would have increased probability.
Sample run:
Test 2(0.2): 1975 (19.75%)
Test 1(0.1): 1042 (10.42%)
Test 3(0.2): 1981 (19.81%)
default(0.0): 5002 (50.02%)
I think that it is easy to understand how it works.
(Cobalt, 20: means 1 of 20 -> 5%)
Dictionary<string, double> ore = new Dictionary<string, double>();
Random random = new Random();
private void AddOre(string Name, double Value)
{
ore.Add(Name, 1.0 / Value);
}
private string GetOreType()
{
double probSum = 0;
double rand = random.NextDouble();
foreach (var pair in ore)
{
probSum += pair.Value;
if (probSum >= rand)
return pair.Key;
}
return "Normal Ore"; //Reaches this point only if an error occurs.
}
private void Action()
{
AddOre("Cobalt", 20);
AddOre("Stone", 10);
AddOre("Iron", 100);
AddOre("GreenOre", 300);
//Add Common ore and sort Dictionary
AddOre("Common ore", 1 / (1 - ore.Values.Sum()));
ore = ore.OrderByDescending(x => x.Value).ToDictionary(x => x.Key, x => x.Value);
Console.WriteLine(GetOreType());
}
Edit:
I add section "Add Common ore and sort Dictionary".
I recently had to do something similar, and I ended up with this generic "spawn generator".
public interface ISpawnable : ICloneable
{
int OneInThousandProbability { get; }
}
public class SpawnGenerator<T> where T : ISpawnable
{
private class SpawnableWrapper
{
readonly T spawnable;
readonly int minThreshold;
readonly int maxThreshold;
public SpawnableWrapper(T spawnable, int minThreshold)
{
this.spawnable = spawnable;
this.minThreshold = minThreshold;
this.maxThreshold = this.minThreshold + spawnable.OneInThousandProbability;
}
public T Spawnable { get { return this.spawnable; } }
public int MinThreshold { get { return this.minThreshold; } }
public int MaxThreshold { get { return this.maxThreshold; } }
}
private ICollection<SpawnableWrapper> spawnableEntities;
private Random r;
public SpawnGenerator(IEnumerable<T> objects, int seed)
{
Debug.Assert(objects != null);
r = new Random(seed);
var cumulativeProbability = 0;
spawnableEntities = new List<SpawnableWrapper>();
foreach (var o in objects)
{
var spawnable = new SpawnableWrapper(o, cumulativeProbability);
cumulativeProbability = spawnable.MaxThreshold;
spawnableEntities.Add(spawnable);
}
Debug.Assert(cumulativeProbability <= 1000);
}
//Note that it can spawn null (no spawn) if probabilities dont add up to 1000
public T Spawn()
{
var i = r.Next(0, 1000);
var retVal = (from s in this.spawnableEntities
where (s.MaxThreshold > i && s.MinThreshold <= i)
select s.Spawnable).FirstOrDefault();
return retVal != null ? (T)retVal.Clone() : retVal;
}
}
And you'd use it like:
public class Gem : ISpawnable
{
readonly string color;
readonly int oneInThousandProbability;
public Gem(string color, int oneInThousandProbability)
{
this.color = color;
this.oneInThousandProbability = oneInThousandProbability;
}
public string Color { get { return this.color; } }
public int OneInThousandProbability
{
get
{
return this.oneInThousandProbability;
}
}
public object Clone()
{
return new Gem(this.color, this.oneInThousandProbability);
}
}
var RedGem = new Gem("Red", 250);
var GreenGem = new Gem("Green", 400);
var BlueGem = new Gem("Blue", 100);
var PurpleGem = new Gem("Purple", 190);
var OrangeGem = new Gem("Orange", 50);
var YellowGem = new Gem("Yellow", 10);
var spawnGenerator = new SpawnGenerator<Gem>(new[] { RedGem, GreenGem, BlueGem, PurpleGem, OrangeGem, YellowGem }, DateTime.Now.Millisecond);
var randomGem = spawnGenerator.Spawn();
Evidently the spawn algorithm was not considered critical code so the overhead of this implementation was of no concern when compared to the ease of use. Spawns were run on world creation and it was easily more than fast enough.
Astrotrain already gave my answer but since I coded it up already I'll post it. Sorry for the syntax, I work mostly in Powershell and that is the context currently in my mind. Consider this psuedo code:
// Define the odds for each loot type
// Description,Freq,Range
LootOddsArray = "Bloodstone",1,100,
"Copper",1,15,
"Emeraldite,"1,35,
"Gold",1,50,
"Heronite",1,60,
"Platinum",1,60,
"Shadownite",1,75,
"Silver",1,35,
"Soranite",1,1000,
"Umbrarite",1,1000,
"Cobalt",1,75,
"Iron",1,15
// Define your lookup table. It should be as big as your largest odds range.
LootLookupArray(1000)
// Fill all the 'default' values with "Nothing"
for (i=0;i<LootLookupArray.length;i++) {
LootOddsArray(i) = "Nothing"
}
// Walk through your various treasures
for (i=0;i<LootOddsArray.length;i++)
// Calculate how often the item will appear in the table based on the odds
// and place that many of the item in random places in the table, not overwriting
// any other loot already in the table
NumOccsPer1000 = Round(LootOddsArray(i).Freq * 1000/LootOddsArray(i).Range)
for (l=0;l<NumOccsPer1000;l++) {
// Find an empty slot for the loot
do
LootIndex = Random(1000)
while (LootLookupArray(LootIndex) != "Nothing")
// Array(Index) is empty, put loot there
LootLookupArray(LootIndex) = LootOddsArray(i).Description
}
}
// Roll for Loot
Loot = LootLookupArray(Random(1000))
Use Random.Next http://msdn.microsoft.com/en-us/library/2dx6wyd4(v=vs.110).aspx:
Random rnd = new Random();
if (rnd.Next(1, 101) == 1)
// spawn Bloodstone
if (rnd.Next(1, 16) == 1)
// spawn Copper
if (rnd.Next(1, 36) == 1)
// spawn Emeraldite
The minimum value should always be 1, the maximum value is the odds of spawning the item + 1 (minValue is inclusive, maxValue is exclusive). Always test the return value for 1, e.g., for Bloodstone there is a 1 in 100 chance that the randomly generated number is 1. Of course, this uses a pseudo random number generator, which should be good enough for a game.
A slightly different approach to Astrotrains idea would be that instead of an array to use if statements . The upside is then that you need less memory, the downside that it will need more cpu time to calc the value of the node.
Thus:
Random rnd = new Random();
var number = rnd.next(1,1000);
if (number >= 1 && number <10)
{
// empty
}
else
{
if (number >= 10 && number <100)
{
// bloodstone
}
else
{
//......
}
}
Also a downside to this variant comapred to the array variant is that this one takes more place codewise at the location where you use it and is more prone to errors / corrections (try to add something inside it you need to update all variants).
Thus this here is metnioned for completeness sake but the array vairant (memory usage aside) is less prone to the problems that the if variant has.
I'm working on an academic open source project and now I need to create a fast blocking FIFO queue in C#. My first implementation simply wrapped a synchronized queue (w/dynamic expansion) within a reader's semaphore, then I decided to re-implement in the following (theorically faster) way
public class FastFifoQueue<T>
{
private T[] _array;
private int _head, _tail, _count;
private readonly int _capacity;
private readonly Semaphore _readSema, _writeSema;
/// <summary>
/// Initializes FastFifoQueue with the specified capacity
/// </summary>
/// <param name="size">Maximum number of elements to store</param>
public FastFifoQueue(int size)
{
//Check if size is power of 2
//Credit: http://stackoverflow.com/questions/600293/how-to-check-if-a-number-is-a-power-of-2
if ((size & (size - 1)) != 0)
throw new ArgumentOutOfRangeException("size", "Size must be a power of 2 for this queue to work");
_capacity = size;
_array = new T[size];
_count = 0;
_head = int.MinValue; //0 is the same!
_tail = int.MinValue;
_readSema = new Semaphore(0, _capacity);
_writeSema = new Semaphore(_capacity, _capacity);
}
public void Enqueue(T item)
{
_writeSema.WaitOne();
int index = Interlocked.Increment(ref _head);
index %= _capacity;
if (index < 0) index += _capacity;
//_array[index] = item;
Interlocked.Exchange(ref _array[index], item);
Interlocked.Increment(ref _count);
_readSema.Release();
}
public T Dequeue()
{
_readSema.WaitOne();
int index = Interlocked.Increment(ref _tail);
index %= _capacity;
if (index < 0) index += _capacity;
T ret = Interlocked.Exchange(ref _array[index], null);
Interlocked.Decrement(ref _count);
_writeSema.Release();
return ret;
}
public int Count
{
get
{
return _count;
}
}
}
This is the classic FIFO queue implementation with static array we find on textbooks. It is designed to atomically increment pointers, and since I can't make the pointer go back to zero when reached (capacity-1), I compute modulo apart. In theory, using Interlocked is the same as locking before doing the increment, and since there are semaphores, multiple producers/consumers may enter the queue but only one at a time is able to modify the queue pointers.
First, because Interlocked.Increment first increments, then returns, I already understand that I am limited to use the post-increment value and start store items from position 1 in the array. It's not a problem, I'll go back to 0 when I reach a certain value
What's the problem with it?
You wouldn't believe that, running on heavy loads, sometimes the queue returns a NULL value. I am SURE, repeat, I AM SURE, that no method enqueues null into the queue. This is definitely true because I tried to put a null check in Enqueue to be sure, and no error was thrown. I created a test case for that with Visual Studio (by the way, I use a dual core CPU like maaaaaaaany people)
private int _errors;
[TestMethod()]
public void ConcurrencyTest()
{
const int size = 3; //Perform more tests changing it
_errors = 0;
IFifoQueue<object> queue = new FastFifoQueue<object>(2048);
Thread.CurrentThread.Priority = ThreadPriority.AboveNormal;
Thread[] producers = new Thread[size], consumers = new Thread[size];
for (int i = 0; i < size; i++)
{
producers[i] = new Thread(LoopProducer) { Priority = ThreadPriority.BelowNormal };
consumers[i] = new Thread(LoopConsumer) { Priority = ThreadPriority.BelowNormal };
producers[i].Start(queue);
consumers[i].Start(queue);
}
Thread.Sleep(new TimeSpan(0, 0, 1, 0));
for (int i = 0; i < size; i++)
{
producers[i].Abort();
consumers[i].Abort();
}
Assert.AreEqual(0, _errors);
}
private void LoopProducer(object queue)
{
try
{
IFifoQueue<object> q = (IFifoQueue<object>)queue;
while (true)
{
try
{
q.Enqueue(new object());
}
catch
{ }
}
}
catch (ThreadAbortException)
{ }
}
private void LoopConsumer(object queue)
{
try
{
IFifoQueue<object> q = (IFifoQueue<object>)queue;
while (true)
{
object item = q.Dequeue();
if (item == null) Interlocked.Increment(ref _errors);
}
}
catch (ThreadAbortException)
{ }
}
Once a null is got by the consumer thread, an error is counted.
When performing the test with 1 producer and 1 consumer, it succeeds. When performing the test with 2 producers and 2 consumers, or more, a disaster happens: even 2000 leaks are detected. I found that the problem can be in the Enqueue method. By design contract, a producer can write only into a cell that is empty (null), but modifying my code with some diagnostics I found that sometimes a producer is trying to write on a non-empty cell, which is then occupied by "good" data.
public void Enqueue(T item)
{
_writeSema.WaitOne();
int index = Interlocked.Increment(ref _head);
index %= _capacity;
if (index < 0) index += _capacity;
//_array[index] = item;
T leak = Interlocked.Exchange(ref _array[index], item);
//Diagnostic code
if (leak != null)
{
throw new InvalidOperationException("Too bad...");
}
Interlocked.Increment(ref _count);
_readSema.Release();
}
The "too bad" exception happens then often. But it's too strange that a conflict raises from concurrent writes, because increments are atomic and writer's semaphore allows only as many writers as the free array cells.
Can somebody help me with that? I would really appreciate if you share your skills and experience with me.
Thank you.
I must say, this struck me as a very clever idea, and I thought about it for a while before I started to realize where (I think) the bug is here. So, on one hand, kudos on coming up with such a clever design! But, at the same time, shame on you for demonstrating "Kernighan's Law":
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
The issue is basically this: you are assuming that the WaitOne and Release calls effectively serialize all of your Enqueue and Dequeue operations; but that isn't quite what is going on here. Remember that the Semaphore class is used to restrict the number of threads accessing a resource, not to ensure a particular order of events. What happens between each WaitOne and Release is not guaranteed to occur in the same "thread-order" as the WaitOne and Release calls themselves.
This is tricky to explain in words, so let me try to provide a visual illustration.
Let's say your queue has a capacity of 8 and looks like this (let 0 represent null and x represent an object):
[ x x x x x x x x ]
So Enqueue has been called 8 times and the queue is full. Therefore your _writeSema semaphore will block on WaitOne, and your _readSema semaphore will return immediately on WaitOne.
Now let's suppose Dequeue is called more or less concurrently on 3 different threads. Let's call these T1, T2, and T3.
Before proceeding let me apply some labels to your Dequeue implementation, for reference:
public T Dequeue()
{
_readSema.WaitOne(); // A
int index = Interlocked.Increment(ref _tail); // B
index %= _capacity;
if (index < 0) index += _capacity;
T ret = Interlocked.Exchange(ref _array[index], null); // C
Interlocked.Decrement(ref _count);
_writeSema.Release(); // D
return ret;
}
OK, so T1, T2, and T3 have all gotten past point A. Then for simplicity let's suppose they each reach line B "in order", so that T1 has an index of 0, T2 has an index of 1, and T3 has an index of 2.
So far so good. But here's the gotcha: there is no guarantee that from here, T1, T2, and T3 are going to get to line D in any specified order. Suppose T3 actually gets ahead of T1 and T2, moving past line C (and thus setting _array[2] to null) and all the way to line D.
After this point, _writeSema will be signaled, meaning you have one slot available in your queue to write to, right? But your queue now looks like this!
[ x x 0 x x x x x ]
So if another thread has come along in the meantime with a call to Enqueue, it will actually get past _writeSema.WaitOne, increment _head, and get an index of 0, even though slot 0 is not empty. The result of this will be that the item in slot 0 could actually be overwritten, before T1 (remember him?) reads it.
To understand where your null values are coming from, you need only to visualize the reverse of the process I just described. That is, suppose your queue looks like this:
[ 0 0 0 0 0 0 0 0 ]
Three threads, T1, T2, and T3, all call Enqueue nearly simultaneously. T3 increments _head last but inserts its item (at _array[2]) and calls _readSema.Release first, resulting in a signaled _readSema but a queue looking like:
[ 0 0 x 0 0 0 0 0 ]
So if another thread has come along in the meantime with a call to Dequeue (before T1 and T2 are finished doing their thing), it will get past _readSema.WaitOne, increment _tail, and get an index of 0, even though slot 0 is empty.
So there's your problem. As for a solution, I don't have any suggestions at the moment. Give me some time to think it over... (I'm posting this answer now because it's fresh in my mind and I feel it might help you.)
(+1 to Dan Tao who I vote has the answer)
The enqueue would be changed to something like this...
while (Interlocked.CompareExchange(ref _array[index], item, null) != null)
;
The dequeue would be changed to something like this...
while( (ret = Interlocked.Exchange(ref _array[index], null)) == null)
;
This builds upon Dan Tao's excellent analysis. Because the indexes are atomically obtained, then (assuming that no threads die or terminate in the enqueue or dequeue methods) a reader is guaranteed to eventually have his cell filled in, or the writer is guaranteed to eventually have his cell freed (null).
Thank you Dan Tao and Les,
I really appreciated your help a lot. Dan, you opened my mind: it's not important how many producers/consumers are inside the critical section, the important is that the locks are released in order. Les, you found the solution to the problem.
Now it's time to finally answer my own question with the final code I made thanks to the help of both of you. Well, it's not much but it's a little enhancement from Les's code
Enqueue:
while (Interlocked.CompareExchange(ref _array[index], item, null) != null)
Thread.Sleep(0);
Dequeue:
while ((ret = Interlocked.Exchange(ref _array[index], null)) == null)
Thread.Sleep(0);
Why Thread.Sleep(0)? When we find that an element cannot be retrieved/stored, why immediately checking again? I need to force context switch to allow other threads to read/write. Obviously, the next thread that will be scheduled could be another thread unable to operate, but at least we force it. Source: http://progfeatures.blogspot.com/2009/05/how-to-force-thread-to-perform-context.html
I also tested the code of the previous test case to get proof of my claims:
without sleep(0)
Read 6164150 elements
Wrote 6322541 elements
Read 5885192 elements
Wrote 5785144 elements
Wrote 6439924 elements
Read 6497471 elements
with sleep(0)
Wrote 7135907 elements
Read 6361996 elements
Wrote 6761158 elements
Read 6203202 elements
Wrote 5257581 elements
Read 6587568 elements
I know this is not a "great" discover and I will wiln no Turing prize for these numbers. Performance increment is not dramatical, but is greater than zero. Forcing context switch allows more RW operations to be performed (=higher throughput).
To be clear: in my test, I merely evaluate the performance of the queue, not simulate a producer/consumer problem, so don't care if at the end of the test after a minute there are still elements in queue. But I just demonstrated my approach works, thanks to you all.
Code available open source as MS-RL: http://logbus-ng.svn.sourceforge.net/viewvc/logbus-ng/trunk/logbus-core/It.Unina.Dis.Logbus/Utils/FastFifoQueue.cs?revision=461&view=markup