We Keep Coding

efficient way of matching coordinates

I have 2 lists of coordinates in C# one as of coordinates of Drivers and the other as of coordinates of cafes. I am looking for an efficient way of populating a static Dictionary with its key as of a Driver from the first list and its associated values of all Cafes in 500 meters radius.
public void ManageList() {
GlobalList.Clear();
foreach (var driver in driverList)
{
var driverCoords = new GeoCoordinate(driver.Latitude, driver.Longitude);
List<Cafe> matchedCafes = new List<Cafe>();
foreach (var cafe in cafeList)
{
var cafeCoords = new GeoCoordinate(cafe.Latitude, cafe.Longitude);
if (cafeCoords.GetDistanceTo(driverCoords) <= 500) {
matchedCafes.Add(cafeCoords);
}
}
GlobalList.Add(driverCoords, matchedCafes);
}
}
the above works fine as long as drivers are not movable objects. If I want to send the driver's coordinates every 5 seconds and update the GlobalList per driver the above algorithm fails as I am basically clearing the whole list and populate it again.

More of a pointer than an answer. It's unclear how many items you are talking about.
But really what you describe is a spatial hashing problem.
This is a basic of game engine, physics, programming.
It is a big topic, but you could google to get started,
https://gamedevelopment.tutsplus.com/tutorials/redesign-your-display-list-with-spatial-hashes--cms-27586
http://zufallsgenerator.github.io/2014/01/26/visually-comparing-algorithms/
https://gamedev.stackexchange.com/a/69794/86883
As a matter of fact, you could probably ask your question on gamedev, since, it is exactly that type of question.
I'll try to make an extremely simple explanation:
Say your system performs perfectly fine (no performance problems) when you have, example, 20 cafes.
But, in fact you have 2000 cafes.
So break down the map into about 100 "boxes".
When you do a taxi, only do the cafes in that box.
You've immediately eliminated 1980 of the cafes which are so far away they are not even in the box in question. (Naturally what I have stated is a simplification, there are a huge number of details to address in the basic approach.)
Actually this article -
https://dzone.com/articles/algorithm-week-spatial
very nicely explains both quadtrees and geohashing for you.

Finding insertion points in a sorted array faster than O(n)?

This is for game programming. Lets say I have a Unit that can track 10 enemies within it's range. Each enemy has a priority between 0-100. So the array currently looks like this (numbers represent priority):
Enemy - 96
Enemy - 78
Enemy - 77
Enemy - 73
Enemy - 61
Enemy - 49
Enemy - 42
Enemy - 36
Enemy - 22
Enemy - 17
Say a new enemy wanders within range and has a priority of 69, this will be inserted between 73 and 61, and 17 will be removed from the array (Well, the 17 would be removed before the insertion, I believe).
Is there any way to figure out that it needs to be inserted between 73 and 61 without an O(n) operation?

I feel you're asking the wrong question here. You have to both first find the spot to insert into and then insert the element. These are two operation that are both tied together and I feel you shouldn't be asking about how to find where to do one faster without the other. It'll make sense why towards the end of the question. But I'm addressing the question of actually inserting faster.
Short Answer: No
Answer you'll get from someone that's too smart for themselves:
The only way to accomplish this is to not use an array. In an array unless you are inserting into the first or last permissions the insert will be O(n). This is because the array consists of its elements occupying contiguous space in memory. That is how you are able to reference a particular element in O(1) time, you know exactly where that element is. The cost is to insert in the middle you need to move half the elements in the array. So while you can look up with a binary search in log(n) time you cannot insert in that time.
So if you're going to do anything, you'll need a different data structure. A simple binary tree may be the solution it will do the insertion in log(n) time. On the other hand if you're feeding it a sorted array you have to worry about tree balancing, so not you might need a red and black tree. Or if you are always popping the element that is the closest or the furthest then you can use heap sort. A heap sort is the best algorithm for a priority queue. It has an additional advantage of fitting a tree structure in an array so it has far better spatial locality (more on this later).
The truth:
You'll most likely have a dozen maybe a few dozen enemies in the vicinity at most. At that level the asymptotic performance does not matter because it is designed especially for large values of 'n'. What you're looking at is a religious adherence to your CS 201 professor's calls about Big Oh. Linear search and insertion will be the fastest method, and the answer to will it scale is, who the hell cares. If you try to implement a complicated algorithm to scale it, you will almost always be slower since what is determining your speed is not the software, it is the hardware, and you're better off sticking to doing things that the hardware knows how to deal with well: "linearly going down memory". In fact after the prefetchers do their thing it would be faster to linearly go through each element even if there were a couple of thousand elements than to implement a red and black tree. Because a data structure like a tree would allocate memory all over the place without any regard to spatial locality. And the calls to allocate more memory for a node are in themselves more expensive than the time it takes to read through a thousand elements. Which is why graphics cards use insert sort all over the place.
Heap Sort
Heap sort might actually be faster depending on the input data since it is using a linear array although it may confuse the prefetchers so it's hard to say. The only limitation is that you can only pop the highest priority element. Obviously you can define highest priority to be either the lowest or the largest element. Heap sort is too fancy for me to try and describe it over here, just Google it. It does separate insertion and removal into two O(log(n)) operations. The biggest downside of heap sort is it will seriously decrease the debugability of the code. A heap is not a sorted array, it has an order to it, but other than heap sort being a complicated unintuitive algorithm, it is not apparently visible to a human being if a heap is setup correctly. So you would introduce more bugs for in the best case little benefit. Hell, the last time I had to do a heap sort I copied the code for it and that had bugs in it.
Insertion Sort With Binary Search
So this is what it seems like you're trying to do. The truth is this is a very bad idea. On average insertion sort takes O(n). And we know this is a hard limit for inserting a random element into a sorted array. Yes we can find the element we want to insert into faster by using a binary search. But then the average insertion still takes O(n). Alternatively, in the best case, if you are inserting and the element goes into the last position insertion sort takes O(1) time because when you inserted, it is already in the correct place. However, if you do a binary search to find the insertion location, then finding out you're supposed to insert in the last position takes O(log(n)) time. And the insertion itself takes O(1) time. So in trying to optimize it, you've severely degraded the best case performance. Looking at your use case, this queue holds the enemies with their priorities. The priority of an enemy is likely a function of their strength and their distance. Which means when an enemy enters into the priority queue, it will likely have a very low priority. This plays very well into the best case of insertion of O(1) performance. If you decrease the best case performance you will do more harm than good because it is also your most general case.
Preoptimization is the root of all evil -- Donald Knuth

Since you are maintaining a sorted search pool at all times, you can use binary search. First check the middle element, then check the element halfway between the middle element and whichever end of the array is closer, and so on until you find the location. This will give you O(log2n) time.

Sure, assuming you are using an Array type to house the list this really easy.
I will assume Enemy is your class name, and that is has a property called Priority to perform the sort. We will need an IComparer<Enemy> that looks like the following:
public class EnemyComparer : IComparer<Enemy>
{
int IComparer<Enemy>.Compare(Enemy x, Enemy y)
{
return y.Priority.CompareTo(x.Priority); // reverse operand to invert ordering
}
}
Then we can write a simple InsertEnemy routine as follows:
public static bool InsertEnemy(Enemy[] enemies, Enemy newEnemy)
{
// binary search in O(logN)
var ix = Array.BinarySearch(enemies, newEnemy, new EnemyComparer());
// If not found, the bit-wise compliment is the insertion index
if (ix < 0)
ix = ~ix;
// If the insertion index is after the list we bail out...
if (ix >= enemies.Length)
return false;// Insert is after last item...
//Move enemies down the list to make room for the insertion...
if (ix + 1 < enemies.Length)
Array.ConstrainedCopy(enemies, ix, enemies, ix + 1, enemies.Length - (ix + 1));
//Now insert the newEnemy into the position
enemies[ix] = newEnemy;
return true;
}
There are other data structures that would make this a bit faster, but this should prove efficient enough. A B-Tree or binary tree would be ok if the list will get large, but for 10 items it's doubtful it would be faster.
The method above was tested with the addition of the following:
public class Enemy
{
public int Priority;
}
public static void Main()
{
var rand = new Random();
// Start with a sorted list of 10
var enemies = Enumerable.Range(0, 10).Select(i => new Enemy() {Priority = rand.Next(0, 100)}).OrderBy(e => e.Priority).ToArray();
// Insert random entries
for (int i = 0; i < 100; i++)
InsertEnemy(enemies, new Enemy() {Priority = rand.Next(100)});
}

How come my class take so much space in memory?

I will have literally tens of millions of instances of some class MyClass and want to minimize its memory size. The question of measuring how much space an object takes in the memory was discussed in Find out the size of a .net object
I decided to follow Jon Skeet's suggestion, and this is my code:
// Edit: This line is "dangerous and foolish" :-)
// (However, commenting it does not change the result)
// [StructLayout(LayoutKind.Sequential, Pack = 1)]
public class MyClass
{
public bool isit;
public MyClass nextRight;
public MyClass nextDown;
}
class Program
{
static void Main(string[] args)
{
var a1 = new MyClass(); //to prevent JIT code mangling the result (Skeet)
var before = GC.GetTotalMemory(true);
MyClass[] arr = new MyClass[10000];
for (int i = 0; i < 10000; i++)
arr[i] = new MyClass();
var after = GC.GetTotalMemory(true);
var per = (after - before) / 10000.0;
Console.WriteLine("Before: {0} After: {1} Per: {2}", before, after, per);
Console.ReadLine();
}
}
I run the program on 64 bit Windows, Choose "release", platform target: "any cpu", and choose "optimize code" (The options only matter if I explicitly target x86) The result is, sadly, 48 bytes per instance.
My calculation would be 8 bytes per reference, plus 1 byte for bool plus some ~8byte overhead. What is going on? Is this a conspiracy to keep RAM prices high and/or let non-Microsoft code bloat? Well, ok, I guess my real question is: what am I doing wrong, or how can I minimize the size of MyClass?
Edit: I apologize for being sloppy in my question, I edited a couple of identifier names. My concrete and immediate concern was to build a "2-dim linked-list" as a sparse boolean matrice implementation, where I can get an enumeration of set values in a given row/column easily. [Of course that means I have to also store the x,y coordinates on the class, which makes my idea even less feasible]

Approach the problem from the other end. Rather than asking yourself "how can I make this data structure smaller and still have tens of millions of them allocated?" ask yourself "how can I represent this data using a completely different data structure that is far more compact?"
It looks like you are building a doubly-linked list of bools, which, as you note, uses thirty to fifty times more memory than it needs to. Is there some reason why you're not simply using a BitArray to store your list of bools?
UPDATE:
in fact I was trying to implement a sparse boolean two-dimensional matrix
Well why didn't you say so in the first place?
When I want to make a sparse Boolean two-d matrix of enormous size, I build an immutable persistent boolean quadtree with a memoized factory. If the array is sparse, or even if it is dense but self-similar in some way, you can achieve enormous compressions. Square arrays of 264 x 264 Booleans are easily representable even though obviously as a real array, that would be more memory than exists in the world.
I have been toying with the idea of doing a series of blog articles on this technique; I will likely do so in late March. (UPDATE: I did not write that article in March 2012; I wrote it in August 2020. https://ericlippert.com/2020/08/17/life-part-32/)
Briefly, the idea is to make an abstract class Quad that has two subclasses: Single, and Multi. "Single" is a doubleton -- like a singleton, but with exactly two instances, called True and False. A Multi is a Quad that has four sub-quads, called NorthEast, SouthEast, SouthWest and NorthWest.
Each Quad has an integer "level"; the level of a Single is zero, and a multi of level n is required to have all of its children be Quads of level n-1.
The Multi factory is memoized; when you ask it to make a new Multi with four children, it consults a cache to see if it has made it before. If it has, it does not construct a new one; it hands out the old one. Since Quads are immutable, you do not have to worry about someone changing the Quad on you after it is in the cache.
Consider now how many memory words (a word is 4 or 8 bytes depending on architecture) an "all false" Multi of level n consumes. A level 1 "all false" multi consumes four words for the links to its children, a word for the level count (if necessary; you are not required to keep the level in the multi, though it helps for debugging) and a couple words for the sync block and so on. Let's call it eight words. (Plus the memory for the False Single quad, which we can assume is a constant two or three words, and thereby may be ignored.)
A level 2 "all false" multi consumes the same eight words, but each of its four children is the same level 1 multi. Therefore the total consumption of the level 2 "all false" multi is let's say 16 words.
The same for the level 3, 4,... and so on. The total memory consumption for a level 64 multi that is logically a 264 x 264 square array of Booleans is only 64 x 16 memory words!
Make sense? Hopefully that is enough of a sketch to get you going. If not, see my blog link above.

8 (object reference) + 8 (object reference) + 1 (bool) + 16 (header) + 8 (reference in array itself) = 41
Even if it's misaligned internally, each will be aligned on the heap. So we're looking at least 48bytes.
I can't for the life of me see why you'd want a linked list of bools though. A list of them would take 48times less space, and that's before you get to optimisations of storing a bool per bit which would make it 384 times smaller. And easier to manipulate.

If these hundreds of millions of instances of the class are mostly copies of the class with minor variations in class property values, then your system is a prime candidate to use what is called the Flyweight pattern. This pattern minimizes memory use by using the same instanes over and over, and just changing the properties as needed...

Are there common methods for hashing an input file to a fixed set of values?

Let's say I'm trying to generate a monster for use in a roleplaying game from an arbitrary piece of input data. Think Barcode Battler or a more-recent iPod game whose name escapes me.
It seems to me like the most straightforward way to generate a monster would be to use a hash function on the input data (say, an MP3 file) and use that hash value to pick from some predetermined set of monsters, or use pieces of the hash value to generate statistics for a custom monster.
The question is, are there obvious methods for taking an arbitrary piece of input data and hashing it to one of a fixed set of values? The primary goal of hashing algorithms is, after all, to avoid collisions. Instead, I'm suggesting that we want to guarantee them - that, given a predetermined set of 100 monsters, we want any given MP3 file to map to one of them.
This question isn't bound to a particular language, but I'm working in C#, so that would be my preference for discussion. Thanks!

Hash the file using any hash function of your choice, convert the result into an integer, and take the result modulo 100.
monsterId = hashResult % 100;
Note that if you later decide to add a new monster and change the code to % 101, nearly all hashes will suddenly map to different monsters.

Okay, that's a very nice question. I would say: don't use hash, because this won't be a nice way for the player to predict patterns. From cognitive theory we know that one thing that is interesting in games is that player can learn by trial and error. So if player gives the input of an image of a red dragon and another image of a red dragon with slightly different pixels, he would like to have the same monster appearing, right? If you use hashes that would not be the case.
Instead, I would recommend doing much simpler things. Imagine that your raw piece of input is just a byte[] , it is itself already a list of numbers. Unfortunately it's only a list of numbers from 0 to 255, so if you for example do an average, you can get 1 number from 0 to 255 . That you could map to a number of monsters already, if you need more, you can read pairs of bytes and just compose Int16, that way you will be able to go up to 65536 possible monsters :)

You can use the MD5, SHA1, or SHA2 of a file as a unique finger print for the file. Each hash function will give you a larger, less overlapping fingerprint and each can be obtained by library functions already in the base libraries.
In truth you could probably hash a much smaller portion of the file, for instance the first 1-3MB of the file and still get a fairly unique fingerprint, without the expense of processing a larger file (like an AVI).
Look in the System.Security namespace for the MD5Crypto provider for an example of how to generate a MD5 from a byte sequence.
Edit: If you want to ensure that the hash collides in a relatively short order you can use CRC2, 4, 6, 8, 16, 32 which will collide fairly frequently (especially CRC2 :)) but be the same for the same file. It is easy to generate.

Using RegEx to get Poker Hand score

I'm developing a poker game in C#. At the moment I'm trying to get the players hand score using RegEx. I search the string (composed of the cards suit and number) and look for suits or numbers to match the RegEx. If i get 2 matches then the player has a pair, 3 matches he has 3 of a kind.
I have 3 classes at the moment, a Card class (with number and suit), a Deck class (that contains 52 Cards) and a Hand class that gets five cards from the shuffled deck.
Deck class has a shuffleDeck();
Hand class has the functions to calculate the score (is in these functions that I am using RegEx).
I generate the string on which I use RegEx by adding the 5 suits and numbers that the hand has.
Is this a good idea or should I do it another way, if so, how?
Thank you for your help
PS. I am one of the unexperienced programmers that want to use a newly learned tool for everything

I do not think that a regex is the appropriate way to deal with this. You probably should be using a more sophisticated representation of a hand than a string.

You have not provided much detail, but what from what I have read, I assume you're not pushing the OOP very far...
I would have a Card class that has a Rank and Suit class instances. I would then have a deck class that handles shuffling / dealing...
I would then have a Hand class that would contain your poker hand of n Card objects...
In this way you can build up rules to evaluate each hand object, thus being more flexible and more extensible in the future...say if you want to make another card game / add support for another variant of poker...
Using Regular expressions to do all of this seems to be a pretty poor choice.

I would agree with the others, that Regex seems like a bad choice. It might work with pairs, 3 of a kind, 4 of a kind. However, it might get kind of tricky (or impossible) once you start looking at hands like flushes, straights, and 2 pair.
I think the best solution would be to evaluate the cards from best hand to worst hand, as shown here, and as soon as your find a match, then that is your hand. This ensures that you don't mistake 4 of a kind for 2 pair. Or a straight flush for just a straight, or just a flush. I would go with mmattax and create an object for the card, and an object for the hand, then you can evaluate the cards in each hand to see if they meet the required criteria for each hand.

I think prime numbers are a good solution for that.
consider :
// D H S C
colors = [7,5,3,2]
// A Q K J T 9 8 7 6 5 4 3 2
ranks = [61,59,53,43,41,37,31,29,23,19,17,13,11,61]
a unique card is identified by a color prime number * a rank prime number.
(for example, As of Diamonds : prime = 7 * 61)
so an entiere unique deck or combinaison are identified by prime * prime * prime * prime * prime
if there is a flush of Diamonds, the 5 cards deck's primes ID must by divisble ( mod = 0 ) by the flush of diamonds ID ( 7 ^ 5 because diamonds color prime is 7 )

Using a string to represent the hand seems like a poor decision. My recommendation would be to use an Enum to represent the Suit and another to represent the numeric value of the card.