We Keep Coding

Generate a unique, random friend code from a user ID algorithmically

I am looking for a way to generate a random, unique 9 digit friend code for a user from a sequential user ID. The idea behind this is so people can't enumerate users by searching the friend codes one by one. If there are 1000 possible codes and 100 registered users, searching a random code should have a 10% chance of finding a user.
A possible way to do this is to generate a code randomly, check if the code is already in use, and if it is, try again. I am looking for an approach (mostly out of curiosity) where the friend code is generated algorithmically and is guarenteed to be unique for that user ID first try.
Specifically, given a range of numbers (1 to 999,999,999), running the function on this number should return another number in the same range, which is paired and unique to the input number. This pairing should only differ if the range changes and/or an input seed to the randomness changes.
An individual should ideally not be able to easily reverse engineer the user ID from the friend ID without knowing the seed and algorithm (or having a very large pool of samples and a lot of time - this does not need to be cryptographically secure), so simply subtracting the user ID from the maximum range is not a valid solution.
Here is some c# code that accomplishes what I am after by generating the entire range of numbers, shuffling the list, then retrieving a friend ID by treating the user ID as the list index:
int start = 1; // Starting number (inclusive)
int end = 999999999; // End number (inclusive)
Random random = new Random(23094823); // Random with a given seed
var friendCodeList = new List<int>();
friendCodeList.AddRange(Enumerable.Range(start, end + 1)); // Populate list
int n = friendCodeList.Count;
// Shuffle the list, this should be the same for a given start, end and seed
while (n > 1)
{
n--;
int k = random.Next(n + 1);
int value = friendCodeList[k];
friendCodeList[k] = friendCodeList[n];
friendCodeList[n] = value;
}
// Retrieve friend codes from the list
var userId = 1;
Console.WriteLine($"User ID {userId}: {friendCodeList[userId]:000,000,000}");
userId = 99999999;
Console.WriteLine($"User ID {userId}: {friendCodeList[userId]:000,000,000}");
userId = 123456;
Console.WriteLine($"User ID {userId}: {friendCodeList[userId]:000,000,000}");
User ID 1: 054,677,867
User ID 99999999: 237,969,637
User ID 123456: 822,632,399
Unfortunately, this is unsuitable for large ranges - this program takes 8GB of RAM to run, with a 10 or 12 digit friend code it would not be feasible to pre-generate the list either in memory or a database. I am looking for a solution that does not require this pre-generation step.
I am interested in solutions that use either a seeded random number generator or bitwise trickery to achieve this, if it is possible. The above function is reversible (by searching the values of the list) but the solution does not need to be.

Quick mathematics lesson!
You're thinking of developing a way to map one integer value (the original "secret" UserId value) to another (the (encrypted) "public" value) and back again. This is exactly what a block-cipher does (except each "block" is usually 16 bytes big instead of being a single character or integer value). So in other words, you want to create your own cryptosystem.
(Note that even if you're thinking of converting UserId 123 into a string instead of an integer, for example, a YouTube Video Id like "dQw4w9WgXcQ") - it's still an integer: because every scalar value stored in a computer, including strings, can be represented as an integer - hence the "illegal primes" problem back in the late-1990s).
And the biggest, most important take-away from any undergraduate-level computer-science class on cryptography is never create your own cryptosystem!.
With that out of the way...
Provided that security is not a top-concern...
...and you're only concerned with preventing disclosure of incrementing integer Id values (e.g. so your visitors and users don't see how many database records you really have) then use a Hashids library: https://hashids.org/
For .NET, use this NuGet package: https://www.nuget.org/packages/Hashids.net/
Overview for .NET: https://hashids.org/net/
Project page: https://github.com/ullmark/hashids.net
In your code, construct a single Hashids object (I'd use a public static readonly field or property - or better yet: a singleton injectable service) and use the .Encode method to convert any integer int/Int32 value into a string value.
To convert the string value back to the original int/Int32, use the .Decode method.
As an aside, I don't like how the library is called "Hashids" when hashes are meant to be one-way functions - because the values are still reversible - albeit by using a secret "salt" value (why isn't it called a "key"?) it isn't really a hash, imo.
If security really matters...
Then you need to treat each integer value as a discrete block in a block cipher (not a stream-cipher, because each value needs to be encrypted and decrypted independently by itself).
For the purposes of practicality, you need to use a symmetric block cipher with a small block-size. Unfortunately many block ciphers with small block sizes aren't very good (TripleDES has a block size of 64-bits - but it's weak today), so let's stick with AES.
AES has a block-size of 128 bits (16 bytes) - that's the same as two Int64 integers concatenated with each other. Assuming you use base64url encoding on a 16-byte value then your output will be 22 characters long (as Base64 uses 6 bits per character). If you're comfortable with strings of this length then you're all set. The shortest URL-safe string you can generate from a 128-bit value is 21 (hardly an improvement at all) because Base-73 is the most you can safely use in a URL that will survive all modern URL-transmission systems (never automatically assume Unicode is supported anywhere when dealing with plaintext).
It is possible to adapt AES to generate smaller output block-sizes, but it won't work in this case because using techniques like CTR Mode mean that the generated output needs to include extra state information (IV, counter, etc) which will end-up taking up the same amount of space as was gained.
Here's the code:
Very important notes:
This uses CBC Mode - which means the same input results in the same output (that's required by-design!). CBC is useful when encrypting blocks independently.
This re-uses the same IV - this is intentional and actually desirable for this application - but generally speaking do not reuse IVs when using AES for any other purpose and make sure you understand what you're doing.
*
private static readonly Byte[] _key = new Byte[] { }. // Must be 128, 192 or 256 bits (16, 24, or 32 bytes) in length.
private static readonly Byte[] _iv = new Byte[8]; // You could use the default all-zeroes.
// Note that this method works with Int32 arguments.
private static Byte[] ProcessBlock( Byte[] inputBlock, Boolean encrypt )
{
Byte[] outputBlock;
using( Aes aes = Aes.Create() )
{
aes.Key = _key;
aes.IV = _iv;
using( ICryptoTransform xform = encrypt ? aes.CreateEncryptor() : aes.CreateDecryptor() )
{
outputBlock = xform.TransformFinalBlock( inputBlock, 0, inputBlock.Length );
}
}
}
public static Byte[] EncryptInteger( Int64 value )
{
Byte[] inputBlock = new Byte[16];
inputBlock[0] = (Byte)(value >> 0 & 0xFF);
inputBlock[1] = (Byte)(value >> 8 & 0xFF);
inputBlock[2] = (Byte)(value >> 16 & 0xFF);
inputBlock[3] = (Byte)(value >> 24 & 0xFF);
inputBlock[4] = (Byte)(value >> 32 & 0xFF);
inputBlock[5] = (Byte)(value >> 40 & 0xFF);
inputBlock[6] = (Byte)(value >> 48 & 0xFF);
inputBlock[7] = (Byte)(value >> 56 & 0xFF);
return ProcessBlock( inputBlock, encrypt: true );
}
public static Int64 DecryptInteger( Byte[] block )
{
Byte[] outputBlock = ProcessInteger( value, encrypt: false );
return
(Int64)outputBlock[0] << 0 |
(Int64)outputBlock[1] << 8 |
(Int64)outputBlock[2] << 16 |
(Int64)outputBlock[3] << 24 |
(Int64)outputBlock[4] << 32 |
(Int64)outputBlock[5] << 40 |
(Int64)outputBlock[6] << 48 |
(Int64)outputBlock[7] << 56;
};
public static String EncryptIntegerToString( Int64 value ) => Convert.ToBase64String( EncryptInteger( value ) ).Replace( '+', '-' ).Replace( '/', '_' );
public static Int64 DecryptIntegerFromString( String base64Url )
{
if( String.IsNullOrWhiteSpace( base64Url ) ) throw new ArgumentException( message: "Invalid string.", paramName: nameof(base64Url) );
// Convert Base64Url to Base64:
String base64 = base64Url.Replace( '-', '+' ).Replace( '_', '/' );
Byte[] block = Convert.FromBase64String( base64 );
return DecryptInteger( block );
}

A simple method like this can produce a long sequence of numbers provided you get the constants right.
ulong Next(ulong current)
{
unchecked
{
return (999_999_937L * current + 383_565_383L) % 999_999_999L;
}
};
From memory, this kind of function can produce a sequence of 999_999_999 digits if the values in the function are chosen correctly.
My test code shows that this method can produce 500_499 numbers without repeating.
My computer can produce the entire sequence in just under 9 milliseconds so it is a fairly fast algorithm.
The first ten elements of this sequence (with leading '0's padded) is:
383565383, 602511613, 027845340, 657154301, 639998680, 703647183, 757439993, 422285770, 201847617, 869013116
5_960_464 * current + 383_565_383L gives a sequence length of 1_000_998 before repetition.

C# change the first 32bit Int of a GUID

I have a GUID which I created with GUID.NewGUID(). Now I want to replace the first 32 bit of it with a specific 32-bit Integer while keeping the rest as they are.
Is there a function to do this?

You can use ToByteArray() function and then the Guid constructor.
byte[] buffer = Guid.NewGuid().ToByteArray();
buffer[0] = 0;
buffer[1] = 0;
buffer[2] = 0;
buffer[3] = 0;
Guid guid = new Guid(buffer);

Since the Guid struct has a constructor that takes a byte array and can return its current bytes, it's actually quite easy:
//Create a random, new guid
Guid guid = Guid.NewGuid();
Console.WriteLine(guid);
//The original bytes
byte[] guidBytes = guid.ToByteArray();
//Your custom bytes
byte[] first4Bytes = BitConverter.GetBytes((UInt32) 0815);
//Overwrite the first 4 Bytes
Array.Copy(first4Bytes, guidBytes, 4);
//Create new guid based on current values
Guid guid2 = new Guid(guidBytes);
Console.WriteLine(guid2);
Fiddle
Keep in mind however, that the order of bytes returned from BitConverter depends on your processor architecture (BitConverter.IsLittleEndian) and that your Guid's entropy decreases by 232 if you use the same number every time (which, depending on your application might not be as bad as it sounds, since you have 2128 to begin with).

The question is about replacing bits, but if someone wants to replace first characters of guid directly, this can be done by converting it to string, replacing characters in string and converting back. Note that replaced characters should be valid in hex, i.e. numbers 0 - 9 or letters a - f.
var uniqueGuid = Guid.NewGuid();
var uniqueGuidStr = "1234" + uniqueGuid.ToString().Substring(4);
var modifiedUniqueGuid = Guid.Parse(uniqueGuidStr);

Generate unique 16 character strings across multiple runs

I have to generate 16 character strings, about 1,00,000 a month. They should be such that they don't repeat across multiple runs (once a month, every month). What is the best method to achieve this? Is using hash functions a good idea?
The string can have A-Z and 0-9 only.
This is to be done using C#.
EDIT: The strings should be random. So, keeping a simple counter is not an option.

Since you're limited to 16 alphanumeric characters, a GUID is probably not an option - it requires the full 128 bits to be unique and whilst that will generate a 16 character string, it will not necessarily fit the alphanumeric constraint.
You could have a simple counter and return the last 64 bits of an MD5 hash and check for uniqueness each time.
//parse out hex digits in calling code
static long NextHash(HashSet<long> hashes, int count)
{
System.Security.Cryptography.MD5 md5 = System.Security.Cryptography.MD5.Create();
long l = BitConverter.ToInt64(md5.ComputeHash(IntToArray(count)));
if(!hashes.Contains(l)){
hashes.Add(l);
return l;
} else return -1; //check this in calling code for failure
}
static byte[] IntToArray(int i)
{
byte[] bytes = new byte[4];
for(int j=0;j<4;j++){
bytes[j] = (byte)i;
i>>=8;
}
}
You could do something similar for GUIDS, but I don't know how likely collisions are when you're only looking at a substring.
MD5 hashes have the advantage of "appearing" more random, if that's at all relevant.

You have not specified the language.
PHP,
http://php.net/manual/en/function.uniqid.php
echo rand(0,999).uniqid();
rand(0,999) = 3 characters randomly
uniqid() = 13 randomly characters

I don't know if that satisfies you, but I came up with sth like that
static List<string> generate(int count)
{
List<string> strings = new List<string>();
while (strings.Count < count)
{
Guid g = Guid.NewGuid();
string GuidString = g.ToString();
GuidString = GuidString.Replace("-", "");
GuidString = GuidString.Remove(16);
if (!strings.Contains(GuidString))
strings.Add(GuidString);
}
return strings;
}

Encrypting and Decrypting a 19-digits long BigInteger

How can I take a maximum 19-digits long BigInteger and encrypt it with the following rules:
The result must be based on digits and lower-case English letters only.
All outputs must have the same length to any input. The length must be between 11 to 16 characters, depending on your method, but should be consistent for all possible inputs.
No easy patterns. For example, if you encrypt 000...1 and 000...2 the results should look completely different.
No collisions at all
Should be able to decrypt back to the original BigInteger.
Things that I have tried
Take the original number, XOR it by some key, multiply it by a factor and convert it to a base 36 string. The purpose of the factor is to expand the range so there won't be too much 0 padding. The factor must be between 1 to 36^16/10^19. The problem with this method is that a) it's not 'secure' enough, and b) close numbers have very similar results.
This answer. However, the result was often too short or too long, and the factor method used before didn't work here.

19 digits is slightly less than 64 bits, so you can simply use a 8 byte block cipher like TDEA in ECB mode to encrypt the BigInteger values. First retrieve a default 64 bit encoding of the BigInteger, then encrypt with the secret key, and finally base 36 encode it. The result will be a few characters less than 16 characters, but you can always pad with any value.
Note that if you encrypt the same value twice that you will get the same result, so in that respect the ciphertext does leak some information about the plain text.

The technique you want is format perserving encryption. This will allow you to encrypt a 19 digit number as another 19 digit number.
Unfortunately, the efficient version of this technique is somewhat difficult to implement and in fact can be done very insecurely if you pick the wrong paramaters. There are libraries for it.
This one is open source. It is in C++ unfortunately and its not clear if it runs on windows. Voltage has a library as well, though it presumably costs money and I'm not sure what languages they support.

Here is a piece of code that seems to do it, provided you can convert the BigInteger into an ulong (9999999999999999999 is in fact an ulong). The result is always a fixed 16 characters string (hexadecimal).
byte[] key = // put your 16-bytes private key here
byte[] iv = Guid.NewGuid().ToByteArray(); // or anything that varies and you can carry
string s = EncryptUInt64(ul, key, iv); // encode
ulong dul = DecryptUInt64(s, key, iv).Value; // decode if possible
public static string EncryptUInt64(ulong ul, byte[] key, byte[] iv)
{
using (MemoryStream output = new MemoryStream())
using (var algo = TripleDES.Create())
{
algo.Padding = PaddingMode.None;
using (CryptoStream stream = new CryptoStream(output, algo.CreateEncryptor(key, iv), CryptoStreamMode.Write))
{
byte[] ulb = BitConverter.GetBytes(ul);
stream.Write(ulb, 0, ulb.Length);
}
return BitConverter.ToUInt64(output.ToArray(), 0).ToString("x16");
}
}
public static ulong? DecryptUInt64(string text, byte[] key, byte[] iv)
{
if (text == null)
return null;
ulong ul;
if (!ulong.TryParse(text, NumberStyles.HexNumber, null, out ul))
return null;
using (MemoryStream input = new MemoryStream(BitConverter.GetBytes(ul)))
using (var algo = TripleDES.Create())
{
algo.Padding = PaddingMode.None;
using (CryptoStream stream = new CryptoStream(input, algo.CreateDecryptor(key, iv), CryptoStreamMode.Read))
{
byte[] olb = new byte[8];
try
{
stream.Read(olb, 0, olb.Length);
}
catch
{
return null;
}
return BitConverter.ToUInt64(olb, 0);
}
}
}

Search ReadAllBytes for specific values

I am writing a program that reads '.exe' files and stores their hex values in an array of bytes for comparison with an array containing a series of values. (like a very simple virus scanner)
byte[] buffer = File.ReadAllBytes(currentDirectoryContents[j]);
I have then used BitConverter to create a single string of these values
string hex = BitConverter.ToString(buffer);
The next step is to search this string for a series of values(definitions) and return positive for a match. This is where I am running into problems. My definitions are hex values but created and saved in notepad as defintions.xyz
string[] definitions = File.ReadAllLines(#"C:\definitions.xyz");
I had been trying to read them into a string array and compare the definition elements of the array with string hex
bool[] test = new bool[currentDirectoryContents.Length];
test[j] = hex.Contains(definitions[i]);
This IS a section from a piece of homework, which is why I am not posting my entire code for the program. I had not used C# before last Friday so am most likely making silly mistakes at this point.
Any advice much appreciated :)

It is pretty unclear exactly what kind of format you use of the definitions. Base64 is a good encoding for a byte[], you can rapidly convert back and forth with Convert.ToBase64String and Convert.FromBase64String(). But your question suggests the bytes are encoded in hex. Let's assume it looks like "01020304" for a new byte[] { 1, 2, 3, 4}. Then this helper function converts such a string back to a byte[]:
static byte[] Hex2Bytes(string hex) {
if (hex.Length % 2 != 0) throw new ArgumentException();
var retval = new byte[hex.Length / 2];
for (int ix = 0; ix < hex.Length; ix += 2) {
retval[ix / 2] = byte.Parse(hex.Substring(ix, 2), System.Globalization.NumberStyles.HexNumber);
}
return retval;
}
You can now do a fast pattern search with an algorithm like Boyer-Moore.

I expect you understand that this is a very inefficient way to do it. But except for that, you should just do something like this:
bool[] test = new bool[currentDirectoryContents.Length];
for(int i=0;i<test.Length;i++){
byte[] buffer = File.ReadAllBytes(currentDirectoryContents[j]);
string hex = BitConverter.ToString(buffer);
test[i] = ContainsAny(hex, definitions);
}
bool ContainsAny(string s, string[] values){
foreach(string value in values){
if(s.Contains(value){
return true;
}
}
return false;
}
If you can use LINQ, you can do it like this:
var test = currentDirectoryContents.Select(
file=>definitions.Any(
definition =>
BitConverter.ToString(
File.ReadAllBytes(file)
).Contains(definition)
)
).ToArray();
Also, make sure that your definitions-file is formatted in a way that matches the output of BitConverter.ToString(): upper-case with dashes separating each encoded byte:
12-AB-F0-34
54-AC-FF-01-02