We Keep Coding

How to compare bits without loop

How did this function compare bits without loop? Please describe how it done. And what are this numbers mean: 6148914691236517205uL, 3689348814741910323uL, 1085102592571150095uL, 72340172838076673uL.
public static int HashDistance(ulong hash1, ulong hash2)
{
ulong num = hash1 ^ hash2;
num -= (num >> 1 & 6148914691236517205uL);
num = (num & 3689348814741910323uL) + (num >> 2 & 3689348814741910323uL);
num = (num + (num >> 4) & 1085102592571150095uL);
return Convert.ToInt32(num * 72340172838076673uL >> 56);
}

Pack and unpack multiple integers into and from an Uint64

Need to pack and unpack the following into an UInt64
UInt25
UInt5
UInt7
UInt27
Have the following for packing and unpacking UInt27 and UInt5 to from UInt32
But I cannot get past 2
My background is math (not computer science)
UInt32 highlow;
UInt32 high;
byte low;
int two27 = (Int32)Math.Pow(2, 27);
for (UInt32 i = 0; i < two27; i++)
{
highlow = ((UInt32)i) << 5;
high = highlow >> 5;
if (high != i)
{
Debug.WriteLine("high wrong A " + high.ToString() + " " + i.ToString());
}
for (byte j = 0; j < 32; j++)
{
highlow = (((UInt32)i) << 5) | j;
high = highlow >> 5;
if (high != i)
{
Debug.WriteLine("high wrong B " + high.ToString() + " " + i.ToString());
}
low = (byte)(highlow & 0x1f);
if (low != j)
{
Debug.WriteLine("low wrong " + low.ToString() + " " + j.ToString());
}
}
}
Code based on accepted answer (did not test the full loop the i27 loop got to 2)
UInt32 bits27;
UInt32 bits25;
UInt32 bits7;
UInt32 bits5;
UInt32 int27 = (UInt32)Math.Pow(2,27);
UInt32 int25 = (UInt32)Math.Pow(2,25);
UInt32 int7 = (UInt32)Math.Pow(2,7);
UInt32 int5 = (UInt32)Math.Pow(2,5);
UInt64 packed;
//ulong packed = (bits27) | ((ulong)bits25 << 27) | ((ulong)bits7 << 52) | ((ulong)bits5 << 59);
for (UInt32 i27 = 0; i27 < int27; i27++)
{
for (UInt32 i25 = 0; i25 < int25; i25++)
{
for (UInt32 i7 = 0; i7 < int7; i7++)
{
for (UInt32 i5 = 0; i5 < int5; i5++)
{
packed = (UInt64)(i27) | ((UInt64)i25 << 27) | ((UInt64)i7 << 52) | ((UInt64)i5 << 59);
bits27 = (UInt32)(packed & ((1 << 27) - 1));
bits25 = (UInt32)((packed >> 27) & ((1 << 25) - 1));
bits7 = (UInt32)((packed >> 52) & ((1 << 7) - 1));
bits5 = (UInt32)((packed >> 59) & ((1 << 5) - 1));
if (bits27 != i27) Debug.WriteLine("bits27 != i27");
if (bits25 != i25) Debug.WriteLine("bits25 != i25");
if (bits7 != i7) Debug.WriteLine("bits7 != i7");
if (bits5 != i5) Debug.WriteLine("bits5 != i5");
}
}
}
}

The shift operators are the right solution, but note that they won't automatically make the result wider than the inputs -- you need to cast the input.
Pack:
ulong packed = (bits27) | ((ulong)bits25 << 27) | ((ulong)bits7 << 52) | ((ulong)bits5 << 59);
Unpack:
bits27 = (uint) (packed & ((1 << 27) - 1));
bits25 = (uint)((packed >> 27) & ((1 << 25) - 1));
bits7 = (uint)((packed >> 52) & ((1 << 7) - 1));
bits5 = (uint)((packed >> 59) & ((1 << 5) - 1));

It seems like it would be far easier to convert the numbers to binary, pad or truncate to the correct length, concatenate them and then construct your 64-bit type from binary.
var packedInt64 = Convert.ToInt64(Convert.ToString(ui25, 2).PadLeft(25, '0') +
Convert.ToString(ui5, 2).PadLeft(5, '0') +
Convert.ToString(ui7, 2).PadLeft(7, '0') +
Convert.ToString(ui27, 2).PadLeft(2, '0'), 2);
To unpack:
var binary = Convert.ToString(packedInt64, 2);
ui25 = Convert.ToUInt32(binary.Substring(0, 24));
ui5 = Convert.ToUInt32(binary.Substring(24, 5));
etc.

Send binary data as parameter to a method in Web Service?

In server side (C#.NET,Windows 2003) I have a web service with a method and in client side (Visual C++ v6, WinINet, POST) I want to call that method and pass binary data as a parameter to it.
when I send Binary data an error rise and when I send ASCII data it called successful.
How can I send binary data as parameter of a method?

To send binary data to a web method you can base64 encode it and then decode it in the web service. If your variable is a byte array called data then you would do the following.
In C++ you need to create a base64 header and cpp file. The following example is from http://www.adp-gmbh.ch/cpp/common/base64.html
base64.h
#include <string>
std::string base64_encode(unsigned char const* , unsigned int len);
std::string base64_decode(std::string const& s);
base64.cpp
#include "base64.h"
#include <iostream>
static const std::string base64_chars =
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"abcdefghijklmnopqrstuvwxyz"
"0123456789+/";
static inline bool is_base64(unsigned char c) {
return (isalnum(c) || (c == '+') || (c == '/'));
}
std::string base64_encode(unsigned char const* bytes_to_encode, unsigned int in_len) {
std::string ret;
int i = 0;
int j = 0;
unsigned char char_array_3[3];
unsigned char char_array_4[4];
while (in_len--) {
char_array_3[i++] = *(bytes_to_encode++);
if (i == 3) {
char_array_4[0] = (char_array_3[0] & 0xfc) >> 2;
char_array_4[1] = ((char_array_3[0] & 0x03) << 4) + ((char_array_3[1] & 0xf0) >> 4);
char_array_4[2] = ((char_array_3[1] & 0x0f) << 2) + ((char_array_3[2] & 0xc0) >> 6);
char_array_4[3] = char_array_3[2] & 0x3f;
for(i = 0; (i <4) ; i++)
ret += base64_chars[char_array_4[i]];
i = 0;
}
}
if (i)
{
for(j = i; j < 3; j++)
char_array_3[j] = '\0';
char_array_4[0] = (char_array_3[0] & 0xfc) >> 2;
char_array_4[1] = ((char_array_3[0] & 0x03) << 4) + ((char_array_3[1] & 0xf0) >> 4);
char_array_4[2] = ((char_array_3[1] & 0x0f) << 2) + ((char_array_3[2] & 0xc0) >> 6);
char_array_4[3] = char_array_3[2] & 0x3f;
for (j = 0; (j < i + 1); j++)
ret += base64_chars[char_array_4[j]];
while((i++ < 3))
ret += '=';
}
return ret;
}
std::string base64_decode(std::string const& encoded_string) {
int in_len = encoded_string.size();
int i = 0;
int j = 0;
int in_ = 0;
unsigned char char_array_4[4], char_array_3[3];
std::string ret;
while (in_len-- && ( encoded_string[in_] != '=') && is_base64(encoded_string[in_])) {
char_array_4[i++] = encoded_string[in_]; in_++;
if (i ==4) {
for (i = 0; i <4; i++)
char_array_4[i] = base64_chars.find(char_array_4[i]);
char_array_3[0] = (char_array_4[0] << 2) + ((char_array_4[1] & 0x30) >> 4);
char_array_3[1] = ((char_array_4[1] & 0xf) << 4) + ((char_array_4[2] & 0x3c) >> 2);
char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];
for (i = 0; (i < 3); i++)
ret += char_array_3[i];
i = 0;
}
}
if (i) {
for (j = i; j <4; j++)
char_array_4[j] = 0;
for (j = 0; j <4; j++)
char_array_4[j] = base64_chars.find(char_array_4[j]);
char_array_3[0] = (char_array_4[0] << 2) + ((char_array_4[1] & 0x30) >> 4);
char_array_3[1] = ((char_array_4[1] & 0xf) << 4) + ((char_array_4[2] & 0x3c) >> 2);
char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];
for (j = 0; (j < i - 1); j++) ret += char_array_3[j];
}
return ret;
}
After you have your base64 implementation you can create the string to pass.
std::string encoded = base64_encode(data, sizeof(data));
You have to be mindful however of the fact that your data will expand by encoding it to Base64 so your uploads will take longer as the data will be larger. It will be approximately 37% larger (see the MIME section of http://en.wikipedia.org/wiki/Base64)
To decode the data on the other end you would simply do the following.
byte[] data = System.Convert.FromBase64String(yourParameterName);

C# Function to Visual Basic 2010 Error with BitShift Operator "<<"

I am currently translating a C# code to Visual Basic 2010. Works good but now I dont get the correct line. everytime i got a bitshift error
Original Code looks like:
Code:
private static string Checksum(string url)
{
uint Magic = 0xE6359A60;
uint a, b;
uint c = Magic;
a = b = 0x9E3779B9;
int k = 0;
int length = url.Length;
//Algorithm
while (length >= 12)
{
a += (uint)(url[k + 0] + (url[k + 1] << 8) + (url[k + 2] << 16) + (url[k + 3] << 24));
b += (uint)(url[k + 4] + (url[k + 5] << 8) + (url[k + 6] << 16) + (url[k + 7] << 24));
c += (uint)(url[k + 8] + (url[k + 9] << 8) + (url[k + 10] << 16) + (url[k + 11] << 24));
Mix(ref a, ref b, ref c);
k += 12;
length -= 12;
}
and my translation is:
Private Function Checksum(ByVal url) As String
Dim Magic As UInteger = &HE6359A60UI
Dim a As UInteger, b As UInteger
Dim c As UInteger =Magic
a = b = &H9E3779B9UI
Dim k As Integer = 0
Dim length As Integer = url.Length
'Algorithm
While length >= 12
a += CUInt(url(k + 0) + (url(k + 1) << 8) + (url(k + 2) << 16) + (url(k + 3) << 24))
b += CUInt(url(k + 4) + (url(k + 5) << 8) + (url(k + 6) << 16) + (url(k + 7) << 24))
c += CUInt(url(k + 8) + (url(k + 9) << 8) + (url(k + 10) << 16) + (url(k + 11) << 24))
Mix(a, b, c)
k += 12
length -= 12
End While
But when the debugger comes to
a += CUInt(url(k + 0) + (url(k + 1) << 8) + (url(k + 2) << 16) + (url(k + 3) << 24))
it throw an exception:The operator "<<" is not defined for the Typ Char.

In C# the compiler automatically converts from char to int. As the VB compiler doesn't do that, you have to do an explicit conversion:
a += CUInt(AscW((url(k + 0)) + (AscW(url(k + 1)) << 8) + (AscW(url(k + 2)) << 16) + (AscW(url(k + 3)) << 24))

a += CUInt(AscW(url(k + 0))) + (CUInt(AscW(url(k + 1))) << 8) + (CUInt(AscW(url(k + 2))) << 16) + (CUInt(AscW(url(k + 3))) << 24)
Assuming your characters are Unicode. If ASCII, use Asc() instead of AscW()

ushort array to byte array

I have an array of ushorts, with each ushort representing a 12-bit word. This needs to be tightly packed into an array of bytes. It should look like this in the end:
| word1 | word2 | word3 | word4 |
| byte1 | byte2 | byte3 | byte4 | byte5 | byte6|
Since each word only uses 12 bits, 2 words will be packed into 3 bytes.
Could someone help? I'm a bit stuck on how to do this in C#.

You're probably going to have to brute-force it.
I'm not a C# guy, but you are looking at something along the lines of (in C):
unsigned incursor, outcursor;
unsigned inlen = length(inputarray); // not literally
for(incursor=0,outcursor=0;incursor < inlen; incursor+=2,outcursor+=3{
outputarray[outcursor+0] = ((inputarray[incursor+0]) >> 4) & 0xFF;
outputarray[outcursor+1] = ((inputarray[incursor+0] & 0x0F)<<4 | ((inputarray[incursor+1]>>8) & 0x0F);
outputarray[outcursor+2] = inputarray[incursor+1] & 0xFF;
}

If you want to use the array as an array of UInt16 while in-memory, and then convert it to a packed byte array for storage, then you'll want a function to do one-shot conversion of the two array types.
public byte[] PackUInt12(ushort[] input)
{
byte[] result = new byte[(input.Length * 3 + 1) / 2]; // the +1 leaves space if we have an odd number of UInt12s. It's the unused half byte at the end of the array.
for(int i = 0; i < input.Length / 2; i++)
{
result[i * 3 + 0] = (byte)input[i * 2 + 0];
result[i * 3 + 1] = (byte)(input[i * 2 + 0] >> 8 | input[i * 2 + 1] << 4);
result[i * 3 + 2] = (byte)(input[i * 2 + 1] >> 4);
}
if(input.Length % 2 == 1)
{
result[i * 3 + 0] = (byte)input[i * 2 + 0];
result[i * 3 + 1] = (byte)(input[i * 2 + 0] >> 8);
}
return result;
}
public ushort[] UnpackUInt12(byte[] input)
{
ushort[] result = new ushort[input.Length * 2 / 3];
for(int i = 0; i < input.Length / 3; i++)
{
result[i * 2 + 0] = (ushort)(((ushort)input[i * 3 + 1]) << 8 & 0x0F00 | input[i * 3 + 0]);
result[i * 2 + 1] = (ushort)(((ushort)input[i * 3 + 1]) << 4 | input[i * 3 + 1] >> 4;)
}
if(result.Length % 2 == 1)
{
result[i * 2 + 0] = (ushort)(((ushort)input[i * 3 + 1]) << 8 & 0x0F00 | input[i * 3 + 0]);
}
return result;
}
If, however, you want to be efficient about memory usage while the application is running, and access this packed array as an array, then you'll want to have a class that returns ushorts, but stores them in byte[].
public class UInt12Array
{
// TODO: Constructors, etc.
private byte[] storage;
public ushort this[int index]
{
get
{
// TODO: throw exceptions if the index is off the array.
int i = index * 2 / 3;
if(index % 2 == 0)
return (ushort)(((ushort)storage[i * 3 + 1]) << 8 & 0x0F00 | storage[i * 3 + 0]);
else
return (ushort)(((ushort)storage[i * 3 + 1]) << 4 | storage[i * 3 + 1] >> 4;)
}
set
{
// TODO: throw exceptions if the index is off the array.
int i = index * 2 / 3;
if(index % 2 == 0)
storage[i * 3 + 0] = (byte)value;
storage[i * 3 + 1] = (byte)(value >> 8 | storage[i * 3 + 1] & 0xF0);
else
storage[i * 3 + 1] = (byte)(storage[i * 3 + 1] & 0x0F | value << 4);
storage[i * 3 + 2] = (byte)(value >> 4);
}
}
}

Why not store the 12-bit words in a byte array and provide a getter and a setter method that read and write the ushort's byte to the correct index in the array?

Trying to solve this with LINQ was fun!
Warning: For entertainment purposes only - do not use the below performance abominations in real code!
First try - group pairs of uints, create three bytes out of each pair, flatten list:
byte[] packedNumbers = (from i in Enumerable.Range(0, unpackedNumbers.Length)
group unpackedNumbers[i] by i - (i % 2) into pairs
let n1 = pairs.First()
let n2 = pairs.Skip(1).First()
let b1 = (byte)(n1 >> 4)
let b2 = (byte)(((n1 & 0xF) << 4) | (n2 & 0xF00) >> 8)
let b3 = (byte)(n2 & 0xFFFF)
select new[] { b1, b2, b3 })
.SelectMany(b => b).ToArray();
Or slightly more compact, but less readable:
byte[] packedNumbers = unpackedNumbers
.Select((Value, Index) => new { Value, Index })
.GroupBy(number => number.Index - (number.Index % 2))
.SelectMany(pair => new byte[] {
(byte)(pair.First().Value >> 4),
(byte)(((pair.First().Value & 0xF) << 4) | (pair.Skip(1).First().Value & 0xF00) >> 8),
(byte)(pair.Skip(1).First().Value & 0xFFFF) }).ToArray();
Strings anyone?
char[] hexChars = unpackedNumbers.SelectMany(n => n.ToString("X4").Substring(1, 3)).ToArray();
byte[] packedNumbers = (from i in Enumerable.Range(0, hexChars.Length / 2)
select byte.Parse(hexChars[i * 2].ToString() + hexChars[i * 2 + 1], NumberStyles.HexNumber))
.ToArray();

According to the comments given, I suppose, the current answers is preferable.
But about this should do it also:
public byte[] ushort2byteArr(ushort[] arr) {
System.IO.MemoryStream ms = new System.IO.MemoryStream();
System.IO.BinaryWriter bw = new System.IO.BinaryWriter(ms);
for (int i = 0; i < arr.Length-1;) { // check upper limit!
// following is wrong! must extend this to pack 8 12 bit words into 3 uint32!
UInt32 tmp = arr[i++] | (arr[i++] << 12) ... ;
bw.Write(tmp);
}
return ms.ToArray();
}
its not tested. take it as pseudocode to get the clue. especially the word -> uint32 conversion. May need some padding at the end?
#edit: made a function out of it for better clearance