We Keep Coding

Convert hashed value to a number for Time Based One Time Password (TOTP)

I've read this Github documentation: Otp.NET
In a section there are these codes:
protected internal long CalculateOtp(byte[] data, OtpHashMode mode)
{
byte[] hmacComputedHash = this.secretKey.ComputeHmac(mode, data);
// The RFC has a hard coded index 19 in this value.
// This is the same thing but also accomodates SHA256 and SHA512
// hmacComputedHash[19] => hmacComputedHash[hmacComputedHash.Length - 1]
int offset = hmacComputedHash[hmacComputedHash.Length - 1] & 0x0F;
return (hmacComputedHash[offset] & 0x7f) << 24
| (hmacComputedHash[offset + 1] & 0xff) << 16
| (hmacComputedHash[offset + 2] & 0xff) << 8
| (hmacComputedHash[offset + 3] & 0xff) % 1000000;
}
I think the last part of above method is convert hashed value to a number but I don't understand the philosophy and the algorithm of it.
1)What is the offset?
2)Why some bytes AND with 0x0f or 0xff?
3)Why in hast line it get Remain for 1000000?
Thanks

RFC 4226 specifies how the data is to be calculated from the HMAC value.
First, the bottom four bits of the last byte are used to determine a starting offset into the HMAC value. This was done so that even if an attacker found a weakness in some fixed portion of the HMAC output, it would be hard to leverage that directly into an attack.
Then, four bytes, big-endian, are read from the HMAC output starting at that offset. The top bit is cleared, to prevent any problems with negative numbers being mishandled, since some languages (e.g., Java) don't provide native unsigned numbers. Finally, the lower N digits are taken (which is typically 6, but sometimes 7 or 8). In this case, the implementation is hard-coded to 6, hence the modulo operation.
Note that due to operator precedence, the bitwise-ors bind more tightly than the modulo operation. This implementer has decided that they'd like to be clever and have not helped us out by adding an explicit pair of parentheses, but in the real world, it's nice to help the reader.

When is a shift operator >> or << useful? [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
When to use Shift operators << >> in C# ?
I've in programming a while and I've never used the shift operator. I could see how it could be helpful for calculating hash codes like in Tuple<T>, but other than that,
When and how is the shift operator useful in C#/.NET?

In general it's not used very often. But it's very useful when dealing with bit level operations. For example printing out the bits in a numeric value
public static string GetBits(int value) {
var builder = new StringBuilder();
for (int i = 0; i < 32; i++) {
var test = 1 << (31 - i);
var isSet = 0 != (test & value);
builder.Append(isSet ? '1' : '0');
}
return builder.ToString();
}

It's useful to write powers of two.
Quick: What's 227?
Answer: 1 << 27
Writing 1 << 27 is both easier and more understandable than 134217728.

I use it rather a lot in dealing with hardware. This isn't something you probably do in C# a lot, but the operator was inherited from C/C++ where it is a fairly common need.
Example 1:
I just got a longword from a little-endian machine, but I'm big endian. How do I convert it? Well, the obvious is call htonl() (you cheater). One of the manual ways to do it is with something like the following:
((source & 0x000000ff) << 24 ) |
((source & 0x0000ff00) << 8) |
((source & 0x00ff0000) >> 8) |
((source & 0xff000000) >> 24);
Example 2:
I have a DMA device that only allows longword accesses of up to 512K. So it requires me to put (for reasons only understood by hardware folks) the modulo 4 of the transfer size into the high order 18 bits of a DMA transfer control register. For the sake of arguement, the low-order bits are to be filled with various flags controlling the DMA operation. That would be accomplished like so:
dma_flags | ((length & 0xffffc) << 14);
These might not be the kind of things you do every day. But for those of us that regularly interface to hardware they are.

If you ever need to multiply without using * How to implement multiplication without using multiplication operator in .NET :)
Or write a Sudoku solver Sudoku validity check algorithm - how does this code works?
In practice, the only time I've seen it used in my (limited) experience was as an (arguably) confusing way to multiply (see first link) or in conjunction with setting BitFlags (the Sudoku solver above).
In .NET I rarely have to work at the bit level; but if you need to, being able to shift is important.

Bitwise operators are good for saving space, but nowadays, space is hardly an issue.
It's useful when multiplying by powers of 2
number<<power;
is number*2^power
And of course division by powers of 2:
number>>power;
Another place is flags in enums.
when you come across code like
Regex re = new Regex(".",RegexOptions.Multiline|RegexOptions.Singleline);
the ability to use RegexOptions.Multiline|RegexOptions.Singleline i.e multiple flags is enabled through the shifting and also this allows them to be unique.
Something like:
enum RegexOptions {
Multiline = (1 << 0),
Singleline = (1<<1)
};

Bit shifts are used when manipulating individual bits is desired. You'll see a lot of bit shifts in many encryption algorithms, for example.
In optimization, it can used in place of multiplication/division. A shift left is equal to multiplying by two. Shift right equals division. You probably don't see this done anymore, since this level of optimization is often unnecessary.
Other than that, I can't think of many reasons to use it. I've seen it used before, but rarely in cases where it was really required and usually a more readable method could have been used.

Whenever you need to multiply by 2 ;)
Really the only use I have is for interoperability code and bitfields:
http://www.codeproject.com/KB/cs/masksandflags.aspx

C# UL and >> operators

What does this meaning mean in words?
(SomeVariable * 330UL >> 10)
Is it: SomeVariable times 3.3 shift right 10 bit??

No.
It means SomeVariable times 330, promote to long and shift non-cyclically right 10bits.
(it would be cyclic, or arithmetic shift without the promotion).

Right-shifting an integral value by one is equivalent to dividing it by 2. Two shifts equivalent to dividing by 4. Etcetera. Which makes the expression equivalent to:
ulong value = ((ulong)SomeVariable * 330) / 1024;

UL stands for Unsigned Long. >> yes it is bitwise arithmetic shift.

SomeVariable times 330 as an unsigned long shift right 10 bits

Fastest way to calculate an X-bit bitmask?

I have been trying to solve this problem for a while, but couldn't with just integer arithmetic and bitwise operators. However, I think its possible and it should be fairly easy. What am I missing?
The problem: to get an integer value of arbitrary length (this is not relevant to the problem) with it's X least significant bits sets to 1 and the rest to 0. For example, given the number 31, I need to get an integer value which equals 0x7FFFFFFF (31 least significant bits are 1 and the rest zeros).
Of course, using a loop OR-ing a shifted 1 to an integer X times will do the job. But that's not the solution I'm looking for. It should be more in the direction of (X << Y - 1), thus using no loops.

Try this: (1 << X) - 1

Try this:
uint.MaxValue >> (32 - something)

I think the following should work:
int mask = (int)Math.Pow(2, 31) - 1;
This is a single mathematical expression, but it isn't particularly efficient because calculating the power in this way is not really a good idea. However, since we're calculating a power of 2, we can do the same thing using shift:
int mask = (1 << 31) - 1;

C/C++ Date Solution/Conversion

I need to come up with a way to unpack a date into a readable format. unfortunately I don't completely understand the original process/code that was used.
Per information that was forwarded to me the date was packed using custom C/Python code as follows;
date = year << 20;
date |= month << 16;
date |= day << 11;
date |= hour << 6;
date |= minute;
For example, a recent packed date is 2107224749 which equates to Tuesday Sept. 22 2009 10:45am
I understand....or at least I am pretty sure....the << is shifting the bits but I am not sure what the "|" accomplishes.
Also, in order to unpack the code the notes read as follows;
year = (date & 0xfff00000) >> 20;
month = (date & 0x000f0000) >> 16;
day = (date & 0x0000f800) >> 11;
hour = (date & 0x000007c0) >> 6;
minute = (date & 0x0000003f);
Ultimately, what I need to do is perform the unpack and convert to readable format using either JavaScript or ASP but I need to better understand the process above in order to develop a solution.
Any help, hints, tips, pointers, ideas, etc. would be greatly appreciated.

The pipe (|) is bitwise or, it is used to combine the bits into a single value.
The extraction looks straight-forward, except I would recommend shifting first, and masking then. This keeps the constant used for the mask as small as possible, which is easier to manage (and can possibly be a tad more efficient, although for this case that hardly matters).
Looking at the masks used written in binary reveals how many bits are used for each field:
0xfff00000 has 12 bits set, so 12 bits are used for the year
0x000f0000 has 4 bits set, for the month
0x0000f800 has 5 bits set, for the day
0x000007c0 has 5 bits set, for the hour
0x0000003f has 6 bits set, for the minute

The idea is exactly what you said. Performing "<<" just shifts the bits to the left.
What the | (bitwise or) is accomplishing is basically adding more bits to the number, but without overwriting what was already there.
A demonstration of this principle might help.
Let's say we have a byte (8 bits), and we have two numbers that are each 4 bits, which we want to "put together" to make a byte. Assume the numbers are, in binary, 1010, and 1011. So we want to end up with the byte: 10101011.
Now, how do we do this? Assume we have a byte b, which is initialized to 0.
If we take the first number we want to add, 1010, and shift it by 4 bits, we get the number 10100000 (the shift adds bytes to the right of the number).
If we do: b = (1010 << 4), b will have the value 10100000.
But now, we want to add the 4 more bits (0011), without touching the previous bits. To do this, we can use |. This is because the | operator "ignores" anything in our number which is zero. So when we do:
10100000 (b's current value)
|
00001011 (the number we want to add)
We get:
10101011 (the first four numbers are copied from the first number,
the other four numbers copied from the second number).
Note: This answer came out a little long, I'm wikiing this, so, if anyone here has a better idea how to explain it, I'd appreciate your help.

These links might help:
http://www.gamedev.net/reference/articles/article1563.asp
http://compsci.ca/v3/viewtopic.php?t=9893

In the decode section & is bit wise and the 0xfff00000 is a hexadecimal bit mask. Basically each character in the bit mask represents 4 bits of the number. 0 being 0000 in binary and f being 1111 so if you look at the operation in binary you are anding 1111 1111 1111 0000 0000 ... with whatever is in date so basically you are getting the upper three nibbles(half bytes) and shifting them down so that 00A00000 gives you 10(A in hex) for the year.
Also note that |= is like += it is bit wise or then assignment rolled in to one.

Just to add some practical tips:
minute = value & ((1 << 6)-1);
hour = (value >> 6) & ((1<<5)-1); // 5 == 11-6 == bits reserved for hour
...
1 << 5 creates a bit at position 5 (i.e. 32=00100000b),
(1<<5)-1 cretaes a bit mask where the 5 lowest bits are set (i.e. 31 == 00011111b)
x & ((1<<5)-1) does a bitwise 'and' preserving only the bits set in the lowest five bits, extracting the original hour value.

Yes the << shifts bits and the | is the bitwise OR operator.