I have been tasked to rewrite a small program written in C++ to C#.
But I came across this line that I couldn't understand fully. Is it concatenating the string length to the string or the pointer?
int n = _keyData * int(*(int*)(_chap + strlen(_chap) - 4));
This is the variables:
short _ver = 12;
short _keyData = short(_ver * _ver);
char _chap[100]; // Hold time with format [%02d-%02d %02d:%02d:%02d:%03d]
*(int*)(_chap + strlen(_chap) - 4) is a strict aliasing violation. Reinterpreting raw bytes as an int is type punning and is not allowed in C++ (even though some compilers tolerate it).
To fix it (assuming a little-endian system), you can rewrite it like this:
short _ver = 12;
short _keyData = short(_ver * _ver);
char _chap[100]; // Hold time with format [%02d-%02d %02d:%02d:%02d:%03d]
int len = strlen(_chap);
int x = (int)(((unsigned)_chap[len - 1] << 24) |
((unsigned)_chap[len - 2] << 16) |
((unsigned)_chap[len - 3] << 8) |
(unsigned)_chap[len - 4]);
int n = _keyData * x;
Coincidently, it should be easy to port this to C# now.
I have a code that turns an integer to its binary representation, but I was wondering if there is a more simple or "easier" way of doing so. I know that there is a built-in method in C# that does this for you automatically, but that is not what I want to use.
This version loops over each o the 32-bit positions while writing ones and zeros and uses TrimStart to remove leading zeroes.
For example, converting the integer 10 to its string representation in
binary as "1010".
static string IntToBinary(int n)
{
char[] b = new char[32];
int pos = 31;
int i = 0;
while (i < 32) // Loops over each of the 32-bit positions while writing ones and zeros.
{
if ((n & (1 << i)) != 0)
{
b[pos] = '1';
}
else
{
b[pos] = '0';
}
pos--;
i++;
}
return new string(b).TrimStart('0'); // TrimStart removes leading zeroes.
}
static void Main()
{
Console.WriteLine(IntToBinary(300));
}
I suppose you could use a nibble lookup table:
static string[] nibbles = {
"0000", "0001", "0010", "0011",
"0100", "0101", "0110", "0111",
"1000", "1001", "1010", "1011",
"1100", "1101", "1110", "1111"
};
public static string IntToBinary(int n)
{
return
nibbles[(n >> 28) & 0xF] +
nibbles[(n >> 24) & 0xF] +
nibbles[(n >> 20) & 0xF] +
nibbles[(n >> 16) & 0xF] +
nibbles[(n >> 12) & 0xF] +
nibbles[(n >> 8) & 0xF] +
nibbles[(n >> 4) & 0xF] +
nibbles[(n >> 0) & 0xF]
.TrimStart('0');
}
Here is a simple LINQ implementation:
static string IntToBinary(int n)
{
return string.Concat(Enumerable.Range(0, 32)
.Select(i => (n & (1 << (31 - i))) != 0 ? '1' : '0')
.SkipWhile(ch => ch == '0'));
}
Another one using for loop:
static string IntToBinary(int n)
{
var chars = new char[32];
int start = chars.Length;
for (uint bits = (uint)n; bits != 0; bits >>= 1)
chars[--start] = (char)('0' + (bits & 1));
return new string(chars, start, chars.Length - start);
}
I have a password hashing method in C# that I'm trying to port it to PHP for my website, which will allow both my website and application to use the passwords from the same database (application requires a website account to use). The problem is, once the password gets over 7 characters in length, the result is different in php then what I'm getting in C#, but any password less then 8 characters, matches the C# encryption exactly.
here my method in C#
public static byte[] PassEncode(byte[] pass)
{
int a = 0;
int num = 0x79707367; // starting num
for (int i = 0; i < pass.Length; i++)
{
num = PassLame(num);
a = num % 0xFF;
pass[i] ^= (byte)a;
}
return pass;
}
private static int PassLame(int num)
{
int c = (num >> 16) & 0xffff;
int a = num & 0xffff;
c *= 0x41a7;
a *= 0x41a7;
a += ((c & 0x7fff) << 16);
if (a < 0)
{
a &= 0x7fffffff;
a++;
}
a += (c >> 15);
if (a < 0)
{
a &= 0x7fffffff;
a++;
}
return a;
}
And my methods in PHP:
function PassEncode($pass)
{
$a = 0;
$num = 0x79707367;
for ($i = 0; $i < sizeof($pass); $i++)
{
$num = PassLame($num);
$a = $num % 0xFF;
$pass[$i] ^= $a;
}
return $pass;
}
function PassLame($num)
{
$c = ($num >> 16) & 0xffff;
$a = $num & 0xffff;
$c *= 0x41a7;
$a *= 0x41a7;
$a += (($c & 0x7fff) << 16);
if ($a < 0)
{
$a &= 0x7fffffff;
$a++;
}
$a += ($c >> 15);
if ($a < 0)
{
$a &= 0x7fffffff;
$a++;
}
return $a;
}
The bytes I'm using is for the word "testing".
bytes = ([0]=> 116 [1]=> 101 [2]=> 115 [3]=> 116 [4]=> 105 [5]=> 110 [6]=> 103)
When I plug these in, the 8th digit returned (and beyond if using a larger pass) are a lot different then in C#./ My results
C#:
[0]=> int(98) [1]=> int(151) [2]=> int(135) [3]=> int(134) [4]=> int(66) [5]=> int(181) [6]=> int(113)
PHP:
[0]=> int(98) [1]=> int(151) [2]=> int(135) [3]=> int(134) [4]=> int(66) [5]=> int(181) [6]=> int(11)
Can anyone help me solve this? I'm using a 32bit webserver and compiling my application in 32bit as well.
I believe PHP handles integer overflow by converting the value to float. That is almost certainly the cause of your problem.
You could write custom functions for the arithmetic that check for overflow and mimic c#'s wraparound behavior.
I would like to implement this in C#
I have looked here:
http://www.codeproject.com/KB/cpp/PEChecksum.aspx
And am aware of the ImageHlp.dll MapFileAndCheckSum function.
However, for various reasons, I would like to implement this myself.
The best I have found is here:
http://forum.sysinternals.com/optional-header-checksum-calculation_topic24214.html
But, I don't understand the explanation. Can anyone clarify how the checksum is calculated?
Thanks!
Update
I from the code example, I do not understand what this means, and how to translate it into C#
sum -= sum < low 16 bits of CheckSum in file // 16-bit borrow
sum -= low 16 bits of CheckSum in file
sum -= sum < high 16 bits of CheckSum in file
sum -= high 16 bits of CheckSum in file
Update #2
Thanks, came across some Python code that does similar too here
def generate_checksum(self):
# This will make sure that the data representing the PE image
# is updated with any changes that might have been made by
# assigning values to header fields as those are not automatically
# updated upon assignment.
#
self.__data__ = self.write()
# Get the offset to the CheckSum field in the OptionalHeader
#
checksum_offset = self.OPTIONAL_HEADER.__file_offset__ + 0x40 # 64
checksum = 0
# Verify the data is dword-aligned. Add padding if needed
#
remainder = len(self.__data__) % 4
data = self.__data__ + ( '\0' * ((4-remainder) * ( remainder != 0 )) )
for i in range( len( data ) / 4 ):
# Skip the checksum field
#
if i == checksum_offset / 4:
continue
dword = struct.unpack('I', data[ i*4 : i*4+4 ])[0]
checksum = (checksum & 0xffffffff) + dword + (checksum>>32)
if checksum > 2**32:
checksum = (checksum & 0xffffffff) + (checksum >> 32)
checksum = (checksum & 0xffff) + (checksum >> 16)
checksum = (checksum) + (checksum >> 16)
checksum = checksum & 0xffff
# The length is the one of the original data, not the padded one
#
return checksum + len(self.__data__)
However, it's still not working for me - here is my conversion of this code:
using System;
using System.IO;
namespace CheckSumTest
{
class Program
{
static void Main(string[] args)
{
var data = File.ReadAllBytes(#"c:\Windows\notepad.exe");
var PEStart = BitConverter.ToInt32(data, 0x3c);
var PECoffStart = PEStart + 4;
var PEOptionalStart = PECoffStart + 20;
var PECheckSum = PEOptionalStart + 64;
var checkSumInFile = BitConverter.ToInt32(data, PECheckSum);
Console.WriteLine(string.Format("{0:x}", checkSumInFile));
long checksum = 0;
var remainder = data.Length % 4;
if (remainder > 0)
{
Array.Resize(ref data, data.Length + (4 - remainder));
}
var top = Math.Pow(2, 32);
for (int i = 0; i < data.Length / 4; i++)
{
if (i == PECheckSum / 4)
{
continue;
}
var dword = BitConverter.ToInt32(data, i * 4);
checksum = (checksum & 0xffffffff) + dword + (checksum >> 32);
if (checksum > top)
{
checksum = (checksum & 0xffffffff) + (checksum >> 32);
}
}
checksum = (checksum & 0xffff) + (checksum >> 16);
checksum = (checksum) + (checksum >> 16);
checksum = checksum & 0xffff;
checksum += (uint)data.Length;
Console.WriteLine(string.Format("{0:x}", checksum));
Console.ReadKey();
}
}
}
Can anyone tell me where I'm being stupid?
Ok, finally got it working ok... my problem was that I was using ints not uints!!!
So, this code works (assuming data is 4-byte aligned, otherwise you'll have to pad it out a little) - and PECheckSum is the position of the CheckSum value within the PE (which is clearly not used when calculating the checksum!!!!)
static uint CalcCheckSum(byte[] data, int PECheckSum)
{
long checksum = 0;
var top = Math.Pow(2, 32);
for (var i = 0; i < data.Length / 4; i++)
{
if (i == PECheckSum / 4)
{
continue;
}
var dword = BitConverter.ToUInt32(data, i * 4);
checksum = (checksum & 0xffffffff) + dword + (checksum >> 32);
if (checksum > top)
{
checksum = (checksum & 0xffffffff) + (checksum >> 32);
}
}
checksum = (checksum & 0xffff) + (checksum >> 16);
checksum = (checksum) + (checksum >> 16);
checksum = checksum & 0xffff;
checksum += (uint)data.Length;
return (uint)checksum;
}
The code in the forum post is not strictly the same as what was noted during the actual disassembly of the Windows PE code. The CodeProject article you reference gives the "fold 32-bit value into 16 bits" as:
mov edx,eax ; EDX = EAX
shr edx,10h ; EDX = EDX >> 16 EDX is high order
and eax,0FFFFh ; EAX = EAX & 0xFFFF EAX is low order
add eax,edx ; EAX = EAX + EDX High Order Folded into Low Order
mov edx,eax ; EDX = EAX
shr edx,10h ; EDX = EDX >> 16 EDX is high order
add eax,edx ; EAX = EAX + EDX High Order Folded into Low Order
and eax,0FFFFh ; EAX = EAX & 0xFFFF EAX is low order 16 bits
Which you could translate into C# as:
// given: uint sum = ...;
uint high = sum >> 16; // take high order from sum
sum &= 0xFFFF; // clear out high order from sum
sum += high; // fold high order into low order
high = sum >> 16; // take the new high order of sum
sum += high; // fold the new high order into sum
sum &= 0xFFFF; // mask to 16 bits
Java code below from emmanuel may not work. In my case it hangs and does not complete. I believe this is due to the heavy use of IO in the code: in particular the data.read()'s. This can be swapped with an array as solution. Where the RandomAccessFile fully or incrementally reads the file into a byte array(s).
I attempted this but the calculation was too slow due to the conditional for the checksum offset to skip the checksum header bytes. I would imagine that the OP's C# solution would have a similar problem.
The below code removes this also.
public static long computeChecksum(RandomAccessFile data, int checksumOffset)
throws IOException {
...
byte[] barray = new byte[(int) length];
data.readFully(barray);
long i = 0;
long ch1, ch2, ch3, ch4, dword;
while (i < checksumOffset) {
ch1 = ((int) barray[(int) i++]) & 0xff;
...
checksum += dword = ch1 | (ch2 << 8) | (ch3 << 16) | (ch4 << 24);
if (checksum > top) {
checksum = (checksum & 0xffffffffL) + (checksum >> 32);
}
}
i += 4;
while (i < length) {
ch1 = ((int) barray[(int) i++]) & 0xff;
...
checksum += dword = ch1 | (ch2 << 8) | (ch3 << 16) | (ch4 << 24);
if (checksum > top) {
checksum = (checksum & 0xffffffffL) + (checksum >> 32);
}
}
checksum = (checksum & 0xffff) + (checksum >> 16);
checksum = checksum + (checksum >> 16);
checksum = checksum & 0xffff;
checksum += length;
return checksum;
}
I still however think that code was too verbose and clunky so I swapped out the raf with a channel and rewrote the culprit bytes to zero's to eliminate the conditional. This code could still probably do with a cache style buffered read.
public static long computeChecksum2(FileChannel ch, int checksumOffset)
throws IOException {
ch.position(0);
long sum = 0;
long top = (long) Math.pow(2, 32);
long length = ch.size();
ByteBuffer buffer = ByteBuffer.wrap(new byte[(int) length]);
buffer.order(ByteOrder.LITTLE_ENDIAN);
ch.read(buffer);
buffer.putInt(checksumOffset, 0x0000);
buffer.position(0);
while (buffer.hasRemaining()) {
sum += buffer.getInt() & 0xffffffffL;
if (sum > top) {
sum = (sum & 0xffffffffL) + (sum >> 32);
}
}
sum = (sum & 0xffff) + (sum >> 16);
sum = sum + (sum >> 16);
sum = sum & 0xffff;
sum += length;
return sum;
}
No one really answered the original question of "Can anyone define the Windows PE Checksum Algorithm?" so I'm going to define it as simply as possible. A lot of the examples given so far are optimizing for unsigned 32-bit integers (aka DWORDs), but if you just want to understand the algorithm itself at its most fundamental, it is simply this:
Using an unsigned 16-bit integer (aka a WORD) to store the checksum, add up all of the WORDs of the data except for the 4 bytes of the PE optional header checksum. If the file is not WORD-aligned, then the last byte is a 0x00.
Convert the checksum from a WORD to a DWORD and add the size of the file.
The PE checksum algorithm above is effectively the same as the original MS-DOS checksum algorithm. The only differences are the location to skip and replacing the XOR 0xFFFF at the end and adding the size of the file instead.
From my WinPEFile class for PHP, the above algorithm looks like:
$x = 0;
$y = strlen($data);
$val = 0;
while ($x < $y)
{
// Skip the checksum field location.
if ($x === $this->pe_opt_header["checksum_pos"]) $x += 4;
else
{
$val += self::GetUInt16($data, $x, $y);
// In PHP, integers are either signed 32-bit or 64-bit integers.
if ($val > 0xFFFF) $val = ($val & 0xFFFF) + 1;
}
}
// Add the file size.
$val += $y;
I was trying to solve the same issue in Java. Here is Mark's solution translated into Java, using a RandomAccessFile instead of a byte array as input:
static long computeChecksum(RandomAccessFile data, long checksumOffset) throws IOException {
long checksum = 0;
long top = (long) Math.pow(2, 32);
long length = data.length();
for (long i = 0; i < length / 4; i++) {
if (i == checksumOffset / 4) {
data.skipBytes(4);
continue;
}
long ch1 = data.read();
long ch2 = data.read();
long ch3 = data.read();
long ch4 = data.read();
long dword = ch1 + (ch2 << 8) + (ch3 << 16) + (ch4 << 24);
checksum = (checksum & 0xffffffffL) + dword + (checksum >> 32);
if (checksum > top) {
checksum = (checksum & 0xffffffffL) + (checksum >> 32);
}
}
checksum = (checksum & 0xffff) + (checksum >> 16);
checksum = checksum + (checksum >> 16);
checksum = checksum & 0xffff;
checksum += length;
return checksum;
}
private unsafe static int GetSetPEChecksum(byte[] Array) {
var Value = 0;
var Count = Array.Length;
if(Count >= 64)
fixed (byte* array = Array) {
var Index = 0;
var Coff = *(int*)(array + 60);
if(Coff >= 64 && Count >= Coff + 92) {
*(int*)(array + Coff + 88) = 0;
var Bound = Count >> 1;
if((Count & 1) != 0) Value = array[Count & ~1];
var Short = (ushort*)array;
while(Index < Bound) {
Value += Short[Index++];
Value = (Value & 0xffff) + (Value >> 16);
Value = (Value + (Value >> 16)) & 0xffff;
}
*(int*)(array + Coff + 88) = Value += Count;
}
}
return Value;
}
If you need short unsafe... (Not need use Double and Long integers and not need Array aligning inside algorithm)
The Java example is not entirely correct. Following Java implementation corresponds with the result of Microsoft's original implementation from Imagehlp.MapFileAndCheckSumA.
It's important that the input bytes are getting masked with inputByte & 0xff and the resulting long masked again when it's used in the addition term with currentWord & 0xffffffffL (consider the L):
long checksum = 0;
final long max = 4294967296L; // 2^32
// verify the data is DWORD-aligned and add padding if needed
final int remainder = data.length % 4;
final byte[] paddedData = Arrays.copyOf(data, data.length
+ (remainder > 0 ? 4 - remainder : 0));
for (int i = 0; i <= paddedData.length - 4; i += 4)
{
// skip the checksum field
if (i == this.offsetToOriginalCheckSum)
continue;
// take DWORD into account for computation
final long currentWord = (paddedData[i] & 0xff)
+ ((paddedData[i + 1] & 0xff) << 8)
+ ((paddedData[i + 2] & 0xff) << 16)
+ ((paddedData[i + 3] & 0xff) << 24);
checksum = (checksum & 0xffffffffL) + (currentWord & 0xffffffffL);
if (checksum > max)
checksum = (checksum & 0xffffffffL) + (checksum >> 32);
}
checksum = (checksum & 0xffff) + (checksum >> 16);
checksum = checksum + (checksum >> 16);
checksum = checksum & 0xffff;
checksum += data.length; // must be original data length
In this case, Java is a bit inconvenient.
The CheckSum field is 32 bits long and is calculated as follows
1. Add all dwords (32 bit pieces) of the entire file to a sum
Add all dwords of the entire file not including the CheckSum field itself, including all headers and all of the contents, to a dword. If the dword overflows, add the overflowed bit back to the first bit (2^0) of the dword.
If the file is not entirely divisible into dwords (4 bit pieces) see 2.
The best way I know to realize this is by using the GNU C Compilers Integer Overflow Builtin function __builtin_uadd_overflow.
In the original ChkSum function documented by Jeffrey Walton the sum
was calculated by performing an add (%esi),%eax where
esi contains the base address of the file and eax is 0 and adding the rest of the file like this
adc 0x4(%esi),%eax
adc 0x8(%esi),%eax
adc 0xc(%esi),%eax
adc 0x10(%esi),%eax
...
adc $0x0,%eax
The first add adds the first dword ignoring any carry flag. The next dwords
are added by the adc instruction which does the same thing as add but
adds any carry flag that was set before executing the instruction in addition
to the summand. The last adc $0x0,%eax adds only the last carry flag if it
was set and cannot be discarded.
Please keep in mind that the dword of CheckSum field itself should not be added.
2. Add the remainder to the sum if there is one
If the file is not entirely divisible into dwords, add the remainder as a
zero-padded dword. For example: say your file is 15 bytes long and looks like this
0E 1F BA 0E | 00 B4 09 CD | 21 B8 01 4C | CD 21 54
You need to add the remainder as 0x005421CD to the sum. My system is a
little-endian system. I do not know if the checksum would change because of the
this order of the bytes on big-endian systems, or you would just simulate this
behaviour.
I do this by rounding up the buffer_size to the next bytecount divisible by 4
without remainder or put differently: the next whole dword count represented
in bytes. Then I allocate with calloc because it initializes the memory block
with all zeros.
if(buffer_size%4)
{buffer_size+=4-(buffer_size%4);
...
calloc(buffer_size,1)
3. Add the lower word (16 bit piece) and the higher word of the sum together.
sum=(sum&0xffff)+(sum>>16);
4. Add the new higher word once again
sum+=(sum>>16);
5. Only keep the lower word
sum&=0xffff;
6. Add the number of bytes in the file to the sum
return(sum+size);
This is how I wrote it. It is not C#, but C. off_t size is the number of bytes in the file. uint32_t *base is a pointer to the file loaded into memory. The block of memory should be padded with zeros at the end to the next bytecount divisible by 4.
uint32_t pe_header_checksum(uint32_t *base,off_t size)
{uint32_t sum=0;
off_t i;
for(i=0;i<(size/4);i++)
{if(i==0x36)
{continue;}
sum+=__builtin_uadd_overflow(base[i],sum,&sum);}
if(size%4)
{sum+=base[i];}
sum=(sum&0xffff)+(sum>>16);
sum+=(sum>>16);
sum&=0xffff;
return(sum+size);}
If you want you can see the code in action and read a little bit more here.
I have the code below to convert a 32 bit BCD value (supplied in two uint halves) to a uint binary value.
The values supplied can be up to 0x9999, to form a maximum value of 0x99999999.
Is there a better (ie. quicker) way to achieve this?
/// <summary>
/// Convert two PLC words in BCD format (forming 8 digit number) into single binary integer.
/// e.g. If Lower = 0x5678 and Upper = 0x1234, then Return is 12345678 decimal, or 0xbc614e.
/// </summary>
/// <param name="lower">Least significant 16 bits.</param>
/// <param name="upper">Most significant 16 bits.</param>
/// <returns>32 bit unsigned integer.</returns>
/// <remarks>If the parameters supplied are invalid, returns zero.</remarks>
private static uint BCD2ToBin(uint lower, uint upper)
{
uint binVal = 0;
if ((lower | upper) != 0)
{
int shift = 0;
uint multiplier = 1;
uint bcdVal = (upper << 16) | lower;
for (int i = 0; i < 8; i++)
{
uint digit = (bcdVal >> shift) & 0xf;
if (digit > 9)
{
binVal = 0;
break;
}
else
{
binVal += digit * multiplier;
shift += 4;
multiplier *= 10;
}
}
}
return binVal;
}
If you've space to spare for a 39,322 element array, you could always just look the value up.
If you unroll the loop, remember to keep the bit shift.
value = ( lo & 0xF);
value += ((lo >> 4 ) & 0xF) * 10;
value += ((lo >> 8 ) & 0xF) * 100;
value += ((lo >> 12) & 0xF) * 1000;
value += ( hi & 0xF) * 10000;
value += ((hi >> 4 ) & 0xF) * 100000;
value += ((hi >> 8 ) & 0xF) * 1000000;
value += ((hi >> 12) & 0xF) * 10000000;
Your code seems rather complicated; do you require the specific error checking?
Otherwise, you could just use the following code which shouldn't be slower, in fact, it's mostly the same:
uint result = 0;
uint multiplier = 1;
uint value = lo | hi << 0x10;
while (value > 0) {
uint digit = value & 0xF;
value >>= 4;
result += multiplier * digit;
multiplier *= 10;
}
return result;
I suppose you could unroll the loop:
value = ( lo & 0xF);
value+= ((lo>>4) & 0xF) *10;
value+= ((lo>>8) & 0xF) *100;
value+= ((lo>>12)& 0xF) *1000;
value+= ( hi & 0xF) *10000;
value+= ((hi>>4 & 0xF) *100000;
value+= ((hi>>8) & 0xF) *1000000;
value+= ((hi>>12)& 0xF) *10000000;
And you can check for invalid BCD digits like this:
invalid = lo & ((lo&0x8888)>>2)*3
This sets invalid to a non-zero value if any single hex digit > 9.
Try this:
public static int bcd2int(int bcd) {
return int.Parse(bcd.ToString("X"));
}
public static uint BCDToNum(int num)
{
return uint.Parse(num.ToString(), System.Globalization.NumberStyles.HexNumber);
}
Of course, there are a more efficient method. this is just a example of course, so you can tune it as a lesson ^^
function bcd_to_bin ($bcd) {
$mask_sbb = 0x33333333;
$mask_msb = 0x88888888;
$mask_opp = 0xF;
for($i=28;$i;--$i) {
$mask_msb <<= 1;
$mask_opp <<= 1;
$mask_sbb <<= 1;
for($j=0;$j<$i;$j+=4) {
$mask_opp_j = $mask_opp << $j;
if ($bcd & $mask_msb & $mask_opp_j ) {
$bcd -= $mask_sbb & $mask_opp_j;
}
}
}
return $bcd;
}