To convert an integer to a hex formatted string I am using ToString("X4") like so:
int target = 250;
string hexString = target.ToString("X4");
To get an integer value from a hex formatted string I use the Parse method:
int answer = int.Parse(data, System.Globalization.NumberStyles.HexNumber);
However the machine that I'm exchanging data with puts the bytes in reverse order.
To keep with the sample data, If I want to send the value 250 I need a string of "FA00" (not 00FA which is what hexString is) Likewise if I get "FA00" I need to convert that to 250 not 64000.
How do I set the endianness of these two converstion methods?
Marc's answer seems, by virtue of having been accepted, to have addressed the OP's original issue. However, it's not really clear to me from the question text why. That still seems to require swapping of bytes, not pairs of bytes as Marc's answer does. I'm not aware of any reasonably common scenario where swapping bits 16 at a time makes sense or is useful.
For the stated requirements, IMHO it would make more sense to write this:
int target = 250; // 0x00FA
// swap the bytes of target
target = ((target << 8) | (target >> 8)) & 0xFFFF;
// target now is 0xFA00
string hexString = target.ToString("X4");
Note that the above assumes we're actually dealing with 16-bit values, stored in a 32-bit int variable. It will handle any input in the 16-bit range (note the need to mask off the upper 16 bits, as they get set to non-zero values by the << operator).
If swapping 32-bit values, one would need something like this:
int target = 250; // 0x00FA
// swap the bytes of target
target = (int)((int)((target << 24) & 0xff000000) |
((target << 8) & 0xff0000) |
((target >> 8) & 0xff00) |
((target >> 24) & 0xff));
// target now is 0xFA000000
string hexString = target.ToString("X8");
Again, masking is required to isolate the bits we are moving to specific positions. Casting the << 24 result back to int before or-ing with the other three bytes is needed because 0xff000000 is a uint (UInt32) literal and causes the & expression to be extended to long (Int64). Otherwise, you'll get compiler warnings with each of the | operators.
In any case, as this comes up most often in networking scenarios, it is worth noting that .NET provides helper methods that can assist with this operation: HostToNetworkOrder() and NetworkToHostOrder(). In this context, "network order" is always big-endian, and "host order" is whatever byte order is used on the computer hosting the current process.
If you know that you are receiving data that's big-endian, and you want to be able to interpret in as correct values in your process, you can call NetworkToHostOrder(). Likewise, if you need to send data in a context where big-endian is expected, you can call HostToNetworkOrder().
These methods work only with the three basic integer types: Int16, Int32, and Int64 (in C#, short, int, and long, respectively). They also return the same type passed to them, so you have to be careful about sign extension. The original example in the question could be solved like this:
int target = 250; // 0x00FA
// swap the bytes of target
target = IPAddress.HostToNetworkOrder((short)target) & 0xFFFF;
// target now is 0xFA00
string hexString = target.ToString("X4");
Once again, masking is required because otherwise the short value returned by the method will be sign-extended to 32 bits. If bit 15 (i.e. 0x8000) is set in the result, then the final int value would otherwise have its highest 16 bits set as well. This could be addressed without masking simply by using more appropriate data types for the variables (e.g. short when the data is expected to be signed 16-bit values).
Finally, I will note that the HostToNetworkOrder() and NetworkToHostOrder() methods, since they are only ever swapping bytes, are equivalent to each other. They both swap bytes, when the machine architecture is little-endian† . And indeed, the .NET implementation is simply for the NetworkToHostOrder() to call HostToNetworkOrder(). There are two methods mainly so that the .NET API matches the original BSD sockets API, which included functions like htons() and ntohs(), and that API in turn included functions for both directions of conversion mainly so that it was clear in code whether one was receiving data from the network or sending data to the network.
† And do nothing when the machine architecture is big-endian…they aren't useful as generalized byte-swapping functions. Rather, the expectation is that the network protocol will always be big-endian, and these functions are used to ensure the data bytes are swapped to match whatever the machine architecture is.
That isn't an inbuilt option. So either do string work to swap the characters around, or so some bit-shifting, I.e.
int otherEndian = (value << 16) | (((uint)value) >> 16);
Related
I've read this Github documentation: Otp.NET
In a section there are these codes:
protected internal long CalculateOtp(byte[] data, OtpHashMode mode)
{
byte[] hmacComputedHash = this.secretKey.ComputeHmac(mode, data);
// The RFC has a hard coded index 19 in this value.
// This is the same thing but also accomodates SHA256 and SHA512
// hmacComputedHash[19] => hmacComputedHash[hmacComputedHash.Length - 1]
int offset = hmacComputedHash[hmacComputedHash.Length - 1] & 0x0F;
return (hmacComputedHash[offset] & 0x7f) << 24
| (hmacComputedHash[offset + 1] & 0xff) << 16
| (hmacComputedHash[offset + 2] & 0xff) << 8
| (hmacComputedHash[offset + 3] & 0xff) % 1000000;
}
I think the last part of above method is convert hashed value to a number but I don't understand the philosophy and the algorithm of it.
1)What is the offset?
2)Why some bytes AND with 0x0f or 0xff?
3)Why in hast line it get Remain for 1000000?
Thanks
RFC 4226 specifies how the data is to be calculated from the HMAC value.
First, the bottom four bits of the last byte are used to determine a starting offset into the HMAC value. This was done so that even if an attacker found a weakness in some fixed portion of the HMAC output, it would be hard to leverage that directly into an attack.
Then, four bytes, big-endian, are read from the HMAC output starting at that offset. The top bit is cleared, to prevent any problems with negative numbers being mishandled, since some languages (e.g., Java) don't provide native unsigned numbers. Finally, the lower N digits are taken (which is typically 6, but sometimes 7 or 8). In this case, the implementation is hard-coded to 6, hence the modulo operation.
Note that due to operator precedence, the bitwise-ors bind more tightly than the modulo operation. This implementer has decided that they'd like to be clever and have not helped us out by adding an explicit pair of parentheses, but in the real world, it's nice to help the reader.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 5 years ago.
Improve this question
I need to pack and unpack several values to/from a single 64-bit value. I have 3 signed integers (x,y,z). I would like to pack them into a single 64-bit value (signed or unsigned doesn't matter to me) using 24, 16, and 24 bits for the values respectively. Here are my requirements:
1) I can ensure ahead of times that the values being stored do not exceed the limits of the number of bits I am using to store them into the 64-bit value, so no additional checks need to be made.
2) The initial values are signed, so I'm thinking some kind of bit magic may need to be done in order to ensure that nothing is lost.
3) This conversion is going to take place a LOT, so it needs to be fast. I know in C++ this can pretty easily be done by storing the values in a struct that specifies the integer length, and then establishing a pointer that just points to the first value that can be used for the 64-bit value. With this method, there really isn't any math that needs done, everything is just memory read or right. As far as I can tell, this can't be done quite so simply in C#, but C# is what I have to work with for this project.
4) I don't really care if the 64-bit value is signed or unsigned, so long as I can go both ways with the operation and recover the initial values, and whatever type is used can be used for a Dictionary key.
Masks and shifts are probably your best option. You can create explicit layout structs in C#, but there's no 24-bit primitive, so you'd be tripping over yourself and have to mask anyway. As soon as you're shifting, it is usually best to work unsigned (especially when right-shifting), so:
ulong val = ((((ulong)x) & 0xFFFFFF) << 40) // 24 bits of x, left-shifted by 40
| ((((ulong)y) & 0xFFFF) << 24) // 16 bits of y, left-shifted by 24
| (((ulong)z) & 0xFFFFFF); // 24 bits of z, no left-shift
and to reverse that (assuming that we want uint values):
uint a = (uint)((val >> 40) & 0xFFFFFF),
b = (uint)((val >> 24) & 0xFFFF),
c = (uint)(val & 0xFFFFFF);
With this method, there really isn't any math that needs done, everything is just memory read or write.
Not really, the math is done when you set partial integers into bitfields, so there's quite a bit of math going on.
As far as I can tell, this can't be done quite so simply in C#, but C# is what I have to work with for this project.
Correct, in C# you would need to write code that combines bits into a long manually. Assuming that you have taken care of range checking, this is relatively straightforward:
static long Pack(long a24, long b16, long c24) {
// a24 can go with no masking, because its MSB becomes
// the MSB of the 64-bit number. The other two numbers
// need to be truncated to deal with 1s in the upper bits of negatives.
return a24<<40 | (b16&0xffffL)<<24 | (c24&0xffffffL);
}
static void Unpack(long packed, out int a24, out int b16, out int c24) {
a24 = (int)(packed >> 40); // Sign extension is done in the long
b16 = ((int)(packed >> 8)) >> 16; // Sign extension is done in the int
c24 = ((int)(packed << 8)) >> 8; // Sign extension is done in the int
}
Demo.
These values are byte-aligned inside the long, you'll want to take advantage of Intel/AMD processors being able to directly access them to make the code as fast as possible. The killer requirement is the 24 bit size, the processor can only directly read/write 8, 16, 32 or 64 bits.
That is a problem in C++ as well, you'd have to use bit-fields. C# does not support them, you'll have to write the code that the C++ compiler emits automatically. Like this:
[StructLayout(LayoutKind.Explicit)]
struct MyPackedLong {
[FieldOffset(0)] uint item1; // 24-bit field
[FieldOffset(3)] uint item2; // 24-bit field
[FieldOffset(6)] ushort item3; // 16-bit field
public uint Item1 {
get { return item1 & 0xffffff; }
set { item1 = (item1 & 0xff000000) | value; }
}
public uint Item2 {
get { return item2 & 0xffffff; }
set { item2 = (item2 & 0xff000000) | value; }
}
public ushort Item3 {
get { return item3; }
set { item3 = value; }
}
}
Some trickorama here, note how item2 has an intentional offset of 3 so that no shift is necessary. I ordered the fields so their access is optimal, putting the 16-bit value either first or last is best. Not thoroughly tested, ought to be in the ballpark. Be careful in threaded code, the writes are not atomic.
I have such code in C++
#define dppHRESULT(Code) \
MAKE_HRESULT(1, 138, (Code))
long x = dppHRESULT(101);
result being x = -2138439579.
MAKE_HRESULT is a windows function and defined as
#define MAKE_HRESULT(sev,fac,code) \
((HRESULT) (((unsigned long)(sev)<<31) | ((unsigned long)(fac)<<16) | ((unsigned long)(code))) )
I need to replicate this in C#. So I wrote this code:
public static long MakeHResult(uint facility, uint errorNo)
{
// Make HR
uint result = (uint)1 << 31;
result |= (uint)facility << 16;
result |= (uint)errorNo;
return (long) result;
}
And call like:
// Should return type be long actually??
long test = DppUtilities.MakeHResult(138, 101);
But I get different result, test = 2156527717.
Why? Can someone please help me replicate that C++ function also in C#? Such that I get similar output on similar inputs?
Alternative implementation.
If I use this implementation
public static long MakeHResult(ulong facility, ulong errorNo)
{
// Make HR
long result = (long)1 << 31;
result |= (long)facility << 16;
result |= (long)errorNo;
return (long) result;
}
this works on input 101.
But if I input -1, then C++ returns -1 as result while C# returns 4294967295. Why?
I would really appreciate some help as I am stuck with it.
I've rewritten the function to be the C# equivalent.
static int MakeHResult(uint facility, uint errorNo)
{
// Make HR
uint result = 1U << 31;
result |= facility << 16;
result |= errorNo;
return unchecked((int)result);
}
C# is more strict about signed/unsigned conversions, whereas the original C code didn't pay any mind to it. Mixing signed and unsigned types usually leads to headaches.
As Ben Voigt mentions in his answer, there is a difference in type naming between the two languages. long in C is actually int in C#. They both refer to 32-bit types.
The U in 1U means "this is an unsigned integer." (Brief refresher: signed types can store negative numbers, unsigned types cannot.) All the arithmetic in this function is done unsigned, and the final value is simply cast to a signed value at the end. This is the closest approximation to the original C macro posted.
unchecked is required here because otherwise C# will not allow you to convert the value if it's out of range of the target type, even if the bits are identical. Switching between signed and unsigned will generally require this if you don't mind that the values differ when you deal with negative numbers.
In Windows C++ compilers, long is 32-bits. In C#, long is 64-bits. Your C# conversion of this code should not contain the type keyword long at all.
SaxxonPike has provided the correct translation, but his explanation(s) are missing this vital information.
Your intermediate result is a 32-bit unsigned integer. In the C++ version, the cast is to a signed 32-bit value, resulting in the high bit being reinterpreted as a sign bit. SaxxonPike's code does this as well. The result is negative if the intermediate value had its most significant bit set.
In the original code in the question, the cast is to a 64-bit signed version, which preserves the old high bit as a normal binary digit, and adds a new sign bit (always zero). Thus the result is always positive. Even though the low 32-bits exactly match the 32-bit result in C++, in the C# version returning long, what would be the sign bit in C++ isn't treated as a sign bit.
In the new attempt in the question, the same thing happens (sign bit in the 64-bit number is always zero), but it happens in intermediate calculations instead of at the end.
You're calculating it inside an unsigned type (uint). So shifts are going to behave accordingly. Try using int instead and see what happens.
The clue here is that 2156527717 as an unsigned int is the same as -2138439579 as a signed int. They are literally the same bits.
I currently have the following:
N.Sockets.UdpClient UD;
UD = new N.Sockets.UdpClient();
UD.Connect("xxx.xxx.xxx.xxx", 5255);
UD.Send( data, data.Length );
How would I send data in hex? I cannot just save it straight into a Byte array.
Hex is just an encoding. It's simply a way of representing a number. The computer works with bits and bytes only -- it has no notion of "hex".
So any number, whether represented in hex or decimal or binary, can be encoded into a series of bytes:
var data = new byte[] { 0xFF };
And any hex string can be converted into a number (using, e.g. int.Parse()).
Things get more interesting when a number exceeds one byte: Then there has to be an agreement of how many bytes will be used to represent the number, and the order they should be in.
In C#, ints are 4 bytes. Internally, depending on the endianness of the CPU, the most significant byte (highest-valued digits) might be stored first (big-endian) or last (little-endian). Typically, big-endian is used as the standard for communication over the network (remember the sender and receiver might have CPUs with different endianness). But, since you are sending the raw bytes manually, I'll assume you are also reading the raw bytes manually on the other end; if that's the case, you are of course free to use any arbitrary format you like, providing that the client can understand that format unambiguously.
To encode an int in big-endian order, you can do:
int num = 0xdeadbeef;
var unum = (uint)num; // Convert to uint for correct >> with negative numbers
var data = new[] {
(byte)(unum >> 24),
(byte)(unum >> 16),
(byte)(unum >> 8),
(byte)(unum)
};
Be aware that some packets might never reach the client (this is the main practical difference between TCP and UDP), possibly leading to misinterpretation of the bytes. You should take steps to improve the robustness of your message-sending (e.g. by adding a checksum, and ignoring values whose checksums are invalid or missing).
I need to send an integer through a NetworkStream. The problem is that I only can send bytes.
Thats why I need to split the integer in four byte's and send those and at the other end convert it back to a int.
For now I need this only in C#. But for the final project I will need to convert the four bytes to an int in Lua.
[EDIT]
How about in Lua?
BitConverter is the easiest way, but if you want to control the order of the bytes you can do bit shifting yourself.
int foo = int.MaxValue;
byte lolo = (byte)(foo & 0xff);
byte hilo = (byte)((foo >> 8) & 0xff);
byte lohi = (byte)((foo >> 16) & 0xff);
byte hihi = (byte)(foo >> 24);
Also.. the implementation of BitConverter uses unsafe and pointers, but it's short and simple.
public static unsafe byte[] GetBytes(int value)
{
byte[] buffer = new byte[4];
fixed (byte* numRef = buffer)
{
*((int*) numRef) = value;
}
return buffer;
}
Try
BitConverter.GetBytes()
http://msdn.microsoft.com/en-us/library/system.bitconverter.aspx
Just keep in mind that the order of the bytes in returned array depends on the endianness of your system.
EDIT:
As for the Lua part, I don't know how to convert back. You could always multiply by 16 to get the same functionality of a bitwise shift by 4. It's not pretty and I would imagine there is some library or something that implements it. Again, the order to add the bytes in depends on the endianness, so you might want to read up on that
Maybe you can convert back in C#?
For Lua, check out Roberto's struct library. (Roberto is one of the authors of Lua.) It is more general than needed for the specific case in question, but it isn't unlikely that the need to interchange an int is shortly followed by the need to interchange other simple types or larger structures.
Assuming native byte order is acceptable at both ends (which is likely a bad assumption, incidentally) then you can convert a number to a 4-byte integer with:
buffer = struct.pack("l", value)
and back again with:
value = struct.unpack("l", buffer)
In both cases, buffer is a Lua string containing the bytes. If you need to access the individual byte values from Lua, string.byte is your friend.
To specify the byte order of the packed data, change the format from "l" to "<l" for little-endian or ">l" for big-endian.
The struct module is implemented in C, and must be compiled to a DLL or equivalent for your platform before it can be used by Lua. That said, it is included in the Lua for Windows batteries-included installation package that is a popular way to install Lua on Windows systems.
Here are some functions in Lua for converting a 32-bit two's complement number into bytes and converting four bytes into a 32-bit two's complement number. A lot more checking could/should be done to verify that the incoming parameters are valid.
-- convert a 32-bit two's complement integer into a four bytes (network order)
function int_to_bytes(n)
if n > 2147483647 then error(n.." is too large",2) end
if n < -2147483648 then error(n.." is too small",2) end
-- adjust for 2's complement
n = (n < 0) and (4294967296 + n) or n
return (math.modf(n/16777216))%256, (math.modf(n/65536))%256, (math.modf(n/256))%256, n%256
end
-- convert bytes (network order) to a 32-bit two's complement integer
function bytes_to_int(b1, b2, b3, b4)
if not b4 then error("need four bytes to convert to int",2) end
local n = b1*16777216 + b2*65536 + b3*256 + b4
n = (n > 2147483647) and (n - 4294967296) or n
return n
end
print(int_to_bytes(256)) --> 0 0 1 0
print(int_to_bytes(-10)) --> 255 255 255 246
print(bytes_to_int(255,255,255,246)) --> -10
investigate the BinaryWriter/BinaryReader classes
Convert an int to a byte array and display : BitConverter ...
www.java2s.com/Tutorial/CSharp/0280__Development/Convertaninttoabytearrayanddisplay.htm
Integer to Byte - Visual Basic .NET answers
http://bytes.com/topic/visual-basic-net/answers/349731-integer-byte
How to: Convert a byte Array to an int (C# Programming Guide)
http://msdn.microsoft.com/en-us/library/bb384066.aspx
As Nubsis says, BitConverter is appropriate but has no guaranteed endianness.
I have an EndianBitConverter class in MiscUtil which allows you to specify the endianness. Of course, if you only want to do this for a single data type (int) you could just write the code by hand.
BinaryWriter is another option, and this does guarantee little endianness. (Again, MiscUtil has an EndianBinaryWriter if you want other options.)
To convert to a byte[]:
BitConverter.GetBytes(int)
http://msdn.microsoft.com/en-us/library/system.bitconverter.aspx
To convert back to an int:
BitConverter.ToInt32(byteArray, offset)
http://msdn.microsoft.com/en-us/library/system.bitconverter.toint32.aspx
I'm not sure about Lua though.
If you are concerned about endianness use John Skeet's EndianBitConverter. I've used it and it works seamlessly.
C# supports their own implementation of htons and ntohs as:
system.net.ipaddress.hosttonetworkorder()
system.net.ipaddress.networktohostorder()
But they only work on signed int16, int32, int64 which means you'll probably end up doing a lot of unnecessary casting to make them work, and if you're using the highest order bit for anything other than signing the integer, you're screwed. Been there, done that. ::tsk:: ::tsk:: Microsoft for not providing better endianness conversion support in .NET.