I'm getting back data in the (expected) following format:
\u0001\u0001\u0004\0\u0001\0\0\0
Each segment represents a byte. The first two segments \u0001\u0001 represents the service version number, the second two \u0004\0 represent the status code, and the final 4 u0001\0\0\0 equals the request id.
How can I take the fields I KNOW go together and make a logical value out of the result? For example, the status code \u0004\0 should be a signed-short and the request id should be an int.
What I've played around with, but I don't know the validity:
byte s1 = 0004;
byte s2 = 0;
short statusCode = (short)(s1 | (s2 << 8));
byte r1 = 0001;
byte r2 = 0;
byte r3 = 0;
byte r4 = 0;
int requestId = (int)(r1 | (r2 << 8) | (r3 << 16) | (r4 << 24));
While your logic seems fine, manual bit shifting can become quite tedious, especially when the amount of data you have to handle increases. It’s simple enough for 8 bytes, but for everything else, I would suggest you to look into marshalling bytes directly into objects.
For this, define a value type that explains your data:
public struct Data
{
public short serviceVersion;
public short statusCode;
public int requestId;
}
Then, you can convert the string into a byte array, and marshall it as a Data object:
// raw input, as a string
string s = "\u0001\u0001\u0004\0\u0001\0\0\0";
// convert string into byte array
byte[] bytes = Encoding.UTF8.GetBytes(s);
// interpret byte array as `Data` object
GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
Data data = (Data)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(Data));
handle.Free();
// access the data!
Console.WriteLine(data.serviceVersion); // 257
Console.WriteLine(data.statusCode); // 4
Console.WriteLine(data.requestId); // 1
Exanding on #poke's post above; if you want to get really fancy you can do a union like effect
using System.Runtime.InteropServices;
[StructLayout(LayoutKind.Explicit, Size = 8)]
public struct Data
{
[FieldOffset(0)]
public short serviceVersion;
[FieldOffset(2)]
public short statusCode;
[FieldOffset(4)]
public int requestId;
[FieldOffset(0)]
public ulong Value;
}
Where the use of the Data.Value field reads all of the bits you care about.
Additionally, using unsafe you can avoid the marshalling.
// raw input, as a string
string s = "\u0001\u0001\u0004\0\u0001\0\0\0";
// convert string into byte array
byte[] bytes = Encoding.UTF8.GetBytes(s);
Data data = new Data();
unsafe
{
Data* d = &data;
fixed(byte* b = bytes)
{
*d = *((Data*)b);
}
}
Related
According to this question C# will assign 4 byte size to field of type Fruits whether it is defined like this:
enum Fruits : byte { Apple, Orange, Banana }
or like this:
enum Fruits { Apple, Orange, Banana }
I'm still curious if there is any way of sidesteping this and making the size of enum smaller than 4 bytes. I know that this probably wouldn't be very efficient or desirable but it's still interesting to know if it's possible at all.
Data alignment (typically on 1, 2, 4 byte border) is used for the faster access to the data (int should be aligned on 4 bytes border).
For instance
(let me use byte and int instead of enum for readability and struct instead of class - it's an easy way to get size of struct with a help of sizeof):
// sizeof() == 8 == 1 + 3 (padding) + 4
public struct MyDemo {
public byte A; // Padded with 3 unused bytes
public int B; // Aligned on 4 byte
}
// sizeof() == 8 == 1 + 1 + 2 (padding) + 4
public struct MyDemo {
public byte A; // Bytes should be aligned on 1 Byte Border
public byte B; // Padded with 2 unused bytes
public int C; // Aligned on 4 byte
}
// sizeof() == 2 == 1 + 1
public struct MyDemo {
public byte A; // Bytes should be aligned on 1 Byte Border
public byte B; // Bytes should be aligned on 1 Byte Border
}
So far so good you can have an effect even in case of fields within class (struct), e.g.
public struct MyClass {
// 4 Byte in total: 1 + 1 + 2 (we are lucky: no padding here)
private Fruits m_Fruits; // Aligned on 1 Byte border
private byte m_MyByte // Aligned on 1 Byte border
private short m_NyShort; // Aligned on 2 Byte border
}
In case of a collection (array) all the values are of the same type which should be aligned in the same way, that's why no padding is required:
// Length * 1Byte == Length byte in total
byte[] array = new [] {
byte1, // 1 Byte alignment
byte2, // 1 Byte alignment
byte3, // 1 Byte alignment
...
byteN, // 1 Byte alignment
}
For the vast majority of applications the size overhead will not matter at all. For some specialized applications, like image processing, it may make sense to use constant byte values and do bit-manipulations instead. This can also be a way to pack multiple values into a single byte, or combine flag-bits with values:
const byte Apple = 0x01;
const byte Orange= 0x02;
const byte Banana= 0x03;
const byte FruitMask = 0x0f; // bits 0-3 represent the fruit value
const byte Red = 0x10;
const byte Green = 0x20;
const byte ColorMask = 0x70; // bits 4-6 represents color
const byte IsValidFlag = 0x80; // bit 7 represent value flag
...
var fruitValue = myBytes[i] & FruitMask;
var IsRed = (myBytes[i] & ColorMask) == Red ;
var isValid = myBytes[i] & IsValidFlag > 0;
According to this question C# will assign 4 byte size to field of type Fruits whether it is defined like this
I would say that this is not what actually is written there. The post describes the memory alignment on stack which seems to align 4 bytes for byte variable too (can be platform depended):
byte b = 1;
results in the same IL_0000: ldc.i4.1 instruction as the var fb1 = FruitsByte.Apple and int i = 1; (see at sharplab.io) and the same 4 bytes difference (Core CLR 6.0.322.12309 on x86) in the move instructions.
Though using corresponding enum as struct fields will result in them being aligned to corresponding borders:
Console.WriteLine(Unsafe.SizeOf<C>()); // prints 2
Console.WriteLine(Unsafe.SizeOf<C1>()); // prints 8
public enum Fruits : byte { Apple, Orange, Banana }
public enum Fruits1 { Apple, Orange, Banana }
public struct C {
public Fruits f1;
public Fruits f2;
}
public struct C1 {
public Fruits1 f1;
public Fruits1 f2;
}
The same will happen for arrays, which will allocate continuous region of memory without aligning different elements.
Useful reading:
StructLayoutAttribute
Blittable and Non-Blittable Types
Article about blittable types with a lot of links
I need to combine two Bytes into one int value.
I receive from my camera a 16bit Image were two successive bytes have the intensity value of one pixel. My goal is to combine these two bytes into one "int" vale.
I manage to do this using the following code:
for (int i = 0; i < VectorLength * 2; i = i + 2)
{
NewImageVector[ImagePointer] = ((int)(buffer.Array[i + 1]) << 8) | ((int)(buffer.Array[i]));
ImagePointer++;
}
My image is 1280*960 so VectorLength==1228800 and the incomming buffer size is 2*1228800=2457600 elements...
Is there any way that I can speed this up?
Maybe there is another way so I don't need to use a for-loop.
Thank you
You could use the equivalent to the union of c. Im not sure if faster, but more elegant:
[StructLayout(LayoutKind.Explicit)]
struct byte_array
{
[FieldOffset(0)]
public byte byte1;
[FieldOffset(1)]
public byte byte2;
[FieldOffset(0)]
public short int0;
}
use it like this:
byte_array ba = new byte_array();
//insert the two bytes
ba.byte1 = (byte)(buffer.Array[i]);
ba.byte2 = (byte)(buffer.Array[i + 1]);
//get the integer
NewImageVector[ImagePointer] = ba.int1;
You can fill your two bytes and use the int. To find the faster way take the StopWatch-Class and compare the two ways like this:
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
//The code
stopWatch.Stop();
MessageBox.Show(stopWatch.ElapsedTicks.ToString()); //Or milliseconds ,...
Assuming you can (re-)define NewImageVector as a short[], and every two consecutive bytes in Buffer should be transformed into a short (which basically what you're doing now, only you cast to an int afterwards), you can use Buffer.BlockCopy to do it for you.
As the documentation tells, you Buffer.BlockCopy copies bytes from one array to another, so in order to copy your bytes in buffer you need to do the following:
Buffer.BlockCopy(Buffer, 0, NewImageVector, 0, [NumberOfExpectedShorts] * 2)
This tells BlockCopy that you want to start copying bytes from Buffer, starting at index 0, to NewImageVector starting at index 0, and you want to copy [NumberOfExpectedShorts] * 2 bytes (since every short is two bytes long).
No loops, but it does depend on the ability of using a short[] array instead of an int[] array (and indeed, on using an array to begin with).
Note that this also requires the bytes in Buffer to be in little-endian order (i.e. Buffer[index] contains the low byte, buffer[index + 1] the high byte).
You can achieve a small performance increase by using unsafe pointers to iterate the arrays. The following code assumes that source is the input byte array (buffer.Array in your case). It also assumes that source has an even number of elements. In production code you would obviously have to check these things.
int[] output = new int[source.Length / 2];
fixed (byte* pSource = source)
fixed (int* pDestination = output)
{
byte* sourceIterator = pSource;
int* destIterator = pDestination;
for (int i = 0; i < output.Length; i++)
{
(*destIterator) = ((*sourceIterator) | (*(sourceIterator + 1) << 8));
destIterator++;
sourceIterator += 2;
}
}
return output;
This is code snippet from c++
struct
{
short m_nVersion;
short m_nMOT_Addr;
q_address m_oDistribution_Q;
short m_nOffset;
char m_cOperator;
short m_cLength;
long m_lOperand;
char m_cReq_Ack;
char m_cReq_SeqGap;
char m_cReq_AutoDereg;
} oSbsReg;
const short nMsgSize = 20; //20 byte long structure when packed
char sMessageBuffer[nMsgSize]; // I created stringbuilder in c#
memset(&oSbsReg,0,sizeof(oSbsReg));
memset(sMessageBuffer,0,sizeof(sMessageBuffer));
oSbsReg.m_nVersion = 20; //Version
oSbsReg.m_nMOT_Addr = (short) lMot; //MOT in which to register with
oSbsReg.m_oDistribution_Q.au.queue = m_oQueue.au.queue; //Current queue number
oSbsReg.m_oDistribution_Q.au.group = m_oQueue.au.group; //Current group
//changed to 0 to resolve Oswego issue when moved to DMQ 5.0
oSbsReg.m_nOffset = 0; //Start byte for telegram match
oSbsReg.m_cOperator = PSEL_OPER_EQ; //Exact match
oSbsReg.m_cLength = 4; //Number of bytes in match
oSbsReg.m_lOperand = lTelegramId; //Telegram Id to match against
oSbsReg.m_cReq_Ack = 1; //TRUE: Request an acknowledge of registration
oSbsReg.m_cReq_SeqGap = 1; //TRUE: Request an acknowledge of a sequence gap
oSbsReg.m_cReq_AutoDereg = 1; //TRUE: Request an auto deregistration on a connection loss
//default compile option is 8 byte packing, didn't want to change it
//manually pack bytes in structure to satisfy the SBS server's expectations
memcpy(sMessageBuffer, &oSbsReg, 11);
memcpy(sMessageBuffer+11, &(oSbsReg.m_cLength), 2);
memcpy(sMessageBuffer+13, &(oSbsReg.m_lOperand), 7); //twenty bytes max!
I need to convert above code in c#,
i created above structure in c# and initialized too but now i want to put it in stringbuilder variable, same way above structure in put into char array. But as you can see they use memset and mem cpy functions.
How can i achieve the same thing in c# ?
How can i put above structure in stringbuilder variable c#, how i extract bytes from structure and put it in string builder variable ?
I would not recommend using chars for storing bytes in C# (char in C# is actually two bytes because it's unicode).
There's a datatype for this kind of data in .NET, it's called byte, which is more similar to C++'s char, because it also contains one byte.
If you replace T with the name of your struct type, you can convert an object to a byte array like this (btw you can make a quite useful generic method out of this):
int size = Marshal.SizeOf(typeof(T));
byte[] rawData = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(obj, ptr, true);
Marshal.Copy(ptr, rawData, 0, size);
Marshal.FreeHGlobal(ptr);
From a library I'm working with I recieve an array of ushort.
I want to convert them in an array of float: The first ushort represents the 16 MSB of the first float and the second ushort is the 16 LSB of the first float, and so on.
I tried with something like the following, but the value is cast as the value of the integer, not the raw bits:
ushort[] buffer = { 0xBF80, 0x0000 };
float f = (uint)buffer[0] << 16 | buffer[1];
// expected result => f == -1 (0xBF800000)
// effective result => f == 3.21283686E+9 (0x4F3F8000)
Any suggestion?
Have a look at the System.BitConverter class.
In particular, the ToSingle method which takes a sequence of bytes and converts them to a float.
ushort[] buffer = {0xBF80, 0x0000};
byte[] bytes = new byte[4];
bytes[0] = (byte)(buffer[1] & 0xFF);
bytes[1] = (byte)(buffer[1] >> 8);
bytes[2] = (byte)(buffer[0] & 0xFF);
bytes[3] = (byte)(buffer[0] >> 8);
float value = BitConverter.ToSingle( bytes, 0 );
EDIT
In the example, I had reversed the MSB/LSB order.. Now it is correct
You should use the BitConverter class for that.
Convert the two ushorts to byte arrays with BitConverter.GetBytes(UInt16), concatenate the two arrays and use BitConverter.ToSingle(byte[] value,int startIndex) to convert the 4 bytes in the resulting array to a float.
I'd look at the System.BitConverter class. You can use BitConverter.GetBytes to turn your ushorts into byte arrays, then combine your byte arrays and use BitConverter to turn the byte array into a float.
Use a C# union:
[System.Runtime.InteropServices.StructLayout(LayoutKind.Explicit)]
public struct FloatUShortUnion {
[System.Runtime.InteropServices.FieldOffset(0)]
float floatValue;
[System.Runtime.InteropServices.FieldOffset(0)]
ushort short1;
[System.Runtime.InteropServices.FieldOffset(16)]
ushort short2;
}
You will need to use System.BitConverter, and convert the shorts to bytes.
http://msdn.microsoft.com/en-us/library/system.bitconverter.tosingle.aspx
I know this threads super old but I had a similar problem nothing above fixed. I had to send a decimal value to a plc using modbus and my only option to write to the registers was a ushort array. Luckily the software on the plc could convert the ushort array back to the decimal, the issue was reading the data from the plc. Here is what I was able to get to work. ConvertTo takes the users input and creates the ushort array for the write and ConvertFrom takes the ushort array the program receives from the plc and converts it back to the float.
private ushort[] ConvertTo(string value)
{
var bytes = BitConverter.GetBytes(float.Parse(value));
return new ushort[] {
BitConverter.ToUInt16(bytes, 0),
BitConverter.ToUInt16(bytes, 2)
};
}
private float ConvertFrom(ushort valueOne, ushort valueTwo)
{
byte[][] final = Array.ConvertAll(new ushort[] { valueOne, valueTwo }, delegate (ushort item) { return BitConverter.GetBytes(item); });
return BitConverter.ToSingle(new byte[4] { final[0][0], final[0][1], final[1][0], final[1][1] }, 0);
}
I am reading a binary log file produced by a piece of equipment.
I have the data in a byte[].
If I need to read two bytes to create a short I can do something like this:
short value = (short)(byte[1] << 8);
value += byte[2];
Now I know the value is the correct for valid data.
How would I know if the file was messed up and lets say the values FF FF were in those two places in the byte array?
When I look at the resultant value of converting FF FF to a short, I get a -1.
Is this a normal value for FF FF?, or did the computer just hit some kind of short bound and roll over with invalid data?
For my purposes all of theses numbers are going to be positive. If FF FF is actually a short -1, then I just need to validate that all my results are postitive.
Thank you,
Keith
BTW, I am also reading other number data types. I'll show them here just because.
The Read function is the basic part of reading from the byte[]. All the other data type reads use the basic Read() function.
public byte Read()
{
//advance position and then return byte at position
byte returnValue;
if (_CurrentPosition < _count - 1)
{
returnValue= _array[_offset + ++_CurrentPosition];
return returnValue;
}
else
throw new System.IO.EndOfStreamException
("Cannot Read Array, at end of stream.");
}
public float ReadFloat()
{
byte[] floatTemp = new byte[4];
for (int i = 3; i >= 0; i--)
{
floatTemp[i] = Read();
}
float returnValue = System.BitConverter.ToSingle
(floatTemp, 0);
if (float.IsNaN(returnValue))
{
throw new Execption("Not a Number");
}
return returnValue;
}
public short ReadInt16()
{
short returnValue = (short)(Read() << 8);
returnValue += Read();
return returnValue;
}
public int ReadInt32()
{
int returnValue = Read() << 24;
returnValue += Read() << 16;
returnValue += Read() << 8;
returnValue += Read();
return returnValue;
}
0xffff (all bits equal to 1) is -1 for signed shorts, yes. Read up on Two's complement to learn more about the details. You can switch to a larger datatype, or (as suggested by Grzenio) just use an unsigned type.
Well, you seemed to have found BitConverter for singles. Now let's see if we can get to to use it for everything else as well...
MemoryStream mem = new MemoryStream(_array);
float ReadFloat(Stream str)
{
byte[] bytes = str.Read(out bytes, 0, 4);
return BitConverter.ToSingle(bytes, 0)
}
public int ReadInt32(Stream str)
{
byte[] bytes = str.Read(out bytes, 0, 4);
return BitConverter.ToInt32(bytes, 0)
}
Did you try to use ushort (unsigned short)?
I think you would be better served using the System.BitConverter class.
Specifically for shorts the ToInt16 method.
You didn't mention it in your question, but you should also make sure that you know what endianess the hardware device is writing its data in. From your examples it looks like the float is in little endian, but the integers are in big endian? I doubt a hardware device would mix endianess in its output of binary data.
Yes 0xFFFF is -1 for a signed short.
Also, why wouldn't you use the BinaryReader class?
The value of FFFF for a "short" is -1 but the value of FFFF for an "unsigned short" is 65536.
In this case you should make sure you are using an unsigned short if you are sure that all of your values will be positive.