There is one C++ function as below which returns me data in unsigned char**
MyCPPFunc(unsigned char** ppData, int &nSize)
I want to convert it to float array in C#. Problem is that internal representation of data returned by CPP could be char\ushort\uint\RGB etc. If I would use as below,
var srcArray = new byte[nSize];
Marshal.Copy(pCPPData, srcArray, 0, nSize/4);
outDataArray = Array.ConvertAll<byte, float>(srcArray, Convert.ToSingle);
It would convert every four consecutive bytes to float, while data in memory could be of different data type length (could be ushort, uchar, RGB etc.)
How to do this in a best performing manner, considering c++ library do support lots of data types and returns data in memory of that type.(although that is represented by uchar**)
I need something of this sort as below. where dataTypeLen can be 1 for char, 2 for short and so on.
Array.Convert(pCPPData, 0, pFloatArray, dataTypeLen, floatArrLen);
Unsafe code will also suffice.
I dont recall any such intelligent Convert'er in the Marshaller. Except for some bold exceptions like String or StringBuilder or Pointers/Handles itself, the Marshaller does not convert datatypes. It gets some bytes and interpretes them as the format(datatype) you requested and returns the format(datatype) you requested. If you ask it to read SHORT then it will decode as SHORT and return a SHORT. It is "not intelligent enough" to know how to convert CHAR into FLOAT. It's just not the job of Marshaller. And, also, not the job of any standard Converter. They can convert/cast simple types one to each other (like double to decimal), but they will not be able to understand more complex structures like "RGB", or, worse, some RGB* with padding to 32bits or BMP with padding at end of the row!
Unless you take some intelligent converter that understands the exact format you have at input, you have to do it manually. You first need to precisely know what type of data it is (uchar, ushort, RGB, etc) and then receive (i.e. marshal) the array in that precise format (uchar[], ushort[], ..) and only then you will be able to convert the elements to floats on the C# side. If you try reading the bytes "just like that", you might run into endianess problems. Of course, you might not care, depending on your needs.
So, for example: if you know that the pCPPData points to uchar array, then unmarshal it as uchar[]; if you know that the pCPPData points to ushort array, then unmarshal it as ushort[]; rgb? unmarshal it as bytes[] or as ints[], depending on the bitdepth of a single color. Then, take the resulting array, loop over it and convert to a new array of floats. (or, just LINQize with .Cast<float>().ToArray() and forget).
However, looking a bit out of the box, you seem to with with some kind of bitmaps. uchar-8bit grayscale, ushort-16bit grayscale, RGB-blah, I guess.. So why dont you try using some Bitmap processors? I dont precisely recall now, but there are functions and classes in .Net that handle/wrap raw byte arrays as Image/Bitmap objects.
For example, see ie. https://stackoverflow.com/a/16300450/717732 or https://stackoverflow.com/a/9560495/717732 - in the latter note the comments, they suggest that you could even set the scan0 to an unmanaged pointer, so you might completely escape the need of marshalling/copying anything. You might get a Bitmap object what just reads directly from the pointer you got from C++ library - and it would read in the format as specified by the ImageFormat specifier you provided. I have not ever tried this, so I say just "might".
But, of course, that would not give you float[] but Bitmap with pixels. If you really need floats, then you'll have to convert it "manually": branch over format, then read them out as the format specifies, then store them in format you want.
Related
Alright, so I basically want to read any file with a specific extension. Going through all the bytes and reading the file is basically easy, but what about getting the type of the next byte? For example:
while ((int)reader.BaseStream.Position != RecordSize * RecordsCount)
{
// How do I check what type is the next byte gonna be?
// Example:
// In every file, the first byte is always a uint:
uint id = reader.GetUInt32();
// However, now I need to check for the next byte's type:
// How do I check the next byte's type?
}
Bytes don't have a type. When data in some language type, such as a char or string or Long is converted to bytes and written to a file, there is no strict way to tell what the type was : all bytes look alike, a number from 0-255.
In order to know, and to convert back from bytes to structured language types, you need to know the format that the file was written in.
For example, you might know that the file was written as an ascii text file, and hence every byte represents one ascii character.
Or you might know that your file was written with the format {uint}{50 byte string}{linefeed}, where the first 2 bytes represent a uint, the next 50 a string, followed by a linefeed.
Because all bytes look the same, if you don't know the file format you can't read the file in a semantically correct way. For example, I might send you a file I created by writing out some ascii text, but I might tell you that the file is full of 2-byte uints. You would write a program to read those bytes as 2-byte uints and it would work : any 2 bytes can be interpreted as a uint. I could tell someone else that the same file was composed of 4-byte longs, and they could read it as 4-byte longs : any 4 bytes can be interpreted as a long. I could tell someone else the file was a 2 byte uint followed by 6 ascii characters. And so on.
Many types of files will have a defined format : for example, a Windows executable, or a Linux ELF binary.
You might be able to guess the types of the bytes in the file if you know something about the reason the file exists. But somehow you have to know, and then you interpret those bytes according to the file format description.
You might think "I'll write the bytes with a token describing them, so the reading program can know what each byte means". For example, a byte with a '1' might mean the next 2 bytes represent a uint, a byte with a '2' might mean the following byte tells the length of a string, and the bytes after that are the string, and so on. Sure, you can do that. But (a) the reading program still needs to understand that convention, so everything I said above is true (it's turtles all the way down), (b) that approach uses a lot of space to describe the file, and (c) The reading program needs to know how to interpret a dynamically described file, which is only useful in certain circumstances and probably means there is a meta-meta format describing what the embedded meta-format means.
Long story short, all bytes look the same, and a reading program has to be told what those bytes represent before it can use them meaningfully.
I know question is a bit weird, I'm asking out of pure curiosity, as I couldn't find any relevant info around. Also, please feel free to edit title, I know its terrible, but could not make up any better.
Let say I have variable foo of type object, which is either short or ushort. I need to send it over network, so I use BitConverter to transform it into byte[]:
byte[] b = new byte[2];
if(foo is short){
BitConverter.GetBytes((short)foo, 0);
}else{
BitConverter.GetBytes((ushort)foo, 0);
}
Network/Socket magic happens, I want my variable back. I know type I am expecting to get, so I call BitConverter.GetUInt16 or GetInt16 properly.
Now, question is - does it actually matter, how I serialized the variable? I mean, bits are the same, so it shouldn't have any meaning, am I correct? So that I could
BitConverter.GetBytes((short)foo, 0);
and then do
BitConverter.GetUInt16(myByteArray, 0);
Anyone?
To serialize your variable, you should assign the result of BitConverter.GetBytes() to your byte[].
It doesn't matter if your variable is short or ushort, as those are the same size and hold the same values between 0 and 32767. As long as the size is ok, you should have no problems.
So you may make your code as simple as this:
byte[] b;
if(foo is short || foo is ushort)
b = BitConverter.GetBytes((short)foo); // You get proper results for ushort as well
However at the decoding site you must know which type you need, for short, you need:
short foo = BitConverter.ToInt16(b, 0);
but if you need an ushort, then you write:
ushort foo = BitConverter.ToUInt16(b, 0);
When you send multibyte variables over the network, you should also ensure that they are in network byte order as #itsme86 mentioned in his answer.
If you need to send both shorts and ushorts, then you also need to send type information to the other end to know if the data type is signed or not.
I don't write about it now in detail as it would complicate the code.
If you're transmitting it over the network, you could run into endianness issues (i.e. multibyte values might be stored in different byte order on different architectures). The standard convention when sending a multibyte value over a network is to transform it to Network Byte Order.
The receiver of the multibyte value would then convert it to Host Byte Order.
I'm porting some C# decompression code to AS3, and since it's doing some pretty complex stuff, it's using a range of datatypes such as byte and short. The problem is, AS3 doesn't have those datatypes.
For the most part I can use uint to hold these values. However, at some points, I get a line such as:
length[symbol++] = (short)len;
To my understanding, this means that len must be read and assigned to the length array as a short. So I'm wondering, how would I do this in AS3? I'm guessing perhaps to do:
length[symbol++] = len & 0xFF;
But I'm unsure if this would give a proper result.
So basically, my question is this: how do I make sure to keep the the correct number of bytes when doing this sort of stuff in AS3? Maybe I should use ByteArrays instead?
Depending on reason why cast is in C# code you may or may not need to keep cast in AS3 code. If cast is purely to adjust type to type of elements of length array (i.e. there is no loss of precision) than you don't need cast. If len can actually be bigger than 0x7FFF you'll need to perform some cast.
I think ByteArray maybe a reasonable option if you need to handle result similar to C# StreamReader, random access may be harder than necessary.
Note that short is 2 bytes long (synonym for System.Int16) so to convert to it using bit manipulations you need to do & 0xFFFF. Be also very careful if casting between signed and unsigned types...
What is the safest way to guarantee that the following operation will be performed correctly:
When I read in 4 bytes as a uint32, I will write it out to a text file.
Later I will open this text file, read the number I wrote out previously, and then convert it back into the 4 bytes for use in other processing.
There is the BitConverter class to help you convert between primitive types and bytes.
Since you are storing this as a string, there isn't a whole lot to this. Obviously there is no issue converting the number into a string using .ToString(). So the only question I assume is how to go back in a reliable fashion. The solution is to use uint.Parse. i.e.:
var s = "12343632423432";
uint i = uint.Parse(s);
(PS: BitConverter is not helpful for conversion from strings)
I'm working on a networking application in C#, sending a lot of plain numbers across the network. I discovered the IPAddress.HostToNetworkOrder and IPAddress.NetworkToHostOrder methods, which are very useful, but they left me with a few questions:
I know I need to encode and decode integers, what about unsigned ones? I think yes, so at the moment I'm doing it by casting a pointer to the unsigned int into a pointer to an int, and then doing a network conversion for the int (since there is no method overload that takes unsigned ints)
public static UInt64 HostToNetworkOrder(UInt64 i)
{
Int64 a = *((Int64*)&i);
a = IPAddress.HostToNetworkOrder(a);
return *((UInt64*)&a);
}
public static UInt64 NetworkToHostOrder(UInt64 a)
{
Int64 i = *((Int64*)&a);
i = IPAddress.HostToNetworkOrder(i);
return *((UInt64*)&i);
}
2. What about floating point numbers (single and double). I think no, however If I do need to should I do a similar method to the unsigned ints and cast a single pointer into a int pointer and convert like so?
EDIT:: Jons answer doesn't answer the second half of the question (it doesn't really answer the first either!), I would appreciate someone answering part 2
I suspect you'd find it easier to use my EndianBinaryReader and EndianBinaryWriter in MiscUtil - then you can decide the endianness yourself. Alternatively, for individual values, you can use EndianBitConverter.
You'd better read several RFC documents to see how different TCP/IP protocols (application level, for example, HTTP/FTP/SNMP and so on).
This is generally speaking, a protocol specific question (both your questions), as your packet must encapsulate the integers or floating point number in a protocol defined format.
For SNMP, this is a conversion that changing an integer/float number to a few bytes and changing it back. ASN.1 is used.
http://en.wikipedia.org/wiki/Abstract_Syntax_Notation_One