When I'm trying to convert an object into byte array I'm getting a wierd array.
this is the code:
using (MemoryStream ms = new MemoryStream())
{
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(ms, obj);
Console.WriteLine(ByteArrayToString(ms.ToArray()));
}
//int obj = 50;
//string ByteArrayToString(byte[] byteArr) the functionality of this method is pretty obvious
the result is this:
"00 01 00 00 00 FF FF FF FF 01 00 00 00 00 00 00 00 04 01 00 00 00 0C 53 79 73 74 65 6D 2E 49 6E 74 33 32 01 00 00 00 07 6D 5F 76 61 6C 75 65 00 08 32 00 00 00 0B "
Can somebody explain to me WHY?:) the optimal result should be only "32 00 00 00".
Since serializer needs to provide enough information to deserialize the data back, it must include some metadata about the object being serialized. Specifically, the
53 79 73 74 65 6D 2E 49 6E 74 33 32
part stands for System.Int32
If you use BinaryWriter and its Write(Int32) method instead, you'll get the desired effect: your memory stream will contain just the four bytes from your integer. You wouldn't be able to deserialize it without knowing that you wrote an Int32 into the stream.
You're conflating BinaryFormatter serialization with an object's in memory format. What is written to the stream is merely an implementation detail of the BinaryFormatter and should not be relied upon for any interprocess communication not using BinaryFormatter.
If you're looking for the byte representation of the built-in types, use BitConverter.GetBytes (for strings use the appropriate Encoding.GetBytes).
The serialized byte array has both the data itself and the type info. That's why you get more info than you expect. That's neccessary for later deserializing.
The extra stuff in the results would be the BinaryFormatter object. You're not just outputting int obj = 50, you're outputting everything included in the BinaryFormatter as well.
Serialization process uses extra bytes to store information about types - it's the only way to ensure that serialized data will be deserialized into same objects of same types.
If you absolutely sure in what you doing and want to avoid any extra bytes, you may use your own serialization and make your formatter and serializers, which is very complicated. Or, you could use marshalling:
var size = Marshal.SizeOf(your_object);
// Both managed and unmanaged buffers required.
var bytes = new byte[size];
var ptr = Marshal.AllocHGlobal(size);
// Copy object byte-to-byte to unmanaged memory.
Marshal.StructureToPtr(font.LogFont, ptr, false);
// Copy data from unmanaged memory to managed buffer.
Marshal.Copy(ptr, bytes, 0, size);
// Release unmanaged memory.
Marshal.FreeHGlobal(ptr);
And to convert bytes to object:
var bytes = new byte[size];
var ptr = Marshal.AllocHGlobal(size);
Marshal.Copy(bytes, 0, ptr, size);
var your_object = (YourType)Marshal.PtrToStructure(ptr, typeof(YourType));
Marshal.FreeHGlobal(ptr);
This is quite slow and unsafe to use in most cases, but it's easiest way to strictly convert object to byte[] without implementing serialization and without [Serializable] attribute.
Related
I have the following piece of code:
using (Stream inputFileStream = File.OpenRead("C:\\Users\\User\\Downloads\\test.txt"))
{
using (Stream transcodingStream = Encoding.CreateTranscodingStream(inputFileStream, Encoding.GetEncoding(500), new UnicodeEncoding(bigEndian: true, byteOrderMark: true)))
{
using (Stream outputStream = File.OpenWrite("C:\\Users\\User\\Downloads\\test.txt"))
{
await transcodingStream.CopyToAsync(outputStream, cancellationToken);
}
}
}
My file before transcoding has the following first 16 bytes and is of Ebcdic type encoding (code page 500):
F5 F1 F1 F0 F2 C2 D4 E6 40 40 40 40 40 40 F1 F1 = 51102BMW 11
After performing the transcoding to Unicode with Big-Endiant and Byte Order Markings, I expect the file to begin with:
FF FE
However, I get:
00 35 00 31 00 31 00 30 00 32 00 42 00 4D 00 57 = �5�1�1�0�2�B�M�W
Where am I going wrong with this?
It seems like the transcoding stream does not care about maintaining the BOM for target encoding and it's something you have to manage yourself.
I've implemented the following solution:
targetEncoding is of type Encoding
outputStream.Seek(0, SeekOrigin.Begin);
await outputStream.WriteAsync(targetEncoding.Preamble.ToArray(), 0, targetEncoding.Preamble.Length, cancellationToken);
I use a NFC-Reader to read data. The data will be stores as byte[] bytes_credentials. Receiving data is working fine but now I want to encode the 16 byte Array to String. I use string str = Encoding.Default.GetString(new_data); for decoding the array. The string output is working fine if the byte Array is fully "booked up" (16 of 16 bytes used), for example: This is an test! -> 54 68 69 73 20 69 73 20 61 6e 20 74 65 73 74 21. The problem i am facing is, that if the NFC-Array is not fully "booked up", the Array look like this: This is! -> 54 68 69 73 20 69 73 21 00 00 00 00 00 00 00 00. The output in console looks like this: This is!????????.
How can I CHECK and REMOVE the empty bytes (00) from the Array. So that the Output looks like this: This is!.
Thank you for helping!!!
I have checked previous posts but no of them are working for me...
You can use LINQ to filter your array further:
using System.Text;
var bytes_credentials = new byte[]
{
0x54,
0x68,
0x69,
0x73,
0x20,
0x69,
0x73,
0x21,
0,
0,
0,
0,
0,
0,
0,
0
};
Console.WriteLine(Encoding.Default.GetString(
bytes_credentials.Where(b => b != 0).ToArray()
));
To learn more about LINQ, take a look here: https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/linq/
assume i got the following byte[]
0C 00 21 08 01 00 00 00 86 1B 06 00 54 51 53 65 72 76 65 72
with bitconverter BitConverter.ToString i can convert it to
0C-00-21-08-01-00-00-00-86-1B-06-00-54-51-53-65-72-76-65-72
how do i convert it back from string to byte[] to get
0C 00 21 08 01 00 00 00 86 1B 06 00 54 51 53 65 72 76 65 72
ascii encoding and other methods always getting me the equivalent bytes to the string but what i really need is the string to be byte[] as it is, i know if i did a reversing operation (using getbytes then tostring) ill end up with the same string but what i care about is while at getbytes to get the exact bytes
as i said
to put
0C-00-21-08-01-00-00-00-86-1B-06-00-54-51-53-65-72-76-65-72
AS string
and get
0C 00 21 08 01 00 00 00 86 1B 06 00 54 51 53 65 72 76 65 72
As byte[]
thanks in advance
You need this
byte[] bytes = str.Split('-').Select(s => Convert.ToByte(s, 16)).ToArray();
You can use SoapHexBinary class in System.Runtime.Remoting.Metadata.W3cXsd2001 namespace
string s = "0C-00-21-08-01-00-00-00-86-1B-06-00-54-51-53-65-72-76-65-72";
byte[] buf = SoapHexBinary.Parse(s.Replace("-"," ")).Value;
Remenber that BitConverter.ToString returns an equivalent hexadecimal string representation,so
if you decide to stick with it converting back as follow:
string temp = BitConverter.ToString(buf);//buf is your array.
byte[] newbuf = temp.Split('-').Select(s => Convert.ToByte(s,16)).ToArray();
But the safest way to convert bytes to string and back is base64:
string str = Convert.ToBase64String(buf);
byte[] result = Convert.FromBase64String(str);
Hello I'm trying to pass data from a pointer to a struct but the values seem to be different.
struct somestruct
{
public file header;
public uint version;
}
unsafe struct file
{
public fixed char name[8];
public uint type;
public uint size;
}
Then in code somewhere..
public unsafe int ReadFile(string filepath)
{
somestruct f = new somestruct();
byte[] fdata = System.IO.ReadAllBytes( filepath );
fixed( byte* src = fdata )
{
f.header = *(file*)src;
MessageBox.Show( new string(f.header.name) ); //should be 'FILENAME' but it's like japanese.
}
return 0;
}
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 46 49 4C 45 4E 41 4D 45 00 00 00 01 00 00 00 30 FILENAME.......0
00000010 74 27 9F EF 74 77 F1 D7 C5 86 93 3D 39 0D 72 A9 t'Ÿïtwñ×ņ“=9.r©
00000020 63 8B 92 CF F6 7D 8A 14 45 9D 68 51 A4 8E A4 EE c‹’Ïö}Š.E.hQ¤Ž¤î
00000030 4E FE D0 66 45 0E C9 8D 96 BB F4 EE 52 1F 89 D3 NþÐfE.É.–»ôîR.‰Ó
00000040 5C 80 1A 71 8A 16 B1 8B 3A A8 1B A4 48 11 B8 E8 \€.qŠ.±‹:¨.¤H.¸è
Do you have any idea what's going on?
Each char is 2 bytes - a fixed buffer of 8 chars is 16 bytes. You are reading the first 8 bytes as only the first 4 characters in that buffer, and the high bytes will make it look. Like the eastern Unicode ranges.
I would say: deserialize it at the stream level. Don't do this.
Basically, read (at least) 20 bytes into a buffer, then decode manually, using:
string s = Encoding.ASCII.GetString(buffer, 0, 8);
For the string, and probably shift operations for the unsigned integers.
You could also use unsafe code to read the integers from the buffer, via the other meaning of fixed and a pointer-cast.
A char is UTF-16 and is 2 bytes. You need to convert the UTF-8/ANSI (1 byte) string to a UTF-16 string.
I need to write a program that will generate 108 combinaisons of icons (standard windows .ico files) based on a tileset image.
I use the class System.Drawing.Bitmap to build each combinaison, and I save them like this:
Bitmap IconBitmap = new Bitmap(16, 16);
// Some processing, writing different parts of the source tileset
// ...
IconBitmap.Save(Path.Combine(TargetPath, "Icon" + Counter + ".ico"),
ImageFormat.Icon);
But I found out that the file saved is actually a PNG. Neither Windows Explorer nor Visual Studio can display it correctly, but GIMP can, and if I open it in an Hex viewer, here is what i see:
00000000 89 50 4E 47 0D 0A 1A 0A 00 00 00 0D 49 48 44 52 ‰PNG........IHDR
00000010 00 00 00 10 00 00 00 10 08 06 00 00 00 1F F3 FF ..............óÿ
00000020 61 00 00 00 01 73 52 47 42 00 AE CE 1C E9 00 00 a....sRGB.®Î.é..
00000030 00 04 67 41 4D 41 00 00 B1 8F 0B FC 61 05 00 00 ..gAMA..±..üa...
00000040 00 09 70 48 59 73 00 00 0E C3 00 00 0E C3 01 C7 ..pHYs...Ã...Ã.Ç
00000050 6F A8 64 00 00 00 15 49 44 41 54 38 4F 63 60 18 o¨d....IDAT8Oc`.
00000060 05 A3 21 30 1A 02 A3 21 00 09 01 00 04 10 00 01 .£!0..£!........
00000070 72 A5 13 76 00 00 00 00 49 45 4E 44 AE 42 60 82 r¥.v....IEND®B`‚
Also if I rename the .ico to .png Windows Explorer can display it properly.
I have this result even if I do NOTHING on the bitmap (I construct it with new and Save it directly, that gives me a black png).
What am I doing wrong?
I also tried this, which gave me awful 16 color icons, but I would prefer to avoid this solution anyway (using handles) :
Icon NewIcon = Icon.FromHandle(IconBitmap.GetHicon());
FileStream FS = new FileStream(Path.Combine(Target, "Icon" + Counter + ".ico"),
FileMode.Create);
NewIcon.Save(FS);
I made a quick-and-dirty workaround myself, I post it here for the record (it might help someone that need a quick solution, like me).
I won't accept this as the correct answer, it's not an actual icon writer.
It just write a 32bits ARGB bitmap into an ico file, using PNG format (works on Vista or later)
It is based on the ICO file format article from Wikipedia, and some fails and retry.
void SaveAsIcon(Bitmap SourceBitmap, string FilePath)
{
FileStream FS = new FileStream(FilePath, FileMode.Create);
// ICO header
FS.WriteByte(0); FS.WriteByte(0);
FS.WriteByte(1); FS.WriteByte(0);
FS.WriteByte(1); FS.WriteByte(0);
// Image size
FS.WriteByte((byte)SourceBitmap.Width);
FS.WriteByte((byte)SourceBitmap.Height);
// Palette
FS.WriteByte(0);
// Reserved
FS.WriteByte(0);
// Number of color planes
FS.WriteByte(0); FS.WriteByte(0);
// Bits per pixel
FS.WriteByte(32); FS.WriteByte(0);
// Data size, will be written after the data
FS.WriteByte(0);
FS.WriteByte(0);
FS.WriteByte(0);
FS.WriteByte(0);
// Offset to image data, fixed at 22
FS.WriteByte(22);
FS.WriteByte(0);
FS.WriteByte(0);
FS.WriteByte(0);
// Writing actual data
SourceBitmap.Save(FS, ImageFormat.Png);
// Getting data length (file length minus header)
long Len = FS.Length - 22;
// Write it in the correct place
FS.Seek(14, SeekOrigin.Begin);
FS.WriteByte((byte)Len);
FS.WriteByte((byte)(Len >> 8));
FS.Close();
}
Here's a simple ICO file writer I wrote today that outputs multiple System.Drawing.Image images to a file.
// https://en.wikipedia.org/wiki/ICO_(file_format)
public static class IconWriter
{
public static void Write(Stream stream, IReadOnlyList<Image> images)
{
if (images.Any(image => image.Width > 256 || image.Height > 256))
throw new ArgumentException("Image cannot have height or width greater than 256px.", "images");
//
// ICONDIR structure
//
WriteInt16(stream, 0); // reserved
WriteInt16(stream, 1); // image type (icon)
WriteInt16(stream, (short) images.Count); // number of images
var encodedImages = images.Select(image => new
{
image.Width,
image.Height,
Bytes = EncodeImagePng(image)
}).ToList();
//
// ICONDIRENTRY structure
//
const int iconDirSize = 6;
const int iconDirEntrySize = 16;
var offset = iconDirSize + (images.Count*iconDirEntrySize);
foreach (var image in encodedImages)
{
stream.WriteByte((byte) image.Width);
stream.WriteByte((byte) image.Height);
stream.WriteByte(0); // no pallete
stream.WriteByte(0); // reserved
WriteInt16(stream, 0); // no color planes
WriteInt16(stream, 32); // 32 bpp
// image data length
WriteInt32(stream, image.Bytes.Length);
// image data offset
WriteInt32(stream, offset);
offset += image.Bytes.Length;
}
//
// Image data
//
foreach (var image in encodedImages)
stream.Write(image.Bytes, 0, image.Bytes.Length);
}
private static byte[] EncodeImagePng(Image image)
{
var stream = new MemoryStream();
image.Save(stream, ImageFormat.Png);
return stream.ToArray();
}
private static void WriteInt16(Stream stream, short s)
{
stream.WriteByte((byte) s);
stream.WriteByte((byte) (s >> 8));
}
private static void WriteInt32(Stream stream, int i)
{
stream.WriteByte((byte) i);
stream.WriteByte((byte) (i >> 8));
stream.WriteByte((byte) (i >> 16));
stream.WriteByte((byte) (i >> 24));
}
}
It's true that the ImageFormat.Icon does not work for writing as you'd suppose, .NET simply does not support writing .ico files and simply dumps the PNG data.
There are a few projects on CodeProject (and this one) (and another one) and that let you write an .ico file, it's actually not that hard. The file format is pretty straight-forward, and supports BMP and PNG data.