We Keep Coding

Serial Ports - There is an error in XML document (5, 3870)

Here I have a class I've created for the purpose of sending text-files and images over a serial port:
public class SendItem
{
private byte[] bytes;
private string _fileName;
private string _extension;
private int bytesSin;
public string Extension
{
get { return _extension; }
set { _extension = value; }
}
public string FileName
{
get { return _fileName; }
set { _fileName = value; }
}
public byte[] Bytes
{
get { return bytes; }
set { bytes = value; }
}
public SendItem()
{
}
public void SendFile(SerialPort serialPort1)
{
if (serialPort1.IsOpen)
{
OpenFileDialog OFDialog = new OpenFileDialog();
OFDialog.Title = "Open File";
OFDialog.Filter = "Text Files (*.txt)" + "|*.txt|All files (*.*)|*.*";
OFDialog.InitialDirectory = #"C:\";
bool? userClickedOK = OFDialog.ShowDialog();
if (userClickedOK == true)
{
serialPort1.DiscardInBuffer();
string chosenFile = OFDialog.FileName;
string ext = Path.GetExtension(OFDialog.FileName);
_extension = ext;
byte[] data = File.ReadAllBytes(chosenFile);
SendItem newSendItem = new SendItem();
newSendItem._extension = ext;
newSendItem._fileName = System.IO.Path.GetFileNameWithoutExtension(chosenFile);
newSendItem.bytes = data;
byte[] view = newSendItem.Serialize();
string test = Convert.ToBase64String(data);
serialPort1.Write(newSendItem.Serialize(), 0, newSendItem.Serialize().Length);
//serialPort1.Write(data, 0, data.Length);
}
}
}
public override string ToString()
{
return string.Format("File: {0}{1}",_fileName, _extension);
}
}
There is quite a few redundant things in the SendFile method, but I ask that it would be ignored in favour of the following issue I keep on having.
Whenever I send a text file or just plain chat text from a textbox, the following block of code is executed without triggering the catch:
void _sPort_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
SerialPort sp = (SerialPort)sender;
byte[] data = new byte[sp.BytesToRead];
string message = string.Empty;
sp.Read(data, 0, data.Length);
try
{
SendItem receivedObject = data.Deserialize<SendItem>();
File.WriteAllBytes(#"d:\" + receivedObject.FileName + receivedObject.Extension, receivedObject.Bytes);
message = "File has been recieved.";
sp.Write("File sent.");
}
catch (Exception exp)
{
errors = exp.Message;
message = Encoding.UTF8.GetString(data);
}
App.Current.Dispatcher.Invoke(new Action(() => _response.Add("Friend: " + message)));
}
The problem comes in when I try to send an image... It trigger the catch and gives the exception There is an error in XML document (5, 3870).
My serializer and deserializer I've written is as follow:
public static byte[] Serialize<T>(this T source)
{
XmlSerializer serializer = new XmlSerializer(typeof(T));
MemoryStream stream = new MemoryStream();
serializer.Serialize(stream, source);
byte[] buffer = stream.GetBuffer();
stream.Close();
return buffer;
}
public static T Deserialize<T>(this byte[] source)
{
XmlSerializer serializer = new XmlSerializer(typeof(T));
MemoryStream stream = new MemoryStream(source);
T result = (T)serializer.Deserialize(stream);
stream.Close();
return result;
}
Can anyone point out where I'm making my mistake? I've been debugging it for ages and I can't wrap my head around it.
**EDIT:
I am also including the serialised data for an image I tried to send, after looking at it, it appears my Byte might be too big for the serial port - is there any way to adjust this or allow the serial port to take all of the data sent over?
/9j/4AAQSkZJRgABAgAAAQABAAD//gAEKgD/4gIcSUNDX1BST0ZJTEUAAQEAAAIMbGNtcwIQAABtbnRyUkdCIFhZWiAH3AABABkAAwApADlhY3NwQVBQTAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA9tYAAQAAAADTLWxjbXMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAApkZXNjAAAA/AAAAF5jcHJ0AAABXAAAAAt3dHB0AAABaAAAABRia3B0AAABfAAAABRyWFlaAAABkAAAABRnWFlaAAABpAAAABRiWFlaAAABuAAAABRyVFJDAAABzAAAAEBnVFJDAAABzAAAAEBiVFJDAAABzAAAAEBkZXNjAAAAAAAAAANjMgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB0ZXh0AAAAAEZCAABYWVogAAAAAAAA9tYAAQAAAADTLVhZWiAAAAAAAAADFgAAAzMAAAKkWFlaIAAAAAAAAG+iAAA49QAAA5BYWVogAAAAAAAAYpkAALeFAAAY2lhZWiAAAAAAAAAkoAAAD4QAALbPY3VydgAAAAAAAAAaAAAAywHJA2MFkghrC/YQPxVRGzQh8SmQMhg7kkYFUXdd7WtwegWJsZp8rGm/fdPD6TD////bAEMACAYGBwYFCAcHBwkJCAoMFA0MCwsMGRITDxQdGh8eHRocHCAkLicgIiwjHBwoNyksMDE0NDQfJzk9ODI8LjM0Mv/bAEMBCQkJDAsMGA0NGDIhHCEyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMv/CABEIAjoCOgMAIgABEQECEQH/xAAbAAEBAAMBAQEAAAAAAAAAAAAAAQIDBAUGB//EABgBAQEBAQEAAAAAAAAAAAAAAAABAgME/8QAGAEBAQEBAQAAAAAAAAAAAAAAAAECAwT/2gAMAwAAARECEQAAAfvRySZY1mKSgAKAWUBICgAAAiiUAAAAAAECgAAAAQAUECUKCCUECXIKxz17KAACgFgWLCgAIAAllCKqUCAAABCygAKCAAJZSBQQFACAoIggBMsayY5AAUCBKSGTFGTC1kkM5BUFuNoBYKgqCoKlsSoAItqVCFqVAgCBRUgUAICgiCBRKrDPDMCgSyiFJjkXUzwwWFqSXOa4u5oq7bokdE0Q6Zoh0zRa3TSl3NNN2WjbrO1L05kKAsqRQEABREAWWKAUkACwZWLQGNCillQCBQsYZzN1S4Ztwwwm9pJ0qCpQQqCsBmxpbjYy3aN2ue2x15gALKhC1KgkUCURQlECggALBkstICZYmQoEsAUgEsl16tmnFnP0cs6dTGulQVAjEZTXbsQVjYyy15Js26N9xuHXiAAsACyoEAAAAQKKRYgLBlYVQSWGQsFIBQQEsl06durnqc3TzN9AvVZCwFirp2a1zSqsJbjYz6ObfeXQs6cQAAAFgqVAiUAIoixVhAUCCLAoJKFxypYASpSAYZa83Xrywxpy9PK30ovWooFVgmNZNAALCZbtG2Y68sMtecNgAAAFgqVAgABKIogKDESiRbFATLDOgAQFCMdWerLDHLHOpwd3mt+hTXUFBBqXbJVAAFkcnX5aaPqPzb6xx+hHXmAAAAAsqBBFtSoEAQFQSVKAAAsFAFASXCMdWeGbiM3T5/f5zp6iXXQhalGGYwti0AJRDxvZ8evj/rfmvvLy9Uu+MAAAAAsFSpFAQABFEBIS0AACWGQAoIxwzwjXjnM3Wzi8vn+j5s36qXXaBQARKUCxZAR5Hr+Rb8z97+dfaOX0Vl6cIFAAAAFJYKgqVAgACKMFSkoAAlFY5AAgxsjGZM3GWry8ffxze+VrtFEUQAAKsJUsPI9fyq+M+n+X9jXL9CuOV4wUAAAAsAAoEgAAAGFiWygAACZQVKiCyWZRZLFGnk7edrC+X8w9H3j852an6C+bxxr6afN7D6B4/bHW59yZTDmOp4/CfTeT43RqeN6HD6u8febIvAKAAAAAFJZUCAAACDES2WFSgBKTLGiwCRFkosQpjMh8l4H0fDn0/Nehwe50xz6PT0Z6eZr9nk1Nfs+NlNfT7vM7OTHwXL1mjX27948/0MfRjwvrPkfv5n3BriVZAopAAALKkpAAAAEpbgmWUAsoShKSwUCURUSgBKHF5nv+Hy6/F+3w+trpweZ9pyx4XF9bq1n5+/RbJt2bts5/B8f0fJvp8/2eznceL7k7sX5L9J+C+4Z7S9eIJBVEQWikUkpAAABKSyhLWBM2gqQqwFIUAFIUlKlIgp819L5kvz/bo18fV9Ew3Xj5+v15XH0Z60zz15acpv5a8nm+hmnmb+njy8P6bxPct9Kx6PPRAAEVQQIVKCCygEUAEpgJZYKAAlAGWNRZQQotIAGrdjHy/nbuPn6Po+3wfWznuapvKcvTm7ctDU16tnPjXo3h3pPJ6PFuur2/l/oNZ+hS9OQiZQKgqCoKgWBYFgBbAsEEXEQAAqFAAsFCCkFoAhRJ8x4/1PhcvRwe38t6ur9BePt5TL436PwOl6NXlYbdnofP91n2nPz3heLyenye09z6j5j62Y9Ea5CqS2BKFBAAUAAEBQAGsQBUoBUFAlJUW2xIKQKFAeV819P8xz6+P0XRrfqavKwj3/P5s16NGVrg6ZLnq2ePnG7Vs3Wer9V8P9VMe+S81lWwAFgsCoAKgAVCoFlGNGCBUFCoKBZUJVlxyqyyS2AAS2zHwo2eBhw57dnldXntze26x6no+dv5dO3R5c1OzkVfJ5O+dOfN382s+gy+U9LOf0vo/PPeZ+lcnXcyiRYqwVAoERUFgWBYtEsEprS0lhklJQURcaoIsoBUhlNfkL7Pk/O+a37Pl+RorbpNT03H6OOurZr0ntbfIyzeycuxNuGnVb06efOzPy/S8vWIjWM+vip7/p/H7M39E9n8v2n6i+H+nmfQVElARKhYogLIZSoJSVKxhVgVKLBbCAUEqkmOlc/H5vkl9jg8vVddPNi1KhAq9vBsl9HCY56btGrXZumqp046Kt36tKXQXIWAluI2Z6Kd3b42Uv6F9L+Rfex78yxzEWICZYiglgsQyRFxyhjY1FlIFoSgASwyjUuHF1eFm/OeRnp60hFkMoVUAhlt57L0YMJWWpZta8TdqgCxYMmNBEqFyuNTPs4afrXd+cfpGWMymEBLAIVKEsCFxzxIl1CVQRYKsKgWBo26Jdfyn0/wU35WNx65rEqwlSgUlglRACAFRVQUIAQWC241Llhkuz9M/MPpJP0WZY4QkkosWAFxsKiLAljUmWNWpUJRZQlBDHRt0y8H579j8RrWvG46lsFbequF62B5j07J5c9XBfOnpLPMdkOSdkl5Has4naOK9mRwvTHmPR1HE3ajELYiZZYZLl08mcfs2Xk+rmJZmLiWy4lISkACmA1FgtgJS3FbbBYkmrTt5835D5j0/L6akssAZY09PzeziKQuWHZW3i2c6d/B18q4ki9nF6Fmvk2aoZY1ero4MU9rW85NunVLUsUItgyywyPvvrPhfusyQzIUhFWUkIygEyNdjapZFgoJlFLLCIujl6vOy/POXPX1ogsKybE6eTfsrlwysYbcZXZwdNTPi261xlRM8Ok6fP6uUgq3HI9LzvT8xJjYSEtsFSlyxp9T+hfm36RmSS5kuIyiKsCWCxFiBLsssgAluSCkkyxsOfwvc+czfhpZ1qCiGzZpqdDRsrPbpyM7q3

It may or may not be the whole problem, but this is definitely a problem:
byte[] buffer = stream.GetBuffer();
That's quite possibly returning more data than you want - because it's not limiting itself to the length of the stream. MemoryStream has the handy MemoryStream.ToArray method to simplify this:
public static byte[] Serialize<T>(this T source)
{
XmlSerializer serializer = new XmlSerializer(typeof(T));
MemoryStream stream = new MemoryStream();
serializer.Serialize(stream, source);
return stream.ToArray();
}
(There's no benefit from closing a MemoryStream that you've created like this - and if you did feel you needed to, you should use a using statement.)
Likewise your Deserialize method can be simplified to:
public static T Deserialize<T>(this byte[] source)
{
XmlSerializer serializer = new XmlSerializer(typeof(T));
return (T) serializer.Deserialize(new MemoryStream(source));
}
Given that you were effectively trying to deserialize "the XML document - and then probably a load of "blank" data, that may well be the issue.
EDIT: Now there's a second problem:
byte[] data = new byte[sp.BytesToRead];
string message = string.Empty;
sp.Read(data, 0, data.Length);
You're assuming that all of the data is ready to read in a single go. That won't be true if there's more data than the buffer size of the serial port. Instead, if you're trying to put multiple "messages" onto what is effectively a stream, you'll need to either indicate the end of a message (and keep reading until you find that) or write the length of the message before the message itself - then when reading, read that length and then make sure you read that many bytes.
This is a standard issue when you try to use a stream-oriented protocol as a message-oriented protocol.

You should use ToByteArray() instead of GetBuffer().

Load File Not working - The magic number in GZip header is not correct

I am attempting to create a Save/Load class that has the option for saving & load files compressed files. Below is what I have so far. Stepping through it seems to work just fine, except that I get a "The magic number in GZip header is not correct" exception. I don't understand how this can be as I am checking to make sure that the number is there before I pass it over, and I have verified via an external program that it is a GZip file.
Any assistance in finding out where I went wrong would be appreciated. Constructive criticism of my code is always welcome - Thanks!
public static class SaveLoad
{
public static void Save(string fileName, object savefrom, bool compress)
{
FileStream stream = new FileStream(fileName, FileMode.Create);
BinaryFormatter formatter = new BinaryFormatter();
if (compress)
{
GZipStream compressor = new GZipStream(stream, CompressionMode.Compress);
formatter.Serialize(compressor, savefrom);
compressor.Close();
}
else { formatter.Serialize(stream, savefrom); }
stream.Close();
}
public static object Load(string fileName)
{
object loadedObject = null;
try
{
FileStream stream = new FileStream(fileName, FileMode.Open);
BinaryFormatter formatter = new BinaryFormatter();
if (stream.Length > 4)
{
byte[] data = new byte[4];
stream.Read(data, 0, 4);
if (BitConverter.ToUInt16(data, 0) == 0x8b1f) //GZIP_LEAD_BYTES == 0x8b1f
{
GZipStream decompressor = new GZipStream(stream, CompressionMode.Decompress);
loadedObject = formatter.Deserialize(decompressor); //Exception
decompressor.Close();
}
else { loadedObject = formatter.Deserialize(stream); }
}
stream.Close();
}
catch (Exception e)
{
Logger.StaticLog.AddEvent(new Logger.lEvent(null, Logger.lEvent.EventTypes.Warning, "Failed to load file: " + fileName, e)
{
SendingObject = "SaveLoad"
});
Logger.StaticLog.WriteLog();
throw;
}
return loadedObject;
}
}

It seems that you read the magic number before passing the stream to decompressor (which won't read the magic number then, because you've already read it).
Use stream.Seek(0,SeekOrigin.Begin) before you decompress.

What is the best way of reading/writing structured binary data in C#

Like in C we could use structure pointers to read or write structured binary data like file headers etc, is there a similar way to do this in C#?

Using BinaryReader and BinaryWriter over a MemoryStream tends to be the best way in my opinion.
Parsing binary data:
byte[] buf = f // some data from somewhere
using (var ms = new MemoryStream(buf, false)) { // Read-only
var br = new BinaryReader(ms);
UInt32 len = br.ReadUInt32();
// ...
}
Generating binary data:
byte[] result;
using (var ms = new MemoryStream()) { // Expandable
var bw = new BinaryWriter(ms);
UInt32 len = 0x1337;
bw.Write(len);
// ...
result = ms.GetBuffer(); // Get the underlying byte array you've created.
}
They allow you to read and write all of the primitive types you'd need for most file headers, etc. such as (U)Int16, 32, 64, Single, Double, as well as byte, char and arrays of those.  There is direct support for strings, however only if 
The string is prefixed with the length, encoded as an integer seven bits at a time.
This only seems useful to me if you wrote the string in this way from BinaryWriter in this way. It's easy enough however, say your string is prefixed by a DWord length, followed by that many ASCII characters:
int len = (int)br.ReadUInt32();
string s = Encoding.ASCII.GetString(br.ReadBytes(len));
Note that I do not have the BinaryReader and BinaryWriter objects wrapped in a using() block. This is because, although they are IDisposable, all their Dispose() does is call Dispose() on the underlying stream (in these examples, the MemoryStream).
Since all the BinaryReader/BinaryWriter are is a set of Read()/Write() wrappers around the underlying streams, I don't see why they're IDisposable anyway. It's just confusing when you try to do The Right Thing and call Dispose() on all your IDisposables, and suddenly your stream is disposed.

To read arbitrarily-structured data (a struct) from a binary file, you first need this:
public static T ToStructure<T>(byte[] data)
{
unsafe
{
fixed (byte* p = &data[0])
{
return (T)Marshal.PtrToStructure(new IntPtr(p), typeof(T));
}
};
}
You can then:
public static T Read<T>(BinaryReader reader) where T: new()
{
T instance = new T();
return ToStructure<T>(reader.ReadBytes(Marshal.SizeOf(instance)));
}
To write, convert the struct object to a byte array:
public static byte[] ToByteArray(object obj)
{
int len = Marshal.SizeOf(obj);
byte[] arr = new byte[len];
IntPtr ptr = Marshal.AllocHGlobal(len);
Marshal.StructureToPtr(obj, ptr, true);
Marshal.Copy(ptr, arr, 0, len);
Marshal.FreeHGlobal(ptr);
return arr;
}
...and then just write the resulting byte array to a file using a BinaryWriter.

Here is an simple example showing how to read and write data in Binary format to and from a file.
using System;
using System.IO;
namespace myFileRead
{
class Program
{
static void Main(string[] args)
{
// Let's create new data file.
string myFileName = #"C:\Integers.dat";
//check if already exists
if (File.Exists(myFileName))
{
Console.WriteLine(myFileName + " already exists in the selected directory.");
return;
}
FileStream fs = new FileStream(myFileName, FileMode.CreateNew);
// Instantialte a Binary writer to write data
BinaryWriter bw = new BinaryWriter(fs);
// write some data with bw
for (int i = 0; i < 100; i++)
{
bw.Write((int)i);
}
bw.Close();
fs.Close();
// Instantiate a reader to read content from file
fs = new FileStream(myFileName, FileMode.Open, FileAccess.Read);
BinaryReader br = new BinaryReader(fs);
// Read data from the file
for (int i = 0; i < 100; i++)
{
//read data as Int32
Console.WriteLine(br.ReadInt32());
}
//close the file
br.Close();
fs.Close();
}
}
}

Equivalent of StringBuilder for byte arrays

This is a simple one, and one that I thought would have been answered. I did try to find an answer on here, but didn't come up with anything - so apologies if there is something I have missed.
Anyway, is there an equivalent of StringBuilder but for byte arrays?
I'm not bothered about all the different overloads of Append() - but I'd like to see Append(byte) and Append(byte[]).
Is there anything around or is it roll-your-own time?

Would MemoryStream work for you? The interface might not be quite as convenient, but it offers a simple way to append bytes, and when you are done you can get the contents as a byte[] by calling ToArray().
A more StringBuilder-like interface could probably be achieved by making an extension method.
Update
Extension methods could look like this:
public static class MemoryStreamExtensions
{
public static void Append(this MemoryStream stream, byte value)
{
stream.Append(new[] { value });
}
public static void Append(this MemoryStream stream, byte[] values)
{
stream.Write(values, 0, values.Length);
}
}
Usage:
MemoryStream stream = new MemoryStream();
stream.Append(67);
stream.Append(new byte[] { 68, 69 });
byte[] data = stream.ToArray(); // gets an array with bytes 67, 68 and 69

The MemoryStream approach is good, but if you want to have StringBuilder-like behavior add a BinaryWriter. BinaryWriter provides all the Write overrides you could want.
MemoryStream stream = new MemoryStream();
BinaryWriter writer = new BinaryWriter(stream);
writer.Write((byte)67);
writer.Write(new byte[] { 68, 69 });

Probably List<byte>:
var byteList = new List<byte>();
byteList.Add(42);
byteList.AddRange(new byte[] { 1, 2, 3 });

List<byte> Then when you want it as an array you can call ToArray()

using System;
using System.IO;
public static class MemoryStreams
{
public static MemoryStream Append(
this MemoryStream stream
, byte value
, out bool done)
{
try
{
stream.WriteByte(value);
done = true;
}
catch { done = false; }
return stream;
}
public static MemoryStream Append(
this MemoryStream stream
, byte[] value
, out bool done
, uint offset = 0
, Nullable<uint> count = null)
{
try
{
var rLenth = (uint)((value == null) ? 0 : value.Length);
var rOffset = unchecked((int)(offset & 0x7FFFFFFF));
var rCount = unchecked((int)((count ?? rLenth) & 0x7FFFFFFF));
stream.Write(value, rOffset, rCount);
done = true;
}
catch { done = false; }
return stream;
}
}

Look at this project at codeproject.
It's just one class that implements the solution with a MemoryStream as suggested in other answers.

C#: Save multiple images to a single file

I am working on a class that maintains a dictionary of images.
This dictionary should be saved to and loaded from a file.
I implemented the below solution, but the problem is that according to MSDN
documentation for Image.FromStream();
http://msdn.microsoft.com/en-us/library/93z9ee4x(v=VS.80).aspx
"The stream is reset to zero if this method is called successively with the same stream."
Any ideas how to fix this? The speed of loading the dictionary is critical.
class ImageDictionary
{
private Dictionary<string, Image> dict = new Dictionary<string, Image>();
public void AddImage(string resourceName, string filename)
{
//...
}
public Image GetImage(string resourceName)
{
//...
}
public void Save(string filename)
{
var stream = new FileStream(filename, FileMode.Create);
var writer = new BinaryWriter(stream);
writer.Write((Int32) dict.Count);
foreach (string key in dict.Keys)
{
writer.Write(key);
Image img;
dict.TryGetValue(key, out img);
img.Save(stream,System.Drawing.Imaging.ImageFormat.Png);
}
writer.Close();
stream.Close();
}
public void Load(string filename)
{
var stream = new FileStream(filename, FileMode.Open);
var reader = new BinaryReader(stream);
Int32 count = reader.ReadInt32();
dict.Clear();
for (int i = 0; i < count; i++)
{
string key = reader.ReadString();
Image img = Image.FromStream(stream);
dict.Add(key, img);
}
reader.Close();
stream.Close();
}
}

The Image.FromStream method expects a valid image stream. You are concatenating multiple images into a single file and if you want to reconstruct them you will also need to save their size in addition to their number. An easier solution would be to simply binary serialize the image dictionary:
public void Save(string filename)
{
var serializer = new BinaryFormatter();
using (var stream = File.Create(filename))
{
serializer.Serialize(stream, dict);
}
}
public void Load(string filename)
{
var serializer = new BinaryFormatter();
using (var stream = File.Open(filename, FileMode.Open))
{
dict = (Dictionary<string, Image>)serializer.Deserialize(stream);
}
}

You can try to use BinaryFormatter to serialize/deserialize your dictionary dict to/from file.

An off-the-wall idea that might work (I have definitely not tested this):
Create a Substream class that derives from Stream and also wraps an underlying Stream. Its constructor would take a Stream and an offset into that stream that the Substream treats as zero. Substream is basically a constrained view or window into another stream (in this case, your file stream).
Initially, create a Substream over your FileStream with an offset of zero.
When you call Image.FromStream, the position of your Substream will advance to some new position (call this p).
Create a new Substream over your FileStream with an offset of p.
Loop until finished.
The idea is that even if Image.FromStream resets the underlying stream, it will reset the Substream to some offset into the FileStream, which is what you really want.

why not create custom header for that file which includes (number of images,starting address of the actual images and make between each image a line separator)