I'm trying to serialize an object to memory, pass it to another process as a string, and deserialize it.
I've discovered that the XML Serialization process strips the \r off of the newlines for strings in the object.
byte[] b;
// serialize to memory.
using (MemoryStream ms = new MemoryStream())
{
XmlSerializer xml = new XmlSerializer(this.GetType());
xml.Serialize(ms, this);
b = ms.GetBuffer();
}
// I can now send the bytes to my process.
Process(b);
// On the other end, I use:
using (MemoryStream ms = new MemoryStream(b))
{
XmlSerializer xml = new XmlSerializer(this.GetType());
clone = (myObject)xml.Deserialize(ms);
}
How do I serialize an object without serializing it to disk just like this, but without mangling the newlines in the strings?
The strings should be wrapped in CDATA sections to preserve the newlines.
The answer came from anther SO post, but I'm reposting it here because I had to tweak it a little.
I had to create a new class to manage XML read/write to memory stream. Here it is:
public class SafeXmlSerializer : XmlSerializer
{
public SafeXmlSerializer(Type type) : base(type) { }
public new void Serialize(StreamWriter stream, object o)
{
XmlWriterSettings ws = new XmlWriterSettings();
ws.NewLineHandling = NewLineHandling.Entitize;
using (XmlWriter xmlWriter = XmlWriter.Create(stream, ws))
{
base.Serialize(xmlWriter, o);
}
}
}
Since it is built on top of XmlSerializer, it should behave exactly as expected. It's just that when I serialize with a StreamWriter, I will use the "safe" version of the serialization, thus saving myself the headache.
I hope this helps someone else.
Related
I'm using serialization to a string as follows.
public static string Stringify(this Process self)
{
XmlSerializer serializer = new XmlSerializer(typeof(Process));
using (StringWriter writer = new StringWriter())
{
serializer.Serialize(writer, self,);
return writer.ToString();
}
}
Then, I deserialize using this code. Please note that it's not an actual stringification from above that's used. In our business logic, it makes more sense to serialize a path, hence reading in from said path and creating an object based on the read data.
public static Process Processify(this string self)
{
XmlSerializer serializer = new XmlSerializer(typeof(Process));
using (XmlReader reader = XmlReader.Create(self))
return serializer.Deserialize(reader) as Process;
}
}
This works as supposed to except for a small issue with encoding. The string XML that's produced, contains the addition encoding="utf-16" as an attribute on the base tag (the one that's about XML version, not the actual data).
When I read in, I get an exception because of mismatching encodings. As far I could see, there's no way to specify the encoding for serialization nor deserialization in any of the objects I'm using.
How can I do that?
For now, I'm using a very brute work-around by simply cutting of the excessive junk like so. It's Q&D and I want to remove it.
public static string Stringify(this Process self)
{
XmlSerializer serializer = new XmlSerializer(typeof(Process));
using (StringWriter writer = new StringWriter())
{
serializer.Serialize(writer, self,);
return writer.ToString().Replace(" encoding=\"utf-16\"", "");
}
}
I have been working with XML in database LINQ and find that it is very difficult to work with the serializer.
The database LINQ required a field that store XElement.
I have a complex object with many customized structure class, so I would like to use the XmlSerializer to serialize the object.
However, the serializer can only serialize to file ("C:\xxx\xxx.xml") or a memory stream.
However to convert or serialize it to be a XElement so that I can store in the database using LINQ?
And How to do the reverse? i.e. Deserialize an XElement...
Try to use this
using (var stream = new MemoryStream())
{
serializer.Serialize(stream, value);
stream.Position = 0;
using (XmlReader reader = XmlReader.Create(stream))
{
XElement element = XElement.Load(reader);
}
}
deserialize :
XmlSerializer xs = new XmlSerializer(typeof(XElement));
using (MemoryStream ms = new MemoryStream())
{
xs.Serialize(ms, xml);
ms.Position = 0;
xs = new XmlSerializer(typeof(YourType));
object obj = xs.Deserialize(ms);
}
To make what John Saunders was describing more explicit, deserialization is very straightforward:
public static object DeserializeFromXElement(XElement element, Type t)
{
using (XmlReader reader = element.CreateReader())
{
XmlSerializer serializer = new XmlSerializer(t);
return serializer.Deserialize(reader);
}
}
Serialization is a little messier because calling CreateWriter() from an XElement or XDocument creates child elements. (In addition, the XmlWriter created from an XElement has ConformanceLevel.Fragment, which causes XmlSerialize to fail unless you use the workaround here.) As a result, I use an XDocument, since this requires a single element, and gets us around the XmlWriter issue:
public static XElement SerializeToXElement(object o)
{
var doc = new XDocument();
using (XmlWriter writer = doc.CreateWriter())
{
XmlSerializer serializer = new XmlSerializer(o.GetType());
serializer.Serialize(writer, o);
}
return doc.Root;
}
First of all, see Serialize Method to see that the serializer can handle alot more than just memory streams or files.
Second, try using XElement.CreateWriter and then passing the resulting XmlWriter to the serializer.
The SQL has XML data type may be this can help you look at msdn
Proper object disposal removed for brevity but I'm shocked if this is the simplest way to encode an object as UTF-8 in memory. There has to be an easier way doesn't there?
var serializer = new XmlSerializer(typeof(SomeSerializableObject));
var memoryStream = new MemoryStream();
var streamWriter = new StreamWriter(memoryStream, System.Text.Encoding.UTF8);
serializer.Serialize(streamWriter, entry);
memoryStream.Seek(0, SeekOrigin.Begin);
var streamReader = new StreamReader(memoryStream, System.Text.Encoding.UTF8);
var utf8EncodedXml = streamReader.ReadToEnd();
No, you can use a StringWriter to get rid of the intermediate MemoryStream. However, to force it into XML you need to use a StringWriter which overrides the Encoding property:
public class Utf8StringWriter : StringWriter
{
public override Encoding Encoding => Encoding.UTF8;
}
Or if you're not using C# 6 yet:
public class Utf8StringWriter : StringWriter
{
public override Encoding Encoding { get { return Encoding.UTF8; } }
}
Then:
var serializer = new XmlSerializer(typeof(SomeSerializableObject));
string utf8;
using (StringWriter writer = new Utf8StringWriter())
{
serializer.Serialize(writer, entry);
utf8 = writer.ToString();
}
Obviously you can make Utf8StringWriter into a more general class which accepts any encoding in its constructor - but in my experience UTF-8 is by far the most commonly required "custom" encoding for a StringWriter :)
Now as Jon Hanna says, this will still be UTF-16 internally, but presumably you're going to pass it to something else at some point, to convert it into binary data... at that point you can use the above string, convert it into UTF-8 bytes, and all will be well - because the XML declaration will specify "utf-8" as the encoding.
EDIT: A short but complete example to show this working:
using System;
using System.Text;
using System.IO;
using System.Xml.Serialization;
public class Test
{
public int X { get; set; }
static void Main()
{
Test t = new Test();
var serializer = new XmlSerializer(typeof(Test));
string utf8;
using (StringWriter writer = new Utf8StringWriter())
{
serializer.Serialize(writer, t);
utf8 = writer.ToString();
}
Console.WriteLine(utf8);
}
public class Utf8StringWriter : StringWriter
{
public override Encoding Encoding => Encoding.UTF8;
}
}
Result:
<?xml version="1.0" encoding="utf-8"?>
<Test xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<X>0</X>
</Test>
Note the declared encoding of "utf-8" which is what we wanted, I believe.
Your code doesn't get the UTF-8 into memory as you read it back into a string again, so its no longer in UTF-8, but back in UTF-16 (though ideally its best to consider strings at a higher level than any encoding, except when forced to do so).
To get the actual UTF-8 octets you could use:
var serializer = new XmlSerializer(typeof(SomeSerializableObject));
var memoryStream = new MemoryStream();
var streamWriter = new StreamWriter(memoryStream, System.Text.Encoding.UTF8);
serializer.Serialize(streamWriter, entry);
byte[] utf8EncodedXml = memoryStream.ToArray();
I've left out the same disposal you've left. I slightly favour the following (with normal disposal left in):
var serializer = new XmlSerializer(typeof(SomeSerializableObject));
using(var memStm = new MemoryStream())
using(var xw = XmlWriter.Create(memStm))
{
serializer.Serialize(xw, entry);
var utf8 = memStm.ToArray();
}
Which is much the same amount of complexity, but does show that at every stage there is a reasonable choice to do something else, the most pressing of which is to serialise to somewhere other than to memory, such as to a file, TCP/IP stream, database, etc. All in all, it's not really that verbose.
Very good answer using inheritance, just remember to override the initializer
public class Utf8StringWriter : StringWriter
{
public Utf8StringWriter(StringBuilder sb) : base (sb)
{
}
public override Encoding Encoding { get { return Encoding.UTF8; } }
}
I found this blog post which explains the problem very well, and defines a few different solutions:
(dead link removed)
I've settled for the idea that the best way to do it is to completely omit the XML declaration when in memory. It actually is UTF-16 at that point anyway, but the XML declaration doesn't seem meaningful until it has been written to a file with a particular encoding; and even then the declaration is not required. It doesn't seem to break deserialization, at least.
As #Jon Hanna mentions, this can be done with an XmlWriter created like this:
XmlWriter writer = XmlWriter.Create (output, new XmlWriterSettings() { OmitXmlDeclaration = true });
I want to Serialize a 60mb file into XML but it gives me System out of memory exception.
Did any one else had this kind of problem?
Can someone suggest me a workaround for this problem.
Here is the method
static public string Serialize(object obj)
{
string returnValue;
System.Xml.Serialization.XmlSerializer xmlWriter = new System.Xml.Serialization.XmlSerializer(obj.GetType());
System.IO.StringWriter xmlOut = new System.IO.StringWriter();
//this is where the problem is.....
xmlWriter.Serialize(xmlOut, obj);
//return the Serialized XML
returnValue = xmlOut.ToString();
xmlOut.Close();
return returnValue;
}
With a stream that large, I think you should consider serializing to a file first. Serializing takes a lot of memory, and doing so purely in memory is killing you.
using(var file = new FileStream(...))
using(var streamWriter = new StreamWriter(file))
{
xmlWriter.Serialize(streamWriter, obj);
}
I'm trying to serialize a very large IEnumerable<MyObject> using an XmlSerializer without keeping all the objects in memory.
The IEnumerable<MyObject> is actually lazy..
I'm looking for a streaming solution that will:
Take an object from the IEnumerable<MyObject>
Serialize it to the underlying stream using the standard serialization (I don't want to handcraft the XML here!)
Discard the in memory data and move to the next
I'm trying with this code:
using (var writer = new StreamWriter(filePath))
{
var xmlSerializer = new XmlSerializer(typeof(MyObject));
foreach (var myObject in myObjectsIEnumerable)
{
xmlSerializer.Serialize(writer, myObject);
}
}
but I'm getting multiple XML headers and I cannot specify a root tag <MyObjects> so my XML is invalid.
Any idea?
Thanks
The XmlWriter class is a fast streaming API for XML generation. It is rather low-level, MSDN has an article on instantiating a validating XmlWriter using XmlWriter.Create().
Edit: link fixed. Here is sample code from the article:
async Task TestWriter(Stream stream)
{
XmlWriterSettings settings = new XmlWriterSettings();
settings.Async = true;
using (XmlWriter writer = XmlWriter.Create(stream, settings)) {
await writer.WriteStartElementAsync("pf", "root", "http://ns");
await writer.WriteStartElementAsync(null, "sub", null);
await writer.WriteAttributeStringAsync(null, "att", null, "val");
await writer.WriteStringAsync("text");
await writer.WriteEndElementAsync();
await writer.WriteCommentAsync("cValue");
await writer.WriteCDataAsync("cdata value");
await writer.WriteEndElementAsync();
await writer.FlushAsync();
}
}
Here's what I use:
using System;
using System.Collections.Generic;
using System.Xml;
using System.Xml.Serialization;
using System.Text;
using System.IO;
namespace Utils
{
public class XMLSerializer
{
public static Byte[] StringToUTF8ByteArray(String xmlString)
{
return new UTF8Encoding().GetBytes(xmlString);
}
public static String SerializeToXML<T>(T objectToSerialize)
{
StringBuilder sb = new StringBuilder();
XmlWriterSettings settings =
new XmlWriterSettings {Encoding = Encoding.UTF8, Indent = true};
using (XmlWriter xmlWriter = XmlWriter.Create(sb, settings))
{
if (xmlWriter != null)
{
new XmlSerializer(typeof(T)).Serialize(xmlWriter, objectToSerialize);
}
}
return sb.ToString();
}
public static void DeserializeFromXML<T>(string xmlString, out T deserializedObject) where T : class
{
XmlSerializer xs = new XmlSerializer(typeof (T));
using (MemoryStream memoryStream = new MemoryStream(StringToUTF8ByteArray(xmlString)))
{
deserializedObject = xs.Deserialize(memoryStream) as T;
}
}
}
}
Then just call:
string xml = Utils.SerializeToXML(myObjectsIEnumerable);
I haven't tried it with, for example, an IEnumerable that fetches objects one at a time remotely, or any other weird use cases, but it works perfectly for List<T> and other collections that are in memory.
EDIT: Based on your comments in response to this, you could use XmlDocument.LoadXml to load the resulting XML string into an XmlDocument, save the first one to a file, and use that as your master XML file. For each item in the IEnumerable, use LoadXml again to create a new in-memory XmlDocument, grab the nodes you want, append them to the master document, and save it again, getting rid of the new one.
After you're finished, there may be a way to wrap all of the nodes in your root tag. You could also use XSL and XslCompiledTransform to write another XML file with the objects properly wrapped in the root tag.
You can do this by implementing the IXmlSerializable interface on the large class. The implementation of the WriteXml method can write the start tag, then simply loop over the IEnumerable<MyObject> and serialize each MyObject to the same XmlWriter, one at a time.
In this implementation, there won't be any in-memory data to get rid of (past what the garbage collector will collect).