Xml Serialization Problem for 60mb file - c#

I want to Serialize a 60mb file into XML but it gives me System out of memory exception.
Did any one else had this kind of problem?
Can someone suggest me a workaround for this problem.
Here is the method
static public string Serialize(object obj)
{
string returnValue;
System.Xml.Serialization.XmlSerializer xmlWriter = new System.Xml.Serialization.XmlSerializer(obj.GetType());
System.IO.StringWriter xmlOut = new System.IO.StringWriter();
//this is where the problem is.....
xmlWriter.Serialize(xmlOut, obj);
//return the Serialized XML
returnValue = xmlOut.ToString();
xmlOut.Close();
return returnValue;
}

With a stream that large, I think you should consider serializing to a file first. Serializing takes a lot of memory, and doing so purely in memory is killing you.
using(var file = new FileStream(...))
using(var streamWriter = new StreamWriter(file))
{
xmlWriter.Serialize(streamWriter, obj);
}

Related

Well Formed XML using Service Stack

I'm building an MVC5 application which pulls records from a database and allows a user to perform some basic data cleansing edits.
Once the data has been cleansed it needs to be exported as XML, run through a validator and then uploaded to a third party portal.
I'm using Service Stack, and I've found it fairly quick and straightforward in the past, particularly when outputting to CSV.
The one issue I'm having is with the XML serialzer. I'm not sure how to make it generate well formed XML.
The file that i'm getting simply dumps it on one line, which won't validate because it isn't well formed.
below is an extract from my controller action:
Response.Clear();
Response.ContentType = "text/xml";
Response.AddHeader("Content-Disposition", "attachment; filename="myFile.xml"");
XmlSerializer.SerializeToStream(viewModel, Response.OutputStream);
Response.End();
UPDATE: Thanks for the useful comments, as explained I'm not talking about pretty printing, the issue is I need to run the file through a validator before uploading it to a third party. The error message the validator is throwing is Error:0000, XML not well-formed. Cannot have more than one tag on one line.
Firstly, be aware that most white space (including new lines) in XML is insignificant -- it has no meaning, and is only for beautification. The lack of new lines doesn't make the XML ill-formed. See White Space in XML Documents or https://www.w3.org/TR/REC-xml/#sec-white-space. Thus in theory it shouldn't matter whether ServiceStack's XmlSerializer is putting all of your XML on a single line.
That being said, if for whatever reason you must cosmetically break your XML up into multiple lines, you'll need to do a little work. From the source code we can see that XmlSerializer uses DataContractSerializer with a hardcoded static XmlWriterSettings that does not allow for setting XmlWriterSettings.Indent = true. However, since this class is just a very thin wrapper on Microsoft's data contract serializer, you can substitute your own code:
public static class DataContractSerializerHelper
{
private static readonly XmlWriterSettings xmlWriterSettings = new XmlWriterSettings { Indent = true, IndentChars = " " };
public static string SerializeToString<T>(T from)
{
try
{
using (var ms = new MemoryStream())
using (var xw = XmlWriter.Create(ms, xmlWriterSettings))
{
var serializer = new DataContractSerializer(from.GetType());
serializer.WriteObject(xw, from);
xw.Flush();
ms.Seek(0, SeekOrigin.Begin);
var reader = new StreamReader(ms);
return reader.ReadToEnd();
}
}
catch (Exception ex)
{
throw new SerializationException(string.Format("Error serializing \"{0}\"", from), ex);
}
}
public static void SerializeToWriter<T>(T value, TextWriter writer)
{
try
{
using (var xw = XmlWriter.Create(writer, xmlWriterSettings))
{
var serializer = new DataContractSerializer(value.GetType());
serializer.WriteObject(xw, value);
}
}
catch (Exception ex)
{
throw new SerializationException(string.Format("Error serializing \"{0}\"", value), ex);
}
}
public static void SerializeToStream(object obj, Stream stream)
{
if (obj == null)
return;
using (var xw = XmlWriter.Create(stream, xmlWriterSettings))
{
var serializer = new DataContractSerializer(obj.GetType());
serializer.WriteObject(xw, obj);
}
}
}
And then do:
DataContractSerializerHelper.SerializeToStream(viewModel, Response.OutputStream);

How to control the encoding while serializing and deserializing?

I'm using serialization to a string as follows.
public static string Stringify(this Process self)
{
XmlSerializer serializer = new XmlSerializer(typeof(Process));
using (StringWriter writer = new StringWriter())
{
serializer.Serialize(writer, self,);
return writer.ToString();
}
}
Then, I deserialize using this code. Please note that it's not an actual stringification from above that's used. In our business logic, it makes more sense to serialize a path, hence reading in from said path and creating an object based on the read data.
public static Process Processify(this string self)
{
XmlSerializer serializer = new XmlSerializer(typeof(Process));
using (XmlReader reader = XmlReader.Create(self))
return serializer.Deserialize(reader) as Process;
}
}
This works as supposed to except for a small issue with encoding. The string XML that's produced, contains the addition encoding="utf-16" as an attribute on the base tag (the one that's about XML version, not the actual data).
When I read in, I get an exception because of mismatching encodings. As far I could see, there's no way to specify the encoding for serialization nor deserialization in any of the objects I'm using.
How can I do that?
For now, I'm using a very brute work-around by simply cutting of the excessive junk like so. It's Q&D and I want to remove it.
public static string Stringify(this Process self)
{
XmlSerializer serializer = new XmlSerializer(typeof(Process));
using (StringWriter writer = new StringWriter())
{
serializer.Serialize(writer, self,);
return writer.ToString().Replace(" encoding=\"utf-16\"", "");
}
}

How to Json To Xml

I am new for Json and I have a simple problem.
I am trying to convert json file to xml file with c#. But it throw an exception.
The Code is ;
private void TakeXML()
{
string json = ReadText();
XmlDocument doc = (XmlDocument)Newtonsoft.Json.JsonConvert.DeserializeXmlNode(json);
XmlTextWriter writer = new XmlTextWriter("json.xml", null);
writer.Formatting = Formatting.Indented;
doc.Save(writer);
}
The ReadText function is;
private string ReadText()
{
FileStream fs = new FileStream(#"C:\Users\Sinan\Desktop\bina.json", FileMode.Open, FileAccess.Read);
StreamReader sr = new StreamReader(fs);
string json;
try
{
json = sr.ReadToEnd();
return json;
}
catch (Exception)
{
return null;
}
finally
{
sr.Close();
fs.Dispose();
}
}
for XmlDocument doc = (XmlDocument)Newtonsoft.Json.JsonConvert.DeserializeXmlNode(json); line, it said that;
"JSON root object has multiple properties. The root object must have a single property in order to create a valid XML document. Consider specifing a DeserializeRootElementName."
I am searching to solve this problem but ı haven't found it. İf you help me in this regard, I will be glad. Thank you.
In method DeserializeXmlNode specify the root node name in second parameter as shown in below code:
XmlDocument doc =
(XmlDocument)
Newtonsoft.Json.JsonConvert.DeserializeXmlNode(json, "rootNodeName");
// second parameter
Although if you can give json string then it would be easy to give exact answer.
Reference link: Converting JSON to XML

Save and restore values from ArrayList

I store some string values in an array list at runtime. When the application closes I want to store the data from the array list in another location so that I can open and retrieve the data the next time the application starts. What are some possible ways to do this?
You can use Reading and Writing from/to files or storing the values in windows registry.
For Reading/Writing from/to files use:
StreamReader sr = new StreamReader(#"C:/store.dll"); //for reading
StreamWriter sw = new StreamWriter(#"C:/store.dll"); //for writing
This is the basic. Here are two great articles for this:
Reding from files
Writing to files
I used this for storing the High Scores for a simple game. :)
Here is a nice tutorial for using Windows registry.
You could Serialize the ArrayList and write it to disk, then later on load the file and Deserialize it to an object:
public static XDocument Serialize<T>(T objectIn, Type[] extraTypes)
{
try
{
var target = new XDocument();
XmlSerializer s = extraTypes != null ? new XmlSerializer(objectIn.GetType(), extraTypes) : new XmlSerializer(objectIn.GetType());
s = extraTypes != null
? new XmlSerializer(objectIn.GetType(), extraTypes)
: new XmlSerializer(objectIn.GetType());
var writer = target.CreateWriter();
s.Serialize(writer, objectIn);
writer.Close();
return target;
}
catch (Exception ex)
{
throw new Exception(string.Format("Could not serialize object: {0}", ex.Message));
}
}
public static T Deserialize<T>(XDocument xDocument, string defaultNamespace)
{
XmlSerializer s = new XmlSerializer(typeof(T), defaultNamespace);
T result = (T)s.Deserialize(xDocument.CreateReader());
return result;
}

XmlSerializer Converts newlines

I'm trying to serialize an object to memory, pass it to another process as a string, and deserialize it.
I've discovered that the XML Serialization process strips the \r off of the newlines for strings in the object.
byte[] b;
// serialize to memory.
using (MemoryStream ms = new MemoryStream())
{
XmlSerializer xml = new XmlSerializer(this.GetType());
xml.Serialize(ms, this);
b = ms.GetBuffer();
}
// I can now send the bytes to my process.
Process(b);
// On the other end, I use:
using (MemoryStream ms = new MemoryStream(b))
{
XmlSerializer xml = new XmlSerializer(this.GetType());
clone = (myObject)xml.Deserialize(ms);
}
How do I serialize an object without serializing it to disk just like this, but without mangling the newlines in the strings?
The strings should be wrapped in CDATA sections to preserve the newlines.
The answer came from anther SO post, but I'm reposting it here because I had to tweak it a little.
I had to create a new class to manage XML read/write to memory stream. Here it is:
public class SafeXmlSerializer : XmlSerializer
{
public SafeXmlSerializer(Type type) : base(type) { }
public new void Serialize(StreamWriter stream, object o)
{
XmlWriterSettings ws = new XmlWriterSettings();
ws.NewLineHandling = NewLineHandling.Entitize;
using (XmlWriter xmlWriter = XmlWriter.Create(stream, ws))
{
base.Serialize(xmlWriter, o);
}
}
}
Since it is built on top of XmlSerializer, it should behave exactly as expected. It's just that when I serialize with a StreamWriter, I will use the "safe" version of the serialization, thus saving myself the headache.
I hope this helps someone else.

Categories