We Keep Coding

Deserialize on the fly, or LINQ to XML

Working with C# Visual Studio 2008, MVC1.
I'm creating an xml file by fetching one from a WebService and adding some nodes to it. Now I wanted to deserialize it to a class which is the model used to strongtyped the View.
First of all, I'm facing problems to achieve that without storing the xml in the filesystem cause I don't know how this serialize and deserialize work. I guess there's a way and it's a matter of time.
But, searching for the previous in the web I came accross LINQ to XML and now I doubt whether is better to use it.
The xml would be formed by some clients details, and basically I will use all of them.
Any hint?
Thanks!!

You can save a XElement to and from a MemoryStream (no need to save it to a file stream)
MemoryStream ms = new MemoryStream();
XmlWriter xw = XmlWriter.Create(ms);
document.Save(xw);
xw.Flush();
Then if you reset the position back to 0 you can deserialize it using the DataContractSerializer.
ms.Position = 0;
DataContractSerializer serializer = new DataContractSerializer(typeof(Model));
Model model = (model) serializer.ReadObject(ms);
There are other options for how serialization works, so if this is not what you have, let me know what you are using and I will help.

try this:
XmlSerializer xmls = new XmlSerializer(typeof(XElement));
FileStream FStream;
try
{
FStream = new FileStream(doctorsPath, FileMode.Open);
_Doctors = (XElement)xmls.Deserialize(FStream); FStream.Close();
FStream = new FileStream(patientsPath, FileMode.Open);
_Patients = (XElement)xmls.Deserialize(FStream)
FStream.Close();
FStream = new FileStream(treatmentsPath, FileMode.Open);
_Treatments = (XElement)xmls.Deserialize(FStream);
FStream.Close();
}
catch
{ }
This will load all of the XML files into our XElement variables. The try – catch block is a form of exception handling that ensures that if one of the functions in the try block throws an exception, the program will jump to the catch section where nothing will happen. When working with files, especially reading files, it is a good idea to work with try – catch.

LINQ to XML is an excellent feature. You can always rely on that. You don't need to write or read or data from file. You can specify either string or stream to the XDocument
There are enough ways to load an XML element to the XDocument object. See the appropriate Load functions. Once you load the content, you can easily add/remove the elements and later you can save to disk if you want.

Stream StringBuilder to file

I need to create a large text document. I currently use StringBuilder to make the document and then call File.WriteallText(filename,sb.ToString). Unfortunately, this is now starting to throw out of memory exceptions.
Is there a better way to stream a StringBuilder to file or is there some other technique I should be using?

Instead of using StringBuilder, try using TextWriter (which has a broadly similar API, but which can write to a number of underlying destinations, including files) - i.e.
using(TextWriter writer = File.CreateText(path))
{
// loop etc
writer.Write(...);
}
More generally, it is worth separating the code that knows about files from the code that knows about how to write the data, i.e.
using(var writer = File.CreateText(path))
{
Serialize(writer);
}
...
void Serialize(TextWriter writer)
{
...
}
this makes it easier to write to different targets. For example, you can now do in-memory too:
var sw = new StringWriter();
Serialize(sw);
string text = sw.ToString();
The point being: your Serialize code didn't need to change to accomodate a different target. This could also be writing directly to a network, or writing through a compression/encryption stream. Very versatile.

Just use a StreamWriter that writes to a FileStream:
using (StreamWriter writer = new StreamWriter("filename.txt")) {
...
}
This will of course mean that you can't change the text that is already written, like you can do in a StringBuilder, but I assume that you are not using that anyway.

Why not streaming directly into the stream?
You could use the TextWriter.

You can use StreamWriter and write to the file directly.

Writing to XML resource file through serialization

So I'm writing a C# app that has a couple of XML files added as resources. I read from these XML files to populate objects in the code using XML serialization and it works perfectly. So I can access the files like so (I've left some code out, just have the important bits):
using TestApp.Properties;
XmlSerializer serializer = new XmlSerializer(typeof(MapTiles));
StringReader sr = new StringReader(Resources.Maps);
mapTiles = (MapTiles)serializer.Deserialize(sr);
Now however, I'd like to do the opposite. I'd like to take some data and write it to these XML resource files. However, I seem to be running into trouble with this aspect and was hoping someone could see something I'm messing up or let me know what I need to do? Here's what I'm trying to do:
XmlSerializer serializer = new XmlSerializer(typeof(MapTiles));
TextWriter writer = new StreamWriter(Resources.Maps);
serializer.Serialize(writer, tempGroup);
writer.Close();
When I run this code though, I get an error on the 2nd line that says ArgumentException was unhandled - Empty path name is not legal.
So if anyone has any thoughts I would greatly appreciate some tips. Thanks so much.

The exception is being raised because you are passing Resources.Maps into StreamWriter. StreamWriter is handling this as a string and assuming it is file path for the stream. But the file path is empty so it is throwing an exception.
To fix out the StreamWriter line you could specify a local temporary file instead of Resources.Maps or use StringWriter with the default constructor e.g. new StringWriter().
If you are writing .resx files then ResXResourceWriter is the class you need. It will also handle your stream writing too. See http://msdn.microsoft.com/en-us/library/system.resources.resxresourcewriter.aspx and http://msdn.microsoft.com/en-us/library/ekyft91f.aspx. The second page has examples on how to use the class but breifly you would call something like this:
XmlSerializer serializer = new XmlSerializer(typeof(MapTiles));
using (StringWriter stringWriter = new StringWriter())
{
serializer.Serialize(stringWriter, tempGroup);
using (ResXResourceWriter resourceWriter = new ResXResourceWriter("~/App_GlobalResources/some_file.resx"))
{
resourceWriter.AddResource("Maps", stringWriter.ToString());
}
}
If you want to write out an assembly that has a dynamically created resource in it the you can emit a new assembly. In that case have a look here http://msdn.microsoft.com/en-us/library/8ye65dh0.aspx.

The StreamWriter constructor is throwing the exception because Resources.Maps is an empty string.
Check out the documentation on MSDN:
http://msdn.microsoft.com/en-us/library/fysy0a4b.aspx

Hmm, after trying some other things and doing even more research it appears that the problem is that you cannot write to a resource that is part of the application. Something to do with the resource being written into the assembly and therefore it's not writable. So I'll have to modify my approach and write the output to a different file.
Unless of course anyone does know differently. This is just what I've discovered so far. Thanks to those who had some ideas though, much appreciated.

Best approach to write huge xml data to file?

I'm currently exporting a database table with huge data (100000+ records) into an xml file using XmlTextWriter class and I'm writing directly to a file on the physical drive.
_XmlTextWriterObject = new XmlTextWriter(_xmlFilePath, null);
While my code runs ok, my question is that is it the best approach? Should I instead write the whole xml in memory stream first and then write the xml document in physical file from memory stream? And what are the effects on memory/ performance in both cases?
EDIT
Sorry that I could not actually convey what I meant to say.Thanks Ash for pointing out.
I will indeed be using XmlTextWriter but I meant to say whether to pass a physical file path string to the XmlTextWriter constructor (or, as John suggested, to the XmlTextWriter.Create() method) or use stream based api. My current code looks like the following:
XmlWriter objXmlWriter = XmlTextWriter.Create(new BufferedStream(new FileStream(#"C:\test.xml", FileMode.Create, System.Security.AccessControl.FileSystemRights.Write, FileShare.None, 1024, FileOptions.SequentialScan)), new XmlWriterSettings { Encoding = Encoding.Unicode, Indent = true, CloseOutput = true });
using (objXmlWriter)
{
//writing xml contents here
}

The rule of thumb is to use XmlWriter when the document need only be written and not worked with in memory, and to use XmlDocument (or the DOM) where you do need to work with it in memory.
Remember though, XmlWriter implements IDisposable, so do the following:
using (XmlWriter _XmlTextWriterObject = XmlWriter.Create(_xmlFilePath))
{
// Code to do the write here
}

While my code runs ok, my question is that is it the best approach?
As mentioned and your update, XmlWriter.Create is fine.
Or should I write the whole xml in memory stream first and then write the xml document in physical file from memory stream?
Do you have the memory to write the entire file in-memory? If you do then that approach will be faster, otherwise stream it using a FileStream which will take care of it for you.
And what are the effects on memory/ performance in both cases?
Reading the entire XML file in will use more memory, and spike the processor to start with. Streaming to disk will use more processor. But you'll need to be using a huge file for this to be noticeable given even desktop hardware now. If you're worried about the size increasing even more in the future, stick to the FileStream technique to future proof it.

As John Saunders mentioned it is better to use XmlWriter.Create(). That is the recommendation from the MSDN. The XmlWriter.Create() method can also take an XmlWriterSettings object. There you can customize your behavior quite a bit. If you don't need validation and character checking then you can turn it off and get a bit more speed. For example
XmlWriterSettings settings = new XmlWriterSettings();
settings.CheckCharacters = false;
using (XmlWriter writer = XmlWriter.Create("path", settings))
{
//writing code
writer.Flush();
}
Otherwise I think everything is okay.

What's the most streamlined way of performing a XSLT transformation in ASP.NET?

In other words, is there a faster, more concise way of writing the following code:
//Create an object for performing XSTL transformations
XslCompiledTransform xslt = new XslCompiledTransform();
xslt.Load(HttpContext.Current.Server.MapPath("/xslt/" + xsltfile.Value), new XsltSettings(true, false), new XmlUrlResolver());
//Create a XmlReader object to read the XML we want to format
//XmlReader needs an input stream (StringReader)
StringReader sr = new StringReader(node.OuterXml);
XmlReader xr = XmlReader.Create(sr);
//Create a StringWriter object to capture the output from the XslCompiledTransform object
StringWriter sw = new StringWriter();
//Perform the transformation
xslt.Transform(xr, null, sw);
//Retrieve the transformed XML from the StringWriter object
string transformedXml = sw.ToString();
UPDATE (thanks for all the answers so far!):
Sorry for my vagueness: by "faster" and more "concise" I mean, am I including any unnecessary steps? Also, I would love a more "readable" solution if someone has one. I use this code in a small part of a web application I'm developing, and I'm about to move it to a large part of the application, so I want to make sure it's as neat as can be before I make the move.
Also, I get the XML from a static class (in a separate data access class library) which communicates with a database. I also manipulate the transformed XML string before shipping it off to a web page. I'm not sure if the input/response streams are still viable in this case.
One more thing: the XML and the XSLT supplied may change (users of the application can make changes to both), so I think I would be forced to compile each time.

Here's code I did for my ASP.NET, which is very similar to yours:
XDocument xDoc = XDocument.Load("output.xml");
XDocument transformedDoc = new XDocument();
using (XmlWriter writer = transformedDoc.CreateWriter())
{
XslCompiledTransform transform = new XslCompiledTransform();
transform.Load(XmlReader.Create(new StreamReader("books.xslt")));
transform.Transform(xDoc.CreateReader(), writer);
}
// now just output transformedDoc

If you have a large XSLT you can save the overhead of compiling it at runtime by compiling the XSLT into a .NET assembly when you build your project (e.g. as a post-build step). The compiler to do this is called xsltc.exe and is part of Visual Studio 2008.
In order to load such a pre-compiled XSLT you will need .NET Framework 2.0 SP1 or later installed on your server (the feature was introduced with SP1).
For an example check Anton Lapounov's blog article:
XSLTC — Compile XSLT to .NET Assembly
If pre-compiling the XSLT is not an option you should consider caching the XslCompiledTransform after it is loaded so that you don't have to compile it everytime you want to execute the transform.

Don't have time to do a full example, but some notes:
XML is not the same as System.String. Get it from the class library as XDocument or XmlDocument; finish with it as XDocument or XmlDocument.
You can use the ASP.NET Cache to store the compiled XSL, with a cache dependency on when the .XSLT file changes.
Don't convert the XML to a string then back to XML. Use node.CreateNavigator().ReadSubTree().
Similarly, use XPathNavigator.AppendChild to get an XmlWriter that will write into an XML Document.

Since you mention ASP.NET, the question is whether you can use the response stream directly for your transform output and whether you can use the input stream directly if it is a POST...

I'd rewrite the code like this:
string path = HttpContext.Current.Server.MapPath("/xslt/" + xsltfile.Value);
XmlReader reader = CreateXmlReader(node.OuterXml);
string transformedXml = Transform(path, reader);
private XmlReader CreateXmlReader(string text)
{
StringReader reader = new StringReader(text);
return XmlReader.Create(reader);
}
private string Transform(string xsltPath, XmlReader source)
{
XsltCompiledTransform transformer = new XsltCompiledTransform();
transformer.Load(
xsltPath,
new XsltSettings(true, false),
new XmlUrlResolver());
StringWriter writer = new StringWriter();
transformer.Transform(source, null, writer);
return writer.ToString();
}
The reason why I'd rewrite the code like this is because each block of code now has one and only one purpose. This makes it easier to read and understand. Additionally, the code requires less comments because a lot of information can be inferred from the names of the function and its parameters.