Deserialize on the fly, or LINQ to XML - c#

Working with C# Visual Studio 2008, MVC1.
I'm creating an xml file by fetching one from a WebService and adding some nodes to it. Now I wanted to deserialize it to a class which is the model used to strongtyped the View.
First of all, I'm facing problems to achieve that without storing the xml in the filesystem cause I don't know how this serialize and deserialize work. I guess there's a way and it's a matter of time.
But, searching for the previous in the web I came accross LINQ to XML and now I doubt whether is better to use it.
The xml would be formed by some clients details, and basically I will use all of them.
Any hint?
Thanks!!

You can save a XElement to and from a MemoryStream (no need to save it to a file stream)
MemoryStream ms = new MemoryStream();
XmlWriter xw = XmlWriter.Create(ms);
document.Save(xw);
xw.Flush();
Then if you reset the position back to 0 you can deserialize it using the DataContractSerializer.
ms.Position = 0;
DataContractSerializer serializer = new DataContractSerializer(typeof(Model));
Model model = (model) serializer.ReadObject(ms);
There are other options for how serialization works, so if this is not what you have, let me know what you are using and I will help.

try this:
XmlSerializer xmls = new XmlSerializer(typeof(XElement));
FileStream FStream;
try
{
FStream = new FileStream(doctorsPath, FileMode.Open);
_Doctors = (XElement)xmls.Deserialize(FStream); FStream.Close();
FStream = new FileStream(patientsPath, FileMode.Open);
_Patients = (XElement)xmls.Deserialize(FStream)
FStream.Close();
FStream = new FileStream(treatmentsPath, FileMode.Open);
_Treatments = (XElement)xmls.Deserialize(FStream);
FStream.Close();
}
catch
{ }
This will load all of the XML files into our XElement variables. The try – catch block is a form of exception handling that ensures that if one of the functions in the try block throws an exception, the program will jump to the catch section where nothing will happen. When working with files, especially reading files, it is a good idea to work with try – catch.

LINQ to XML is an excellent feature. You can always rely on that. You don't need to write or read or data from file. You can specify either string or stream to the XDocument
There are enough ways to load an XML element to the XDocument object. See the appropriate Load functions. Once you load the content, you can easily add/remove the elements and later you can save to disk if you want.

Related

Big Data Xml file (file size over 20GB) convert to Json File

I have an XML file. I want to convert it to JSON with C#. However, the XML file is over 20 GB.
I have tried to read XML with XmlReader, then append every node to a JSON file. I wrote the following code:
var path = #"c:\result.json";
TextWriter tw = new StreamWriter(path, true, Encoding.UTF8);
tw.Write("{\"A\":");
using (XmlTextReader xmlTextReader = new XmlTextReader("c:\\muslum.xml"))
{
while (xmlTextReader.Read())
{
if (xmlTextReader.Name == "A")
{
var xmlDoc = new XmlDocument();
var v = xmlTextReader.ReadInnerXml();
string json = Newtonsoft.Json.JsonConvert.SerializeXmlNode(xmlDoc, Newtonsoft.Json.Formatting.None, true);
tw.Write(json);
}
}
}
tw.Write("}");
tw.Close();
This code not working. I am getting error while converting json. Is there any best way to perform the conversion?
I would do it the following way
generate classes out of xsd schema using xsd.exe
open file and read top level ( i e your document level ) tags one by one (with XmlTextReader or XmlReader)
serialize each tag into object using generated classes
deserialize resulting object to json and save to whatever
consider saving in batches of 1000-2000 tags
you are right about serialize/deserialize being slow. still doing work in several threads, preferably using TPL will give you good speed. Also consider using json.net serializer, it is really a lot faster than standard ones ( it is standard for web.api though)
I can put some code snippets in the morning if you need them.
We are processing big ( 1-10gigs) files this manner in order to save data to sql server database.

Writing to XML resource file through serialization

So I'm writing a C# app that has a couple of XML files added as resources. I read from these XML files to populate objects in the code using XML serialization and it works perfectly. So I can access the files like so (I've left some code out, just have the important bits):
using TestApp.Properties;
XmlSerializer serializer = new XmlSerializer(typeof(MapTiles));
StringReader sr = new StringReader(Resources.Maps);
mapTiles = (MapTiles)serializer.Deserialize(sr);
Now however, I'd like to do the opposite. I'd like to take some data and write it to these XML resource files. However, I seem to be running into trouble with this aspect and was hoping someone could see something I'm messing up or let me know what I need to do? Here's what I'm trying to do:
XmlSerializer serializer = new XmlSerializer(typeof(MapTiles));
TextWriter writer = new StreamWriter(Resources.Maps);
serializer.Serialize(writer, tempGroup);
writer.Close();
When I run this code though, I get an error on the 2nd line that says ArgumentException was unhandled - Empty path name is not legal.
So if anyone has any thoughts I would greatly appreciate some tips. Thanks so much.
The exception is being raised because you are passing Resources.Maps into StreamWriter. StreamWriter is handling this as a string and assuming it is file path for the stream. But the file path is empty so it is throwing an exception.
To fix out the StreamWriter line you could specify a local temporary file instead of Resources.Maps or use StringWriter with the default constructor e.g. new StringWriter().
If you are writing .resx files then ResXResourceWriter is the class you need. It will also handle your stream writing too. See http://msdn.microsoft.com/en-us/library/system.resources.resxresourcewriter.aspx and http://msdn.microsoft.com/en-us/library/ekyft91f.aspx. The second page has examples on how to use the class but breifly you would call something like this:
XmlSerializer serializer = new XmlSerializer(typeof(MapTiles));
using (StringWriter stringWriter = new StringWriter())
{
serializer.Serialize(stringWriter, tempGroup);
using (ResXResourceWriter resourceWriter = new ResXResourceWriter("~/App_GlobalResources/some_file.resx"))
{
resourceWriter.AddResource("Maps", stringWriter.ToString());
}
}
If you want to write out an assembly that has a dynamically created resource in it the you can emit a new assembly. In that case have a look here http://msdn.microsoft.com/en-us/library/8ye65dh0.aspx.
The StreamWriter constructor is throwing the exception because Resources.Maps is an empty string.
Check out the documentation on MSDN:
http://msdn.microsoft.com/en-us/library/fysy0a4b.aspx
Hmm, after trying some other things and doing even more research it appears that the problem is that you cannot write to a resource that is part of the application. Something to do with the resource being written into the assembly and therefore it's not writable. So I'll have to modify my approach and write the output to a different file.
Unless of course anyone does know differently. This is just what I've discovered so far. Thanks to those who had some ideas though, much appreciated.

XMLReader from a string content

I'm trying to generate XML from another XML using a XslTransform. I get both files (source XML and XSL transformation file) as string content, so I'm trying to pass the XSL file to XslTransform.Load() method as XmlReader. Now the XmlReader has to be created form a source string containing XSL file, so i try doing it like this:
MemoryStream memStream = new MemoryStream();
byte[] data = Encoding.Default.GetBytes(transformation.XsltContent);
memStream.Write(data, 0, data.Length);
memStream.Position = 0;
XmlReader reader = XmlReader.Create(memStream);
and also tried using a StringReader:
XmlReader reader = XmlReader.Create(new StringReader(transformation.XsltContent));
Unfortunately, bot methods don't seems to work, the input seems to be ok, I even tried creating some basic one-element XML to pass, won't work either - reader contains {None}.
Could someone point out what seems to be the problem here?
XmlReader xmlReader = XmlReader.Create(new StringReader(YourStringValue));
The StringReader -> XmlReader approach is fine, you should stick to it. The reader reports none because it hasn't been read yet. Try calling Read() on it to see what happens then. The transformation will also call read on it.

xml and files on disc interaction

Its been a while since i've needed to do this so i was looking at the old school methods of writing XMLDocument from code down to a File.
In my application i am writing alot to an XMLdocument with new elements and values and periodically saving it down to disc and also reading from the file and depending on the data i am doing things.
I am using methods like File.Exists(...) _xmldoc.LoadFile(..) etc...
Im wondering probably now a days there are better methods for this with regards
Parsing the XML
Checking its format for saving down
rather then the data being saved down being treated as text but as XML
maybe what i am doing is fine but its been a while and wondered if there are other methods :)
thanks
Well there's LINQ to XML which is a really nice XML API introduced in .NET 3.5. I don't think the existing XMLDocument API has changed much, beyond some nicer ways of creating XmlReaders and XmlWriters (XmlReader.Create/XmlWriter.Create).
I'm not sure what you really mean by your second and third bullets though. What exactly are you doing in code which feels awkward?
Have you looked at the Save method of your XmlDocument? It will save whatever is in your XmlDocument as a valid formatted file.
If your program is able to use the XmlDocument class, the XmlDocument class will be able to save your file. You won't need to worry about validating before saving, and you can give it whatever file extension you want. As to your third point... an XML file is really just a text file. It won't matter how the OS sees it.
I was a big fan of XmlDocument due to its facility to use but recently I got a huge memory problem with that class so I started to use XmlReader and XmlWriter.
XmlReader can be a little bit tricky to use if your Xml file is complex because you read the Xml file sequentially. In that case, the method ReadSubTree of XmlReader can be very useful because this method returns only the xml tree under the current node so you send the new xmlreader to a function to parse the subnode content and once it is done, you continue to the next node.
XmlReader Example:
string xmlcontent = "<BigXml/>";
using(StringReader strContent = new StringReader(xmlcontent))
{
using (XmlReader reader = XmlReader.Create(strContent))
{
while (reader.Read())
{
if (reader.Name == "SomeName" && reader.NodeType == XmlNodeType.Element)
{
//Send the XmlReader created by ReadSubTree to a function to read it.
ReadSubContentOfSomeName(reader.ReadSubtree());
}
}
}
}
XmlWriter Example:
StringBuilder builder = new StringBuilder();
using (XmlWriter writer = XmlWriter.Create(builder))
{
writer.WriteStartDocument();
writer.WriteStartElement("BigXml");
writer.WriteAttributeString("someAttribute", "42");
writer.WriteString("Some Inner Text");
//Write nodes under BigXml
writer.WriteStartElement("SomeName");
writer.WriteEndElement();
writer.WriteEndElement();
writer.WriteEndDocument();
}

How to inspect XML streams from the debugger in Visual Studio 2003

I've got to edit an XSLT stylesheet, but I'm flying blind because the XML input only exists fleetingly in a bunch of streams. I can debug into the code, but can't figure out how to get the contents of the streams out into text I can look at (and run through the XSLT manually while I'm editing them).
The code is part of a big old legacy system, I can modify it in a debug environment if absolutely necessary, but it runs in a windows service connected up to a bunch of MSMQs. So for various reasons I'd rather be able to use the debugger to see the XML without having to change the code first.
Code much simplified, is something like this: (C# - but remember it's .net 1.1 in VS 2003.)
This is the function that gets the XML as a stream, which is then fed into some sort of XSLT transform object. I've tried looking at the writer and xmlStream objects in the watch windows and the immediate window, but can't quite fathom how to see the actual XML.
private MemoryStream GetXml()
{
MemoryStream xmlStream;
xmlStream = new MemoryStream();
XmlWriter writer = new XmlTextWriter(xmlStream, Encoding.UTF8);
writer.WriteStartDocument();
//etc etc...
writer.WriteEndDocument();
writer.Flush();
xmlStream.Position = 0;
return xmlStream; //Goes off to XSLT transform thingy!
}
All help much appreciated.
You could simply add this expression to your watch window after the MemoryStream is ready:
(new StreamReader(xmlStream)).ReadToEnd();
Watch expressions don't need to be simple variable values. They can be complex expressions, but they will have side-effects. As you've noted, this will interrupt execution, since the stream contents will be read out completely. You could recreate the stream after the interruption with another expression, if you need to re-start execution.
This situation arises frequently when debuging code with streams, so I avoid them for simple, self-contained tasks. Unfortunately, for large systems, it's not always easy to know in advance whether you should make your code stream-oriented or not, since it depends greatly on how it will be used. I consider the use of streams to be a premature optimization in many cases, however.
OK, I've not succeeded in using the debugger without modifying the code. I added in the following snippet, which lets me either put a breakpoint in or use debugview.
private MemoryStream GetXml()
{
MemoryStream xmlStream;
xmlStream = new MemoryStream();
XmlWriter writer = new XmlTextWriter(xmlStream, Encoding.UTF8);
writer.WriteStartDocument();
//etc etc...
writer.WriteEndDocument();
writer.Flush();
xmlStream.Position = 0;
#if DEBUG
string temp;
StreamReader st=new StreamReader(xmlStream);
temp=st.ReadToEnd();
Debug.WriteLine(temp);
#endif
return xmlStream; //Goes off to XSLT transform thingy!
}
I'd still prefer to simply look at the xmlstream object in the debugger somehow, even if it disrupts the flow of execution, but in the meantime this is the best I've managed.

Categories