We Keep Coding

deserialize xml node with unknown child nodes

I have some xml that looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<response>
<video key="8bJ8OyXI">
<custom>
<legacyID>50898311001</legacyID>
</custom>
<date>1258497567</date>
<description>some description</description>
<duration>486.20</duration>
<md5>bc89fde37ef103db26b8a9d98065d006</md5>
<mediatype>video</mediatype>
<size>99416259</size>
<sourcetype>file</sourcetype>
<status>ready</status>
<views>0</views>
</video>
</response>
I am using XmlSerializer to serialize the xml to class objects, and would prefer to stick with it if possible since everything else works just fine. The node custom is just custom metadata added to the video, and pretty much anything could potentially end up in there (only strings, just a name and value). I used xsd.exe to generate class objects from my xml, which generated a unique class for the <custom> tag with just one ulong property for the legacyID value. The thing is, potentially any arbitrary number of values could be there and I can't and don't need to account for them all (but I may need to read particular values later).
Is it possible to set up the Video.Custom property in my class so that the serializer can deserialize those values into say something like a Dictionary<string, string>? I don't need type information for those particular values, saving the node names + values are more than enough for my purposes.

You can handle UnknownElement event and there deserialize custom element to your dictionary
serializer.UnknownElement += (s, e) =>
{
if (e.Element.LocalName == "custom" && e.ObjectBeingDeserialized is Video)
{
Video video = (Video)e.ObjectBeingDeserialized;
if (video.Custom == null)
{
video.Custom = new Dictionary<string, string>();
}
foreach (XmlElement element in e.Element.OfType<XmlElement>())
{
XmlText text = (XmlText)element.FirstChild;
video.Custom.Add(element.LocalName, text.Value);
}
}
};

Trouble Parsing XML

I am having some trouble parsing some XML from centovacast v3 XML API. I've worked with their 2.x API and parsed it, but the responses have totally changed and I cannot seem to make any of my existing parsers work. Every example I've tried I cannot seem to get at the data correctly.
I'm using .NET 3.5 (4.0 is acceptable as well), any examples would be greatly appreciated.
An example XML document is:
<?xml version=""1.0"" encoding=""UTF-8""?>
<centovacast version=""3.0.0"" host=""0.0.0.0:2199"">
<response type=""success"">
<message>OK</message>
<data>
<row>
<id>1</id>
<parameters>
<ipaddress>127.0.0.1</ipaddress>
<port>2198</port>
<title>Local server</title>
<isrelay>1</isrelay>
<ismaster>1</ismaster>
<defaultip>0.0.0.0</defaultip>
<daemontype>RPC</daemontype>
<hostname/>
</parameters>
<status>
<memfree>101879808</memfree>
<memtotal>1073741824</memtotal>
<memavail>778653696</memavail>
<swapfree>1077501952</swapfree>
<swaptotal>1077501952</swaptotal>
<buffers>172535808</buffers>
<cpuload>0.00</cpuload>
<uptime>13372713</uptime>
<machine>Intel(R) Xeon(R) CPU E5620</machine>
<osbrief>Linux</osbrief>
<osdetails>2.6.18</osdetails>
<other>
<Processes>
<field>n</field>
<field>72</field>
</Processes>
<Kernel>
<field>s</field>
<field>Linux version 2.6.18</field>
</Kernel>
<row>
<field>f</field>
<field>0.000000</field>
</row>
<row>
<field>f</field>
<field>0.000000</field>
</row>
<row>
<field>f</field>
<field>0.000000</field>
</row>
</other>
<online>1</online>
</status>
<accounts>
<licensed>-1</licensed>
<active>1</active>
<inactive>0</inactive>
</accounts>
</row>
</data>
</response>
</centovacast>
I've tried using the following code:
var xml = XDocument.Parse(xmldata);
var query = from p in xml.Descendants("status")
select p;
foreach (var record in query)
MessageBox.Show(record.Value);
but it returns all the data inside the <status> and <parameters> in one big jumble, rather then in separate values.
I would love to serialize / deserialize, as the XML call I'm making returns the above for each server in the cluster, so it could be quite a large result set, but I am not picky, I would be happy just being able to get the data into the correct variables so I can use them.

Here's an example storing some of the elements in an anonymous type:
var data =
XDocument.Parse(xml)
.Root
.Element("response")
.Element("data")
.Elements("row")
.Select(row =>
new
{
Id = Int32.Parse(row.Element("id").Value),
Parameters = new
{
IpAddress = row.Element("parameters").Element("ipaddress").Value,
port = Int32.Parse(row.Element("parameters").Element("port").Value),
},
Status = new
{
MemFree = Int32.Parse(row.Element("status").Element("memfree").Value),
},
});
You can always plug in your own concrete types and null checks where option values might be.

xml.Descendants("status") returns the whole element status together with it's child elements. If you want to enumerate it's elements use the following code:
xml.Descendants("status").Descendants();

Detecting structural differences in XML using Linq and XElement

I'm trying to audit some XML that is used in a bespoke piece of software. Im able to detect changes in identical structures using 'XNode.DeepEquals' and then adding an extra attribute to the elements that have changed so I can highlight them.
My problem is that, when the structure does change this methodology fails. ( I'm enumerating over both XElements at the same time performing a DeepEquals, if they are not equal - recursively calling the same method to filter out where the exact changes occurr )
Obviously this now falls apart when I'm enumerating and the nodes being compared are not the same. See Below Sample:
Before
<?xml version="1.0" encoding="utf-16"?>
<Prices xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Price default="true">
<Expression operator="Addition">
<LeftOperand>
<AttributeValue field="ccx_bandwidth" />
</LeftOperand>
<RightOperand>
<Constant value="10" type="Integer" />
</RightOperand>
</Expression>
</Price>
<Price default="false">
<Expression operator="Addition">
<LeftOperand>
<AttributeValue field="ccx_bandwidth" />
</LeftOperand>
<RightOperand>
<Constant value="99" type="Integer" />
</RightOperand>
</Expression>
</Price>
<RollupChildren />
After
<?xml version="1.0" encoding="utf-16"?>
<Prices xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Price default="true">
<Expression operator="Addition">
<LeftOperand>
<AttributeValue field="ccx_bandwidth" />
</LeftOperand>
<RightOperand>
<Constant value="10" type="Integer" />
</RightOperand>
</Expression>
</Price>
<RollupChildren />
So you can see that the latter Price Node has been removed and I need to show this change.
At the moment I have access to both pieces of xml and modify them on load of the audit application with an 'auditchanged' attribute which in my silverlight app i bind the background too with a converter.
I'd been playing around with Linq to Xml and looking at joining the two XElements in a query but wasn't sure how to proceed.
Ideally what I would like to do is merge the two XElements together but add a seperate attribute depending on if it's added or removed which i can then bind to with a converter to say highlight in red or green appropriately.
Does anyone have any bright ideas on this one? ( I'd been looking at XmlDiff however I can't use that in Silverlight, I don't think? )

I have a generic differ class in the codeblocks library http://codeblocks.codeplex.com
Loading your XML documents and treating each document as an IEnumerable (flattened XML tree) should allow you to use the differ as shown here: http://codeblocks.codeplex.com/wikipage?title=Differ%20Sample&referringTitle=Home
Here's the source code for differ.cs: http://codeblocks.codeplex.com/SourceControl/changeset/view/96119#1887406
Diff prototype is:
static IEnumerable<DiffEntry> Diff(IEnumerable<T> oldData, IEnumerable<T> newData, Comparison<T> identity, Comparison<T> different)

The important part here is the descendants query. It turns every element in the first doc in a list of its ancestors, where every item contains the name of the element and it's index among its siblings of the same name. I think this can be somehow used for joining, though I have no idea how to do full outer join with linq. So instead i just use these lists to find elements in the second document, and then depending on the result, probably mark it as either deleted or changed.
var doc = XDocument.Load(in_A);
var doc2 = XDocument.Load(in_B);
var descendants = doc.Descendants().Select(d =>
d.AncestorsAndSelf().Reverse().Select(el =>
new {idx = el.ElementsBeforeSelf(el.Name).Count(), el, name = el.Name}).ToList());
foreach (var list in descendants) {
XContainer el2 = doc2;
var el = list.Last().el;
foreach (var item in list) {
if (el2 == null) break;
el2 = el2.Elements(item.name).Skip(item.idx).FirstOrDefault();
}
string changed = "";
if (el2 == null) changed += " deleted";
else {
var el2e = el2 as XElement;
if (el2e.Attributes().Select(a => new { a.Name, a.Value })
.Except(el.Attributes().Select(a => new { a.Name, a.Value })).Count() > 0) {
changed += " attributes";
}
if (!el2e.HasElements && el2e.Value != el.Value) {
changed += " value";
}
el2e.SetAttributeValue("found", "found");
}
if (changed != "") el.SetAttributeValue("changed", changed.Trim());
}
doc.Save(out_A);
doc2.Save(out_B);

How can I get the Status element from the following XML?

I'm getting the following block of xml back from a web service:
<?xml version="1.0" encoding="utf-16"?>
<ArrayOfItemResponse xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<ItemResponse>
<Item xmlns="http://www.xyz.com/ns/2006/05/01/webservices/abc/def">
<RequestKey Name="ESM.PA" Service="" />
<QoS>
<TimelinessInfo Timeliness="REALTIME" TimeInfo="0" />
<RateInfo Rate="TIME_CONFLATED" TimeInfo="10" />
</QoS>
<Status>
<StatusMsg>OK</StatusMsg>
<StatusCode>0</StatusCode>
</Status>
<Fields>
<Field DataType="Utf8String" Name="DSPLY_NAME">
<Utf8String>D15 |ASDFDSAA ETF </Utf8String>
</Field>
</Fields>
</Item>
</ItemResponse>
</ArrayOfItemResponse>
I'm trying to capture the Status element in an object as follows, but it's not working.
var _xml = XDocument.Parse(xmlFromService);
var stat = _xml
.Descendants("ArrayOfItemResponse")
.Descendants("ItemResponse")
.Descendants("Item")
.Descendants("Status");
What's the best way for me to get this element?

If you want to use System.Xml.Linq, you can use this expression:
var stat = (from n in _xml.Descendants() where n.Name.LocalName == "Status" select n).ToList();

You are not able to get the required results with your existing code because of the xmlns attribute in Item
<Item xmlns="http://www.xyz.com/ns/2006/05/01/webservices/abc/def">
This question addresses the problem you are actually facing. If you don't know the namespace then you should take a look at this question.

I don't know the best way, but you can read it like this
XmlDocument xdoc = new XmlDocument();
xdoc.Load(stream);
var statMsg = xdoc.GetElementsByTagName("StatusMsg")[0].InnerText;
var statCode = xdoc.GetElementsByTagName("StatusCode")[0].InnerText;

use xpath, something like
var stat = _xml.SelectSingleNode(#"ArrayOfItemResponse/ItemResponse/ItemStatus/StatusCode").Value;
that should put the value 0 into stat.

Your xml code use Namespace.
XNamespace ns = "http://www.xyz.com/ns/2006/05/01/webservices/abc/def";
var stat = _xml.Descendants(ns + "Status");

Analysis of XML structured data

My work uses software to fill out records that are expressed as XML documents. I have the task of trawling through these XML files to pull statistics out of them. The files themselves adhere to no schema and if a form field doesn't get filled out then the XML corresponding to that field is not created.
What's my best approach?
Example XML:
<Form>
<Field>
<id>Field ID</id>
<value>Entered Value</value>
</Field>
</Form>
I have been attempting to write software that I can use to query the files but have not been able to come up with anything even remotely useful.
Any thoughts appreciated.
EDIT: In terms of C#, what I would like (Though I'm sure it isn't possible) is a Dictionary that has a string as the key and the corresponding value could EITHER be a string or another Dictionary.

Is like this ↓ ?
XML:
<?xml version="1.0" encoding="utf-8" ?>
<Form>
<Field>
<id>People1</id>
<value>C Sharp</value>
</Field>
<Field>
<id>People2</id>
<value>C Sharp</value>
</Field>
<Field>
<id>People3</id>
<value>C</value>
</Field>
Source:
static void Main(string[] args)
{
var doc = XDocument.Load("test.xml");
var result = from p in doc.Descendants("Form").Descendants("Field")
select new { ID = p.Element("id").Value, VALUE = p.Element("value").Value };
foreach (var x in result)
Console.WriteLine(x);
var gr = from p in result
group p by p.VALUE into g
select new { Language=g.Key , Count=g.Count() };
foreach (var x in gr)
Console.WriteLine(string.Format("Language:{0} Count:{1}" , x.Language , x.Count));
Console.Read();
}

If the file is not too big, I would suggest perl and the XML::Simple module. This will map the XML to a perl array of hashes, and then you can simply loop through it like normal. Something like:
my $xml = XML::Simple::XmlIn( 'file.xml', force_array => [ 'Form', 'Field' ] );
my %fld_counts;
foreach my $form ( #{$xml->{Form}} )
{
# Any start record processing...
foreach my $fld ( #{$form->{Field}} )
{
my $id = $fld->{id}
my $val = $fld->{value}
# Do something with id/value... like...
$fld_counts{$id}++;
}
}
So just adjust that structure based on the stats you want to gather

For parsing XML I prefer using plain XmlReader. Granted, it's more verbose, but it's super efficient and transparent, at least for me. For example:
using(var xr = XmlReader.Create('your stream here'))
while(xr.Read())
if(xr.NodeType == XmlNodeType.Element)
switch(xr.Name) {
case "brands":
// do something here with this element,
// like possibly reading the whole subtree...
using(var xrr = xr.ReadSubtree())
while(xrr.Read()) {
// working here...
}
break;
case "products":
// that is another element
break;
case "some-other-element":
// and so on
break;
} // switch