C# XSLT Transforming Large XML Files Quickly - c#

I'm transforming a > 2GB file with a lookup template in the XSLT.
I would like this to run faster but can't find any low hanging fruit to improve performance. Any help would be greatly appreciated.
I'm a newb when it comes to transformations.
This is the current format of the XML file.
<?xml version="1.0" encoding="utf-8" ?>
<contacts>
<contact>
<attribute>
<name>text12</name>
<value>B00085590</value>
</attribute>
<attribute>
<name>text34</name>
<value>Atomos</value>
</attribute>
<attribute>
<name>date866</name>
<value>02/21/1991</value>
</attribute>
</contact>
<contact>
<attribute>
<name>text12</name>
<value>B00058478</value>
</attribute>
<attribute>
<name>text34</name>
<value>Balderas</value>
</attribute>
<attribute>
<name>date866</name>
<value>11/24/1997</value>
</attribute>
</contact>
</contacts>
The xslt I used for the transformation.
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:msxsl="urn:schemas-microsoft-com:xslt" exclude-result-prefixes="msxsl"
>
<xsl:output method="xml" indent="yes"/>
<!--Identify location of the lookup xml-->
<xsl:param name="lookupDoc" select="document('C:\Projects\Attributes.xml')" />
<!--Main Template-->
<xsl:template match="/contacts">
<!--Apply Formatted Contacts Template-->
<xsl:apply-templates select="contact" />
</xsl:template>
<!--Formatted Contacts Template-->
<xsl:template match="contact">
<contact>
<xsl:for-each select="attribute">
<!--Create variable to hold New Name after passing the Data Name to the Lookup Template-->
<xsl:variable name="newName">
<xsl:apply-templates select="$lookupDoc/attributes/attribute">
<xsl:with-param name="nameToMatch" select="name" />
</xsl:apply-templates>
</xsl:variable>
<!--Format Contact Element with New Name variable-->
<xsl:element name="{$newName}">
<xsl:value-of select="value"/>
</xsl:element>
</xsl:for-each>
</contact>
</xsl:template>
<!--Lookup Template-->
<xsl:template match="attributes/attribute">
<xsl:param name="nameToMatch" />
<xsl:value-of select='translate(translate(self::node()[name = $nameToMatch]/mappingname, "()*%$##!~<>&apos;&,.?[]=-+/\:1234567890", "")," ","")' />
</xsl:template>
</xsl:stylesheet>
Sample Lookup XML
<?xml version="1.0" encoding="utf-8" ?>
<attributes>
<attribute>
<name>text12</name>
<mappingname>ID</mappingname>
<datatype>Varchar2</datatype>
<size>30</size>
</attribute>
<attribute>
<name>text34</name>
<mappingname>Last Name</mappingname>
<datatype>Varchar2</datatype>
<size>30</size>
</attribute>
<attribute>
<name>date866</name>
<mappingname>DOB</mappingname>
<datatype>Date</datatype>
<size></size>
</attribute>
</attributes>
Transformed XML
<?xml version="1.0" encoding="utf-8" ?>
<contacts>
<contact>
<ID>B00085590</ID>
<LastName>Brady</LastName>
<DOB>02/21/1991</DOB>
</contact>
<contact>
<ID>B00058478</ID>
<LastName>Balderas</LastName>
<DOB>11/24/1997</DOB>
</contact>
</contacts>
C#
XsltSettings settings = new XsltSettings(true, true);
XslCompiledTransform ContactsXslt = new XslCompiledTransform();
ContactsXslt.Load(#"C:\Projects\ContactFormat.xslt", settings, new XmlUrlResolver());
using (XmlReader r = XmlReader.Create(#"C:\Projects\Contacts.xml")){
using (XmlWriter w = XmlWriter.Create(#"C:\Projects\FormattedContacts.xml")) {
w.WriteStartElement("contacts");
while (r.Read()) {
if (r.NodeType == XmlNodeType.Element && r.Name == "contact") {
XmlReader temp = new XmlTextReader(new StringReader(r.ReadOuterXml()));
ContactsXslt.Transform(temp, null, w);
}
}
}
}
The approach I'm taking is transforming 1 node at a time to avoid an OutOfMemoryException. Should I be feeding larger chunks through to speed up the process? Or am I going about this all wrong?

I think you can simplify the XSLT code
<xsl:for-each select="attribute">
<!--Create variable to hold New Name after passing the Data Name to the Lookup Template-->
<xsl:variable name="newName">
<xsl:apply-templates select="$lookupDoc/attributes/attribute">
<xsl:with-param name="nameToMatch" select="name" />
</xsl:apply-templates>
</xsl:variable>
using the template
<xsl:template match="attributes/attribute">
<xsl:param name="nameToMatch" />
<xsl:value-of select='translate(translate(self::node()[name = $nameToMatch]/mappingname, "()*%$##!~<>&apos;&,.?[]=-+/\:1234567890", "")," ","")' />
</xsl:template>
to
<xsl:for-each select="attribute">
<!--Create variable to hold New Name after passing the Data Name to the Lookup Template-->
<xsl:variable name="newName">
<xsl:apply-templates select="$lookupDoc/attributes/attribute[name = current()/name]"/>
</xsl:variable>
with the template being simplified to
<xsl:template match="attributes/attribute">
<xsl:value-of select='translate(translate(mappingname, "()*%$##!~<>&apos;&,.?[]=-+/\:1234567890", "")," ","")' />
</xsl:template>
I think that for sure is a more concise and XSLT way of expressing the approach, whether it improves performance is something you would have to test.
In general with XSLT to improve performance of cross-references/lookups it is recommended to use a key so you would use
<xsl:key name="att-lookup" match="attributes/attribute" use="name"/>
and then use it as
<xsl:variable name="name" select="name"/>
<xsl:variable name="newName">
<!-- in XSLT 1 we need to change the context doc for the key lookup -->
<xsl:for-each select="$lookupDoc">
<xsl:apply-templates select="key('att-lookup', $name)"/>
</xsl:variable>
I think that would considerable speed up the lookup in a single transformation, as you combine XmlReader and XSLT to run the XSLT many times on as many elements your XmlReader finds I can't tell whether it helps a lot, you would need to try.
As pointed out in the XSLT 3 suggestion, I would also consider transforming the lookup file first and once to avoid the repetition of all those translate calls to create proper XML element names. Either do that outside of the existing XSLT or do it inside by using a variable and then exsl:node-set to convert the result tree fragment into a variable. But in your case as you run the XSLT repeatedly I think it is probably better to first transform the lookup document outside of the main XSLT, to avoid having to do all those translates again and again.

When reading huge xml files always use XmlReader. I like using a combination of XmlReader and Xml linq. I also like using dictionaries. See code below :
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
namespace ConsoleApplication1
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
XmlReader reader = XmlReader.Create(FILENAME);
while (!reader.EOF)
{
if (reader.Name != "contact")
{
reader.ReadToFollowing("contact");
}
if (!reader.EOF)
{
XElement xContact = (XElement)XElement.ReadFrom(reader);
Contact newContact = new Contact();
Contact.contacts.Add(newContact);
newContact.attributes = xContact.Descendants("attribute")
.GroupBy(x => (string)x.Element("name"), y => (string)y.Element("value"))
.ToDictionary(x => x.Key, y => y.FirstOrDefault());
}
}
}
}
public class Contact
{
public static List<Contact> contacts = new List<Contact>();
public Dictionary<string, string> attributes { get; set; }
}
}

As an alternative, you might want to look into solving the task with XSLT 3 and its streaming feature (https://www.w3.org/TR/xslt-30/#streaming-concepts) as there you could process the huge input file in a forwards only but declarative way where you only in the template for the attribute element you need to ensure you work with a intentionally created full copy of that element to allow XPath navigation to the child elements. Additionally I think it makese sense to read in the lookup document only once and do the translate calls to create the proper element names only once. So the following is a streaming XSLT 3 solution runnable with Saxon 9.8 EE which transforms the lookup document into an XPath 3.1 map (https://www.w3.org/TR/xpath-31/#id-maps) and otherwise uses a streamable mode to process the large, main input:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:map="http://www.w3.org/2005/xpath-functions/map"
exclude-result-prefixes="xs map"
version="3.0">
<!-- could of course load the document using select="document('lookup.xml')" instead of inlining it as done here just for the example and testing -->
<xsl:param name="lookup-doc">
<attributes>
<attribute>
<name>text12</name>
<mappingname>ID</mappingname>
<datatype>Varchar2</datatype>
<size>30</size>
</attribute>
<attribute>
<name>text34</name>
<mappingname>Last Name</mappingname>
<datatype>Varchar2</datatype>
<size>30</size>
</attribute>
<attribute>
<name>date866</name>
<mappingname>DOB</mappingname>
<datatype>Date</datatype>
<size></size>
</attribute>
</attributes>
</xsl:param>
<xsl:variable
name="lookup-map"
as="map(xs:string, xs:string)"
select="map:merge(
$lookup-doc/attributes/attribute
!
map {
string(name) : translate(translate(mappingname, '()*%$##!~<>''&,.?[]=-+/\:1234567890', ''), ' ','')
}
)"/>
<xsl:mode on-no-match="shallow-copy" streamable="yes"/>
<xsl:output method="xml" indent="yes"/>
<xsl:template match="contact/attribute">
<xsl:variable name="attribute-copy" select="copy-of()"/>
<xsl:element name="{$lookup-map($attribute-copy/name)}">
<xsl:value-of select="$attribute-copy/value"/>
</xsl:element>
</xsl:template>
</xsl:stylesheet>
Online sample (there running with Saxon 9.8 HE which ignores the streaming and does normal XSLT processing) is at https://xsltfiddle.liberty-development.net/bFDb2Ct/1.
To run streaming XSLT 3 with Saxon 9.8 and C# you use http://saxonica.com/html/documentation/dotnetdoc/Saxon/Api/Xslt30Transformer.html and set up ApplyTemplates on an input Stream with your huge input XML (http://saxonica.com/html/documentation/dotnetdoc/Saxon/Api/Xslt30Transformer.html#ApplyTemplates(System.IO.Stream,Saxon.Api.XmlDestination)).

Related

How can I replace xmlns namespace attributes with prefixes?

I have been trying to write a utility in C# which takes an XML file, removes the xmlns attributes from the tags, sets the prefix of these attributes in the root tag, and then uses these prefixes in the tags instead.
Source XML file:
<?xml version="1.0" encoding="utf-8"?>
<Main version="1.0" xmlns="urn:root:v1">
<Report>
<Title>Some Value</Title>
</Report>
<Content>
<Address>
<CountryName xmlns="urn:location:v2">Australia</CountryName>
</Address>
</Content>
</Main>
Target XML file:
<?xml version="1.0" encoding="utf-8"?>
<root:Main version="1.0" xmlns:root="urn:root:v1" xmlns:loc="urn:location:v2">
<root:Report>
<root:Title>Some Value</root:Title>
</root:Report>
<root:Content>
<root:Address>
<loc:CountryName>Australia</loc:CountryName>
</root:Address>
</root:Content>
</root:Main>
I've managed to get part of the way there with the following code. I have replaced all tags with no attributes with the root prefix, and added the xmlns attribute to the root tag, but have not been successful with removing the xmlns attribute from CountryName tag and using the prefix there instead.
XDocument doc = XDocument.Load(#"C:\Temp\Source.xml");
var content = XElement.Parse(doc.ToString());
content.Attributes("xmlns").Remove();
content.Add(new XAttribute(XNamespace.Xmlns + "root", "urn:root:v1"));
content.Add(new XAttribute(XNamespace.Xmlns + "loc", "urn:location:v2"));
foreach (var node in doc.Root.Descendants().Where(n => n.Name.NamespaceName == "urn:location:v2"))
{
node.Attribute("xmlns").Remove();
node.Add(new XAttribute(XNamespace.Xmlns + "loc", "urn:location:v2"));
}
content.Save(#"C:\Temp\Target.xml");
Any help would be appreciated - thanks!
You're not a million miles away. All you need to do is remove any existing namespace declaration attributes and then add the ones you want to the root. The rest will be taken care of.
var doc = XDocument.Load(#"C:\Temp\Source.xml");
doc.Descendants().Attributes().Where(x => x.IsNamespaceDeclaration).Remove();
doc.Root.Add(new XAttribute(XNamespace.Xmlns + "root", "urn:root:v1"));
doc.Root.Add(new XAttribute(XNamespace.Xmlns + "loc", "urn:location:v2"));
doc.Save(#"C:\Temp\Target.xml");
See this fiddle for a demo.
Consider XSLT, the special-purpose language designed to transform XML files. While I personally do not know or use C#, I do know it can run XSLT 1.0 scripts. See answers here. Also, the XSLT processor you choose to use must allow the document() function for this solution.
XSLT (save as .xsl file; notice namespaces declared in header)
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:root="urn:root:v1" xmlns:local="urn:location:v2">
<xsl:output omit-xml-declaration="no" indent="yes"/>
<xsl:strip-space elements="*"/>
<xsl:template match="node()|#*">
<xsl:copy>
<xsl:apply-templates select="node()|#*"/>
</xsl:copy>
</xsl:template>
<xsl:template match="*">
<xsl:element name="root:{name()}" namespace="urn:root:v1">
<xsl:copy-of select="document('')/*/namespace::local"/>
<xsl:apply-templates select="node()|#*"/>
</xsl:element>
</xsl:template>
<xsl:template match="*[local-name()='CountryName']">
<xsl:element name="local:{name()}" namespace="urn:location:v2">
<xsl:apply-templates select="node()|#*"/>
</xsl:element>
</xsl:template>
</xsl:stylesheet>
C# (see link above)
var myXslTrans = new XslCompiledTransform();
myXslTrans.Load("XSLTScript.xsl");
myXslTrans.Transform("Input.xml", "Output.xml");
XML Output
<?xml version="1.0"?>
<root:Main xmlns:root="urn:root:v1" xmlns:local="urn:location:v2" version="1.0">
<root:Report>
<root:Title>Some Value</root:Title>
</root:Report>
<root:Content>
<root:Address>
<local:CountryName>Australia</local:CountryName>
</root:Address>
</root:Content>
</root:Main>

Append template from a XSLT into another XSLT

I am working in Visual Studio 2012 with C#.
I have two xslt files.
One has a few templates.
Another has some nodes defined there.
All i want is to build a function in C# using which i pass the template name. Using that name it search in the one xslt and if there is a template with the given name, it copies it over into the second xslt.
F("GetMonth") should result as following:
XSLT1:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template name="GetMonth">
<xsl:param name="Month"/>
<xsl:param name="PutCall"/>
<xsl:value-of select ="'A'"/>
</xsl:template>
XSLT2:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
<DocumentElement>
// Some Code written
</DocumentElement>
</xsl:template>
</xsl:stylesheet>
Resultant XSLT2:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template name="GetMonth">
<xsl:param name="Month"/>
<xsl:param name="PutCall"/>
<xsl:value-of select ="'A'"/>
</xsl:template>
<xsl:template match="/">
<DocumentElement>
// Some Tags defined here
</DocumentElement>
</xsl:template>
</xsl:stylesheet>
My attempt:
XmlDocument xslDoc1 = new XmlDocument();
XmlDocument xslDoc2 = new XmlDocument();
xslDoc1.Load("XSLT1.xslt");
xslDoc2.Load("XSLT2.xslt");
XmlNamespaceManager nsMg1r = new XmlNamespaceManager(xslDoc1.NameTable);
nsMgr1.AddNamespace("xsl", "http://www.w3.org/1999/XSL/Transform");
XmlNamespaceManager nsMgr2 = new XmlNamespaceManager(xslDoc2.NameTable);
nsMgr2.AddNamespace("xsl", "http://www.w3.org/1999/XSL/Transform");
XmlNodeList template = (XmlNodeList)xslDoc.SelectNodes("/xsl:stylesheet/xsl:template[#name = templateName]", nsMgr);
if(template != null)
{
// What code should be written here???
}
Please suggest.
string templateName = "GetMonth";
XmlNode template = xslDoc.SelectSingleNode(string.Format("/xsl:stylesheet/xsl:template[#name = '{0}']", templateName), nsMgr);
if (template != null)
{
// will append the template as last child of xsl:stylesheet
xslDoc2.DocumentElement.AppendChild(xslDoc2.ImportNode(template, true));
// as alternative to insert as the first child use
// xslDoc2.DocumentElement.InsertBefore(xslDoc2.ImportNode(template, true), xslDoc2.DocumentElement.FirstChild);
// now Save
xslDoc2.Save("XSLT2.xslt");
}

XML parsing and transforming (XSLT or otherwise)

I have several xml files that are formated this way:
<ROOT>
<OBJECT>
<identity>
<id>123</id>
</identity>
<child2 attr = "aa">32</child2>
<child3>
<childOfChild3 att1="aaa" att2="bbb" att3="CCC">LN</childOfChild3>
</child3>
<child4>
<child5>
<child6>3ddf</child6>
<child7>
<childOfChild7 att31="RR">1231</childOfChild7>
</child7>
</child5>
</child4>
</OBJECT>
<OBJECT>
<identity>
<id>124</id>
</identity>
<child2 attr = "bb">212</child2>
<child3>
<childOfChild3 att1="ee" att2="ccc" att3="EREA">OP</childOfChild3>
</child3>
<child4>
<child5>
<child6>213r</child6>
<child7>
<childOfChild7 att31="EE">1233</childOfChild7>
</child7>
</child5>
</child4>
</OBJECT>
</ROOT>
How can i format it this way?:
<ROOT>
<OBJECT>
<id>123</id>
<child2>32</child2>
<attr>aa</attr>
<child3></child3>
<childOfChild3>LN</childOfChild3>
<att1>aaa</att1>
<att2>bbb</att2>
<att3>CCC</att3>
<child4></child4>
<child5></child5>
<child6>3ddf</child6>
<child7></child7>
<childOfChild7>1231</childOfChild7>
<att31>RR</att31>
</OBJECT>
<OBJECT>
<id>124</id>
<child2>212</child2>
<attr>bb</attr>
<child3></child3>
<childOfChild3>LN</childOfChild3>
<att1>ee</att1>
<att2>ccc</att2>
<att3>EREA</att3>
<child4></child4>
<child5></child5>
<child6>213r</child6>
<child7></child7>
<childOfChild7>1233</childOfChild7>
<att31>EE</att31>
</OBJECT>
</ROOT>
I know some C# so maybe a parser there? or some generic xslt?
The xml files are some data received from a client, so i can't control the way they are sending it to me.
L.E. Basically when i am trying to test this data in excel (for example i want to make sure that the attribute of childOfChild7 corresponds to the correct identity id) i am getting a lot of blank spaces. If i am importing in access to get only the data i want out, i have to do a thousands subqueries to get them all in a nice table. Basically i just want to see for one Object all its data (one object - One row) and then just delete/hide the columns i don't need.
Here is a pure XSLT 1.0 solution:
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output omit-xml-declaration="yes" indent="yes"/>
<xsl:strip-space elements="*"/>
<xsl:template match="node()|#*">
<xsl:copy>
<xsl:apply-templates select="node()|#*"/>
</xsl:copy>
</xsl:template>
<xsl:template match="OBJECT//*[not(self::identity) and text()]">
<xsl:copy>
<xsl:apply-templates select="text()"/>
</xsl:copy>
<xsl:apply-templates select="#* | node()[not(self::text())]"/>
</xsl:template>
<xsl:template match="OBJECT//*[not(self::identity) and (not(text()))]">
<xsl:copy/>
<xsl:apply-templates select="#*"/>
<xsl:apply-templates select="node()"/>
</xsl:template>
<xsl:template match="#*">
<xsl:element name="{name()}">
<xsl:value-of select="."/>
</xsl:element>
</xsl:template>
<xsl:template match="identity">
<xsl:apply-templates/>
</xsl:template>
</xsl:stylesheet>
When this transformation is applied on the provided XML document:
<ROOT>
<OBJECT>
<identity>
<id>123</id>
</identity>
<child2 attr = "aa">32</child2>
<child3>
<childOfChild3 att1="aaa" att2="bbb" att3="CCC">LN</childOfChild3>
</child3>
<child4>
<child5>
<child6>3ddf</child6>
<child7>
<childOfChild7 att31="RR">1231</childOfChild7>
</child7>
</child5>
</child4>
</OBJECT>
<OBJECT>
<identity>
<id>124</id>
</identity>
<child2 attr = "bb">212</child2>
<child3>
<childOfChild3 att1="ee" att2="ccc" att3="EREA">OP</childOfChild3>
</child3>
<child4>
<child5>
<child6>213r</child6>
<child7>
<childOfChild7 att31="EE">1233</childOfChild7>
</child7>
</child5>
</child4>
</OBJECT>
</ROOT>
the wanted, correct result is produced:
<ROOT>
<OBJECT>
<id>123</id>
<child2>32</child2>
<attr>aa</attr>
<child3/>
<childOfChild3>LN</childOfChild3>
<att1>aaa</att1>
<att2>bbb</att2>
<att3>CCC</att3>
<child4/>
<child5/>
<child6>3ddf</child6>
<child7/>
<childOfChild7>1231</childOfChild7>
<att31>RR</att31>
</OBJECT>
<OBJECT>
<id>124</id>
<child2>212</child2>
<attr>bb</attr>
<child3/>
<childOfChild3>OP</childOfChild3>
<att1>ee</att1>
<att2>ccc</att2>
<att3>EREA</att3>
<child4/>
<child5/>
<child6>213r</child6>
<child7/>
<childOfChild7>1233</childOfChild7>
<att31>EE</att31>
</OBJECT>
</ROOT>
You do it by flattening it out. You take all the descendants of the OBJECTS and turn them into elements. You should really try this for yourself instead of just accepting my code, but it works so you can test it vs what you come up with.
XElement root1 = XElement.Load(file1);
XElement root = new XElement("ROOT",
root1.Elements()
.Select(o => new XElement(o.Name, o
.Descendants()
.Select(x =>
{
List<XElement> list = new List<XElement>();
list.Add(new XElement(x.Name, x.HasElements ? "" : x.Value));
if (x.HasAttributes)
list.AddRange(x.Attributes()
.Select(a => new XElement(a.Name, a.Value))
);
return list;
})
))
.ToArray());
PS. You forgot <identity></identity> in your result set.
Just use XmlSerializer to deserialize it into a class, here is an msdn post on it, basically you build a class that matches your xml structure and then let the XmlSerializer class do the conversion, you shouldn't need to create your own parser
I am not familiar with C# but I am hoping this will atleast get you started. I had to do something similar, where I would get a XML file and parse the information into a database.
I used a Java lib ~ StAX. It will allow you read the XML file and parse the information into another XML file quite easily using the parent - child system. I hope this helped a little.

Comparing two XML files & generating a third with XMLDiff in C#

I am trying to write a simple algorithm to read two XML files with the exact same nodes and structure but not necessarily the same data inside the child nodes and not the same order. How could I create a simple implementation for creating a third, temporary XML being the differential between the two first ones, using Microsoft's XML Diff .DLL ?
XML Diff on MSDN:
XML Diff and Patch Tool
XML Diff and Patch GUI Tool
sample XML code of the two different XML files to compare:
<?xml version="1.0" encoding="utf-8" ?>
<Stats Date="2011-01-01">
<Player Rank="1">
<Name>Sidney Crosby</Name>
<Team>PIT</Team>
<Pos>C</Pos>
<GP>39</GP>
<G>32</G>
<A>33</A>
<PlusMinus>20</PlusMinus>
<PIM>29</PIM>
</Player>
</Stats>
<?xml version="1.0" encoding="utf-8" ?>
<Stats Date="2011-01-10">
<Player Rank="1">
<Name>Sidney Crosby</Name>
<Team>PIT</Team>
<Pos>C</Pos>
<GP>42</GP>
<G>35</G>
<A>34</A>
<PlusMinus>22</PlusMinus>
<PIM>30</PIM>
</Player>
</Stats>
Result wanted (difference between the two)
<?xml version="1.0" encoding="utf-8" ?>
<Stats Date="2011-01-10">
<Player Rank="1">
<Name>Sidney Crosby</Name>
<Team>PIT</Team>
<Pos>C</Pos>
<GP>3</GP>
<G>3</G>
<A>1</A>
<PlusMinus>2</PlusMinus>
<PIM>1</PIM>
</Player>
</Stats>
In this case, I would probably use XSLT to convert the resulting XML "differential" file into a sorted HTML file, but I am not there yet. All I want to do is to display in the third XML file the difference of every numerical value of each nodes, starting from the "GP" child-node.
C# code I have so far:
private void CompareXml(string file1, string file2)
{
XmlReader reader1 = XmlReader.Create(new StringReader(file1));
XmlReader reader2 = XmlReader.Create(new StringReader(file2));
string diffFile = StatsFile.XmlDiffFilename;
StringBuilder differenceStringBuilder = new StringBuilder();
FileStream fs = new FileStream(diffFile, FileMode.Create);
XmlWriter diffGramWriter = XmlWriter.Create(fs);
XmlDiff xmldiff = new XmlDiff(XmlDiffOptions.IgnoreChildOrder |
XmlDiffOptions.IgnoreNamespaces |
XmlDiffOptions.IgnorePrefixes);
bool bIdentical = xmldiff.Compare(file1, file2, false, diffGramWriter);
diffGramWriter.Close();
// cleaning up after we are done with the xml diff file
File.Delete(diffFile);
}
That's what I have so far, but the results is garbage... note that for each "Player" node, the first three childs have NOT to be compared... How can I implement this?
There are two immediate solutions:
Solution 1.
You can first apply a simple transform to the two documents that will delete the elements that should not be compared. Then, compare the results ing two documents -- exactly with your current code. Here is the transformation:
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output omit-xml-declaration="yes" indent="yes"/>
<xsl:strip-space elements="*"/>
<xsl:template match="node()|#*">
<xsl:copy>
<xsl:apply-templates select="node()|#*"/>
</xsl:copy>
</xsl:template>
<xsl:template match="Name|Team|Pos"/>
</xsl:stylesheet>
When this transformation is applied to the provided XML document:
<Stats Date="2011-01-01">
<Player Rank="1">
<Name>Sidney Crosby</Name>
<Team>PIT</Team>
<Pos>C</Pos>
<GP>39</GP>
<G>32</G>
<A>33</A>
<PlusMinus>20</PlusMinus>
<PIM>29</PIM>
<PP>10</PP>
<SH>1</SH>
<GW>3</GW>
<Shots>0</Shots>
<ShotPctg>154</ShotPctg>
<TOIPerGame>20.8</TOIPerGame>
<ShiftsPerGame>21:54</ShiftsPerGame>
<FOWinPctg>22.6</FOWinPctg>
</Player>
</Stats>
the wanted resulting document is produced:
<Stats Date="2011-01-01">
<Player Rank="1">
<GP>39</GP>
<G>32</G>
<A>33</A>
<PlusMinus>20</PlusMinus>
<PIM>29</PIM>
<PP>10</PP>
<SH>1</SH>
<GW>3</GW>
<Shots>0</Shots>
<ShotPctg>154</ShotPctg>
<TOIPerGame>20.8</TOIPerGame>
<ShiftsPerGame>21:54</ShiftsPerGame>
<FOWinPctg>22.6</FOWinPctg>
</Player>
</Stats>
Solution 2.
This is a complete XSLT 1.0 solution (for convenience only, the second XML document is embedded in the transformation code):
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output omit-xml-declaration="yes" indent="yes"/>
<xsl:strip-space elements="*"/>
<xsl:variable name="vrtfDoc2">
<Stats Date="2011-01-01">
<Player Rank="2">
<Name>John Smith</Name>
<Team>NY</Team>
<Pos>D</Pos>
<GP>38</GP>
<G>32</G>
<A>33</A>
<PlusMinus>15</PlusMinus>
<PIM>29</PIM>
<PP>10</PP>
<SH>1</SH>
<GW>4</GW>
<Shots>0</Shots>
<ShotPctg>158</ShotPctg>
<TOIPerGame>20.8</TOIPerGame>
<ShiftsPerGame>21:54</ShiftsPerGame>
<FOWinPctg>22.6</FOWinPctg>
</Player>
</Stats>
</xsl:variable>
<xsl:variable name="vDoc2" select=
"document('')/*/xsl:variable[#name='vrtfDoc2']/*"/>
<xsl:template match="node()|#*" name="identity">
<xsl:param name="pDoc2"/>
<xsl:copy>
<xsl:apply-templates select="node()|#*">
<xsl:with-param name="pDoc2" select="$pDoc2"/>
</xsl:apply-templates>
</xsl:copy>
</xsl:template>
<xsl:template match="/">
<xsl:apply-templates select="*">
<xsl:with-param name="pDoc2" select="$vDoc2"/>
</xsl:apply-templates>
-----------------------
<xsl:apply-templates select="$vDoc2">
<xsl:with-param name="pDoc2" select="/*"/>
</xsl:apply-templates>
</xsl:template>
<xsl:template match="Player/*">
<xsl:param name="pDoc2"/>
<xsl:if test=
"not(. = $pDoc2/*/*[name()=name(current())])">
<xsl:call-template name="identity"/>
</xsl:if>
</xsl:template>
<xsl:template match="Name|Team|Pos" priority="20"/>
</xsl:stylesheet>
when this transformation is applied on the same first document as above, the correct diffgrams are produced:
<Stats Date="2011-01-01">
<Player Rank="1">
<GP>39</GP>
<PlusMinus>20</PlusMinus>
<GW>3</GW>
<ShotPctg>154</ShotPctg>
</Player>
</Stats>
-----------------------
<Stats xmlns:xsl="http://www.w3.org/1999/XSL/Transform" Date="2011-01-01">
<Player Rank="2">
<GP>38</GP>
<PlusMinus>15</PlusMinus>
<GW>4</GW>
<ShotPctg>158</ShotPctg>
</Player>
</Stats>
How this works:
The transformation is applied on the first document, passing the second document as parameter.
This produces an XML document whose only leaf element nodes are the ones that have different value than the corresponding leaf element nodes in the second document.
The same processing is performed as in 1. above, but this time on the second document, passing the first document as parameter.
This produces a second diffgram: an XML document whose only leaf element nodes are the ones that have different value** than the corresponding leaf element nodes in the first document
Okay... I finally opted with a pure C# solution to compare the two XML files, without using the XML Diff/Patch .dll and without even needing to use XSL transforms. I will be needing XSL transforms in the next step though, to convert the Xml into HTML for viewing purposes, but I have figured an algorithm using nothing but System.Xml and System.Xml.XPath.
Here is my algorithm:
private void CompareXml(string file1, string file2)
{
// Load the documents
XmlDocument docXml1 = new XmlDocument();
docXml1.Load(file1);
XmlDocument docXml2 = new XmlDocument();
docXml2.Load(file2);
// Get a list of all player nodes
XmlNodeList nodes1 = docXml1.SelectNodes("/Stats/Player");
XmlNodeList nodes2 = docXml2.SelectNodes("/Stats/Player");
// Define a single node
XmlNode node1;
XmlNode node2;
// Get the root Xml element
XmlElement root1 = docXml1.DocumentElement;
XmlElement root2 = docXml2.DocumentElement;
// Get a list of all player names
XmlNodeList nameList1 = root1.GetElementsByTagName("Name");
XmlNodeList nameList2 = root2.GetElementsByTagName("Name");
// Get a list of all teams
XmlNodeList teamList1 = root1.GetElementsByTagName("Team");
XmlNodeList teamList2 = root2.GetElementsByTagName("Team");
// Create an XmlWriterSettings object with the correct options.
XmlWriter writer = null;
XmlWriterSettings settings = new XmlWriterSettings();
settings.Indent = true;
settings.IndentChars = (" ");
settings.OmitXmlDeclaration = false;
// Create the XmlWriter object and write some content.
writer = XmlWriter.Create(StatsFile.XmlDiffFilename, settings);
writer.WriteStartElement("StatsDiff");
// The compare algorithm
bool match = false;
int j = 0;
try
{
// the list has 500 players
for (int i = 0; i < 500; i++)
{
while (j < 500 && match == false)
{
// There is a match if the player name and team are the same in both lists
if (nameList1.Item(i).InnerText == nameList2.Item(j).InnerText)
{
if (teamList1.Item(i).InnerText == teamList2.Item(j).InnerText)
{
match = true;
node1 = nodes1.Item(i);
node2 = nodes2.Item(j);
// Call to the calculator and Xml writer
this.CalculateDifferential(node1, node2, writer);
j = 0;
}
}
else
{
j++;
}
}
match = false;
}
// end Xml document
writer.WriteEndElement();
writer.Flush();
}
finally
{
if (writer != null)
writer.Close();
}
}
XML Results:
<?xml version="1.0" encoding="utf-8"?>
<StatsDiff>
<Player Rank="1">
<Name>Sidney Crosby</Name>
<Team>PIT</Team>
<Pos>C</Pos>
<GP>0</GP>
<G>0</G>
<A>0</A>
<Points>0</Points>
<PlusMinus>0</PlusMinus>
<PIM>0</PIM>
<PP>0</PP>
<SH>0</SH>
<GW>0</GW>
<OT>0</OT>
<Shots>0</Shots>
<ShotPctg>0</ShotPctg>
<ShiftsPerGame>0</ShiftsPerGame>
<FOWinPctg>0</FOWinPctg>
</Player>
<Player Rank="2">
<Name>Steven Stamkos</Name>
<Team>TBL</Team>
<Pos>C</Pos>
<GP>1</GP>
<G>0</G>
<A>0</A>
<Points>0</Points>
<PlusMinus>0</PlusMinus>
<PIM>2</PIM>
<PP>0</PP>
<SH>0</SH>
<GW>0</GW>
<OT>0</OT>
<Shots>4</Shots>
<ShotPctg>-0,6000004</ShotPctg>
<ShiftsPerGame>-0,09999847</ShiftsPerGame>
<FOWinPctg>0,09999847</FOWinPctg>
</Player>
[...]
</StatsDiff>
I have spared to show the implementation for the CalculateDifferential() method, it is rather cryptic but it is fast and efficient. This way I could obtain the results wanted without using any other reference but the strict minimum, without having to use XSL...

C#: How to remove namespace information from XML elements

How can I remove the "xmlns:..." namespace information from each XML element in C#?
Zombiesheep's cautionary answer notwithstanding, my solution is to wash the xml with an xslt transform to do this.
wash.xsl:
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="no" encoding="UTF-8"/>
<xsl:template match="/|comment()|processing-instruction()">
<xsl:copy>
<xsl:apply-templates/>
</xsl:copy>
</xsl:template>
<xsl:template match="*">
<xsl:element name="{local-name()}">
<xsl:apply-templates select="#*|node()"/>
</xsl:element>
</xsl:template>
<xsl:template match="#*">
<xsl:attribute name="{local-name()}">
<xsl:value-of select="."/>
</xsl:attribute>
</xsl:template>
</xsl:stylesheet>
From here http://simoncropp.com/working-around-xml-namespaces
var xDocument = XDocument.Parse(
#"<root>
<f:table xmlns:f=""http://www.w3schools.com/furniture"">
<f:name>African Coffee Table</f:name>
<f:width>80</f:width>
<f:length>120</f:length>
</f:table>
</root>");
xDocument.StripNamespace();
var tables = xDocument.Descendants("table");
public static class XmlExtensions
{
public static void StripNamespace(this XDocument document)
{
if (document.Root == null)
{
return;
}
foreach (var element in document.Root.DescendantsAndSelf())
{
element.Name = element.Name.LocalName;
element.ReplaceAttributes(GetAttributes(element));
}
}
static IEnumerable GetAttributes(XElement xElement)
{
return xElement.Attributes()
.Where(x => !x.IsNamespaceDeclaration)
.Select(x => new XAttribute(x.Name.LocalName, x.Value));
}
}
I had a similar problem (needing to remove a namespace attribute from a particular element, then return the XML as an XmlDocument to BizTalk) but a bizarre solution.
Before loading the XML string into the XmlDocument object, I did a text replacement to remove the offending namespace attribute. It seemed wrong at first as I ended up with XML that could not be parsed by the "XML Visualizer" in Visual Studio. This is what initially put me off this approach.
However, the text could still be loaded into the XmlDocument and I could output it to BizTalk fine.
Note too that earlier, I hit one blind alley when trying to use childNode.Attributes.RemoveAll() to remove the namespace attribute - it just came back again!

Categories