We Keep Coding

How can I create a XML file containing the difference in value between another 2 XML files

Hello there C# enthusiast!
I am trying to create an XML file based on the elements subtraction of another two xml files ( the difference in value of elements)
For example:
The Source Xml files
Sought output:
Expected output
I did a quick research and found that there is a lib for XML content difference but none that fulfill this
functionality.
I know this can be solved by System.xml class , but I don't know exactly how to start.
Any help is appreciated.
XML Text Sample:
<?xml version="1.0" encoding="utf-8"?>
<Report>
<Project>
<Name>P1</Name>
<Runs>10</Runs>
<Errors>5</Errors>
<Successful>5</Successful>
</Project>
.
. Multiple Projects exists here
.
<Project>
<Name>P2</Name>
<Runs>12</Runs>
<Errors>3</Errors>
<Successful>9</Successful>
</Project>
<Timestamp>
<Year>2020</Year>
<Month>6</Month>
<Day>8</Day>
<Hour>12</Hour>
</Timestamp>
</Report>

Try xml linq. I assumed you had same Projects in both input and output files. If not you will need a left outer join. :
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
namespace ConsoleApplication3
{
class Program
{
const string INPUT_FILENAME_1 = #"c:\temp\test.xml";
const string INPUT_FILENAME_2 = #"c:\temp\test1.xml";
const string OUTPUT_FILENAME = #"c:\temp\test.xml2";
static void Main(string[] args)
{
XDocument doc1 = XDocument.Load(INPUT_FILENAME_1);
DateTime date1 = doc1.Descendants("Timestamp")
.Select(x => new DateTime(
(int)x.Element("Year"),
(int)x.Element("Month"),
(int)x.Element("Day"),
(int)x.Element("Hour"),
0, 0)).FirstOrDefault();
XDocument doc2 = XDocument.Load(INPUT_FILENAME_2);
DateTime date2 = doc2.Descendants("Timestamp")
.Select(x => new DateTime(
(int)x.Element("Year"),
(int)x.Element("Month"),
(int)x.Element("Day"),
(int)x.Element("Hour"),
0, 0)).FirstOrDefault();
string ident = "<?xml version=\"1.0\" encoding=\"utf-8\"?><Report></Report>";
XDocument outputDoc = XDocument.Parse(ident);
XElement outputReport = outputDoc.Root;
XElement oldReport;
XElement newReport;
TimeSpan deltaTime;
if (date2 > date1)
{
newReport = doc2.Root;
oldReport = doc1.Root;
deltaTime = date2.Subtract(date1);
}
else
{
newReport = doc1.Root;
oldReport = doc2.Root;
deltaTime = date1.Subtract(date2);
}
var groups = (from n in newReport.Elements("Project")
join o in oldReport.Elements("Project")
on (string)n.Element("Name") equals (string)o.Element("Name")
select new { oldProj = o, newProj = n }
).ToList();
foreach (var group in groups)
{
XElement difference = new XElement("Project", new object[] {
new XElement("Name", (string)group.newProj.Element("Name")),
new XElement("Runs", (int)group.newProj.Element("Runs") - (int)group.oldProj.Element("Runs")),
new XElement("Errors", (int)group.newProj.Element("Errors") - (int)group.oldProj.Element("Errors")),
new XElement("Successful", (int)group.newProj.Element("Successful") - (int)group.oldProj.Element("Successful"))
});
outputReport.Add(difference);
}
XElement newTime = new XElement("Timestamp", new object[] {
new XElement("Year", 0),
new XElement("Month", 0),
new XElement("Day", deltaTime.Days),
new XElement("Hour", deltaTime.Hours)
});
outputReport.Add(newTime);
outputDoc.Save(OUTPUT_FILENAME);
}
}
}

How to get the node value by passing type in XDocument C#

I have below XML.
<subscription>
<subscription_add_ons type="array">
<subscription_add_on>
<add_on_code>premium_support</add_on_code>
<name>Premium Support</name>
<quantity type="integer">1</quantity>
<unit_amount_in_cents type="integer">15000</unit_amount_in_cents>
<add_on_type>fixed</add_on_type>
<usage_percentage nil="true"></usage_percentage>
<measured_unit_id nil="true"></measured_unit_id>
</subscription_add_on>
</subscription_add_ons>
My XMLParse function
public XNode GetXmlNodes(XElement xml, string elementName)
{
List<string> addOnCodes= new List<string>();
//elementName = "subscription_add_ons ";
var addOns = xml.DescendantNodes().Where(x => x.Parent.Name == elementName).FirstOrDefault();
foreach (XNode addOn in addOns)
{
//Needed to do something like this
/*var len = "add_on_code".Length + 2;
var sIndex = addOn.ToString().IndexOf("<add_on_code>") + len;
var eIndex = addOn.ToString().IndexOf("</add_on_code>");
var addOnCode = addOn.ToString().Substring(sIndex, (eIndex - sIndex)).Trim().ToLower();
addOnCodes.Add(addOnCode);*/
}
As mentioned in comments by #JonSkeet, I updated my snippet as below.
var addOns = xml.Descendants(elementName).Single().Elements();
foreach (XNode addOn in addOns)
{
/*addon = {<subscription_add_on>
<add_on_code>premium_support</add_on_code>
<name>Premium Support</name>
<quantity type="integer">1</quantity>
<unit_amount_in_cents type="integer">15000</unit_amount_in_cents>
<add_on_type>fixed</add_on_type>
<usage_percentage nil="true"></usage_percentage>
<measured_unit_id nil="true"></measured_unit_id>
</subscription_add_on>} */
//how to get the addOnCode node value ?
var addOnCode = string.Empty;
addOnCodes.Add(addOnCode);
}
But what I need is from the passed XML, get all the nodes of type subscription_add_on then get the value contained in add_on_code & add it to string collection.
Or in general get the value of node by passing type ? Tried with the available methods coming from VS Intellisense but not getting the exact method that can do this?
Thanks!

Here is solution with Xml Linq (XDOCUMENT) :
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
namespace ConsoleApplication107
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
XDocument doc = XDocument.Load(FILENAME);
var results = doc.Descendants("subscription_add_on").Select(x => new
{
add_on_code = (string)x.Element("add_on_code"),
name = (string)x.Element("name"),
quantity = (int)x.Element("quantity"),
amount = (int)x.Element("unit_amount_in_cents"),
add_on_type = (string)x.Element("add_on_type")
}).ToList();
}
}
}

Web scraping a listings website

I'm trying to scrape a website - ive accomplished this on other projects but i cant seem to get this right. It could be that ive been up for over 2 days working and maybe i am missing something. Please could someone look over my code? Here it is :
using System;
using System.Collections.Generic;
using HtmlAgilityPack;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;
using System.Linq;
using System.Xml.Linq;
using System.IO;
public partial class _Default : System.Web.UI.Page
{
List<string> names = new List<string>();
List<string> address = new List<string>();
List<string> number = new List<string>();
protected void Page_Load(object sender, EventArgs e)
{
string url = "http://www.scoot.co.uk/find/" + "cafe" + " " + "-in-uk?page=" + "4";
var Webget = new HtmlWeb();
var doc = Webget.Load(url);
List<List<string>> mainList = new List<List<string>>();
foreach (HtmlNode node in doc.DocumentNode.SelectNodes("//h2//a"))
{
names.Add(Regex.Replace(node.ChildNodes[0].InnerHtml, #"\s{2,}", " "));
}
foreach (HtmlNode node in doc.DocumentNode.SelectNodes("//p[#class='result-address']"))
{
address.Add(Regex.Replace(node.ChildNodes[0].InnerHtml, #"\s{2,}", " "));
}
foreach (HtmlNode node in doc.DocumentNode.SelectNodes("//p[#class='result-number']"))
{
number.Add(Regex.Replace(node.ChildNodes[0].InnerHtml, #"\s{2,}", " "));
}
XDocument doccy = new XDocument(
new XDeclaration("1.0", "utf-8", "yes"),
new XComment("Business For Sale"),
new XElement("Data",
from data in mainList
select new XElement("data", new XAttribute("data", "data"),
new XElement("Name : ", names[0]),
new XElement("Add : ", address[0]),
new XElement("Number : ", number[0])
)
)
);
var xml = doccy.ToString();
Response.ContentType = "text/xml"; //Must be 'text/xml'
Response.ContentEncoding = System.Text.Encoding.UTF8; //We'd like UTF-8
doccy.Save(Response.Output); //Save to the text-writer
}
}
The website lists business name, phone number and address and they are all defined by a class name (result-address, result-number etc). I am trying to get XML output so i can get the business name, address and phone number from each listing on page 4 for a presentation tomorrow but i cant get it to work at all!
The results are right in all 3 of the for each loops but they wont output in the xml i get an out of range error.

My first piece of advice would be to keep your CodeBehind as light as possible. If you bloat it up with business logic then the solution will become difficult to maintain. That's off topic, but I recommend looking up SOLID principles.
First, I've created a custom object to work with instead of using Lists of strings which have no way of knowing which address item links up with which name:
public class Listing
{
public string Name { get; set; }
public string Address { get; set; }
public string Number { get; set; }
}
Here is the heart of it, a class that does all the scraping and serializing (I've broken SOLID principles but sometimes you just want it to work right.)
using System.Collections.Generic;
using HtmlAgilityPack;
using System.IO;
using System.Xml;
using System.Xml.Serialization;
using System.Linq;
public class TheScraper
{
public List<Listing> DoTheScrape(int pageNumber)
{
List<Listing> result = new List<Listing>();
string url = "http://www.scoot.co.uk/find/" + "cafe" + " " + "-in-uk?page=" + pageNumber;
var Webget = new HtmlWeb();
var doc = Webget.Load(url);
// select top level node, this is the closest we can get to the elements in which all the listings are a child of.
var nodes = doc.DocumentNode.SelectNodes("//*[#id='list']/div/div/div/div");
// loop through each child
if (nodes != null)
{
foreach (var node in nodes)
{
Listing listing = new Listing();
// get each individual listing and manually check for nulls
// listing.Name = node.SelectSingleNode("./div/div/div/div/h2/a")?.InnerText; --easier way to null check if you can use null propagating operator
var nameNode = node.SelectSingleNode("./div/div/div/div/h2/a");
if (nameNode != null) listing.Name = nameNode.InnerText;
var addressNode = node.SelectSingleNode("./div/div/div/div/p[#class='result-address']");
if (addressNode != null) listing.Address = addressNode.InnerText.Trim();
var numberNode = node.SelectSingleNode("./div/div/div/div/p[#class='result-number']/a");
if (numberNode != null) listing.Number = numberNode.Attributes["data-visible-number"].Value;
result.Add(listing);
}
}
// filter out the nulls
result = result.Where(x => x.Name != null && x.Address != null && x.Number != null).ToList();
return result;
}
public string SerializeTheListings(List<Listing> listings)
{
var xmlSerializer = new XmlSerializer(typeof(List<Listing>));
using (var stringWriter = new StringWriter())
using (var xmlWriter = XmlWriter.Create(stringWriter, new XmlWriterSettings { Indent = true }))
{
xmlSerializer.Serialize(xmlWriter, listings);
return stringWriter.ToString();
}
}
}
Then your code behind would look something like this, plus references to the scraper class and model class:
public partial class _Default : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
TheScraper scraper = new TheScraper();
List<Listing> listings = new List<Listing>();
// quick hack to do a loop 5 times, to get all 5 pages. if this is being run frequently you'd want to automatically identify how many pages or start at page one and find / use link to next page.
for (int i = 0; i < 5; i++)
{
listings = listings.Union(scraper.DoTheScrape(i)).ToList();
}
string xmlListings = scraper.SerializeTheListings(listings);
}
}

xml parsing - code refactoring issue

I have the following xml:
<?xml version="1.0" encoding="utf-8"?>
<RootData>
<PassResult>
<FirstOne>P1</FirstOne>
<SecondOne>P2</SecondOne>
<IsMale>false</IsMale>
</PassResult>
<TestResult>
<MarkOne>100</MarkOne>
<MarkTwo>50</MarkTwo>
<Slope>30</Slope>
</TestResult>
<ToneTestResult>
<TotalTime>2</TotalTime>
<CorrectPercentage>90</CorrectPercentage>
</ToneTestResult>
<QuestionnaireResult Version="1">
<Question Id="50">
<Answer Id="B" />
</Question>
<Question Id="60">
<Answer Id="A" />
</Question>
</QuestionnaireResult>
</RootData>
I have the following code which is not at all looking a good one. Lots of repeatative link queries.
How can I refactor this code in a more structured way to fill "OutputData" object. I do not want to change it to XmlSerializer now :-(.
Sample code:
// Gets the root element decendants
var elementRootData = xDocument.Descendants("RootData");
var xElements = elementRootData as IList<XElement> ?? elementRootData.ToList();
// Read first leaf node "ProfileResult"
var passResult = from xElement in xElements.Descendants("PassResult")
select new
{
FirstOne = xElement.Element("FirstOne").GetValue(),
SecondOne = xElement.Element("SecondOne").GetValue(),
IsMale = xElement.Element("IsMale").GetValue()
};
// Read second leaf note
var testResult = from xElement in xElements.Descendants("TestResult")
select new
{
MarkOne = xElement.Element("MarkOne").GetValue(),
MarkTwo = xElement.Element("MarkTwo").GetValue(),
Slope = xElement.Element("Slope").GetValue()
};
// Update OutputData object
var parseOutputData = new OutputData();
foreach (var result in passResult)
{
parseOutputData.FirstOne = result.FirstOne;
parseOutputData.SecondOne = result.SecondOne;
parseOutputData.IsMale = result.IsMale.Equals("True");
}
foreach (var result in testResult)
{
parseOutputData.MarkOne = double.Parse(result.MarkOne);
parseOutputData.MarkTwo = double.Parse(result.MarkTwo);
parseOutputData.Slope = double.Parse(result.Slope);
}
I have to write some more code like this to fill other elements data like ToneTestResult, QuestionnaireResult etc.
Can someone suggest with a sample code?
Best regards,

Given your XML is tiny, you probably don't have to worry too much about performance. You can just do the whole thing in one go, making use of the built in explicit conversions:
var data = new OutputData
{
FirstOne = (string) doc.Descendants("FirstOne").Single(),
SecondOne = (string) doc.Descendants("SecondOne").Single(),
IsMale = (bool) doc.Descendants("IsMale").Single(),
MarkOne = (double) doc.Descendants("MarkOne").Single(),
MarkTwo = (double) doc.Descendants("MarkTwo").Single(),
Slope = (double) doc.Descendants("Slope").Single()
};
As an aside, Descendants will never return anything implementing IList<XElement>, so you can definitely remove that.

Try XML Linq
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
namespace ConsoleApplication1
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
XDocument doc = XDocument.Load(FILENAME);
var root = doc.Elements("RootData").Select(x => new
{
passResults = x.Elements("PassResult").Select(y => new
{
firstOne = (string)y.Element("FirstOne"),
secondOne = (string)y.Element("SecondOne"),
isMale = (Boolean)y.Element("IsMale")
}).FirstOrDefault(),
testResult = x.Elements("TestResult").Select(y => new {
markOne = (int)y.Element("MarkOne"),
markTwo = (int)y.Element("MarkTwo"),
slope = (int)y.Element("Slope")
}).FirstOrDefault(),
toneTestResult = x.Elements("ToneTestResult").Select(y => new {
totalTime = (int)y.Element("TotalTime"),
correctPecentage = (int)y.Element("CorrectPercentage")
}).FirstOrDefault(),
questionnaireResult = x.Elements("QuestionnaireResult").Elements("Question").Select(y => new {
question = (int)y.Attribute("Id"),
answer = (string)y.Descendants("Answer").FirstOrDefault().Attribute("Id")
}).ToList(),
}).FirstOrDefault();
}
}
}

Read a XML file using LINQ

How do I read a XML file (XMLfile.xml) using LINQ in a previously made file in the root of a ASP.net program. Each Element exist in the XML file I have already created (see below for an excerpt).
XDocument xmlElements = XDocument.Parse(System.IO.File.ReadAllText(Server.MapPath("XMLfile.xml")));
var elements = from data in xmlElements.Descendants("/NewDataSet/Table")
select new
{
Number0 = (int)data.Element("Number"),
Name0 = (string)data.Element("Name"),
a0 = (double)data.Element("a"),
e0 = (double)data.Element("e"),
i0 = (double)data.Element("i"),
N0 = (double)data.Element("N"),
w0 = (double)data.Element("w"),
Pyrs0 = (double)data.Element("Pyrs"),
mm0 = (double)data.Element("mm"),
MA0 = (double)data.Element("MA0")
};
foreach (var element in elements)
{
m = m + 1;
num[m] = element.Number0;
nam[m] = element.Name0;
a1[m] = element.a0;
ecc[m] = element.e0;
i[m] = element.i0;
N[m] = element.N0;
w[m] = element.w0;
Pyrs[m] = element.Pyrs0;
mm[m] = element.mm0;
MA0[m] = element.MA0;
}
XMLfile.xml
<?xml version="1.0" standalone="yes"?>
<NewDataSet>
<Table>
<Number>1</Number>
<Name>Ceres</Name>
<a>2.7681117</a>
<e>0.0757544</e>
<i>10.59166</I>
<N>80.3218024</N>
<w>72.73324</w>
<Pyrs>4.61</Pyrs>
<mm>0.2140072</mm>
<MA0>181.38143</MA0>
</Table>
<Table>
<Number>2</Number>
<Name>Pallas</Name>
<a>2.7723622</a>
<e>0.2310236</e>
<i>34.84095</i>
<N>173.0882785</N>
<w>309.98943</w>
<Pyrs>4.62</Pyrs>
<mm>0.2135153</mm>
<MA0>163.60434</MA0>
</Table>
...
</NewDataSet>

What has worked for me is the extension method XPathSelectElements. As in:
var elements = from data in xmlElements.XPathSelectElements("/NewDataSet/Table")
select new
{
Number0 = (int)data.Element("Number"),
Name0 = (string)data.Element("Name"),
a0 = (double)data.Element("a"),
e0 = (double)data.Element("e"),
i0 = (double)data.Element("i"),
N0 = (double)data.Element("N"),
w0 = (double)data.Element("w"),
Pyrs0 = (double)data.Element("Pyrs"),
mm0 = (double)data.Element("mm"),
MA0 = (double)data.Element("MA0")
};

The code below is XML Linq like you requests. XML is case sensitive so you need to change the closing tag "I" to "i". I tested the code below and it works. If you have a string instead of the file then change "Load" to "Parse".
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
namespace ConsoleApplication1
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
XDocument doc = XDocument.Load(FILENAME);
var results = doc.Descendants("Table").Select(x => new
{
number = (int)x.Element("Number"),
name = (string)x.Element("Name"),
a = (double)x.Element("a"),
e = (double)x.Element("e"),
i = (double)x.Element("i"),
n = (double)x.Element("N"),
w = (double)x.Element("w"),
pyrs = (double)x.Element("Pyrs"),
mm = (double)x.Element("mm"),
ma0 = (double)x.Element("MA0")
}).ToList();
}
}
}