twice reading xml file with linq - c#

I am trying to read xml file. and then extract some useful data to draw graphs.. I have achieved the desired output.. But the problem is my program is twice reading the xml file to extract the useful data.. This takes some extra time. Is there some other way to read the file once only. ? Thanks
<?xml version="1.0" encoding="UTF-8"?>
<CanConformanceTesterLog Version="4.1">
<TestProperties>
<Item name="IUT Name" value="Reference"/>
<Item name="PG Clock Period" value="1000 ns"/>
</TestProperties>
<SignalData SamplingPeriod="1000.000 ns" DataWidth="16 bit">
<Signal>
<Id>IUT_RX</Id>
<InitState>1</InitState>
<![CDATA[HQFPAVkBiwGVAZ8BqQHHAdEBAwINAjUCPwJxAnsCrQK3AsEC1QLzAv0CEQMbAzkDTQNrA3UDfwOJA7sDxQPtA/cDKQQzBEcEUQSDBI0EtQTJBN0E5wTxBAUFDwUZBS0FNwVBBUsFVQWHFZEVmxWlFa8VuRXDFc0V1xXhFesV9RX/FTEWOxZFFk8WgRaLFpUWnxapFscW0RbbFuUW7xYDFyEXPxdJF1MXGhgkGC4YTBhWGHQYfhiwGLoY2BjiGBQZHhkoGTIZUBlaGXgZghmgGaoZvhnbGeUZ9RwTHR0dTx1ZHYsdlR29Hccd+R0DHg0eFx5JHlMeZx6ZHsEe6R4lH5Qfsh+8H+4f+B8qIDQgXCBmIJggoiCsILYg6CDyIAYhOCFgIYghxCEzIlEiWyKNIpciySLTIvsiBSM3I0EjSyNVI4cjmyOlI9cj/yMTJB0kpyaxJrsmxSbPJgEnCyc9J0cnZSdvJ6EnqyfdJ+cnGSgjKC0oQShLKF8ocyiHKJsopSivKLkowyjWKOAo8jQkNS41YDVqNZw1pjXENc41ADYKNhQ2HjZGNlA2WjZkNm42eDaWNqo2tDbHNtE2uDd=]]>
</Signal>
<Signal>
<Id>IUT_TX</Id>
<InitState>1</InitState>
<![CDATA[SwVVBYcVkRWbFaUVrxW5FcMVzRXXFeEV6xX1Ff8VMRY7FkUWTxaBFosWlRafFqkWxxbRFtsW5RbvFgMXIRc/FxoYJBguGEwYVhh0GH4YsBi6GNgY4hgUGR4ZKBkyGVAZWhl4GYIZoBmqGb4Z6B4kH4ghxCETJB0kpyaxJrsmxSbPJgEnCyc9J0cnZSdvJ6EnqyfdJ+cnGSgjKC0oQShLKF8ocyiHKJsopSivKLkowyjyNCQ1LjVgNWo1nDWmNcQ1zjUANgo2FDYeNkY2UDZaNmQ2bjZ4NpY2qja0Nrg3]]>
</Signal>
</SignalData>
</CanConformanceTesterLog>
I have function that reads the data of tag "SignalData".then after reading this data it calls another function and pass the name of xml file,dataWidth,samplingPeriod.
The second function then reads "Signal" tag.. and then extract the data from every "Signal". Finally when everything is done then a function is called to draw the graphs...
private bool SignalDataInfo(string fileName)
{
var xdoc = XDocument.Load(fileName);
if (xdoc != null)
{
var signalData = xdoc.Descendants("SignalData");
foreach (var signal in signalData)
{
var width = signal.Attribute("DataWidth").Value;
string dataWidth = width.Substring(0, width.IndexOf(" "));
var period = signal.Attribute("SamplingPeriod").Value;
string samplingPeriod = period.Substring(0, period.IndexOf(" "));
SignalData(fileName,dataWidth, samplingPeriod);
}
return true;
}
else
return false;
}
public bool SignalData(string fileName,string width, string period)
{var xdoc = XDocument.Load(fileName);
if (xdoc != null)
{
var signalData = xdoc.Descendants("Signal");
foreach (var signal in signalData)
{ // extract data from every signal }
return true;
else false;
}

Here I create a sample console app for you to extract data from both of your SignalData and Signal.
I think you would be in search of code like below.
In below code snippet you would use result in your program where you want to read data inside xml.
So by this way you don't need to write two different methods and load your xml each time when your methods called.
class Program
{
static void Main(string[] args)
{
XDocument doc = XDocument.Load(#"Your xml document path");
var result = (from o in doc.Descendants("SignalData")
from i in o.Descendants("Signal")
select new
{
dataWidth = o.Attribute("DataWidth").Value.Substring(0, o.Attribute("DataWidth").Value.IndexOf(" ")),
period = o.Attribute("SamplingPeriod").Value.Substring(0, o.Attribute("SamplingPeriod").Value.IndexOf(" ")),
Id = i.Elements("Id").Select(item => (string)item).FirstOrDefault(),
InitState = i.Elements("InitState").Select(item => (string)item).FirstOrDefault(),
cdata = i.Value
}).ToList();
foreach (var item in result)
{
Console.WriteLine($"dataWidth: { item.dataWidth}, \t period: {item.period}, \t Id: {item.Id}, \t InitState: {item.InitState}");
}
Console.ReadLine();
}
}
Output: cdata excluded from output.

You can just load the XML file once by saving the XDocument that was loaded in a class variable (say private XDocument xDoc;) instead of creating an instance on each method. Also, it would help if you just retrieve the XML data separately, in this case, the loadData() method which you can use to initialize the data once. This will also give your code, somehow, separation of concerns. See code below:
private XDocument xDoc;
private void loadData(string fileName)
{
xDoc = XDocument.Load(fileName);
}
private bool SignalDataInfo()
{
if (xDoc != null)
{
var signalData = xDoc.Descendants("SignalData");
foreach (var signal in signalData)
{
var width = signal.Attribute("DataWidth").Value;
string dataWidth = width.Substring(0, width.IndexOf(" "));
var period = signal.Attribute("SamplingPeriod").Value;
string samplingPeriod = period.Substring(0, period.IndexOf(" "));
SignalData(fileName,dataWidth, samplingPeriod);
}
return true;
}
else
return false;
}
public bool SignalData(string width, string period)
{
if (xDoc != null)
{
var signalData = xDoc.Descendants("Signal");
foreach (var signal in signalData)
{ // extract data from every signal }
return true;
else false;
}
Hope this helps!

Related

How to create a CSV file from a XML file

I am very new at C#. In my project I need to create a csv file which will get data from a xml data. Now, I can get data from XML, and print in looger for some particulaer attributes from xml. But I am not sure how can I store my Data into CSV file for that particular attribues.
Here is my XML file that I need to create a CSV file.
<?xml version="1.0" encoding="utf-8"?>
<tlp:WorkUnits xmlns:tlp="http://www.timelog.com/XML/Schema/tlp/v4_4"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.timelog.com/XML/Schema/tlp/v4_4 http://www.timelog.com/api/xsd/WorkUnitsRaw.xsd">
<tlp:WorkUnit ID="130">
<tlp:EmployeeID>3</tlp:EmployeeID>
<tlp:AllocationID>114</tlp:AllocationID>
<tlp:TaskID>239</tlp:TaskID>
<tlp:ProjectID>26</tlp:ProjectID>
<tlp:ProjectName>LIK Template</tlp:ProjectName>
<tlp:CustomerId>343</tlp:CustomerId>
<tlp:CustomerName>Lekt Corp Inc.</tlp:CustomerName>
<tlp:IsBillable>1</tlp:IsBillable>
<tlp:ApprovedStatus>0</tlp:ApprovedStatus>
<tlp:LastModifiedBy>AL</tlp:LastModifiedBy>
</tlp:WorkUnit>
And my Code where I am getting this value in logger.But I am not sure how can I create a csv file that stores that value in order.
Edited
namespace TimeLog.ApiConsoleApp
{
/// <summary>
/// Template class for consuming the reporting API
/// </summary>
public class ConsumeReportingApi
{
private static readonly ILog Logger = LogManager.GetLogger(typeof(ConsumeReportingApi));
public static void Consume()
{
if (ServiceHandler.Instance.TryAuthenticate())
{
if (Logger.IsInfoEnabled)
{
Logger.Info("Successfully authenticated on reporting API");
}
var customersRaw = ServiceHandler.Instance.Client.GetWorkUnitsRaw(ServiceHandler.Instance.SiteCode,
ServiceHandler.Instance.ApiId,
ServiceHandler.Instance.ApiPassword,
WorkUnit.All,
Employee.All,
Allocation.All,
Task.All,
Project.All,
Department.All,
DateTime.Now.AddDays(-5).ToString(),
DateTime.Now.ToString()
);
if (customersRaw.OwnerDocument != null)
{
var namespaceManager = new XmlNamespaceManager(customersRaw.OwnerDocument.NameTable);
namespaceManager.AddNamespace("tlp", "http://www.timelog.com/XML/Schema/tlp/v4_4");
var workUnit = customersRaw.SelectNodes("tlp:WorkUnit", namespaceManager);
var output = new StringBuilder();
output.AppendLine("AllocationID,ApprovedStatus,CustomerId,CustomerName,EmployeeID");
if (workUnit != null)
{
foreach (XmlNode customer in workUnit)
{
var unit = new WorkUnit();
var childNodes = customer.SelectNodes("./*");
if (childNodes != null)
{
foreach (XmlNode childNode in childNodes)
{
if (childNode.Name == "tlp:EmployeeID")
{
unit.EmployeeID = Int32.Parse(childNode.InnerText);
}
if (childNode.Name == "tlp:EmployeeFirstName")
{
unit.EmployeeFirstName = childNode.InnerText;
}
if (childNode.Name == "tlp:EmployeeLastName")
{
unit.EmployeeLastName = childNode.InnerText;
}
if (childNode.Name == "tlp:AllocationID")
{
unit.AllocationID = Int32.Parse(childNode.InnerText);
}
if (childNode.Name == "tlp:TaskName")
{
unit.TaskName = childNode.InnerText;
}
}
}
output.AppendLine($"{unit.EmployeeID},{unit.EmployeeFirstName},{unit.EmployeeLastName},{unit.AllocationID},{unit.TaskName}");
//Console.WriteLine("---");
}
Console.WriteLine(output.ToString());
File.WriteAllText("c:\\...\\WorkUnits.csv", output.ToString());
}
}
else
{
if (Logger.IsWarnEnabled)
{
Logger.Warn("Failed to authenticate to reporting API");
}
}
}
}
}
}
You want to write the columns in the correct order to the CSV (of course), so you need to process them in the correct order. Two options:
intermediate class
Create a new class (let's call it WorkUnit) with properties for each of the columns that you want to write to the CSV. Create a new instance for every <tlp:WorkUnit> node in your XML and fill the properties when you encounter the correct subnodes. When you have processed the entire WorkUnit node, write out the properties in the correct order.
var output = new StringBuilder();
foreach (XmlNode customer in workUnit)
{
// fresh instance of the class that holds all columns (so all properties are cleared)
var unit = new WorkUnit();
var childNodes = customer.SelectNodes("./*");
if (childNodes != null)
{
foreach (XmlNode childNode in childNodes)
{
if(childNode.Name== "tlp:EmployeeID")
{
// employeeID node found, now write to the corresponding property:
unit.EmployeeId = childNode.InnerText;
}
// etc for the other XML nodes you are interested in
}
// all nodes have been processed for this one WorkUnit node
// so write a line to the CSV
output.AppendLine($"{unit.EmployeeId},{unit.AllocationId}, etc");
}
read in correct order
Instead of using foreach to loop through all subnodes in whatever order they appear, search for specific subnodes in the order you want. Then you can write out the CSV in the same order. Note that even when you don't find some subnode, you still need to write out the separator.
var output = new StringBuilder();
foreach (XmlNode customer in workUnit)
{
// search for value for first column (EmployeeID)
var node = workUnit.SelectSingleNode("tlp:EmployeeID");
if (node != null)
{
output.Append(node.InnerText).Append(',');
}
else
{
output.Append(','); // no content, but we still need a separator
}
// etc for the other columns
And of course watch out for string values that contain the separator.
Assuming that you put your XML data into List
StringBuilder str = new StringBuilder();
foreach (var fin list.ToList())
{
str.Append(fin.listfield.ToString() + ",");
}
to create a new line:
str.Replace(",", Environment.NewLine, str.Length - 1, 1);
to save:
string filename=(DirectoryPat/filename.csv");
File.WriteAllText(Filename, str.ToString());
Try this:
var output = new StringBuilder();
output.AppendLine("AllocationID,ApprovedStatus,CustomerId,CustomerName,EmployeeID");
if (workUnit != null)
{
foreach (XmlNode customer in workUnit)
{
var unit = new WorkUnit();
var childNodes = customer.SelectNodes("./*");
if (childNodes != null)
{
for (int i = 0; i<childNodes.Count; ++i)
{
XmlNode childNode = childNodes[i];
if (childNode.Name == "tlp:EmployeeID")
{
unit.EmployeeID = Int32.Parse(childNode.InnerText);
}
if (childNode.Name == "tlp:EmployeeFirstName")
{
unit.EmployeeFirstName = childNode.InnerText;
}
if (childNode.Name == "tlp:EmployeeLastName")
{
unit.EmployeeLastName = childNode.InnerText;
}
if (childNode.Name == "tlp:AllocationID")
{
unit.AllocationID = Int32.Parse(childNode.InnerText);
}
if (childNode.Name == "tlp:TaskName")
{
unit.TaskName = childNode.InnerText;
}
output.Append(childNode.InnerText);
if (i<childNodes.Count - 1)
output.Append(",");
}
output.Append(Environment.NewLine);
}
}
Console.WriteLine(output.ToString());
File.WriteAllText("c:\\Users\\mnowshin\\projects\\WorkUnits.csv", output.ToString());
}
You can use this sequence:
a. Deserialize (i.e. convert from XML to C# objects) your XML.
b. Write a simple loop to write the data to a file.
The advantages of this sequence:
You can use a list of your data/objects "readable" that you can add any other access code to it.
If you XML schema changed at any time, you can maintain the code very easily.
The solution
a. Desrialize:
Copy you XML file contents. Note You should modify your XML input before coping it.. You should double the WorkUnit node, in order to tell Visual Studio that you would have a list of this node nested inside WorkUnits node.
From Visual Studio Menus select Edit -> Paste Special -> Paste XML as Classes.
Use the deserialize code.
var workUnitsNode = customersRaw.SelectSingleNode("tlp:WorkUnits", namespaceManager);
XmlSerializer ser = new XmlSerializer(typeof(WorkUnits));
WorkUnits workUnits = (WorkUnits)ser.Deserialize(workUnitsNode);
b. Write the csv file
StringBuilder csvContent = new StringBuilder();
// add the header line
csvContent.AppendLine("AllocationID,ApprovedStatus,CustomerId,CustomerName,EmployeeID");
foreach (var unit in workUnits.WorkUnit)
{
csvContent.AppendFormat(
"{0}, {1}, {2}, {3}, {4}",
new object[]
{
unit.AllocationID,
unit.ApprovedStatus,
unit.CustomerId,
unit.CustomerName,
unit.EmployeeID
// you get the idea
});
csvContent.AppendLine();
}
File.WriteAllText(#"G:\Projects\StackOverFlow\WpfApp1\WorkUnits.csv", csvContent.ToString());
You can use Cinchoo ETL - if you have room to use open source library
using (var csvWriter = new ChoCSVWriter("sample1.csv").WithFirstLineHeader())
{
using (var xmlReader = new ChoXmlReader("sample1.xml"))
csvWriter.Write(xmlReader);
}
Output:
ID,tlp_EmployeeID,tlp_AllocationID,tlp_TaskID,tlp_ProjectID,tlp_ProjectName,tlp_CustomerId,tlp_CustomerName,tlp_IsBillable,tlp_ApprovedStatus,tlp_LastModifiedBy
130,3,114,239,26,LIK Template,343,Lekt Corp Inc.,1,0,AL
Disclaimer: I'm the author of this library.

Insert huge files (2G) to mongodb

I have 2GB files (9 of them) which contains approximately 12M records of strings that i want to insert each one as a document to local mongodb (windows).
Now i'm reading line by line and inserting every second line (the first is unnecessary header) like this:
bool readingFlag = false;
foreach (var line in File.ReadLines(file))
{
if (readingflag)
{
String document = "{'read':'" + line + "'}";
var documnt = new BsonDocument(
MongoDB
.Bson
.Serialization
.BsonSerializer
.Deserialize<BsonDocument>(document));
await collection.InsertOneAsync(documnt);
readingflag = false;
}
else
{
readingflag = true;
}
}
This method is working but not as fast as i expected. I'm now in the middle of the file and i assume it will end in about 4 hours for just one file. (40 hours for all my data)
I think that my bottleneck is the file reading but since it is very big file VS doesn't let my load it to memory (out of memory exception).
Is there any other way that i'm missing here?
I think we could utilize those things:
Get some lines and add in a bunch by insert many
insert data on separate thread as we don't need to wait for finish
use a typed class TextData to push serialization to other thread
You can play with limit at once - as this depend of amount of data read from file
public class TextData{
public ObjectId _id {
get;
set;
}
public string read {
get;
set;
}
}
public class Processor{
public async void ProcessData() {
var client = new MongoClient("mongodb://localhost:27017");
var database = client.GetDatabase("test");
var collection = database.GetCollection < TextData > ("Yogevnn");
var readingflag = false;
var listOfDocument = new List < TextData > ();
var limiAtOnce = 100;
var current = 0;
foreach(var line in File.ReadLines( # "E:\file.txt")) {
if (readingflag) {
var dataToInsert = new TextData {
read = line
};
listOfDocument.Add(dataToInsert);
readingflag = false;
Console.WriteLine($ "Current position: {current}");
if (++current == limiAtOnce) {
current = 0;
Console.WriteLine($ "Inserting data");
var listToInsert = listOfDocument;
var t = new Task(() = > {
Console.WriteLine($ "Inserting data START");
collection.InsertManyAsync(listToInsert);
Console.WriteLine($ "Inserting data FINISH");
});
t.Start();
listOfDocument = new List < TextData > ();
}
} else {
readingflag = true;
}
}
// insert remainder
await collection.InsertManyAsync(listOfDocument);
}
}
Any comments welcome!
In my experiments I found Parallel.ForEach(File.ReadLines("path")) to be the fastest.
File size was about 42 GB. I also tried batching a set of 100 lines and save the batch but was slower than Parallel.ForEach.
Another example: Read large txt file multithreaded?

Next node in xml with same element name

I've been working on this problem for a few hours now and I have searched all around with no luck for a solution :(
What I am trying to do is print out names of the nodes, what i have is the amount of nodes that exist so I know how many times to loop but am having the hardest of times retrieving the values
What I have tried:
int num = Convert.ToInt32(queuecount);
var jobs = QueueXML.SelectSingleNode(xpathjobsfilename).InnerText;
PreviousQueue = jobs.ToString();
//foreach(loop < num)
//{
// if (CurrentQueue == PreviousQueue)
// {
// }
// else
// {
// resultsListView.Items.Clear();
// resultsListView.Items.Add(jobs[num]);
// }
// loop++;
//}
foreach (char JobName in jobs.ToString())
{
if (CurrentQueue == PreviousQueue)
{
}
else
{
resultsListView.Items.Clear();
resultsListView.Items.Add(jobs[num]);
}
}
Edit: Example XML
<jobs>
<job>
<timeleft>0:00:00</timeleft>
<mb>1419.60536003</mb>
<msgid></msgid>
<filename>Extended_Final</filename>
<mbleft>1274.33209419</mbleft>
<id>nzo_i7qxxq</id>
</job>
<job>
<timeleft>0:00:00</timeleft>
<mb>9.22459220886</mb>
<msgid></msgid>
<filename>Video2</filename>
<mbleft>9.22459220886</mbleft>
<id>2m3dv5</id>
</job>
</jobs>
I want to retrieve the job details for each individual jobs
Use this code to loop through your job-nodes.
XmlDocument doc = new Windows.Data.Xml.Dom.XmlDocument();
doc.Load(#"/path/to/xml/file");
foreach (XmlNode job in doc.SelectNodes("/jobs/job"))
{
string filename = job.SelectSingleNode("filename").InnerText;
double mbleft = double.Parse(job.SelectSingleNode("mbleft").InnerText);
}
I am not quite sure what you want to do with it. If you want to use that information throughout your program, I'd create a job datatype and parse the XML document to a List<Job>. In any case the above code will enable you to access the information you are after.
Use LINQ2XML
XElement doc=XElement.Load("yourXMLfile.xml");
string timeleft,mb,msgid,filename,mbleft,id;
foreach(XElement elm in doc.Descendants().Elements("job"))
{
timeleft = elm.Element("timeleft").Value; //time left value
mb = elm.Element("mb").Value; //mb value
msgid = elm.Element("msgid").Value; //msgid value
filename = elm.Element("filename").Value; //filename value
mbleft = elm.Element("mbleft").Value; //mbleft value
id = elm.Element("id").Value; //id value
}
Below is the code to add individual job details to a List of dictionary.
It eliminates selecting single node in the loop when there are lot of child nodes. And, more generic.
System.Xml.XmlDocument doc = new System.Xml.XmlDocument();
doc.LoadXml(yourXmlString);
if (doc.HasChildNodes)
{
System.Xml.XmlNodeList jobLst = doc.DocumentElement.ChildNodes;
System.Collections.Generic.Dictionary<string, string> jobDescription;
var lstjobDescription = new System.Collections.Generic.List<System.Collections.Generic.Dictionary<string, string>>();
string name;
for (int i = 0; i < jobLst.Count; i++)
{
var responseDoc = new System.Xml.XmlDocument();
responseDoc.LoadXml(jobLst[i].OuterXml);
jobDescription = new System.Collections.Generic.Dictionary<string, string>();
foreach (System.Xml.XmlNode node in responseDoc.SelectNodes("//job/*")) //select all nodes of Job
{
jobDescription.Add(node.Name, node.InnerText);
}
lstjobDescription.Add(jobDescription);
}
}

Retrieving Data From XML File

I seem to be having a problem with retrieving XML values with C#, which I know it is due to my very limited knowledge of C# and .XML.
I was given the following XML file
<PowerBuilderRunTimes>
<PowerBuilderRunTime>
<Version>12</Version>
<Files>
<File>EasySoap110.dll</File>
<File>exPat110.dll</File>
<File>pbacc110.dll</File>
</File>
</PowerBuilderRunTime>
</PowerBuilderRunTimes>
I am to process the XML file and make sure that each of the files in the exist in the folder (that's the easy part). It's the processing of the XML file that I have having a hard time with. Here is what I have done thus far:
var runtimeXml = File.ReadAllText(string.Format("{0}\\{1}", configPath, Resource.PBRuntimes));
var doc = XDocument.Parse(runtimeXml);
var topElement = doc.Element("PowerBuilderRunTimes");
var elements = topElement.Elements("PowerBuilderRunTime");
foreach (XElement section in elements)
{
//pbVersion is grabbed earlier. It is the version of PowerBuilder
if( section.Element("Version").Value.Equals(string.Format("{0}", pbVersion ) ) )
{
var files = section.Elements("Files");
var fileList = new List<string>();
foreach (XElement area in files)
{
fileList.Add(area.Element("File").Value);
}
}
}
My issue is that the String List is only ever populated with one value, "EasySoap110.dll", and everything else is ignored. Can someone please help me, as I am at a loss.
Look at this bit:
var files = section.Elements("Files");
var fileList = new List<string>();
foreach (XElement area in files)
{
fileList.Add(area.Element("File").Value);
}
You're iterating over each Files element, and then finding the first File element within it. There's only one Files element - you need to be iterating over the File elements within that.
However, there are definitely better ways of doing this. For example:
var doc = XDocument.Load(Path.Combine(configPath, Resource.PBRuntimes));
var fileList = (from runtime in doc.Root.Elements("PowerBuilderRunTime")
where (int) runtime.Element("Version") == pbVersion
from file in runtime.Element("Files").Elements("File")
select file.Value)
.ToList();
Note that if there are multiple matching PowerBuilderRunTime elements, that will create a list with all the files of all those elements. That may not be what you want. For example, you might want:
var doc = XDocument.Load(Path.Combine(configPath, Resource.PBRuntimes));
var runtime = doc.Root
.Elements("PowerBuilderRunTime")
.Where(r => (int) r.Element("Version") == pbVersion)
.Single();
var fileList = runtime.Element("Files")
.Elements("File")
.Select(x => x.Value)
.ToList();
That will validate that there's exactly one matching runtime.
The problem is, there's only one element in your XML, with multiple children. You foreach loop only executes once, for the single element, not for its children.
Do something like this:
var fileSet = files.Elements("File");
foreach (var file in fileSet) {
fileList.Add(file.Value);
}
which loops over all children elements.
I always preferred using readers for reading homegrown XML config files. If you're only doing this once it's probably over kill, but readers are faster and cheaper.
public static class PowerBuilderConfigParser
{
public static IList<PowerBuilderConfig> ReadConfigFile(String path)
{
IList<PowerBuilderConfig> configs = new List<PowerBuilderConfig>();
using (FileStream stream = new FileStream(path, FileMode.Open))
{
XmlReader reader = XmlReader.Create(stream);
reader.ReadToDescendant("PowerBuilderRunTime");
do
{
PowerBuilderConfig config = new PowerBuilderConfig();
ReadVersionNumber(config, reader);
ReadFiles(config, reader);
configs.Add(config);
reader.ReadToNextSibling("PowerBuilderRunTime");
} while (reader.ReadToNextSibling("PowerBuilderRunTime"));
}
return configs;
}
private static void ReadVersionNumber(PowerBuilderConfig config, XmlReader reader)
{
reader.ReadToDescendant("Version");
string version = reader.ReadString();
Int32 versionNumber;
if (Int32.TryParse(version, out versionNumber))
{
config.Version = versionNumber;
}
}
private static void ReadFiles(PowerBuilderConfig config, XmlReader reader)
{
reader.ReadToNextSibling("Files");
reader.ReadToDescendant("File");
do
{
string file = reader.ReadString();
if (!string.IsNullOrEmpty(file))
{
config.AddConfigFile(file);
}
} while (reader.ReadToNextSibling("File"));
}
}
public class PowerBuilderConfig
{
private Int32 _version;
private readonly IList<String> _files;
public PowerBuilderConfig()
{
_files = new List<string>();
}
public Int32 Version
{
get { return _version; }
set { _version = value; }
}
public ReadOnlyCollection<String> Files
{
get { return new ReadOnlyCollection<String>(_files); }
}
public void AddConfigFile(String fileName)
{
_files.Add(fileName);
}
}
Another way is to use a XmlSerializer.
[Serializable]
[XmlRoot]
public class PowerBuilderRunTime
{
[XmlElement]
public string Version {get;set;}
[XmlArrayItem("File")]
public string[] Files {get;set;}
public static PowerBuilderRunTime[] Load(string fileName)
{
PowerBuilderRunTime[] runtimes;
using (var fs = new FileStream(fileName, FileMode.Open, FileAccess.Read))
{
var reader = new XmlTextReader(fs);
runtimes = (PowerBuilderRunTime[])new XmlSerializer(typeof(PowerBuilderRunTime[])).Deserialize(reader);
}
return runtimes;
}
}
You can get all the runtimes strongly typed, and use each PowerBuilderRunTime's Files property to loop through all the string file names.
var runtimes = PowerBuilderRunTime.Load(string.Format("{0}\\{1}", configPath, Resource.PBRuntimes));
You should try replacing this stuff with a simple XPath query.
string configPath;
System.Xml.XPath.XPathDocument xpd = new System.Xml.XPath.XPathDocument(cofigPath);
System.Xml.XPath.XPathNavigator xpn = xpd.CreateNavigator();
System.Xml.XPath.XPathExpression exp = xpn.Compile(#"/PowerBuilderRunTimes/PwerBuilderRunTime/Files//File");
System.Xml.XPath.XPathNodeIterator iterator = xpn.Select(exp);
while (iterator.MoveNext())
{
System.Xml.XPath.XPathNavigator nav2 = iterator.Current.Clone();
//access value with nav2.value
}

How to convert JSON to XML or XML to JSON?

I started to use Json.NET to convert a string in JSON format to object or viceversa. I am not sure in the Json.NET framework, is it possible to convert a string in JSON to XML format and viceversa?
Yes. Using the JsonConvert class which contains helper methods for this precise purpose:
// To convert an XML node contained in string xml into a JSON string
XmlDocument doc = new XmlDocument();
doc.LoadXml(xml);
string jsonText = JsonConvert.SerializeXmlNode(doc);
// To convert JSON text contained in string json into an XML node
XmlDocument doc = JsonConvert.DeserializeXmlNode(json);
Documentation here: Converting between JSON and XML with Json.NET
Yes, you can do it (I do) but Be aware of some paradoxes when converting, and handle appropriately. You cannot automatically conform to all interface possibilities, and there is limited built-in support in controlling the conversion- many JSON structures and values cannot automatically be converted both ways. Keep in mind I am using the default settings with Newtonsoft JSON library and MS XML library, so your mileage may vary:
XML -> JSON
All data becomes string data (for example you will always get "false" not false or "0" not 0) Obviously JavaScript treats these differently in certain cases.
Children elements can become nested-object {} OR nested-array [ {} {} ...] depending if there is only one or more than one XML child-element. You would consume these two differently in JavaScript, etc. Different examples of XML conforming to the same schema can produce actually different JSON structures this way. You can add the attribute json:Array='true' to your element to workaround this in some (but not necessarily all) cases.
Your XML must be fairly well-formed, I have noticed it doesn't need to perfectly conform to W3C standard, but 1. you must have a root element and 2. you cannot start element names with numbers are two of the enforced XML standards I have found when using Newtonsoft and MS libraries.
In older versions, Blank elements do not convert to JSON. They are ignored. A blank element does not become "element":null
A new update changes how null can be handled (Thanks to Jon Story for pointing it out): https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_NullValueHandling.htm
JSON -> XML
You need a top level object that will convert to a root XML element or the parser will fail.
Your object names cannot start with a number, as they cannot be converted to elements (XML is technically even more strict than this) but I can 'get away' with breaking some of the other element naming rules.
Please feel free to mention any other issues you have noticed, I have developed my own custom routines for preparing and cleaning the strings as I convert back and forth. Your situation may or may not call for prep/cleanup. As StaxMan mentions, your situation may actually require that you convert between objects...this could entail appropriate interfaces and a bunch of case statements/etc to handle the caveats I mention above.
You can do these conversions also with the .NET Framework:
JSON to XML: by using System.Runtime.Serialization.Json
var xml = XDocument.Load(JsonReaderWriterFactory.CreateJsonReader(
Encoding.ASCII.GetBytes(jsonString), new XmlDictionaryReaderQuotas()));
XML to JSON: by using System.Web.Script.Serialization
var json = new JavaScriptSerializer().Serialize(GetXmlData(XElement.Parse(xmlString)));
private static Dictionary<string, object> GetXmlData(XElement xml)
{
var attr = xml.Attributes().ToDictionary(d => d.Name.LocalName, d => (object)d.Value);
if (xml.HasElements) attr.Add("_value", xml.Elements().Select(e => GetXmlData(e)));
else if (!xml.IsEmpty) attr.Add("_value", xml.Value);
return new Dictionary<string, object> { { xml.Name.LocalName, attr } };
}
I'm not sure there is point in such conversion (yes, many do it, but mostly to force a square peg through round hole) -- there is structural impedance mismatch, and conversion is lossy. So I would recommend against such format-to-format transformations.
But if you do it, first convert from json to object, then from object to xml (and vice versa for reverse direction). Doing direct transformation leads to ugly output, loss of information, or possibly both.
Thanks for David Brown's answer. In my case of JSON.Net 3.5, the convert methods are under the JsonConvert static class:
XmlNode myXmlNode = JsonConvert.DeserializeXmlNode(myJsonString); // is node not note
// or .DeserilizeXmlNode(myJsonString, "root"); // if myJsonString does not have a root
string jsonString = JsonConvert.SerializeXmlNode(myXmlNode);
I searched for a long time to find alternative code to the accepted solution in the hopes of not using an external assembly/project. I came up with the following thanks to the source code of the DynamicJson project:
public XmlDocument JsonToXML(string json)
{
XmlDocument doc = new XmlDocument();
using (var reader = JsonReaderWriterFactory.CreateJsonReader(Encoding.UTF8.GetBytes(json), XmlDictionaryReaderQuotas.Max))
{
XElement xml = XElement.Load(reader);
doc.LoadXml(xml.ToString());
}
return doc;
}
Note: I wanted an XmlDocument rather than an XElement for xPath purposes.
Also, this code obviously only goes from JSON to XML, there are various ways to do the opposite.
Here is the full c# code to convert xml to json
public static class JSon
{
public static string XmlToJSON(string xml)
{
XmlDocument doc = new XmlDocument();
doc.LoadXml(xml);
return XmlToJSON(doc);
}
public static string XmlToJSON(XmlDocument xmlDoc)
{
StringBuilder sbJSON = new StringBuilder();
sbJSON.Append("{ ");
XmlToJSONnode(sbJSON, xmlDoc.DocumentElement, true);
sbJSON.Append("}");
return sbJSON.ToString();
}
// XmlToJSONnode: Output an XmlElement, possibly as part of a higher array
private static void XmlToJSONnode(StringBuilder sbJSON, XmlElement node, bool showNodeName)
{
if (showNodeName)
sbJSON.Append("\"" + SafeJSON(node.Name) + "\": ");
sbJSON.Append("{");
// Build a sorted list of key-value pairs
// where key is case-sensitive nodeName
// value is an ArrayList of string or XmlElement
// so that we know whether the nodeName is an array or not.
SortedList<string, object> childNodeNames = new SortedList<string, object>();
// Add in all node attributes
if (node.Attributes != null)
foreach (XmlAttribute attr in node.Attributes)
StoreChildNode(childNodeNames, attr.Name, attr.InnerText);
// Add in all nodes
foreach (XmlNode cnode in node.ChildNodes)
{
if (cnode is XmlText)
StoreChildNode(childNodeNames, "value", cnode.InnerText);
else if (cnode is XmlElement)
StoreChildNode(childNodeNames, cnode.Name, cnode);
}
// Now output all stored info
foreach (string childname in childNodeNames.Keys)
{
List<object> alChild = (List<object>)childNodeNames[childname];
if (alChild.Count == 1)
OutputNode(childname, alChild[0], sbJSON, true);
else
{
sbJSON.Append(" \"" + SafeJSON(childname) + "\": [ ");
foreach (object Child in alChild)
OutputNode(childname, Child, sbJSON, false);
sbJSON.Remove(sbJSON.Length - 2, 2);
sbJSON.Append(" ], ");
}
}
sbJSON.Remove(sbJSON.Length - 2, 2);
sbJSON.Append(" }");
}
// StoreChildNode: Store data associated with each nodeName
// so that we know whether the nodeName is an array or not.
private static void StoreChildNode(SortedList<string, object> childNodeNames, string nodeName, object nodeValue)
{
// Pre-process contraction of XmlElement-s
if (nodeValue is XmlElement)
{
// Convert <aa></aa> into "aa":null
// <aa>xx</aa> into "aa":"xx"
XmlNode cnode = (XmlNode)nodeValue;
if (cnode.Attributes.Count == 0)
{
XmlNodeList children = cnode.ChildNodes;
if (children.Count == 0)
nodeValue = null;
else if (children.Count == 1 && (children[0] is XmlText))
nodeValue = ((XmlText)(children[0])).InnerText;
}
}
// Add nodeValue to ArrayList associated with each nodeName
// If nodeName doesn't exist then add it
List<object> ValuesAL;
if (childNodeNames.ContainsKey(nodeName))
{
ValuesAL = (List<object>)childNodeNames[nodeName];
}
else
{
ValuesAL = new List<object>();
childNodeNames[nodeName] = ValuesAL;
}
ValuesAL.Add(nodeValue);
}
private static void OutputNode(string childname, object alChild, StringBuilder sbJSON, bool showNodeName)
{
if (alChild == null)
{
if (showNodeName)
sbJSON.Append("\"" + SafeJSON(childname) + "\": ");
sbJSON.Append("null");
}
else if (alChild is string)
{
if (showNodeName)
sbJSON.Append("\"" + SafeJSON(childname) + "\": ");
string sChild = (string)alChild;
sChild = sChild.Trim();
sbJSON.Append("\"" + SafeJSON(sChild) + "\"");
}
else
XmlToJSONnode(sbJSON, (XmlElement)alChild, showNodeName);
sbJSON.Append(", ");
}
// Make a string safe for JSON
private static string SafeJSON(string sIn)
{
StringBuilder sbOut = new StringBuilder(sIn.Length);
foreach (char ch in sIn)
{
if (Char.IsControl(ch) || ch == '\'')
{
int ich = (int)ch;
sbOut.Append(#"\u" + ich.ToString("x4"));
continue;
}
else if (ch == '\"' || ch == '\\' || ch == '/')
{
sbOut.Append('\\');
}
sbOut.Append(ch);
}
return sbOut.ToString();
}
}
To convert a given XML string to JSON, simply call XmlToJSON() function as below.
string xml = "<menu id=\"file\" value=\"File\"> " +
"<popup>" +
"<menuitem value=\"New\" onclick=\"CreateNewDoc()\" />" +
"<menuitem value=\"Open\" onclick=\"OpenDoc()\" />" +
"<menuitem value=\"Close\" onclick=\"CloseDoc()\" />" +
"</popup>" +
"</menu>";
string json = JSON.XmlToJSON(xml);
// json = { "menu": {"id": "file", "popup": { "menuitem": [ {"onclick": "CreateNewDoc()", "value": "New" }, {"onclick": "OpenDoc()", "value": "Open" }, {"onclick": "CloseDoc()", "value": "Close" } ] }, "value": "File" }}
For convert JSON string to XML try this:
public string JsonToXML(string json)
{
XDocument xmlDoc = new XDocument(new XDeclaration("1.0", "utf-8", ""));
XElement root = new XElement("Root");
root.Name = "Result";
var dataTable = JsonConvert.DeserializeObject<DataTable>(json);
root.Add(
from row in dataTable.AsEnumerable()
select new XElement("Record",
from column in dataTable.Columns.Cast<DataColumn>()
select new XElement(column.ColumnName, row[column])
)
);
xmlDoc.Add(root);
return xmlDoc.ToString();
}
For convert XML to JSON try this:
public string XmlToJson(string xml)
{
XmlDocument doc = new XmlDocument();
doc.LoadXml(xml);
string jsonText = JsonConvert.SerializeXmlNode(doc);
return jsonText;
}
Here is a simple snippet that converts a XmlNode (recursively) into a hashtable, and groups multiple instances of the same child into an array (as an ArrayList).
The Hashtable is usually accepted to convert into JSON by most of the JSON libraries.
protected object convert(XmlNode root){
Hashtable obj = new Hashtable();
for(int i=0,n=root.ChildNodes.Count;i<n;i++){
object result = null;
XmlNode current = root.ChildNodes.Item(i);
if(current.NodeType != XmlNodeType.Text)
result = convert(current);
else{
int resultInt;
double resultFloat;
bool resultBoolean;
if(Int32.TryParse(current.Value, out resultInt)) return resultInt;
if(Double.TryParse(current.Value, out resultFloat)) return resultFloat;
if(Boolean.TryParse(current.Value, out resultBoolean)) return resultBoolean;
return current.Value;
}
if(obj[current.Name] == null)
obj[current.Name] = result;
else if(obj[current.Name].GetType().Equals(typeof(ArrayList)))
((ArrayList)obj[current.Name]).Add(result);
else{
ArrayList collision = new ArrayList();
collision.Add(obj[current.Name]);
collision.Add(result);
obj[current.Name] = collision;
}
}
return obj;
}
Try this function. I just wrote it and haven't had much of a chance to test it, but my preliminary tests are promising.
public static XmlDocument JsonToXml(string json)
{
XmlNode newNode = null;
XmlNode appendToNode = null;
XmlDocument returnXmlDoc = new XmlDocument();
returnXmlDoc.LoadXml("<Document />");
XmlNode rootNode = returnXmlDoc.SelectSingleNode("Document");
appendToNode = rootNode;
string[] arrElementData;
string[] arrElements = json.Split('\r');
foreach (string element in arrElements)
{
string processElement = element.Replace("\r", "").Replace("\n", "").Replace("\t", "").Trim();
if ((processElement.IndexOf("}") > -1 || processElement.IndexOf("]") > -1) && appendToNode != rootNode)
{
appendToNode = appendToNode.ParentNode;
}
else if (processElement.IndexOf("[") > -1)
{
processElement = processElement.Replace(":", "").Replace("[", "").Replace("\"", "").Trim();
newNode = returnXmlDoc.CreateElement(processElement);
appendToNode.AppendChild(newNode);
appendToNode = newNode;
}
else if (processElement.IndexOf("{") > -1 && processElement.IndexOf(":") > -1)
{
processElement = processElement.Replace(":", "").Replace("{", "").Replace("\"", "").Trim();
newNode = returnXmlDoc.CreateElement(processElement);
appendToNode.AppendChild(newNode);
appendToNode = newNode;
}
else
{
if (processElement.IndexOf(":") > -1)
{
arrElementData = processElement.Replace(": \"", ":").Replace("\",", "").Replace("\"", "").Split(':');
newNode = returnXmlDoc.CreateElement(arrElementData[0]);
for (int i = 1; i < arrElementData.Length; i++)
{
newNode.InnerText += arrElementData[i];
}
appendToNode.AppendChild(newNode);
}
}
}
return returnXmlDoc;
}
I did like David Brown said but I got the following exception.
$exception {"There are multiple root elements. Line , position ."} System.Xml.XmlException
One solution would be to modify the XML file with a root element but that is not always necessary and for an XML stream it might not be possible either. My solution below:
var path = Path.GetFullPath(Path.Combine(Environment.CurrentDirectory, #"..\..\App_Data"));
var directoryInfo = new DirectoryInfo(path);
var fileInfos = directoryInfo.GetFiles("*.xml");
foreach (var fileInfo in fileInfos)
{
XmlDocument doc = new XmlDocument();
XmlReaderSettings settings = new XmlReaderSettings();
settings.ConformanceLevel = ConformanceLevel.Fragment;
using (XmlReader reader = XmlReader.Create(fileInfo.FullName, settings))
{
while (reader.Read())
{
if (reader.NodeType == XmlNodeType.Element)
{
var node = doc.ReadNode(reader);
string json = JsonConvert.SerializeXmlNode(node);
}
}
}
}
Example XML that generates the error:
<parent>
<child>
Text
</child>
</parent>
<parent>
<child>
<grandchild>
Text
</grandchild>
<grandchild>
Text
</grandchild>
</child>
<child>
Text
</child>
</parent>
I have used the below methods to convert the JSON to XML
List <Item> items;
public void LoadJsonAndReadToXML() {
using(StreamReader r = new StreamReader(# "E:\Json\overiddenhotelranks.json")) {
string json = r.ReadToEnd();
items = JsonConvert.DeserializeObject <List<Item>> (json);
ReadToXML();
}
}
And
public void ReadToXML() {
try {
var xEle = new XElement("Items",
from item in items select new XElement("Item",
new XElement("mhid", item.mhid),
new XElement("hotelName", item.hotelName),
new XElement("destination", item.destination),
new XElement("destinationID", item.destinationID),
new XElement("rank", item.rank),
new XElement("toDisplayOnFod", item.toDisplayOnFod),
new XElement("comment", item.comment),
new XElement("Destinationcode", item.Destinationcode),
new XElement("LoadDate", item.LoadDate)
));
xEle.Save("E:\\employees.xml");
Console.WriteLine("Converted to XML");
} catch (Exception ex) {
Console.WriteLine(ex.Message);
}
Console.ReadLine();
}
I have used the class named Item to represent the elements
public class Item {
public int mhid { get; set; }
public string hotelName { get; set; }
public string destination { get; set; }
public int destinationID { get; set; }
public int rank { get; set; }
public int toDisplayOnFod { get; set; }
public string comment { get; set; }
public string Destinationcode { get; set; }
public string LoadDate { get; set; }
}
It works....
Cinchoo ETL - an open source library available to do the conversion of Xml to JSON easily with few lines of code
Xml -> JSON:
using (var p = new ChoXmlReader("sample.xml"))
{
using (var w = new ChoJSONWriter("sample.json"))
{
w.Write(p);
}
}
JSON -> Xml:
using (var p = new ChoJsonReader("sample.json"))
{
using (var w = new ChoXmlWriter("sample.xml"))
{
w.Write(p);
}
}
Sample fiddle: https://dotnetfiddle.net/enUJKu
Checkout CodeProject articles for some additional help.
Disclaimer: I'm the author of this library.
Here's an example of how to convert JSON to XML using .NET built-in libraries (instead of 3rd party libraries like Newtonsoft).
using System.Text.Json;
using System.Text.Json.Nodes;
using System.Xml.Linq;
XDocument xmlDoc = jsonToXml(jsonObj);
private XDocument jsonToXml(JsonObject obj)
{
var xmlDoc = new XDocument();
var root = new XElement("Root");
xmlDoc.Add(root);
foreach (var prop in obj)
{
var xElement = new XElement(prop.Key);
xElement.Value = prop.Value.ToString();
root.Add(xElement);
}
return xmlDoc;
}

Categories