We Keep Coding

How to get text out of Word Document using C# open-xml SDK in json like format?

I'm trying to extract all the text from a word document using C# open-xml SDK, and I want to store the style attributes and innerText data from each of the paragraphs and then for each of the runs inside a paragraph (and eventually the same thing for the tables too).
{
"paragraphs":
[
{
"innerText": "",
"runs": [
{
"runProperties": {},
"innerText": ""
}
],
"paragraphProperties": {}
}
],
"tables":
[
{
​
}
]
}
This is the desired structure.
I'm from python background where doing this using python dict would've been easy since there's no strict type for value of the key in a dict. But I've got little to no idea on how to approach this in C#.
static void Main(string[] args)
{
string fileName = #"<path to file>";
​
using (WordprocessingDocument myDoc = WordprocessingDocument.Open(fileName, true))
{
IEnumerable<Paragraph> paragraphList = myDoc.MainDocumentPart.Document.Body.Elements().Where(c => c is Paragraph).Cast<Paragraph>();
foreach (Paragraph p in paragraphList)
{
string paragraphInnerText = p.InnerText;
IEnumerable<Run> runList = p.ChildElements.Where(c => c is Run).Cast<Run>();
foreach (Run r in runList)
{
string runInnerText = r.InnerText;
IEnumerable<RunProperties> runProperties = r.ChildElements.Where(c => c is RunProperties).Cast<RunProperties>();
Console.WriteLine("Getting Run Data.");
}
IEnumerable<ParagraphProperties> paragraphPropertiesList = p.ChildElements.Where(r => r is ParagraphProperties).Cast<ParagraphProperties>();
}
}
Console.WriteLine("All done. Press a key.");
}
This is the code I've come up with so far, with some help.
Any ideas on how can I store this in the json format, or how should I go about it?
Thanks in advance! :)

Option A, convert all document using source xml with SerializeXmlNode (JSON.NET):
XmlDocument doc = new XmlDocument();
doc.LoadXml(myDoc.MainDocumentPart.Document.OuterXml);
var json = JsonConvert.SerializeXmlNode(doc);
Option B, create anonymus type for all paragraph and run's:
IEnumerable<Paragraph> paragraphList = myDoc.MainDocumentPart.Document.Body.Elements().OfType<Paragraph>();
var proj = paragraphList.Select(p => p.ChildElements.OfType<Run>().Select(r => new
{
r.InnerText,
runProperties = r.ChildElements.OfType<RunProperties>().FirstOrDefault()?.Select(rp => new { rp.GetType().Name, Val = rp.GetAttributes().FirstOrDefault().Value })
}));
var json = JsonConvert.SerializeObject(proj, Newtonsoft.Json.Formatting.Indented);

How to output JSON array as a single field in CSV using ChoETL

I'm using ChoETL to convert JSON to CSV. Currently, if a property in the JSON object is an array it is output into separate fields in JSON.
Example:
{
"id", 1234,
"states": [
"PA",
"VA"
]
},
{
"id", 1235,
"states": [
"CA",
"DE",
"MD"
]
},
This results in CSV like this (using pipe as a delimeter)
"id"|"states_0"|"states_1"|"states_2"
"1234"|"PA"|"VA"
"1235"|"CA"|"DE"|"MD"
What I would like is for the array to be displayed in a single states field as a comma separated string
"id"|"states"
"1234"|"PA,VA"
"1235"|"CA,DE,MD"
Here is the code I have in place to perform the parsing and transformation.
public static class JsonCsvConverter
{
public static string ConvertJsonToCsv(string json)
{
var csvData = new StringBuilder();
using (var jsonReader = ChoJSONReader.LoadText(json))
{
using (var csvWriter = new ChoCSVWriter(csvData).WithFirstLineHeader())
{
csvWriter.WithMaxScanRows(1000);
csvWriter.Configuration.Delimiter = "|";
csvWriter.Configuration.QuoteAllFields = true;
csvWriter.Write(jsonReader);
}
}
return csvData.ToString();
}
}
Edited: Removed test code that wasn't useful

This is how you can produce the expected output using the code below
var csvData = new StringBuilder();
using (var jsonReader = ChoJSONReader.LoadText(json))
{
using (var csvWriter = new ChoCSVWriter(csvData)
.WithFirstLineHeader()
.WithDelimiter("|")
.QuoteAllFields()
.Configure(c => c.UseNestedKeyFormat = false)
.WithField("id")
.WithField("states", m => m.ValueConverter(o => String.Join(",", ((Array)o).OfType<string>())))
)
{
csvWriter.Write(jsonReader);
}
}
Console.WriteLine(csvData.ToString());
Output:
id|states
"1234"|"PA,VA"
"1235"|"CA,DE,MD"
PS: on the next release, this issue will be handled automatically without using valueconverters

Converting XmlDocument as List of objects

At the moment I convert XmlDocument as string List but this is not a good solution becouse if I display it in ListBox this recive me inline string with all parameters and I need to display only of table then when it's checked send it Sql Server
I can't create helper class with properties because parameters can be added/delete dynamically. I don't know what the exact parameters will be in file
Here is example XML
<TechnologyTables>
<Tables>
<TA_ID>3102</TA_ID>
<TA_Name>AL000-01.50M-N2-S0</TA_Name>
<TA_MaterialID>1</TA_MaterialID>
<TA_ThicknessID>4</TA_ThicknessID>
<TA_IsActive>1</TA_IsActive>
<TA_ArchiveID>100</TA_ArchiveID>
<TA_IsArchive>0</TA_IsArchive>
<CL_IsActive>1</CL_IsActive>
<MP_Lens>200</MP_Lens>
<MP_Nozzle>4.0</MP_Nozzle>
<MP_Focal>-0.6</MP_Focal>
<MP_NozzleDist>1.5000000e+000</MP_NozzleDist>
<MP_GasStabilization>1</MP_GasStabilization>
<MP_SensitiveSensor>1</MP_SensitiveSensor>
<MP_SensitiveArea>2.5000000e+001</MP_SensitiveArea>
<GA_ID>1</GA_ID>
<GP_ID>0</GP_ID>
<GP_FlushOn>1</GP_FlushOn>
<GP_FlushTime>2000</GP_FlushTime>
<GP_FlushPressure>1.0000000e+001</GP_FlushPressure>
<GP_FeedbackOn>1</GP_FeedbackOn>
<GP_FeedbackTime>0</GP_FeedbackTime>
<GP_FeedbackPressure>1.0000000e-001</GP_FeedbackPressure>
<GP_MaxPressure>1.0000000e+001</GP_MaxPressure>
<GP_ContinueOn>0</GP_ContinueOn>
<GP_ContinueTime>0</GP_ContinueTime>
<TA_Jerk>100</TA_Jerk>
<TA_Acceleration>100</TA_Acceleration>
<TA_CuttingTechID>3</TA_CuttingTechID>
<TA_FlyCut>1</TA_FlyCut>
<TA_HeadID>1</TA_HeadID>
<TA_MachineID>3</TA_MachineID>
<TA_TypeID>1</TA_TypeID>
<TT_HeadPowerID>7</TT_HeadPowerID>
<TA_CreateDate>2019-08-26T17:10:59.810</TA_CreateDate>
<Description>AL1.5 CATLINE</Description>
<TA_HeadTypeID>2</TA_HeadTypeID>
<CatlineFolder>1</CatlineFolder>
<NozzleNameID>10</NozzleNameID>
<LaserTypeID>1</LaserTypeID>
</Tables>
</TechnologyTables>
Some code when importing file
private async void _ImportTechTables()
{
var open = new OpenFileDialog();
var TableXml = new XmlDocument();
open.Filter = "xml files |*.xml";
if (open.ShowDialog() == DialogResult.OK)
{
TableXml.Load(open.FileName);
}
RSA rsaKey = GetKey();
DecryptXML(TableXml, rsaKey, "XmlKey");
if (TableXml != null)
{
var import = new TechnologyTableImportViewModel();
List<string> xmlNodeLists = new List<string>();
XmlNodeList node = TableXml.SelectNodes("TechnologyTables/Tables");
foreach (XmlNode nodes in node)
{
xmlNodeLists.Add(nodes.InnerXml);
}
import.List = xmlNodeLists;
And element in list look like this:
<Tables><TA_ID><Tables><TA_ID>3102</TA_ID><TA_Name>AL000-01.50M-N2<TA_Name>

I like using a dictionary and create dictionary using xml linq :
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
namespace ConsoleApplication137
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
XDocument doc = XDocument.Load(FILENAME);
Dictionary<string, string> dict = doc.Descendants("Tables").FirstOrDefault().Elements()
.GroupBy(x => x.Name.LocalName, y => (string)y)
.ToDictionary(x => x.Key, y => y.FirstOrDefault());
}
}
}
If you have multiple tables then use following :
List<Dictionary<string, string>> dict = doc.Descendants("Tables").Select(t => t.Elements()
.GroupBy(x => x.Name.LocalName, y => (string)y)
.ToDictionary(x => x.Key, y => y.FirstOrDefault())
).ToList();

You can save the fields to a list this way, although I recommend using a dictionary for this kind of format.
var doc = XDocument.Parse(yourXmlFile);
var table = doc.XPathSelectElement("TechnologyTables/Tables");
var dict = new Dictionary<string, string>();
foreach (var element in table.Elements())
{
dict.Add(element.Name.LocalName, element.Value);
}

Read a JSON file and generate string keys with values in a dictionary like object

I have to read in a JSON file that will be fairly big, with potentially hundreds of key value pairs and nested JSON key/values.
So say I have something like this:
{
"abc": {
"123": {
"donkey": "hello world",
"kong": 123
},
"meta": {
"aaa": "bbb"
}
}
}
I want to read this JSON file, and then initialize a Dictionary with keys that would be like this (based on the above JSON file):
"abc.123.donkey": "hello world"
"abc.123.kong": 123
"abc.meta.aaa": "bbb"
So basically the key is like a namespace based on the number of nested items, and then it has the value.
How should I go about parsing the JSON file when I don't know the shape of the JSON ahead of time, given that I need to create a dictionary out of it using this namespace style keys?

A. System.Text.Json
I'm sure this will get easier, but as of .NET Core 3.0 JsonDocument is a way.
using System.Linq;
using System.Text.Json;
(...)
public static Dictionary<string, JsonElement> GetFlat(string json)
{
IEnumerable<(string Path, JsonProperty P)> GetLeaves(string path, JsonProperty p)
=> p.Value.ValueKind != JsonValueKind.Object
? new[] { (Path: path == null ? p.Name : path + "." + p.Name, p) }
: p.Value.EnumerateObject() .SelectMany(child => GetLeaves(path == null ? p.Name : path + "." + p.Name, child));
using (JsonDocument document = JsonDocument.Parse(json)) // Optional JsonDocumentOptions options
return document.RootElement.EnumerateObject()
.SelectMany(p => GetLeaves(null, p))
.ToDictionary(k => k.Path, v => v.P.Value.Clone()); //Clone so that we can use the values outside of using
}
More expressive version is shown below.
Test
using System.Linq;
using System.Text.Json;
(...)
var json = #"{
""abc"": {
""123"": {
""donkey"": ""hello world"",
""kong"": 123
},
""meta"": {
""aaa"": ""bbb""
}
}
}";
var d = GetFlat(json);
var options2 = new JsonSerializerOptions { WriteIndented = true };
Console.WriteLine(JsonSerializer.Serialize(d, options2));
Output
{
"abc.123.donkey": "hello world",
"abc.123.kong": 123,
"abc.meta.aaa": "bbb"
}
More expressive version
using System.Linq;
using System.Text.Json;
(...)
static Dictionary<string, JsonElement> GetFlat(string json)
{
using (JsonDocument document = JsonDocument.Parse(json))
{
return document.RootElement.EnumerateObject()
.SelectMany(p => GetLeaves(null, p))
.ToDictionary(k => k.Path, v => v.P.Value.Clone()); //Clone so that we can use the values outside of using
}
}
static IEnumerable<(string Path, JsonProperty P)> GetLeaves(string path, JsonProperty p)
{
path = (path == null) ? p.Name : path + "." + p.Name;
if (p.Value.ValueKind != JsonValueKind.Object)
yield return (Path: path, P: p);
else
foreach (JsonProperty child in p.Value.EnumerateObject())
foreach (var leaf in GetLeaves(path, child))
yield return leaf;
}
B. Json.NET
You can do with Json.NET.
B1. SelectTokens
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
(...)
JObject jo = JObject.Parse(json);
var d = jo.SelectTokens("$..*")
.Where(t => t.HasValues == false)
.ToDictionary(k => k.Path, v => v);
Console.WriteLine(JsonConvert.SerializeObject(d, Formatting.Indented));
Output
{
"abc.123.donkey": "hello world",
"abc.123.kong": 123,
"abc.meta.aaa": "bbb"
}
B2. JToken traversal
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
(...)
static IEnumerable<JToken> Traverse(JToken jo)
{
if (!jo.Any()) yield return jo;
foreach (var ch in jo)
foreach (var x in Traverse(ch))
yield return x;
}
Test
var json = #"{
""abc"": {
""123"": {
""donkey"": ""hello world"",
""kong"": 123
},
""meta"": {
""aaa"": ""bbb""
}
}
}";
JObject jo = JObject.Parse(json);
var d = Traverse(jo).ToDictionary(k => k.Path, v => v);
var json2 = JsonConvert.SerializeObject(d, Formatting.Indented);
Console.WriteLine(json2);
Output
{
"abc.123.donkey": "hello world",
"abc.123.kong": 123,
"abc.meta.aaa": "bbb"
}

You can use Json.NET's LINQ to JSON feature to accomplish this.
You can deserialize the object to a JObject, which is a dictionary with better typings for json. A JToken is the base type for any json value.
Once you have the JObject you can iterate its values. If it's another JObject then navigate it recursively, otherwise save the current value to the result dictionary.
After the whole tree is visited we return the flattened result.
Executing the following program:
using System;
using System.Collections.Generic;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
namespace FlattenObject
{
class Program
{
static void Main(string[] args)
{
var json = #"
{
""abc"": {
""123"": {
""donkey"": ""hello world"",
""kong"": 123
},
""meta"": {
""aaa"": ""bbb""
}
}
}
";
var root = JsonConvert.DeserializeObject<JObject>(json);
var flattened = Flatten(root);
Console.WriteLine(JsonConvert.SerializeObject(flattened, Formatting.Indented));
}
static Dictionary<string, JToken> Flatten(JObject root)
{
var result = new Dictionary<string, JToken>();
void FlattenRec(string path, JToken value)
{
if (value is JObject dict)
{
foreach (var pair in dict)
{
string joinedPath = path != null
? path + "." + pair.Key
: pair.Key;
FlattenRec(joinedPath, pair.Value);
}
}
else
{
result[path] = value;
}
}
FlattenRec(null, root);
return result;
}
}
}
gives output:
{
"abc.123.donkey": "hello world",
"abc.123.kong": 123,
"abc.meta.aaa": "bbb"
}
Note: the code above uses local functions which is a recent feature. If you can't use it then create a helper method and pass the result dictionary explicitly.

xml parsing - code refactoring issue

I have the following xml:
<?xml version="1.0" encoding="utf-8"?>
<RootData>
<PassResult>
<FirstOne>P1</FirstOne>
<SecondOne>P2</SecondOne>
<IsMale>false</IsMale>
</PassResult>
<TestResult>
<MarkOne>100</MarkOne>
<MarkTwo>50</MarkTwo>
<Slope>30</Slope>
</TestResult>
<ToneTestResult>
<TotalTime>2</TotalTime>
<CorrectPercentage>90</CorrectPercentage>
</ToneTestResult>
<QuestionnaireResult Version="1">
<Question Id="50">
<Answer Id="B" />
</Question>
<Question Id="60">
<Answer Id="A" />
</Question>
</QuestionnaireResult>
</RootData>
I have the following code which is not at all looking a good one. Lots of repeatative link queries.
How can I refactor this code in a more structured way to fill "OutputData" object. I do not want to change it to XmlSerializer now :-(.
Sample code:
// Gets the root element decendants
var elementRootData = xDocument.Descendants("RootData");
var xElements = elementRootData as IList<XElement> ?? elementRootData.ToList();
// Read first leaf node "ProfileResult"
var passResult = from xElement in xElements.Descendants("PassResult")
select new
{
FirstOne = xElement.Element("FirstOne").GetValue(),
SecondOne = xElement.Element("SecondOne").GetValue(),
IsMale = xElement.Element("IsMale").GetValue()
};
// Read second leaf note
var testResult = from xElement in xElements.Descendants("TestResult")
select new
{
MarkOne = xElement.Element("MarkOne").GetValue(),
MarkTwo = xElement.Element("MarkTwo").GetValue(),
Slope = xElement.Element("Slope").GetValue()
};
// Update OutputData object
var parseOutputData = new OutputData();
foreach (var result in passResult)
{
parseOutputData.FirstOne = result.FirstOne;
parseOutputData.SecondOne = result.SecondOne;
parseOutputData.IsMale = result.IsMale.Equals("True");
}
foreach (var result in testResult)
{
parseOutputData.MarkOne = double.Parse(result.MarkOne);
parseOutputData.MarkTwo = double.Parse(result.MarkTwo);
parseOutputData.Slope = double.Parse(result.Slope);
}
I have to write some more code like this to fill other elements data like ToneTestResult, QuestionnaireResult etc.
Can someone suggest with a sample code?
Best regards,

Given your XML is tiny, you probably don't have to worry too much about performance. You can just do the whole thing in one go, making use of the built in explicit conversions:
var data = new OutputData
{
FirstOne = (string) doc.Descendants("FirstOne").Single(),
SecondOne = (string) doc.Descendants("SecondOne").Single(),
IsMale = (bool) doc.Descendants("IsMale").Single(),
MarkOne = (double) doc.Descendants("MarkOne").Single(),
MarkTwo = (double) doc.Descendants("MarkTwo").Single(),
Slope = (double) doc.Descendants("Slope").Single()
};
As an aside, Descendants will never return anything implementing IList<XElement>, so you can definitely remove that.

Try XML Linq
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
namespace ConsoleApplication1
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
XDocument doc = XDocument.Load(FILENAME);
var root = doc.Elements("RootData").Select(x => new
{
passResults = x.Elements("PassResult").Select(y => new
{
firstOne = (string)y.Element("FirstOne"),
secondOne = (string)y.Element("SecondOne"),
isMale = (Boolean)y.Element("IsMale")
}).FirstOrDefault(),
testResult = x.Elements("TestResult").Select(y => new {
markOne = (int)y.Element("MarkOne"),
markTwo = (int)y.Element("MarkTwo"),
slope = (int)y.Element("Slope")
}).FirstOrDefault(),
toneTestResult = x.Elements("ToneTestResult").Select(y => new {
totalTime = (int)y.Element("TotalTime"),
correctPecentage = (int)y.Element("CorrectPercentage")
}).FirstOrDefault(),
questionnaireResult = x.Elements("QuestionnaireResult").Elements("Question").Select(y => new {
question = (int)y.Attribute("Id"),
answer = (string)y.Descendants("Answer").FirstOrDefault().Attribute("Id")
}).ToList(),
}).FirstOrDefault();
}
}
}