We Keep Coding

After importing DataSet from XML, new rows are not nested properly

I'm using DataSet.ReadXml() to import an XML file into a new DataSet. Then I'm adding a new row to one of the tables inside the DataSet, and then I want to export that DataSet to XML again. The problem is that the new row is not nested properly and just gets appended to the end of the XML file.
Here is the program:
using System;
using System.Data;
using System.IO;
using System.Xml;
public class Program
{
public static void Main()
{
string xml = #"<?xml version=""1.0"" encoding=""UTF-8"" standalone=""yes""?>
<DATAPACKET Version=""2.0"">
<METADATA>
<FIELDS>
<FIELD attrname=""CompanyID"" fieldtype=""string"" WIDTH=""10""/>
<FIELD attrname=""Description"" fieldtype=""string"" WIDTH=""40""/>
</FIELDS>
<PARAMS/>
</METADATA>
<ROWDATA>
<ROW CompanyID=""CC"" Description=""Contoso""/>
</ROWDATA>
</DATAPACKET>
";
XmlReader reader = XmlReader.Create(new StringReader(xml));
DataSet dataSet = new DataSet();
dataSet.ReadXml(reader, XmlReadMode.InferTypedSchema);
var rowTable = dataSet.Tables["ROW"];
var newRow = rowTable.NewRow();
newRow["CompanyID"] = "APPL";
newRow["Description"] = "Apple";
rowTable.Rows.Add(newRow);
Console.WriteLine(dataSet.GetXml());
}
}
And here is the output:
<DATAPACKET Version="2.0">
<METADATA>
<PARAMS />
<FIELDS>
<FIELD attrname="CompanyID" fieldtype="string" WIDTH="10" />
<FIELD attrname="Description" fieldtype="string" WIDTH="40" />
</FIELDS>
</METADATA>
<ROWDATA>
<ROW CompanyID="CC" Description="Contoso" />
</ROWDATA>
</DATAPACKET>
<ROW CompanyID="APPL" Description="Apple" />
What I want is for the new row to be nested with the other rows from that table like this:
<DATAPACKET Version="2.0">
<METADATA>
<PARAMS />
<FIELDS>
<FIELD attrname="CompanyID" fieldtype="string" WIDTH="10" />
<FIELD attrname="Description" fieldtype="string" WIDTH="40" />
</FIELDS>
</METADATA>
<ROWDATA>
<ROW CompanyID="CC" Description="Contoso" />
<ROW CompanyID="APPL" Description="Apple" />
</ROWDATA>
</DATAPACKET>
What am I doing wrong?
How do I get well formed XML out of DataSet.GetXml() ?
Here is the program running over at dotnetfiddle

I solved my own question. The problem was that I was unaware that the auto generated relationships between the tables created foreign key columns that needed to be populated.
For the ROW table, the auto generated foreign key is ROWDATA_Id.
Here is the updated code that works as expected:
using System;
using System.Data;
using System.IO;
using System.Xml;
public class Program
{
public static void Main()
{
string xml = #"<?xml version=""1.0"" encoding=""UTF-8"" standalone=""yes""?>
<DATAPACKET Version=""2.0"">
<METADATA>
<FIELDS>
<FIELD attrname=""CompanyID"" fieldtype=""string"" WIDTH=""10""/>
<FIELD attrname=""Description"" fieldtype=""string"" WIDTH=""40""/>
</FIELDS>
<PARAMS/>
</METADATA>
<ROWDATA>
<ROW CompanyID=""CC"" Description=""Contoso""/>
</ROWDATA>
</DATAPACKET>
";
XmlReader reader = XmlReader.Create(new StringReader(xml));
DataSet dataSet = new DataSet();
dataSet.ReadXml(reader, XmlReadMode.InferTypedSchema);
var rowTable = dataSet.Tables["ROW"];
var newRow = rowTable.NewRow();
newRow["CompanyID"] = "APPL";
newRow["Description"] = "Apple";
newRow["ROWDATA_Id"] = 0; //This is what I was missing. This nests the row properly
rowTable.Rows.Add(newRow);
Console.WriteLine(dataSet.GetXml());
}
}
An alternate solution is to set the DataColumn.DefaultValue to 0 for the foreign key column ROWDATA_Id
var rowTable = dataSet.Tables["ROW"];
rowTable.Columns["ROWDATA_Id"].DefaultValue = 0;
Here is the output for both solutions:
<DATAPACKET Version="2.0">
<METADATA>
<PARAMS />
<FIELDS>
<FIELD attrname="CompanyID" fieldtype="string" WIDTH="10" />
<FIELD attrname="Description" fieldtype="string" WIDTH="40" />
</FIELDS>
</METADATA>
<ROWDATA>
<ROW CompanyID="CC" Description="Contoso" />
<ROW CompanyID="APPL" Description="Apple" />
</ROWDATA>
</DATAPACKET>
Here is the first solution running on dotnetfiddle
Here is the alternate solution running on dotnetfiddle

The ReadXml fragments your xml into a lot of tables. ReadXml does following with following nested tags 1) DataSet name 2) DataTable name 3) Row Data : Column name is tag and innertext is value
See my code below which parses xml with xml linq :
using System;
using System.Data;
using System.IO;
using System.Xml;
using System.Xml.Linq;
namespace ConsoleApplication1
{
class Program
{
public static void Main()
{
string xml = #"<?xml version=""1.0"" encoding=""UTF-8"" standalone=""yes""?>
<DATAPACKET Version=""2.0"">
<METADATA>
<FIELDS>
<FIELD attrname=""CompanyID"" fieldtype=""string"" WIDTH=""10""/>
<FIELD attrname=""Description"" fieldtype=""string"" WIDTH=""40""/>
</FIELDS>
<PARAMS/>
</METADATA>
<ROWDATA>
<ROW CompanyID=""CC"" Description=""Contoso""/>
</ROWDATA>
</DATAPACKET>
";
XmlReader reader = XmlReader.Create(new StringReader(xml));
DataSet dataSet = new DataSet();
dataSet.ReadXml(reader, XmlReadMode.InferTypedSchema);
var rowTable = dataSet.Tables["ROW"];
var newRow = rowTable.NewRow();
newRow["CompanyID"] = "APPL";
newRow["Description"] = "Apple";
rowTable.Rows.Add(newRow);
Console.WriteLine(dataSet.GetXml());
XDocument doc = XDocument.Parse(xml);
DataTable rowTable2 = new DataTable("Table1");
DataRow newRow2 = null;
foreach (XElement field in doc.Descendants("FIELD"))
{
string t = (string)field.Attribute("fieldtype");
Type _type = null;
switch (t)
{
case "string" :
_type = typeof(string);
break;
}
rowTable2.Columns.Add((string)field.Attribute("attrname"), _type);
}
foreach (XElement row in doc.Descendants("ROW"))
{
newRow = rowTable2.Rows.Add();
foreach (XAttribute attribute in row.Attributes())
{
newRow[attribute.Name.LocalName] = (string)attribute;
}
}
newRow = rowTable2.Rows.Add();
newRow["CompanyID"] = "APPL";
newRow["Description"] = "Apple";
DataSet ds = new DataSet();
ds.Tables.Add(rowTable2);
Console.WriteLine(ds.GetXml());
}
}
}

Where did you get that XML from? It is formatted in a way not supported by the DataSet. When nesting tables, you must define a parent-child relationship between the tables and you must set the Nested property of the child table to true. In your XML, the DataSet has no idea which parent the new child row belongs to, so it appends it to the end.
You can read in the MSDN about Nesting DataRelations.
Having said that, your XML doesn't actually have parent and child tables. It has METADATA and ROWDATA. Like I said, that format is not supported by DataSet, you will have to move your meta data to a Schema (XSD). You can read in the MSDN about Deriving DataSet Relational Structure from XML Schema.
Here is an example of how you car represent your data in question with XSD and XML:
using System;
using System.Data;
using System.IO;
using System.Xml;
public class Program
{
public static void Main()
{
string xml = #"<?xml version=""1.0"" encoding=""UTF-8"" standalone=""yes""?>
<MyDataSet>
<Companies>
<CompanyID>CC</CompanyID>
<Description>Contoso</Description>
</Companies>
</MyDataSet>
";
string xsd = #"<?xml version=""1.0"" encoding=""utf-8""?>
<xs:schema id=""SomeID""
xmlns=""""
xmlns:xs=""http://www.w3.org/2001/XMLSchema""
xmlns:msdata=""urn:schemas-microsoft-com:xml-msdata"">
<xs:element name=""MyDataSet"" msdata:IsDataSet=""true"">
<xs:complexType>
<xs:choice minOccurs=""0"" maxOccurs=""unbounded"">
<xs:element name=""Companies"">
<xs:complexType >
<xs:sequence>
<xs:element name=""CompanyID"" type=""xs:string"" minOccurs=""0"" />
<xs:element name=""Description"" type=""xs:string"" minOccurs=""0"" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:complexType>
</xs:element>
</xs:schema>
";
DataSet dataSet = new DataSet();
StringReader sr = new StringReader(xsd);
dataSet.ReadXmlSchema(sr);
sr = new StringReader(xml);
dataSet.ReadXml(sr, XmlReadMode.InferTypedSchema);
var rowTable = dataSet.Tables["Companies"];
var newRow = rowTable.NewRow();
newRow["CompanyID"] = "APPL";
newRow["Description"] = "Apple";
rowTable.Rows.Add(newRow);
Console.WriteLine(dataSet.GetXml());
}
}
In this case, you don't really need a Schema because you only have one table with all columns as string. So if you remove the Schema from the code above and run it again, you'll get the exact same results. However, this gives you an idea on how to define your DataSet structure using a Schema, so you can add more complex tables with relationship between them. For simple tables with no relationships, you don't need a Schema.

C# Load DataTable from LinQ XDocument

I have a complex XML which is loaded into XDocument and I want to turn it into a DataTable please. Here is the actual XML:
<?xml version="1.0"?>
<XMLCONTAINER version="1.0" incCount="3">
<APPLICATION id="APPLICATION_NODE_1" title="" name="55038170812RDOpayment" application="Excel" appfile="%SAAR%\xmlsFs\Dbxmls.xla" command="openDatabaseDesigner">
<MAINDATA id="CONTAINER_NODE_2" title=""/>
<SKY.FORM id="CONTAINER_NODE_3" title="" table="tblMain" database="\\c\E\DARA\Data\Link\RDOpayments.mdb">
<FIELD fieldName="RID">1234567</FIELD>
<FIELD fieldName="DateTime">05/02/2019 09:00:50</FIELD>
<FIELD fieldName="DateOfLetter">29/01/2019</FIELD>
<FIELD fieldName="Name">Mr Joe Bloggs</FIELD>
<FIELD fieldName="NRID">XXX1234X</FIELD>
<FIELD fieldName="paymentAmount">776.40</FIELD>
<FIELD fieldName="Amountfor0809">776.40</FIELD>
<FIELD fieldName="ACE">FALSE</FIELD>
<FIELD fieldName="Telephone">123456789</FIELD>
<FIELD fieldName="AcceptWithheldNumber">FALSE</FIELD>
<FIELD fieldName="PotentialACECase">FALSE</FIELD>
<FIELD childID="CONTAINER_NODE_3_field7option1" fieldName="CustomerType">3</FIELD>
<FIELD fieldName="ContactHistory">05/02/2019</FIELD>
<FIELD fieldName="ACERed">FALSE</FIELD>
</SKY.FORM>
The DataTable needs to look like this:
RID DateTime DateOfLetter Name NRID paymentAmount ACE
--- ----------- ------------ ---- ---- ------------- ----------
1234567 2019-02-05 10:23:51 2019-02-05 MISS LL TEST X1234X 123.45 FALSE
7654321 2019-01-11 11:11:11 2019-02-03 MR I WONG Y4321Y 321.21 TRUE
My current attempts of coding this are not picking up the xml as Node? I think it should be attributes xattribute??
// Conversion Xml file to DataTable
public DataTable CreateDataTableXML(string XmlFile)
{
XmlDocument doc = new XmlDocument();
doc.Load(XmlFile);
DataTable Dt = new DataTable();
try
{
Dt.TableName = GetTableName(XmlFile);
XmlNode NodeStructure = doc.DocumentElement.ChildNodes.Cast<XmlNode>().ToList()[0];
progressBar1.Maximum = NodeStructure.ChildNodes.Count;
progressBar1.Value = 0;
foreach (XmlNode column in NodeStructure.ChildNodes)
{
Dt.Columns.Add(column.Name, typeof(String));
Progress();
}
XmlNode Xnodes = doc.DocumentElement;
progressBar1.Maximum = Xnodes.ChildNodes.Count;
progressBar1.Value = 0;
foreach (XmlNode xnode in Xnodes.ChildNodes)
{
List<string> Values = xnode.ChildNodes.Cast<XmlNode>().ToList().Select(x => x.InnerText).ToList();
Dt.Rows.Add(Values.ToArray());
Progress();
}
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
return Dt;
}

So I finally solved this. For anyone reading this, you need to solve in two parts, first get the column names into the datatable and then the xml values.
For the column names as we already have a table in sql we want to upload to, I simply used a dataadapter to fill the datatable with the schema which made sure every column name was there in the dt.
For the xml values, this proved more tricky as in a normal xml I could have just looped on the fieldname=, however as each attribute tag is different I extracted each manually with .attribute ?.value. It was a simple case then of creating datarows which would coalesce to DBNull if data was null.

Editing xml on live in C#. Deleting nodes that contain specific value

I have an xml document of type like this:
<?xml version="1.0" encoding="UTF-16"?>
<Recordset>
<Table>Recordset</Table>
<Rows>
<Row>
<Fields>
...
<Field>
<Alias>StatusName</Alias>
<Value>Scheduled</Value>
</Field>
<Field>
<Alias>U_Revision</Alias>
<Value>code00</Value>
</Field>
<Field>
<Alias>U_Quantity</Alias>
<Value>10.000000</Value>
</Field>
<Field>
<Alias>U_ActualQty</Alias>
<Value>0.000000</Value>
</Field>
...
</Fields>
</Row>
...
<Row>
<Fields>
...
<Field>
<Alias>StatusName</Alias>
<Value>Scheduled</Value>
</Field>
<Field>
<Alias>U_Revision</Alias>
<Value>code00</Value>
</Field>
<Field>
<Alias>U_Quantity</Alias>
<Value>150.000000</Value>
</Field>
<Field>
<Alias>U_ActualQty</Alias>
<Value>0.000000</Value>
</Field>
...
</Fields>
</Row>
</Rows>
</Recordset>
I have different values in field with alias of StatusName. There are some Scheduled, notScheduled, Realeased, Finished etc values. What I would like to do is to delete each node that contain node with alias StatusName and value lets say Scheduled or Finished.
I was thinking to do this more or less in that way however I am doing something wrong. May anybody let me on right way ?
XmlDocument xmlDocument = new XmlDocument();
xmlDocument.LoadXml(xml);
XmlNodeList nodes = xmlDocument.SelectNodes("//Rows[#StatusName='Finished']");
for (int i = nodes.Count - 1; i >= 0; i--)
{
nodes[i].ParentNode.RemoveChild(nodes[i]);
}
var newXml = nodes.ToString();
I would like to delete the whole node if contains with alias StatusName and specific value lets say Finished.
I would expect the result in new string variable.

I like to work with DataTable with xml, I found it very easy.
I used a DataTable to work with your nodes.
So, I took your xml file and wrote some code for you that might help you:
//READ THE XML FILE
XmlDocument xmlDoc = new XmlDocument();
//My path
xmlDoc.LoadXml(Properties.Resources.test);
//Read the xml file into a dataSet
DataSet ds = new DataSet();
XmlNodeReader xnr = new XmlNodeReader(xmlDoc);
ds.ReadXml(xnr);
//Your data will be store in the 4's dataTable of the dataSet ( the <field> )
for(int i=0;i<ds.Tables[4].Rows.Count;i++)
{
//Check the value as you wish
//Here i want to suppress all the <Field> nodes with <Value> = "Scheduled"
if ( ds.Tables[4].Rows[i]["Value"].ToString().Equals("Scheduled"))
{
//RemoteAt will remove all the node, so the node <Field> in your example data
ds.Tables[4].Rows.RemoveAt(i);
//If you want to only remove the node <Value> (and not all the <Field> node ) just do ds.Tables[4].Rows["Value"]=null;
}
}
//Write your new content in a new xml file
//As you wanted here you just read the new xml file created as a string
using (var stringWriter = new StringWriter())
using (var xmlTextWriter = XmlWriter.Create(stringWriter))
{
ds.WriteXml(xmlTextWriter);
xmlTextWriter.Flush();
stringWriter.GetStringBuilder().ToString();
//Here the result is in stringWriter, and there is 6 <Field> nodes, and not 8 like before the suppress
}
//If you want to create a new xml file with the new content just do
ds.WriteXml(yourPathOfXmlFile);
//( like rewriting the previous xml file )

I assume, you are going to delete entire <Row> which matches your condition
i.e.,
<Row>
<Fields>
...
<Field>
<Alias>StatusName</Alias>
<Value>Finished</Value>
</Field>
</Fields>
</Row>
The required XPath:
//Row[Fields[Field[Alias[text()='StatusName'] and Value[text() = 'Finished']]]]
C#
string xPath = #"//Row[Fields[Field[Alias[text()='StatusName'] and Value[text() = 'Finished']]]]";
var nodes = xmlDocument.SelectNodes(xPath);
for (int i = nodes.Count - 1; i >= 0; i--)
{
nodes[i].ParentNode.RemoveChild(nodes[i]);
}
var newXml = xmlDocument.OuterXml;

Entity Framework returns incomplete xml data

EntityFramework returns incomplete data for xml field.
I Serialize data and save this in an xml field in the db. The missing data is in the Images node value in the xml is also serialized a serilized object. The ecoding of the value happens when i serialze the field object.
Where is my missing data from the Images -> Value field and why does it dissapear?
This is what i have in my ms sql xml field:
<ArrayOfField xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Field>
<Key>Images</Key>
<Value><ArrayOfImage xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Image>
<Name>Penguins.jpg</Name>
<Path>~/Fileshare/Pages/6/Penguins.jpg</Path>
<AltText>Test</AltText>
</Image>
<Image>
<Name>Tulips.jpg</Name>
<Path>~/Fileshare/Pages/6/Tulips.jpg</Path>
<AltText>Test</AltText>
</Image>
</ArrayOfImage></Value>
</Field>
<Field>
<Key>Test</Key>
<Value>Test</Value>
</Field>
<Field>
<Key>MyEditor</Key>
<Value />
</Field>
</ArrayOfField>
This is what EF returns:
<ArrayOfField xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><Field><Key>Images</Key><Value><ArrayOfImage xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" /></Value></Field><Field><Key>Test</Key><Value>Test</Value></Field><Field><Key>MyEditor</Key><Value /></Field></ArrayOfField>
I get data from EntityFramework like this:
public Item Get(int id)
{
using (var context = new Entities())
{
var item = context.Items.SingleOrDefault(x => x.ID == id);
return item;
}
}

I think you need to write a stored procedure to return the xml field data. For Entity Framework, you need to write something like following to get the full list of xml data
EntityCommand command = connection.CreateCommand();
command.CommandText = "XXXX";
command.CommandType = CommandType.StoredProcedure;
using (EntityDataReader reader = command.ExecuteReader(CommandBehavior.SequentialAccess))
{
while (reader.Read())
{
str = str + reader.GetString(0);
}
}

I think either the data is lost during deserialization, or it's a mistake due to fatigue on your part (i mean, you might be looking at a different entity when comparing and you think there is something wrong).
Either way, an alternative to rule out the serialization issue is to save it serialized as something else or avoid having an xml in an xml (even though there shouldn't be any problem) by just adding the ArrayOfImages as a class member and not just another key/value pair in your array.

Update XML attribute in a repeating section using Microsoft.BizTalk.Streaming.ValueMutator

I have a problem where I am attempting to update a set of attributes with a fixed value contained within a repeating section of an XML document using the Microsoft.BizTalk.Streaming.ValueMutator.
For example the XML document which I am attempting to update contains the following input:
<ns0:TestXML xmlns:ns0="http://Test.Schemas">
<ns0:NodeA>
<ns0:NodeB>
<ns0:alpha Id="1" Value="Apple" Type=""></ns0:alpha>
<ns0:alpha Id="2" Value="Banana" Type=""></ns0:alpha>
<ns0:alpha Id="3" Value="Car" Type=""></ns0:alpha>
<ns0:alpha Id="4" Value="Duck" Type=""></ns0:alpha>
</ns0:NodeB>
</ns0:NodeA>
</ns0:TestXML>
The code which I am attempting to use to update the XML document is:
XmlDocument xDocInput = new XmlDocument();
XmlDocument xDocOutput = new XmlDocument();
string inputFileName = #"C:\Input.xml";
string outputFileName = #"C:\Output.xml";
string newValue = "fruit";
string xpathToUpdate = "/*[namespace-uri()='http://Test.Schemas']/*[local-name()='NodeA']/*[local-name()='NodeB']/*[#Type]";
xDocInput.Load(inputFileName);
using (MemoryStream memstream = new MemoryStream())
{
xDocInput.Save(memstream);
memstream.Position = 0;
XPathCollection queries = new XPathCollection();
queries.Add(new XPathExpression(xpathToUpdate));
//ValueMutator xpathMatcher = new ValueMutator(this.XPathCallBack);
//Get resulting stream into response xml
xDocOutput.Load(new XPathMutatorStream(memstream, queries, delegate(int matchIdx, XPathExpression expr, string origValue, ref string finalValue) { finalValue = newValue; }));
//System.Diagnostics.Trace.WriteLine("Trace: " + memstream.Length.ToString());
}
xDocOutput.Save(outputFileName);
The resulting output of this code is the file "Output.xml". Contained within the output document "Output.xml" is the following output:
<ns0:TestXML xmlns:ns0="http://Test.Schemas" >
<ns0:NodeA>
<ns0:NodeB>
<ns0:alpha Id="1" Value="Apple" Type="" >fruit</ns0:alpha>
<ns0:alpha Id="2" Value="Banana" Type="" >fruit</ns0:alpha>
<ns0:alpha Id="3" Value="Child" Type="" >fruit</ns0:alpha>
<ns0:alpha Id="4" Value="Duck" Type="" >fruit</ns0:alpha>
</ns0:NodeB>
</ns0:NodeA>
</ns0:TestXML>
As you will notice the "alpha" element's text value is updated incorrectly. The desired result is to update all attributes named "Type" with the value "Fruit". What is going wrong and how is this problem solved?

You need to include the alpha element in your XPath expression.
I ran your code using the expression below:
string xpathToUpdate = "/*[namespace-uri()='http://Test.Schemas']/*[local-name()='NodeA']/*[local-name()='NodeB']/*[local-name()='alpha']/#Type";
and got the following XML
<ns0:TestXML xmlns:ns0="http://Test.Schemas">
<ns0:NodeA>
<ns0:NodeB>
<ns0:alpha Id="1" Value="Apple" Type="fruit">
</ns0:alpha>
<ns0:alpha Id="2" Value="Banana" Type="fruit">
</ns0:alpha>
<ns0:alpha Id="3" Value="Car" Type="fruit">
</ns0:alpha>
<ns0:alpha Id="4" Value="Duck" Type="fruit">
</ns0:alpha>
</ns0:NodeB>
</ns0:NodeA>
</ns0:TestXML>
Looking at the code you posted, you have probably seen these articles on using the ValueMutator, but just in case there is good info here, here and here.
Heh - Just realised that the last of those is one of my co-workers. Small world.

The XPath expression used:
//namespace-uri()='http://Test.Schemas']/*[local-name()='NodeA']/*[local-name()='NodeB']/*[#Type]
selects only the lements that have a "Type" attribute.
Most probably what is needed is:
/*[namespace-uri()='http://Test.Schemas']/*[local-name()='NodeA']/*[local-name()='NodeB']/#Type
Hope this helped.
Cheers,
Dimitre Novatchev