We Keep Coding

How to read nodes in XSD document [duplicate]

How does XPath deal with XML namespaces?
If I use
/IntuitResponse/QueryResponse/Bill/Id
to parse the XML document below I get 0 nodes back.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<IntuitResponse xmlns="http://schema.intuit.com/finance/v3"
time="2016-10-14T10:48:39.109-07:00">
<QueryResponse startPosition="1" maxResults="79" totalCount="79">
<Bill domain="QBO" sparse="false">
<Id>=1</Id>
</Bill>
</QueryResponse>
</IntuitResponse>
However, I'm not specifying the namespace in the XPath (i.e. http://schema.intuit.com/finance/v3 is not a prefix of each token of the path). How can XPath know which Id I want if I don't tell it explicitly? I suppose in this case (since there is only one namespace) XPath could get away with ignoring the xmlns entirely. But if there are multiple namespaces, things could get ugly.

XPath 1.0/2.0
Defining namespaces in XPath (recommended)
XPath itself doesn't have a way to bind a namespace prefix with a namespace. Such facilities are provided by the hosting library.
It is recommended that you use those facilities and define namespace prefixes that can then be used to qualify XML element and attribute names as necessary.
Here are some of the various mechanisms which XPath hosts provide for specifying namespace prefix bindings to namespace URIs.
(OP's original XPath, /IntuitResponse/QueryResponse/Bill/Id, has been elided to /IntuitResponse/QueryResponse.)
C#:
XmlNamespaceManager nsmgr = new XmlNamespaceManager(doc.NameTable);
nsmgr.AddNamespace("i", "http://schema.intuit.com/finance/v3");
XmlNodeList nodes = el.SelectNodes(#"/i:IntuitResponse/i:QueryResponse", nsmgr);
Google Docs:
Unfortunately, IMPORTXML() does not provide a namespace prefix binding mechanism. See next section, Defeating namespaces in XPath, for how to use local-name() as a work-around.
Java (SAX):
NamespaceSupport support = new NamespaceSupport();
support.pushContext();
support.declarePrefix("i", "http://schema.intuit.com/finance/v3");
Java (XPath):
xpath.setNamespaceContext(new NamespaceContext() {
public String getNamespaceURI(String prefix) {
switch (prefix) {
case "i": return "http://schema.intuit.com/finance/v3";
// ...
}
});
Remember to call
DocumentBuilderFactory.setNamespaceAware(true).
See also:
Java XPath: Queries with default namespace xmlns
JavaScript:
See Implementing a User Defined Namespace Resolver:
function nsResolver(prefix) {
var ns = {
'i' : 'http://schema.intuit.com/finance/v3'
};
return ns[prefix] || null;
}
document.evaluate( '/i:IntuitResponse/i:QueryResponse',
document, nsResolver, XPathResult.ANY_TYPE,
null );
Note that if the default namespace has an associated namespace prefix defined, using the nsResolver() returned by Document.createNSResolver() can obviate the need for a customer nsResolver().
Perl (LibXML):
my $xc = XML::LibXML::XPathContext->new($doc);
$xc->registerNs('i', 'http://schema.intuit.com/finance/v3');
my #nodes = $xc->findnodes('/i:IntuitResponse/i:QueryResponse');
Python (lxml):
from lxml import etree
f = StringIO('<IntuitResponse>...</IntuitResponse>')
doc = etree.parse(f)
r = doc.xpath('/i:IntuitResponse/i:QueryResponse',
namespaces={'i':'http://schema.intuit.com/finance/v3'})
Python (ElementTree):
namespaces = {'i': 'http://schema.intuit.com/finance/v3'}
root.findall('/i:IntuitResponse/i:QueryResponse', namespaces)
Python (Scrapy):
response.selector.register_namespace('i', 'http://schema.intuit.com/finance/v3')
response.xpath('/i:IntuitResponse/i:QueryResponse').getall()
PhP:
Adapted from #Tomalak's answer using DOMDocument:
$result = new DOMDocument();
$result->loadXML($xml);
$xpath = new DOMXpath($result);
$xpath->registerNamespace("i", "http://schema.intuit.com/finance/v3");
$result = $xpath->query("/i:IntuitResponse/i:QueryResponse");
See also #IMSoP's canonical Q/A on PHP SimpleXML namespaces.
Ruby (Nokogiri):
puts doc.xpath('/i:IntuitResponse/i:QueryResponse',
'i' => "http://schema.intuit.com/finance/v3")
Note that Nokogiri supports removal of namespaces,
doc.remove_namespaces!
but see the below warnings discouraging the defeating of XML namespaces.
VBA:
xmlNS = "xmlns:i='http://schema.intuit.com/finance/v3'"
doc.setProperty "SelectionNamespaces", xmlNS
Set queryResponseElement =doc.SelectSingleNode("/i:IntuitResponse/i:QueryResponse")
VB.NET:
xmlDoc = New XmlDocument()
xmlDoc.Load("file.xml")
nsmgr = New XmlNamespaceManager(New XmlNameTable())
nsmgr.AddNamespace("i", "http://schema.intuit.com/finance/v3");
nodes = xmlDoc.DocumentElement.SelectNodes("/i:IntuitResponse/i:QueryResponse",
nsmgr)
SoapUI (doc):
declare namespace i='http://schema.intuit.com/finance/v3';
/i:IntuitResponse/i:QueryResponse
xmlstarlet:
-N i="http://schema.intuit.com/finance/v3"
XSLT:
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:i="http://schema.intuit.com/finance/v3">
...
Once you've declared a namespace prefix, your XPath can be written to use it:
/i:IntuitResponse/i:QueryResponse
Defeating namespaces in XPath (not recommended)
An alternative is to write predicates that test against local-name():
/*[local-name()='IntuitResponse']/*[local-name()='QueryResponse']
Or, in XPath 2.0:
/*:IntuitResponse/*:QueryResponse
Skirting namespaces in this manner works but is not recommended because it
Under-specifies the full element/attribute name.
Fails to differentiate between element/attribute names in different
namespaces (the very purpose of namespaces). Note that this concern could be addressed by adding an additional predicate to check the namespace URI explicitly:
/*[ namespace-uri()='http://schema.intuit.com/finance/v3'
and local-name()='IntuitResponse']
/*[ namespace-uri()='http://schema.intuit.com/finance/v3'
and local-name()='QueryResponse']
Thanks to Daniel Haley for the namespace-uri() note.
Is excessively verbose.
XPath 3.0/3.1
Libraries and tools that support modern XPath 3.0/3.1 allow the specification of a namespace URI directly in an XPath expression:
/Q{http://schema.intuit.com/finance/v3}IntuitResponse/Q{http://schema.intuit.com/finance/v3}QueryResponse
While Q{http://schema.intuit.com/finance/v3} is much more verbose than using an XML namespace prefix, it has the advantage of being independent of the namespace prefix binding mechanism of the hosting library. The Q{} notation is known as Clark Notation after its originator, James Clark. The W3C XPath 3.1 EBNF grammar calls it a BracedURILiteral.
Thanks to Michael Kay for the suggestion to cover XPath 3.0/3.1's BracedURILiteral.

I use /*[name()='...'] in a google sheet to fetch some counts from Wikidata. I have a table like this
thes WD prop links items
NOM P7749 3925 3789
AAT P1014 21157 20224
and the formulas in cols links and items are
=IMPORTXML("https://query.wikidata.org/sparql?query=SELECT(COUNT(*)as?c){?item wdt:"&$B14&"[]}","//*[name()='literal']")
=IMPORTXML("https://query.wikidata.org/sparql?query=SELECT(COUNT(distinct?item)as?c){?item wdt:"&$B14&"[]}","//*[name()='literal']")
respectively. The SPARQL query happens not to have any spaces...
I saw name() used instead of local-name() in Xml Namespace breaking my xpath!, and for some reason //*:literal doesn't work.

Custom SonarQube rules generated with Roslyn SDK Generator have always issue type "Code Smell"

I'm trying to create a custom SonarQube rule in VisualStudio 2015, using the Roslyn SDK Generator.
The generator works fine and I'm able to publish the .jar file to SonarQube server and use my custom rule in daily builds.
Now I would like to categorize the rule as "Vulnerabilty", but it always appear as "Code Smell".
I tried a couple of approaches:
Changed the "Category" of the DiagnosticDescriptor class to "Security"
private const string Category = "Security";
private static DiagnosticDescriptor Rule = new DiagnosticDescriptor(DiagnosticId, Title, MessageFormat, Category, DiagnosticSeverity.Warning, isEnabledByDefault: true, description: Description);
public override ImmutableArray<DiagnosticDescriptor> SupportedDiagnostics { get { return ImmutableArray.Create(Rule); } }
Changed the xml template provided by the generator and regenerated the plugin using the new xml (I tried "SECURITY" and "SECURITY_COMPLIANCE" in place of the generated "MAINTENABILITY_COMPLIANCE")
<sqale xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<chc>
<key>SECURITY</key>
<chc>
<rule-key>MyRule</rule-key>
<prop>
<key>remediationFunction</key>
<txt>CONSTANT_ISSUE</txt>
</prop>
<prop>
<key>offset</key>
<txt />
<val>15min</val>
</prop>
</chc>
</chc>
</sqale>
Nothing worked so far.
I'm using the following configuration:
VS2015 Update 3
SonarQube v. 6.1
SonarLint v. 2.8
Custom C# analyzer developed with SonarQube.Roslyn.SDK v. 1.0

Unfortunately seems that ability to explicitly set category is not yet implemented - see
https://jira.sonarsource.com/browse/SFSRAP-48
As a workaround you can add tag security to a rule and rule will be categorized as Vulnerabilty thanks to automatic conversion of tag into category in SonarQube. However it seems that SonarQube.Plugins.Roslyn.RuleGenerator is not considering the CustomTags property when building the SonarQube rule, but addition of newRule.Tags = diagnostic.CustomTags?.ToArray(); to the method SonarQube.Plugins.Roslyn.RuleGenerator.GetAnalyzerRules and rebuild of sonarqube-roslyn-sdk will do the job.

C# LINQ and XML Getting child nodes

Having issues getting node values. Not sure why the following code is failing to do so.
<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet type='text/xsl' href='STIG_unclass.xsl'?>
<Benchmark xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:cpe="http://cpe.mitre.org/language/2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" id="Windows_7_STIG" xml:lang="en" xsi:schemaLocation="http://checklists.nist.gov/xccdf/1.1 http://nvd.nist.gov/schema/xccdf-1.1.4.xsd http://cpe.mitre.org/dictionary/2.0 http://cpe.mitre.org/files/cpe-dictionary_2.1.xsd" xmlns="http://checklists.nist.gov/xccdf/1.1">
<status date="2015-06-16">accepted</status>
<title>Windows 7 Security Technical Implementation Guide</title>
<description>
The Windows 7 Security Technical Implementation Guide (STIG) is published as a tool to improve the security of Department of Defense (DoD) information systems. The requirements were developed from DoD consensus, as well as the Windows 7 Security Guide and security templates published by Microsoft Corporation. Comments or proposed revisions to this document should be sent via e-mail to the following address: disa.stig_spt#mail.mil.
</description>
<notice id="terms-of-use" xml:lang="en">Developed_by_DISA_for_the_DoD</notice>
<reference href="http://iase.disa.mil">
<dc:publisher>DISA, Field Security Operations</dc:publisher>
<dc:source>STIG.DOD.MIL</dc:source>
</reference>
<plain-text id="release-info">Release: 20 Benchmark Date: 24 Jul 2015</plain-text>
</Benchmark>
Sample XML File.
and the following is my code.
String Title = LoadedXML.Element("Benchmark").Attribute("id").Value;
var XMLData = LoadedXML.Element("Benchmark").Elements("plain-text")
.Single(release => release.Attribute("id").Value == "release-info").Value;
is there a way I can get multiple Node values at the same time? Like getting the Title and Release Value at once instead of having a separate one for each?

Your code is failing because your XML contains Namespace and you can't access your nodes directly. If you want to confirm this simply query LoadedXML.Elements() and examine the values in debugger, you can clearly see the namespaces there:-
So, You need to declare the namespace and use it:-
XNamespace ns = "http://checklists.nist.gov/xccdf/1.1";
If you want both vales to be fetched at once you can project it to a anonymous type like this:-
var result = LoadedXML.Root.Elements(ns + "plain-text")
.Where(x => (string)x.Attribute("id") == "release-info")
.Select(x => new
{
Title = (string)x.Document.Root.Attribute("id"),
XMLData = x.Value
}).FirstOrDefault();
This query is giving me below output:-

Linq-to-xml is generally used to query a XML to filter it's nodes and then get the desired element/values per need. It's more like querying a table with SQL.
If all/most of the XML is required as a result, then the better approach would be to deseralize the XMl into a native (C# here) object and map it to the required model object. XML can always be thought of a serialized version of an object (although it can be manually as well), and can be deserialized back to the actual object.
.Net has native support for all these, see msdn links for XML Serialization and Deserialization for details. You can write a small method to deserialize your object like this.
using System.Xml.Linq;
using System.Xml.Serialization;
public class XMLHelper
{
public T DeserializeData<T>(string data)
{
XmlSerializer serializer = new XmlSerializer(typeof(T));
StringReader reader = new StringReader(data);
var deserializedObject = serializer.Deserialize(reader);
return deserializedObject == null ? default(T) : (T)deserializedObject;
}
}
To get the string you can do like File.ReadAllText(xmlFilePath) or whatever is easier for the situation.
This will give you deseialized object of the whole XML. If you want some other transformed object, you can either manually map that, or use AutoMapper

Cannot modify deposit using interop.qbfc12

In a c# program using interop.qbfc12, the following code:
<?xml version="1.0" encoding="utf-8"?>
<?qbxml version="12.0"?>
<QBXML>
<QBXMLMsgsRq onError = "continueOnError">
<DepositModRq requestID = "0">
<DepositMod>
<TxnID>B2864-1388784731</TxnID>
<EditSequence>1388784731</EditSequence>
<TxnDate>2014-01-03</TxnDate>
<DepositToAccountRef>
<FullName>Checking Acct-CCFCSB</FullName>
</DepositToAccountRef>
<Memo>Test deposit memo for 01032014-1</Memo>
<DepositLineMod>
<TxnLineID>B2866-1388784731</TxnLineID>
</DepositLineMod>
<DepositLineMod>
<TxnLineID>B2867-1388784731</TxnLineID>
</DepositLineMod>
</DepositMod>
</DepositModRq>
</QBXMLMsgsRq>
</QBXML>
produces this error:
DepositMod
DepositLineModList:
element(1) - ORDepositLineMod: required field is missing
End of DepositLineModList
End of DepositMod
When I use the same code in a vba project using qbfc12 type library, I get no error.
Does the interop library differ from the qbfc library? What must I do to resolve this error.

You might need to look at the XML being generated by the QBFC12 library to see what the difference is.
You are giving DepositLineMod's, but what are you modifying?

Visual Studio XML external comments file does not work

By using include tag I am trying to put comments for my code in separate file "docs.xml".
But it does not work. I have been trying both C# and VB.NET projects.
Here is my comments file:
<?xml version="1.0" encoding="utf-8" ?>
<d>
<summary>Demo summary</summary>
</d>
I have a class ABC with one single property Demo. before this property I write:
/// <include file="docs.xml" path="d/*" />
or in VB.NET:
''' <include file="docs.xml" path="d/*" />
However summary for ABC.Demo never appears in InteliSense / Object browser / another project (if I reference my project).
I have a strong feeling I am missing something here.
P.S. I have tried following "path[#name=]" pattern of XML file, but it does not help.

Perhaps you already saw this in the documentation then, but if I understand correctly, you have to do the following:
In Visual Studio, you specify the XML doc comments option in the Build
pane of the Project Designer. When the C# compiler sees the
tag, it will search for documentation comments in xml_include_tag.doc
instead of the current source file.