We Keep Coding

How do I get Schematron.net to trigger diagnostic?

I am using the Schematron.net nuget package to validate some XML but I can't figure out how to use the diagnostic tag correctly. I have the following XML:
<bk:books xmlns:bk="http://www.example.com/books">
<bk:book publisher="QUE">
<bk:title>XML By Example</bk:title>
<!--<bk:author>Benoit Marchal</bk:author>-->
<bk:publication-date>1999-12-31</bk:publication-date>
<bk:retailPrice>9.95</bk:retailPrice>
</bk:book>
<bk:book publisher="Addison Wesley">
<bk:title>Essential C++</bk:title>
<bk:author>Stanley Lippman</bk:author>
<bk:publication-date>2000-10-31</bk:publication-date>
<bk:retailPrice>29.95</bk:retailPrice>
</bk:book>
</bk:books>
And my simplified Schematron schema looks like this:
<schema xmlns="http://www.ascc.net/xml/schematron"
schemaVersion="1.01" >
<title>A Schema for Books</title>
<ns prefix="bk" uri="http://www.example.com/books" />
<pattern id="authorTests">
<rule context="bk:book">
<assert test="count(bk:author)!= 0" diagnostics="bookTest">
A book must have at least one author
</assert>
</rule>
</pattern>
<diagnostics>
<diagnostic id="bookTest">
The book that has no author is <value-of select="bk:title"></value-of>
</diagnostic>
</diagnostics>
</schema>
The code I'm running to exercise this schema looks like this:
try
{
var bookValidator = new Validator();
bookValidator.AddSchema("book.xsd");
bookValidator.Validate("book.xml");
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
When this runs I get the following console output:
Results from Schematron validation: A Schema for Books
From pattern ""
Assert fails: A book must have at least one author
At: /bk:bk:books[1]/bk:bk:book[1]
<bk:book publisher="QUE">...</bk:book>
(Line: 3, Column: 4)
xmlns:bk="http://www.example.com/books"
Why don't I see the output from the diagnostic tag in my console?

It seems as if the NMatrix.Schematron implementation doesn't support the full set of Schematron features. After looking at the source code I can't find any reference to the diagnostics elements at all. In addition the code seems to only support XPath 1.0 and the earlier implementations of Schematron (before the namespace was changed with ISO standardisation). Best advice is to implement a Schematron pipeline using the Schematron "Skeleton" implementation as a guide, which can be found here: http://schematron.com/front-page/the-schematron-skeleton-implementation/

How to read nodes in XSD document [duplicate]

How does XPath deal with XML namespaces?
If I use
/IntuitResponse/QueryResponse/Bill/Id
to parse the XML document below I get 0 nodes back.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<IntuitResponse xmlns="http://schema.intuit.com/finance/v3"
time="2016-10-14T10:48:39.109-07:00">
<QueryResponse startPosition="1" maxResults="79" totalCount="79">
<Bill domain="QBO" sparse="false">
<Id>=1</Id>
</Bill>
</QueryResponse>
</IntuitResponse>
However, I'm not specifying the namespace in the XPath (i.e. http://schema.intuit.com/finance/v3 is not a prefix of each token of the path). How can XPath know which Id I want if I don't tell it explicitly? I suppose in this case (since there is only one namespace) XPath could get away with ignoring the xmlns entirely. But if there are multiple namespaces, things could get ugly.

XPath 1.0/2.0
Defining namespaces in XPath (recommended)
XPath itself doesn't have a way to bind a namespace prefix with a namespace. Such facilities are provided by the hosting library.
It is recommended that you use those facilities and define namespace prefixes that can then be used to qualify XML element and attribute names as necessary.
Here are some of the various mechanisms which XPath hosts provide for specifying namespace prefix bindings to namespace URIs.
(OP's original XPath, /IntuitResponse/QueryResponse/Bill/Id, has been elided to /IntuitResponse/QueryResponse.)
C#:
XmlNamespaceManager nsmgr = new XmlNamespaceManager(doc.NameTable);
nsmgr.AddNamespace("i", "http://schema.intuit.com/finance/v3");
XmlNodeList nodes = el.SelectNodes(#"/i:IntuitResponse/i:QueryResponse", nsmgr);
Google Docs:
Unfortunately, IMPORTXML() does not provide a namespace prefix binding mechanism. See next section, Defeating namespaces in XPath, for how to use local-name() as a work-around.
Java (SAX):
NamespaceSupport support = new NamespaceSupport();
support.pushContext();
support.declarePrefix("i", "http://schema.intuit.com/finance/v3");
Java (XPath):
xpath.setNamespaceContext(new NamespaceContext() {
public String getNamespaceURI(String prefix) {
switch (prefix) {
case "i": return "http://schema.intuit.com/finance/v3";
// ...
}
});
Remember to call
DocumentBuilderFactory.setNamespaceAware(true).
See also:
Java XPath: Queries with default namespace xmlns
JavaScript:
See Implementing a User Defined Namespace Resolver:
function nsResolver(prefix) {
var ns = {
'i' : 'http://schema.intuit.com/finance/v3'
};
return ns[prefix] || null;
}
document.evaluate( '/i:IntuitResponse/i:QueryResponse',
document, nsResolver, XPathResult.ANY_TYPE,
null );
Note that if the default namespace has an associated namespace prefix defined, using the nsResolver() returned by Document.createNSResolver() can obviate the need for a customer nsResolver().
Perl (LibXML):
my $xc = XML::LibXML::XPathContext->new($doc);
$xc->registerNs('i', 'http://schema.intuit.com/finance/v3');
my #nodes = $xc->findnodes('/i:IntuitResponse/i:QueryResponse');
Python (lxml):
from lxml import etree
f = StringIO('<IntuitResponse>...</IntuitResponse>')
doc = etree.parse(f)
r = doc.xpath('/i:IntuitResponse/i:QueryResponse',
namespaces={'i':'http://schema.intuit.com/finance/v3'})
Python (ElementTree):
namespaces = {'i': 'http://schema.intuit.com/finance/v3'}
root.findall('/i:IntuitResponse/i:QueryResponse', namespaces)
Python (Scrapy):
response.selector.register_namespace('i', 'http://schema.intuit.com/finance/v3')
response.xpath('/i:IntuitResponse/i:QueryResponse').getall()
PhP:
Adapted from #Tomalak's answer using DOMDocument:
$result = new DOMDocument();
$result->loadXML($xml);
$xpath = new DOMXpath($result);
$xpath->registerNamespace("i", "http://schema.intuit.com/finance/v3");
$result = $xpath->query("/i:IntuitResponse/i:QueryResponse");
See also #IMSoP's canonical Q/A on PHP SimpleXML namespaces.
Ruby (Nokogiri):
puts doc.xpath('/i:IntuitResponse/i:QueryResponse',
'i' => "http://schema.intuit.com/finance/v3")
Note that Nokogiri supports removal of namespaces,
doc.remove_namespaces!
but see the below warnings discouraging the defeating of XML namespaces.
VBA:
xmlNS = "xmlns:i='http://schema.intuit.com/finance/v3'"
doc.setProperty "SelectionNamespaces", xmlNS
Set queryResponseElement =doc.SelectSingleNode("/i:IntuitResponse/i:QueryResponse")
VB.NET:
xmlDoc = New XmlDocument()
xmlDoc.Load("file.xml")
nsmgr = New XmlNamespaceManager(New XmlNameTable())
nsmgr.AddNamespace("i", "http://schema.intuit.com/finance/v3");
nodes = xmlDoc.DocumentElement.SelectNodes("/i:IntuitResponse/i:QueryResponse",
nsmgr)
SoapUI (doc):
declare namespace i='http://schema.intuit.com/finance/v3';
/i:IntuitResponse/i:QueryResponse
xmlstarlet:
-N i="http://schema.intuit.com/finance/v3"
XSLT:
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:i="http://schema.intuit.com/finance/v3">
...
Once you've declared a namespace prefix, your XPath can be written to use it:
/i:IntuitResponse/i:QueryResponse
Defeating namespaces in XPath (not recommended)
An alternative is to write predicates that test against local-name():
/*[local-name()='IntuitResponse']/*[local-name()='QueryResponse']
Or, in XPath 2.0:
/*:IntuitResponse/*:QueryResponse
Skirting namespaces in this manner works but is not recommended because it
Under-specifies the full element/attribute name.
Fails to differentiate between element/attribute names in different
namespaces (the very purpose of namespaces). Note that this concern could be addressed by adding an additional predicate to check the namespace URI explicitly:
/*[ namespace-uri()='http://schema.intuit.com/finance/v3'
and local-name()='IntuitResponse']
/*[ namespace-uri()='http://schema.intuit.com/finance/v3'
and local-name()='QueryResponse']
Thanks to Daniel Haley for the namespace-uri() note.
Is excessively verbose.
XPath 3.0/3.1
Libraries and tools that support modern XPath 3.0/3.1 allow the specification of a namespace URI directly in an XPath expression:
/Q{http://schema.intuit.com/finance/v3}IntuitResponse/Q{http://schema.intuit.com/finance/v3}QueryResponse
While Q{http://schema.intuit.com/finance/v3} is much more verbose than using an XML namespace prefix, it has the advantage of being independent of the namespace prefix binding mechanism of the hosting library. The Q{} notation is known as Clark Notation after its originator, James Clark. The W3C XPath 3.1 EBNF grammar calls it a BracedURILiteral.
Thanks to Michael Kay for the suggestion to cover XPath 3.0/3.1's BracedURILiteral.

I use /*[name()='...'] in a google sheet to fetch some counts from Wikidata. I have a table like this
thes WD prop links items
NOM P7749 3925 3789
AAT P1014 21157 20224
and the formulas in cols links and items are
=IMPORTXML("https://query.wikidata.org/sparql?query=SELECT(COUNT(*)as?c){?item wdt:"&$B14&"[]}","//*[name()='literal']")
=IMPORTXML("https://query.wikidata.org/sparql?query=SELECT(COUNT(distinct?item)as?c){?item wdt:"&$B14&"[]}","//*[name()='literal']")
respectively. The SPARQL query happens not to have any spaces...
I saw name() used instead of local-name() in Xml Namespace breaking my xpath!, and for some reason //*:literal doesn't work.

C# LINQ and XML Getting child nodes

Having issues getting node values. Not sure why the following code is failing to do so.
<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet type='text/xsl' href='STIG_unclass.xsl'?>
<Benchmark xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:cpe="http://cpe.mitre.org/language/2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" id="Windows_7_STIG" xml:lang="en" xsi:schemaLocation="http://checklists.nist.gov/xccdf/1.1 http://nvd.nist.gov/schema/xccdf-1.1.4.xsd http://cpe.mitre.org/dictionary/2.0 http://cpe.mitre.org/files/cpe-dictionary_2.1.xsd" xmlns="http://checklists.nist.gov/xccdf/1.1">
<status date="2015-06-16">accepted</status>
<title>Windows 7 Security Technical Implementation Guide</title>
<description>
The Windows 7 Security Technical Implementation Guide (STIG) is published as a tool to improve the security of Department of Defense (DoD) information systems. The requirements were developed from DoD consensus, as well as the Windows 7 Security Guide and security templates published by Microsoft Corporation. Comments or proposed revisions to this document should be sent via e-mail to the following address: disa.stig_spt#mail.mil.
</description>
<notice id="terms-of-use" xml:lang="en">Developed_by_DISA_for_the_DoD</notice>
<reference href="http://iase.disa.mil">
<dc:publisher>DISA, Field Security Operations</dc:publisher>
<dc:source>STIG.DOD.MIL</dc:source>
</reference>
<plain-text id="release-info">Release: 20 Benchmark Date: 24 Jul 2015</plain-text>
</Benchmark>
Sample XML File.
and the following is my code.
String Title = LoadedXML.Element("Benchmark").Attribute("id").Value;
var XMLData = LoadedXML.Element("Benchmark").Elements("plain-text")
.Single(release => release.Attribute("id").Value == "release-info").Value;
is there a way I can get multiple Node values at the same time? Like getting the Title and Release Value at once instead of having a separate one for each?

Your code is failing because your XML contains Namespace and you can't access your nodes directly. If you want to confirm this simply query LoadedXML.Elements() and examine the values in debugger, you can clearly see the namespaces there:-
So, You need to declare the namespace and use it:-
XNamespace ns = "http://checklists.nist.gov/xccdf/1.1";
If you want both vales to be fetched at once you can project it to a anonymous type like this:-
var result = LoadedXML.Root.Elements(ns + "plain-text")
.Where(x => (string)x.Attribute("id") == "release-info")
.Select(x => new
{
Title = (string)x.Document.Root.Attribute("id"),
XMLData = x.Value
}).FirstOrDefault();
This query is giving me below output:-

Linq-to-xml is generally used to query a XML to filter it's nodes and then get the desired element/values per need. It's more like querying a table with SQL.
If all/most of the XML is required as a result, then the better approach would be to deseralize the XMl into a native (C# here) object and map it to the required model object. XML can always be thought of a serialized version of an object (although it can be manually as well), and can be deserialized back to the actual object.
.Net has native support for all these, see msdn links for XML Serialization and Deserialization for details. You can write a small method to deserialize your object like this.
using System.Xml.Linq;
using System.Xml.Serialization;
public class XMLHelper
{
public T DeserializeData<T>(string data)
{
XmlSerializer serializer = new XmlSerializer(typeof(T));
StringReader reader = new StringReader(data);
var deserializedObject = serializer.Deserialize(reader);
return deserializedObject == null ? default(T) : (T)deserializedObject;
}
}
To get the string you can do like File.ReadAllText(xmlFilePath) or whatever is easier for the situation.
This will give you deseialized object of the whole XML. If you want some other transformed object, you can either manually map that, or use AutoMapper

Parsing XML (using class generated by xsd.exe) - Error: There is an error in XML document (83, 8)

I have this XML file that I would like to parse into an object in C#:
001-0180914-5787994.xml (http://pastebin.com/bzzAnsQL)
So, I opened this xml file in VS Express 2013 for Desktop and clicked on the XML -> Generate Schema menu option. This generated two XSD schema files:
001-0180914-5787994.xsd (http://pastebin.com/QsvARtyB)
001-0180914-57879941.xsd (http://pastebin.com/FH4XNhvd)
I then wrote a batch script which uses the xsd.exe tool (comes with .NET SDK) to generate a C# class file from the schema like this:
#ECHO off
set xsdFile="C:\Users\Administrator\Desktop\test\001-0180914-5787994.xsd"
set outDirectory="C:\Users\Administrator\Desktop\test\out"
set xsdExeDir="C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools"
set language="CS"
cd %xsdExeDir%
xsd.exe "%xsdFile%" "C:\Users\Administrator\Desktop\test\001-0180914-57879941.xsd" /c /out:"%outDirectory%" /l:"%language%"
pause
When I ran the above batch script, it generated the following C# class:
001-0180914-57879941.cs (http://pastebin.com/wX8N0DAf)
Finally, I added this into a test console app project and I tried to generate an object out of my XML file and this auto-generated class like this:
class Program
{
static void Main(string[] args)
{
try
{
var order = Parse("001-0180914-5787994.xml");
Console.WriteLine("Success !!!");
}
catch (Exception ex)
{
Console.WriteLine("Error: " + ex.Message);
}
Console.WriteLine("Press any key to exit...");
Console.Read();
}
public static OrderResponseDetailComplete Parse(String XMLFile)
{
var settings = new XmlReaderSettings();
var obj = new OrderResponseDetailComplete();
var reader = XmlReader.Create(XMLFile, settings);
var serializer = new XmlSerializer(typeof(OrderResponseDetailComplete));
obj = (OrderResponseDetailComplete)serializer.Deserialize(reader);
reader.Close();
return obj;
}
}
When test program ran, I am getting this error:
What does this error mean? and how do I fix it?

From the first link (XML) I see an xsi:type on line 83.
Look at "Is it valid to specify xsi:type for an local complexType?" for more information about this error.
As far as I can tell you need to define the OrderLineItemItemResponse type in your XSD.
The problem is in the XML but the solution is in the XSD. You have to define the type in order for it to be legit and pass validation. For example the first XSD defines OrderResponseDetailComplete. Where is the XSD for OrderLineItemItemResponse?
This may be a viable solution, depending on the use case: XML Serialized from xsd.exe generated code using substitution groups is invalid (invalid xsi:type error).

Inserting copyright notice/banner in all source code files in Visual Studio 2012

After some Googling I found this: Use a Visual Studio Macro to Insert Copyright Headers into Source Files. It looked promising:
// <copyright file="Sample.cs" company="My Company Name">
// Copyright (c) 2012 All Rights Reserved
// </copyright>
// <author>Leniel Macaferi</author>
// <date>08/30/2012 11:39:58 AM </date>
// <summary>Class representing a Sample entity</summary>
When I tried Tools -> Macros menu option it wasn't there anymore in VS 2012. Here's the proof: Macros in Visual Studio 11 Developer Preview. They just dropped this functionality. :(
So, I'm just curious to know which option I could use to add the copyright info to all existing source code files in my solution using Visual Studio 2012. Is there any standard way of doing this, using a template file (something related to T4 templates) or a PowerShell script? I could write some code to iterate over the files with .cs extension and add the copyright info but that is not what I'm after. I'd like to know about some tool to automate this process.

You could create a new snippet and just type cp + double tab to insert the notice where you want (needless to say you can change the keyword to whatever you want).
The only problem with it is, from what I'm aware, snippets do not support time functions, so getting the current time for your date line seems impossible with this technique. A not so good workaround for this is to make the time fields editable (similar to how the mbox snippet works) and just insert the time manually.
Here's an example on how a snippet looks. The bellow snippet will get the class name automatically and insert the copyright notice in the place where you type 'copyright' and double tab.
Method 1
<?xml version="1.0" encoding="utf-8" ?>
<CodeSnippets xmlns="http://schemas.microsoft.com/VisualStudio/2005/CodeSnippet">
<CodeSnippet Format="1.0.0">
<Header>
<Title>Copyright</Title>
<Shortcut>Copyright</Shortcut>
<Description>Code snippet for Copyright notice</Description>
<Author>author name</Author>
<SnippetTypes>
<SnippetType>Expansion</SnippetType>
</SnippetTypes>
</Header>
<Snippet>
<Declarations>
<Literal Editable="false">
<ID>classname</ID>
<Function>ClassName()</Function>
</Literal>
</Declarations>
<Code Language="csharp">
<![CDATA[// <copyright file="$classname$" company="My Company Name">
// Copyright (c) 2012 All Rights Reserved
// <author>Leniel Macaferi</author>
// </copyright>
]]>
</Code>
</Snippet>
</CodeSnippet>
</CodeSnippets>
Method 2
Also, here's an example of a program you can make to do that for you.
List<string> files = new List<string>()
{
"c:\\Form1.cs",
"c:\\Form2.cs",
};
foreach (string file in files)
{
string tempFile = Path.GetFullPath(file) + ".tmp";
using (StreamReader reader = new StreamReader(file))
{
using (StreamWriter writer = new StreamWriter(tempFile))
{
writer.WriteLine(#"// <copyright file=" + Path.GetFileNameWithoutExtension(file) + #" company=My Company Name>
// Copyright (c) 2012 All Rights Reserved
// </copyright>
// <author>Leniel Macaferi</author>
// <date> " + DateTime.Now + #"</date>
// <summary>Class representing a Sample entity</summary>
");
string line = string.Empty;
while ((line = reader.ReadLine()) != null)
{
writer.WriteLine(line);
}
}
}
File.Delete(file);
File.Move(tempFile, file);
}
Some error catching will be required of course. But this should give you the general idea how to construct an UI around it an add the files you will want to process.
Method 3
It's also possible to change the template for your classes that can be usually be found under:
C:\Program Files (x86)\Microsoft Visual Studio <version>\Common7\IDE\ItemTemplates\CSharp\1033\
Sometimes editing ItemTemplatesCache is also necessary to display the results.
Here's an example template based on your question:
using System;
using System.Collections.Generic;
$if$ ($targetframeworkversion$ >= 3.5)using System.Linq;
$endif$using System.Text;
/* <copyright file=$safeitemrootname$ company="My Company Name">
Copyright (c) 2012 All Rights Reserved
</copyright>
<author>Leniel Macaferi</author>
<date>$time$</date>
<summary>Class representing a Sample entity</summary>*/
namespace $rootnamespace$
{
class $safeitemrootname$
{
}
}

I'm going to add here a PowerShell script I found in this post: Powershell – Copyright header generator script. It captures what I had in mind before posting the question...
param($target = "C:\MyProject", $companyname = "My Company")
$header = "//-----------------------------------------------------------------------
// <copyright file=""{0}"" company=""{1}"">
// Copyright (c) {1}. All rights reserved.
// </copyright>
//-----------------------------------------------------------------------`r`n"
function Write-Header ($file)
{
$content = Get-Content $file
$filename = Split-Path -Leaf $file
$fileheader = $header -f $filename,$companyname
Set-Content $file $fileheader
Add-Content $file $content
}
Get-ChildItem $target -Recurse | ? { $_.Extension -like ".cs" } | % `
{
Write-Header $_.PSPath.Split(":", 3)[2]
}
I wrote about it with minor modifications to adapt it to my needs:
Inserting copyright notice/banner/header in all source code files with PowerShell

In case this might still be interesting, there is the License header plugin that can add a completey customizable header to any file on creation. Currently, this does work with VS2013, but not (yet?) with VS 2015.

Add a .editorconfig file to your Visual Studio project with following line:
file_header_template = // <copyright file="{fileName}.cs" company="xyz Corporation">\n// Copyright (c) Xyz Corporation. All rights reserved.\n// </copyright>
Next, place your cursor on the first line of any C# or Visual Basic file and type (Ctrl+.) to trigger the Quick Actions and Refactorings menu. Select Add file banner. If you would like to apply the file header to all files of an existing project or solution, select Project or Solution under the Fix all occurrences in: option.
This approach will work for new files and even existing files. This is inbuilt in visual studio code so you don't need custom scripts.
Details here: https://learn.microsoft.com/en-us/visualstudio/releases/2019/release-notes-v16.6#net-productivity:~:text=Add%20file%20headers%20to%20existing%20files

As per the release notes the release notes (scroll down a bit), Visual Studio 16.6 allows to define a file_header_template rule in .editorconfig that can be used to create file banners.

After losing work to some obscure PowerShell script (not the one added as an answer), I decided to create copyright.py. Example usage:
C:\>python copyright.py "C:\users\me\documents\path\to\vsproject"
It recursively finds all *.cs files in the specified directory and prefixes them with the copyright text. It does not matter if other text already exists at the start of the file; it will be removed.
Note: As a precaution, you should always backup your code before executing this script

A simpler version of the code by #leniel-macaferi (no filename and company, just header):
param($target = "C:\MyProject")
$header = "//-----------------------------------------------------------------------------
// Copyright (c) The Corporation. All rights reserved.
//-----------------------------------------------------------------------------
"
function Write-Header ($file)
{
$content = Get-Content $file
$filename = Split-Path -Leaf $file
$fileheader = $header
Set-Content $file $fileheader
Add-Content $file $content
}
Get-ChildItem $target -Recurse | ? { $_.Extension -like ".cs" } | % `
{
Write-Header $_.PSPath.Split(":", 3)[2]
}
To run, save the code in a new file (e.g. AddHeader.ps1) in the directory you want to recursively iterate through and execute .\AddHeader.ps1 . from PowerShell window.