We Keep Coding

Processing XML from non-standard DTD in C#

I literally just want to be able to traverse the contents of various XML files that I have been given but having a non-standard DTD means I am hitting some issues - one error being "Reference to undeclared entity 'reg'" and another saying it is unable to locate the .DTD file.
Is it possible to do this sort of thing for XML files when no DTD is available? I have no control over these files and cannot change them. I am looking to grab various amounts of them at a time, move through the contents as efficiently as possible, email out some notifications and thats it.
Sample of XML file below:
<!DOCTYPE Toro-Pub PUBLIC "-//Toro//DTD Toro Publication V1.0//EN//XML" "Toro-Pub.dtd">
<!--Arbortext, Inc., 1988-2011, v.4002-->
<?Pub UDT _nopagebreak _touchup KeepsKeep="yes" KeepsPrev="no" KeepsNext="no" KeepsBoundary="page"?>
<?Pub UDT template _font?>
<?Pub UDT _bookmark _target?>
<?Pub UDT _nocolumnbreak _touchup KeepsKeep="yes" KeepsPrev="no" KeepsNext="no" KeepsBoundary="column"?>
<?Pub UDT instructions _comment FontColor="red"?>
<?Pub EntList alpha bull copy rArr sect trade deg?>
<?Pub Inc?>
<Toro-Pub><PubMeta Brand="Toro" CE="Yes" ClientPubNo="" CopyrightYear="2013" FormNumber="3378-827" Lang="CS" LangParentForm="3378-826" LangParentID="72729" LangParentRev="A" PageSize="" PhoneNoCan="" PhoneNoMex="" PhoneNoUS="" ProductFamily="sample product name" PubID="72730" PublicationType="Operator Manual" RegistrationURL="www.website.com" Rev="A" ServiceURL="www.website.com"><?TranslationData DueDate="07/01/2013" InCarton(1-yes)="0" Author="Mr Smith" EngParent="https://lwww.website.com?vPubID=423&vPubNum=3378-826" ?></PubMeta><Pub-TBlock>
<Body-TB>
...
Many thanks.
UPDATE #1
I have tried the below code taken from the suggested comment:
Stream file = File.OpenRead("4d00fa60800e0a5d_3378-827.xml");
// The next line is the fix!!!
XmlTextReader xmlTextReader = new XmlTextReader(file);
xmlTextReader.XmlResolver = null; // Don't require file in system32\inetsrv
XmlReaderSettings readerSettings = new XmlReaderSettings();
readerSettings.ValidationType = ValidationType.Schema;
//readerSettings.Schemas.Add(null, "");
readerSettings.DtdProcessing = DtdProcessing.Ignore;
readerSettings.XmlResolver = null; // Doesn't help
//readerSettings.ValidationEventHandler += ValidationEventHandle;
XmlReader myXmlReader = XmlReader.Create(xmlTextReader, readerSettings);
XmlDocument myXmlDocument = new XmlDocument();
myXmlDocument.XmlResolver = null; // Doesn't help
myXmlDocument.Load(myXmlReader); // Load doc, no .dtd required on local disk
However, I now get a new error of 'Operation is not valid due to the current state of the object.' on the line 'myXmlDocument.Load(myXmlReader)'.

How to prevent XXE attack (XmlDocument in .NET)

We had a security audit on our code, and they mentioned that our code is vulnerable to EXternal Entity (XXE) attack. I am using following code -
string OurOutputXMLString=
"<ce><input><transaction><length>00000</length><tran_type>Login</tran_type></transaction><user><user_id>ce_userid</user_id><subscriber_name>ce_subscribername</subscriber_name><subscriber_id>ce_subscriberid</subscriber_id><group_id>ce_groupid</group_id><permissions></permissions></user><consumer><login_details><username>UnitTester9</username><password>pDhE5AsKBHw85Sqgg6qdKQ==</password><pin>tOlkiae9epM=</pin></login_details></consumer></input></ce>"
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXml(OurOutputXMLString);
In the audit report they say that it's failing because an XML entity can contain URLs that can resolve outside of intended control. XML entity resolver will attempt to resolve and retrieve external references. If attacker-controlled XML can be submitted to one of these functions, then the attacker could gain access to information about an internal network, local filesystem, or other sensitive data.
To avoid this I wrote the following code but it doesn't work.
MemoryStream stream =
new MemoryStream(System.Text.Encoding.Default.GetBytes(OurOutputXMLString));
XmlReaderSettings settings = new XmlReaderSettings();
settings.DtdProcessing = DtdProcessing.Prohibit;
settings.MaxCharactersFromEntities = 6000;
XmlReader reader = XmlReader.Create(stream, settings);
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.Load(reader);
But I can see here that reader does not have any value to load into xmlDoc(XmlDocument).
Can anyone help where I am missing things?

External resources are resolved using the XmlResolver provided via XmlDocument.XmlResolver property. If your XML documents **should not contain any external resource **(for example DTDs or schemas) simply set this property to null:
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.XmlResolver = null;
xmlDoc.LoadXml(OurOutputXMLString);
If you want to filter where these URLs come from (for example to allow only certain domains) just derive your own class from XmlUrlResolver and override the ResolveUri() method. There you can check what the URL is and sanitize it (for example you can allow only URLs within your local network or from trusted sources).
For example:
class CustomUrlResovler : XmlUrlResolver
{
public override Uri ResolveUri(Uri baseUri, string relativeUri)
{
Uri uri = new Uri(baseUri, relativeUri);
if (IsUnsafeHost(uri.Host))
return null;
return base.ResolveUri(baseUri, relativeUri);
}
private bool IsUnsafeHost(string host)
{
return false;
}
}
Where IsUnsafeHost() is a custom function that check if the given host is allowed or not. See this post here on SO for few ideas. Just return null from ResolveUri() to save your code from this kind of attacks. In case the URI is allowed you can simply return the default XmlUrlResolver.ResolveUri() implementation.
To use it:
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.XmlResolver = new CustomUrlResolver();
xmlDoc.LoadXml(OurOutputXMLString);
For more details about how XML external resources are resolved just read Resolving External Resources on MS Docs. If your code is more complex than this example then you should definitely read Remarks section for XmlDocument.XmlResolver property.

So its better to use
new XmlDocument { XmlResolver = null };
Interestingly from .net 4.5.2 and 4.6, the default resolver behaves differently and does not use an XmlUrlResolver upfront implicitly to resolve any urls or locations as i seen.
//In pre 4.5.2 it is a security issue.
//In 4.5.2 it will not resolve any more the url references in dtd and such,
//Still better to avoid the below since it will trigger security warnings.
new XmlDocument();

Setting the XmlReaderSettings.DtdProcessing to DtdProcessing.Prohibit works totally fine in .NET 4.7.2. Here is what i used to test.
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE demo
[
<!ELEMENT demo ANY >
<!ENTITY % extentity SYSTEM "https://www.hl7.org/documentcenter/public/wg/structure/CDA.xsl">
%extentity;
]>
<test>
Some random content
</test>
Saved the above content in a file and read the file from the following fragment of c# code.
XmlReaderSettings settings = new XmlReaderSettings();
settings.DtdProcessing = DtdProcessing.Prohibit;
settings.MaxCharactersFromEntities = 6000;
//The following stream should be the filestream of the above content.
XmlReader reader = XmlReader.Create(stream, settings);
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.Load(reader);
I get the following exception.
For security reasons DTD is prohibited in this XML document. To enable DTD
processing set the DtdProcessing property on XmlReaderSettings to Parse and
pass the settings into XmlReader.Create method.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.ParseDoctypeDecl()
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace)
at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc)
at System.Xml.XmlDocument.Load(XmlReader reader)

Record and replay human-readable protobuf messages using a file with protobuf-csharp-port

I'm using protobuf-csharp-port and I need the ability to record some protobuf messages for replay later. XML would be ideal for me, but I'm flexible as long as a human could go into the file, make changes, then replay the messages from the file.
Using this C# code:
MyRequest req =
MyRequest.CreateBuilder().SetStr("Lafayette").Build();
using (var stringWriter = new StringWriter())
{
var xmlWriterSettings = new XmlWriterSettings
{
ConformanceLevel = ConformanceLevel.Fragment
};
var xmlWriter = XmlWriter.Create(stringWriter, xmlWriterSettings);
ICodedOutputStream output = XmlFormatWriter.CreateInstance(xmlWriter);
req.WriteTo(output);
output.Flush();
string xml = stringWriter.ToString();
using (var streamWriter = new StreamWriter(#"Requests.txt"))
{
streamWriter.WriteLine(xml);
streamWriter.WriteLine(xml);
streamWriter.WriteLine(xml);
}
}
I produce the Requests.txt file containing:
<str>Lafayette</str>
<str>Lafayette</str>
<str>Lafayette</str>
However when I try to deserialize them back using:
var xmlReaderSettings = new XmlReaderSettings
{
ConformanceLevel = ConformanceLevel.Fragment
};
using (var xmlReader = XmlReader.Create(#"Requests.txt", xmlReaderSettings))
{
ICodedInputStream input = XmlFormatReader.CreateInstance(xmlReader);
MyRequest reqFromFile;
while(!input.IsAtEnd)
{
reqFromFile =
ReverseRequest.CreateBuilder().MergeFrom(input).Build();
}
}
Only one MyRequest gets deserialized and the other two are ignored. (After reqFromFile is built, input.IsAtEnd == true.)
So back to my question: is there some way to read multiple human-readable protobuf messages from a file?

So back to my question: is there some way to read multiple human-readable protobuf messages from a file?
Well you're currently creating a single text file with multiple root elements - it's not a valid XML file.
The simplest approach would probably be to create a single XML document with all those requests in, then load it (e.g. with LINQ to XML) and create an XmlReader positioned at each of the root element's children. Each of those XmlReader objects should be able to then deserialize to a single protobuf message.

You could look at the Protocol buffer Editor
Alternatively there is Google own Text format which is a bit like JSon. You can convert between the 2 using the protoc command (look at encode / decode options, you also need the proto definition)

Read XML values Webservice

I want to read xml on runtime, without save it on a path
After my searching i find that, In console application i need to use Console.Out for displaying result
xmlSerializer.Serialize(Console.Out, patient);
In Windows / Web Application we need to set path like
StreamWriter streamWriter = new StreamWriter(#"C:\test.xml");
but i need to read xml with out save it, i am using Webserive where i need to read it and take a decision that either it is valid or not
I hope i define it clearly..

Use the XmlDocument object.
There are several ways to load the XML, you can use the XmlDocument.Load() and specify your URL in there or use XmlDocument.LoadXml() to load the XML from a string.

You could use the XmlDocument.LoadXml class to read the received xml. There is no need to save it to disk.
try
{
XmlDocument doc = new XmlDocument();
doc.LoadXml(receivedXMLStr);
//valid xml
}
catch (XmlException xe)
{
//invalid xml
}

Use Linq2Xml..
XElement doc;
try
{
doc=XElement.Load(yourStream);
}
catch
{
//invalid XML
}
foreach(XElement node in doc.Descendants())
{
node.Value;//value of this node
nodes.Attributes();//all the attributes of this node
}

Thanks all of you for your reply, i want to laod my XML without save it on a local Path, because saving creating many XML.
Finally i find the solutions for load the XML from class on a Memory stream, I thinn this solution is very easy and optimize
XmlDocument doc = new XmlDocument();
System.Xml.Serialization.XmlSerializer serializer2 = new System.Xml.Serialization.XmlSerializer(Patients.GetType());
System.IO.MemoryStream stream = new System.IO.MemoryStream();
serializer2.Serialize(stream, Patients);
stream.Position = 0;
doc.Load(stream);

You need to use the Deserialize option to read the xml. Follow the below steps to achieve it,
Create a target class. It structure should represent the xml output.
After creating the class, use the below code to load your xml into the target object
TargetType result = null;
XmlSerializer worker = new XmlSerializer(typeof(TargetType));
result = worker.Deserialize("<xml>.....</xml>");
Now the xml is loaded into the object 'result' and use it.

An error occurred while parsing EntityName

I'm trying to load a xml document into an object XPathDocument in C#.
My xml documents include this line:
trés dégagée + rade
and when the parser arrives there it gives me this error:
"An error occurred while parsing EntityName"
I know that's normal cause of the character "é". Does anybody know how can I avoid this error... My idea is to insert into the xml document an entities declaration and after replace all special characters with entities...but it's long and I’m not sure if it's working. Do you have other ideas? Simpler?
Thanks a lot

Was about to post this and just then the servers went down. I think I've rewritten it correctly from memory:
I think that the problem lies within the fact that by default the XPathDocument uses an XmlTextReader to parse the contents of the supplied file and this XmlTextReader uses an EntityHandling setting of ExpandEntities.
In other words, when you rely on the default settings, an XmlTextReader will validate the input XML and try to resolve all entities. The better way is to do this manually by taking full control over the XmlReaderSettings (I always do it manually):
string myXMLFile = "SomeFile.xml";
string fileContent = LoadXML(myXMLFile);
private string LoadXML(string xml)
{
XPathDocument xDoc;
XmlReaderSettings xrs = new XmlReaderSettings();
// The following line does the "magic".
xrs.CheckCharacters = false;
using (XmlReader xr = XmlReader.Create(xml, xrs))
{
xDoc = new XPathDocument(xr);
}
if (xDoc != null)
{
XPathNavigator xNav = xDoc.CreateNavigator();
return xNav.OuterXml;
}
else
// Unable to load file
return null;
}

Typically this is caused by a mismatch between the encoding used to read the file and the files actually encoding.
At a guess I would say the file is UTF-8 encoded but you are reading it with a default encoding.
Try beefing up your question with more details to get a more definitive answer.