We Keep Coding

Create xsd schema from instance of class at runtime

I need to validate some generic sensor input. The requirement is, that the validation cannot happen in my code but with a external validator like xsd from outside the codebase to give users the ability to swap the validation logic without needing to code or recompile the application.
I know that the sensor input is only valid for one specific case and therefore would like to generate the xsd from an Instance of a class, that exists at runtime, that was user validated, to get the valid restrictions.
I tried the Idea from this question, however this only works on types and not on instances of classes.
Therefore my question: Is there a way to take a runtime instance of a C# class and convert it to an xsd that has the values of the properties as the only valid restrictions?
Update:
to clarify: What I have is a class like this:
public sealed class Sensor
{
public int Data { get; set; }
public int otherData { get; set; }
public int MoreData { get; set; }
}
the class gets instanciated somewhere (e.g. like this):
var se = new Sensor()
{
Data = 5,
otherData = 10,
MoreData = 15
};
When I now try to create an xsd using something like the following function:
var schemas = new XmlSchemas();
var exporter = new XmlSchemaExporter(schemas);
var mapping = new XmlReflectionImporter().ImportTypeMapping(typeof(Person));
exporter.ExportTypeMapping(mapping);
var schemaWriter = new StringWriter();
foreach (XmlSchema schema in schemas)
{
schema.Write(schemaWriter);
}
return schemaWriter.ToString();
I receive some xsd like this:
<?xml version="1.0" encoding="utf-8"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="sensor">
<xs:complexType>
<xs:sequence>
<xs:element name="Data" type="xs:integer" />
<xs:element name="otherData" type="xs:integer" />
<xs:element name="moreData" type="xs:integer" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
However this is far from what I want to archieve. I would like to have the proper restrictions built into it (it should look something like this):
<?xml version="1.0" encoding="utf-8"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="sensor">
<xs:complexType>
<xs:sequence>
<xs:element name="Data">
<xs:simpleType>
<xs:restriction base="xs:integer">
<xs:enumeration value="5"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
<xs:element name="otherData">
<xs:simpleType>
<xs:restriction base="xs:integer">
<xs:enumeration value="10"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
<xs:element name="moreData">
<xs:simpleType>
<xs:restriction base="xs:integer">
<xs:enumeration value="15"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
I could obviously go ahead load the generated file into memory, strip some attributes and change how the xsd should look like, but this feels wrong because of the following things:
By me defining Rules to how the xsd should look like I take away
flexibility that I would like to have.
This approach seems quite errorprone to me because it seems like basically a little better than direct string manipulation.
This extra code would make my already large code way complexer and harder to understand.
To sum up: I need either a library or a really clever function that can create a xsd like the one above based on the runitme info I have on the class without writing a lot of things to manipulate the xml directly to avoid errorprone or wrong assumptions about the future usage of the validation.

I took your generate schema and added details using Xml Linq. See code below
using System;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
using System.Xml.Serialization;
using System.Xml.Schema;
using System.IO;
namespace ConsoleApplication131
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
Sensor se = new Sensor()
{
Data = 5,
otherData = 10,
MoreData = 15
};
XmlSchemas schemas = new XmlSchemas();
XmlSchemaExporter exporter = new XmlSchemaExporter(schemas);
XmlTypeMapping mapping = new XmlReflectionImporter().ImportTypeMapping(typeof(Sensor));
exporter.ExportTypeMapping(mapping);
StringWriter schemaWriter = new StringWriter();
foreach (XmlSchema schema in schemas)
{
schema.Write(schemaWriter);
}
XDocument doc = XDocument.Parse(schemaWriter.ToString());
XElement root = doc.Root;
XNamespace xs = root.GetNamespaceOfPrefix("xs");
foreach (XElement _class in doc.Descendants(xs + "complexType"))
{
List<XElement> elements = _class.Descendants(xs + "element").ToList();
if (elements.Count > 0)
{
XElement complexType = new XElement(xs + "complexType");
_class.Add(complexType);
XElement sequence = new XElement(xs + "sequence");
complexType.Add(sequence);
foreach (var prop in se.GetType().GetProperties())
{
string name = prop.Name;
string value = prop.GetValue(se, null).ToString();
XElement element = elements.Where(x => (string)x.Attribute("name") == name).FirstOrDefault();
string strType = (string)element.Attribute("type");
XElement newElement = new XElement(xs + "simpleType", new object[] {
new XElement(xs + "restriction", new object[] {
new XAttribute("base", strType),
new XElement(xs + "enumeration", new XAttribute("value", value))
})
});
sequence.Add(newElement);
}
}
}
doc.Save(FILENAME);
}
}
public sealed class Sensor
{
public int Data { get; set; }
public int otherData { get; set; }
public int MoreData { get; set; }
}
}

compile schemas with included xsd files

I have static method, which i use to validate a XML File against a XSD File. This works fine, until there is an XSD File which includes another XSD File.
Example, where i got troubles:
TYPES.XSD:
<xs:simpleType name="MY_AMOUNT">
<xs:restriction base="xs:decimal">
<xs:maxInclusive value="999999999999.99"/>
<xs:minInclusive value="-999999999999.99"/>
<xs:totalDigits value="14"/>
<xs:fractionDigits value="2"/>
</xs:restriction>
</xs:simpleType>
MAIN.XSD:
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
<xs:include schemaLocation="TYPES.xsd"/>
<xs:element name="ROOT">
<xs:complexType>
<xs:sequence>
<xs:element ref="SOMEREF1"/>
<xs:element ref="SOMEREF2"/>
<xs:element name="AMOUNT" type="MY_AMOUNT" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
VALIDATION CODE:
public static class XmlUtils
{
private static string Errors = string.Empty;
public static bool ValidateAgainstXSD(string xmlFilePath, string xsdFilePath, ref string message)
{
try
{
var settings = new XmlReaderSettings();
settings.ValidationType = ValidationType.Schema;
settings.ValidationFlags = XmlSchemaValidationFlags.ProcessInlineSchema
| XmlSchemaValidationFlags.ProcessInlineSchema
| XmlSchemaValidationFlags.ReportValidationWarnings;
settings.Schemas.Add(null, xsdFilePath);
settings.Schemas.Compile();
settings.ValidationEventHandler += (sender, args) =>
{
if (args.Severity == XmlSeverityType.Error)
{
Errors += args.Message + "\n";
}
};
using (var reader = XmlReader.Create(xmlFilePath, settings))
{
while (reader.Read()) { }
}
message = Errors ?? string.Empty;
return string.IsNullOrEmpty(Errors);
}
catch (Exception e)
{
message = "# error validating xml file: " + e.Message;
return false;
}
}
}
Somehow it seems i have to specify the path of the included XSD File but i have no idea where.
The error occurs at settings.Schemas.Compile(); , where it says that the type "MY_AMOUNT" is not declared. I read about custom XmlResolvers but to be honest i didn't get that working.
If this is important for an answer: The xsd files are always located in the same directory!
The method is called likes this:
string msg = string.Empty;
string basedir = #"C:\Temp";
string xml = Path.Combine(basedir, "XML_FILE.xml");
string xsd = Path.Combine(basedir, "MAIN.xsd");
if (XmlUtils.ValidateAgainstXSD(xml, xsd, ref msg))
{
// do some work
}
else
{
Console.WriteLine(msg);
}
Console.ReadLine();
Any help is highly appreciated - Thank you!
UPDATE 2016-12-05:
I wrote my own XmlUrlResolver, to see what happens behind the scenes:
internal class XUrlResolver : XmlUrlResolver
{
public override object GetEntity(Uri absoluteUri, string role, Type ofObjectToReturn)
{
return base.GetEntity(absoluteUri, role, ofObjectToReturn);
}
public override Uri ResolveUri(Uri baseUri, string relativeUri)
{
return base.ResolveUri(baseUri, relativeUri);
}
}
And the i just try to do:
XmlSchemaSet xset = new XmlSchemaSet();
xset.XmlResolver = new XUrlResolver();
xset.Add("", xsdFilePath);
xset.Compile();
What happens now (on line xset.Add):
XmlUrlResolver.ResolveUri(null,"C:\\Temp\\MAIN.XSD") --> {file:///C:/Temp/MAIN.xsd}
XmlUrlResolver.ResolveUri(null,"C:\\Temp\\MAIN.XSD") --> {file:///C:/Temp/MAIN.xsd}
XmlUrlResolver.GetEntity({file:///C:/Temp/MAIN.xsd}) --> Filestream to MAIN.xsd
XmlUrlResolver.ResolveUri({file:///C:/Temp/MAIN.xsd},"TYPES.XSD") --> {file:///C:/Temp/TYPES.xsd}
XmlUrlResolver.GetEntity({file:///C:/Temp/TYPES.xsd}) --> Filestream to TYPES.xsd
Looks good to me (except the first 2 Calls are equal!?!) - the path to TYPES.XSD is resolved as it should.
Nevertheless, xset.Compile() throws an Exception: "Type MY_AMOUNT is not declared"
And i have no idea why :/

First you need to make your xsd files valid.
Types.xsd (added schema root element and xs namespace)
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:simpleType name="MY_AMOUNT">
<xs:restriction base="xs:decimal">
<xs:maxInclusive value="999999999999.99"/>
<xs:minInclusive value="-999999999999.99"/>
<xs:totalDigits value="14"/>
<xs:fractionDigits value="2"/>
</xs:restriction>
</xs:simpleType>
</xs:schema>
Main.xsd (removed invalid refs).
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
<xs:include schemaLocation="TYPES.xsd"/>
<xs:element name="ROOT">
<xs:complexType>
<xs:sequence>
<xs:element name="AMOUNT" type="MY_AMOUNT" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
After that, given that both xsd files are in the same directory, your schemas will compile fine.

I ran into this very same issue.
I'm not suggesting this is the correct answer, but I got around it by setting the Environment.CurrentDirectory property to be the path where the included XSDs were located. Then it all processed just fine.

Partial XML file validation using XSD

I am trying to use the XDocument class and XmlSchemaSet class to validate an XMl file.
The XML file already exists but I want to add in just a single element consisting of a couple other elements and I only want to validate this node.
Here is an example of the XML file. The piece I would like to validate is the TestConfiguration node:
<?xml version="1.0" encoding="ISO-8859-1"?>
<Root>
<AppType>Test App</AppType>
<LabelMap>
<Label0>
<Title>Tests</Title>
<Indexes>1,2,3</Indexes>
</Label0>
</LabelMap>
<TestConfiguration>
<CalculateNumbers>true</CalculateNumbers>
<RoundToDecimalPoint>3</RoundToDecimalPoint>
</TestConfiguration>
</Root>
Here is my xsd so far:
<?xml version="1.0" encoding="utf-8"?>
<xs:schema id="TestConfiguration"
targetNamespace="MyApp_ConfigurationFiles" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="TestConfiguration">
<xs:complexType>
<xs:sequence>
<xs:element name="CalculateNumbers" type="xs:boolean" minOccurs="1" maxOccurs="1"/>
<xs:element name="RoundToDecimalPoint" type="xs:int" minOccurs="1" maxOccurs="1"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
Here is the code I use to validate it:
private bool ValidateXML(string xmlFile, string xsdFile)
{
string xsdFilePath = Path.Combine(Path.GetDirectoryName(Assembly.GetEntryAssembly().Location) ?? string.Empty, xsdFile);
Logger.Info("Validating XML file against XSD schema file.");
Logger.Info("XML: " + xmlFile);
Logger.Info("XSD: " + xsdFilePath);
try
{
XDocument xsdDocument = XDocument.Load(xsdFilePath);
XmlSchemaSet schemaSet = new XmlSchemaSet();
schemaSet.Add(XmlSchema.Read(new StringReader(xsdDocument.ToString()), this.XmlValidationEventHandler));
XDocument xmlDocument = XDocument.Load(xmlFile);
xmlDocument.Validate(schemaSet, this.XmlValidationEventHandler);
}
catch (Exception e)
{
Logger.Info("Error parsing XML file: " + xmlFile);
throw new Exception(e.Message);
}
Logger.Info("XML validated against XSD.");
return true;
}
Even validating the full XML file, the validation will pass successfully causing me to run into problems when I try to load the XML file into the generated class file created by xsd2code, the error: <Root xmlns=''> was not expected..
How can I validate just the TestConfiguration piece?
Thanks

You have a few issues here:
Validating the entire document succeeds when it should fail.
This happens because the root node is unknown to the schema, and encountering an unknown node is considered a validation warning not a validation error - even if that unknown node is the root element. To enable warnings while validating, you need to set XmlSchemaValidationFlags.ReportValidationWarnings. However, there's no way to pass this flag to XDocument.Validate(). The question XDocument.Validate is always successful shows one way to work around this.
Having done this, you must also throw an exception in your validation handler when ValidationEventArgs.Severity == XmlSeverityType.Warning.
(As for requiring a certain root element in your XSD, this is apparently not possible.)
You need a convenient way to validate elements as well as documents, so you can validate your <TestConfiguration> piece.
Your XSD and XML are inconsistent.
You XSD specifies that your elements are in the XML namespace MyApp_ConfigurationFiles in the line targetNamespace="MyApp_ConfigurationFiles" elementFormDefault="qualified". In fact the XML elements shown in your question are not in any namespace.
If the XSD is correct, your XML root node needs to look like:
<Root xmlns="MyApp_ConfigurationFiles">
If the XML is correct, your XSD needs to look like:
<xs:schema id="TestConfiguration"
elementFormDefault="unqualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
After you have resolved the XSD and XML inconsistency from #3, you can solve issues #1 and #2 by introducing the following extension methods that validate both documents and elements:
public static class XNodeExtensions
{
public static void Validate(this XContainer node, XmlReaderSettings settings)
{
if (node == null)
throw new ArgumentNullException();
using (var innerReader = node.CreateReader())
using (var reader = XmlReader.Create(innerReader, settings))
{
while (reader.Read())
;
}
}
public static void Validate(this XContainer node, XmlSchemaSet schemaSet, XmlSchemaValidationFlags validationFlags, ValidationEventHandler validationEventHandler)
{
var settings = new XmlReaderSettings();
settings.ValidationType = ValidationType.Schema;
settings.ValidationFlags |= validationFlags;
if (validationEventHandler != null)
settings.ValidationEventHandler += validationEventHandler;
settings.Schemas = schemaSet;
node.Validate(settings);
}
}
Then, to validate the entire document, do:
try
{
var xsdDocument = XDocument.Load(xsdFilePath);
var schemaSet = new XmlSchemaSet();
using (var xsdReader = xsdDocument.CreateReader())
schemaSet.Add(XmlSchema.Read(xsdReader, this.XmlSchemaEventHandler));
var xmlDocument = XDocument.Load(xmlFile);
xmlDocument.Validate(schemaSet, XmlSchemaValidationFlags.ReportValidationWarnings, XmlValidationEventHandler);
}
catch (Exception e)
{
Logger.Info("Error parsing XML file: " + xmlFile);
throw new Exception(e.Message);
}
And to validate a specific node, you can use the same extension methods:
XNamespace elementNamespace = "MyApp_ConfigurationFiles";
var elementName = elementNamespace + "TestConfiguration";
try
{
var xsdDocument = XDocument.Load(xsdFilePath);
var schemaSet = new XmlSchemaSet();
using (var xsdReader = xsdDocument.CreateReader())
schemaSet.Add(XmlSchema.Read(xsdReader, this.XmlSchemaEventHandler));
var xmlDocument = XDocument.Load(xmlFile);
var element = xmlDocument.Root.Element(elementName);
element.Validate(schemaSet, XmlSchemaValidationFlags.ReportValidationWarnings, this.XmlValidationEventHandler);
}
catch (Exception e)
{
Logger.Info(string.Format("Error validating element {0} of XML file: {1}", elementName, xmlFile));
throw new Exception(e.Message);
}
Now validating the entire document fails while validating the {MyApp_ConfigurationFiles}TestConfiguration node succeeds, using the following validation event handlers:
void XmlSchemaEventHandler(object sender, ValidationEventArgs e)
{
if (e.Severity == XmlSeverityType.Error)
throw new XmlException(e.Message);
else if (e.Severity == XmlSeverityType.Warning)
Logger.Info(e.Message);
}
void XmlValidationEventHandler(object sender, ValidationEventArgs e)
{
if (e.Severity == XmlSeverityType.Error)
throw new XmlException(e.Message);
else if (e.Severity == XmlSeverityType.Warning)
throw new XmlException(e.Message);
}

.NET can't deserialize nested structs?

I'm running into issues getting C# (VS2008, Compact Framework, .NET is version 3.5 SP1) to successfully deserialize nested structs. The problem only appears in CF when I'm running on the emulator for the mobile device (I'm using the "Pocket PC 2003 Second Edition" emulator), the exact same code running on my Windows box does not have the same problem.
Here's my code:
public struct Fred
{
public string Name;
}
public struct Middle
{
public Fred[] Freds;
}
public struct Top
{
public Middle Middle;
public Fred[] Freds;
}
public static void Test()
{
Top top = new Top();
top.Middle.Freds = new Fred[2];
top.Middle.Freds[0].Name = "Fred20";
top.Middle.Freds[1].Name = "Fred21";
top.Freds = new Fred[2];
top.Freds[0].Name = "Fred10";
top.Freds[1].Name = "Fred11";
StringBuilder sb = new StringBuilder();
System.Xml.Serialization.XmlSerializer x = new System.Xml.Serialization.XmlSerializer(top.GetType());
using (StringWriter sw = new StringWriter(sb))
{
x.Serialize(sw, top);
}
string xml = sb.ToString();
string[] lines = xml.Split(new char[] { '\r', '\n' });
foreach (string line in lines)
{
Debug.WriteLine(" " + line.Trim());
}
MemoryStream ms = new MemoryStream(System.Text.Encoding.ASCII.GetBytes(xml));
StreamReader sr = new StreamReader(ms);
object o = x.Deserialize(sr);
Debug.WriteLine("Deserialized into " + o);
Top go2 = (Top)o;
if (go2.Freds == null)
Debug.WriteLine(" go2.Freds is null");
else
Debug.WriteLine(" go2.Freds[0].Name is \"" + go2.Freds[0].Name + "\"");
if (go2.Middle.Freds == null)
Debug.WriteLine(" go2.Middle.Freds is null");
else
Debug.WriteLine(" go2.Middle.Freds[0].Name is \"" + go2.Middle.Freds[0].Name + "\"");
}
When I run this, the XML it creates looks good:
<?xml version="1.0" encoding="utf-16"?>
<Top xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Middle>
<Freds>
<Fred>
<Name>Fred20</Name>
</Fred>
<Fred>
<Name>Fred21</Name>
</Fred>
</Freds>
</Middle>
<Freds>
<Fred>
<Name>Fred10</Name>
</Fred>
<Fred>
<Name>Fred11</Name>
</Fred>
</Freds>
</Top>
but C# is unable to successfully deserialize this XML - the console output is this:
Deserialized into Top
go2.Freds[0].Name is "Fred10"
go2.Middle.Freds is null
xsd has similar problems:
<?xml version="1.0" encoding="utf-8"?>
<xs:schema id="Top" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xs:element name="Top" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">
<xs:complexType>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="Middle">
<xs:complexType>
<xs:sequence>
<xs:element name="Freds" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="Fred" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="Name" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:complexType>
</xs:element>
</xs:schema>
Have I just encountered a C# bug? Or am I missing something obvious?
Note: It's not a problem with using the name twice, if I create a struct named George that is identical to Fred, and change the contents of Middle to public George[] George, the problem isn't any better.

TLDR (for the skimmers): This post consist of two parts.
Part 1: A quick intro to Protobuf. Attributes are used here.
Part 2: The actual answer to the question: to configure serialization without modifying the inherited library
Ok, I'll give it a shot.
The problem seems to be that you're using the Compact Framework, which does not have the same serialization/deserialization capabilities as the full .NET framework. So we need some custom serialization here.
Going with the philosophy of the Compact Framework, my guess is that you also want something that performs well and has a small footprint. So I picked Protobuf for the task (which is also approximately 12 times faster than XmlSerializer)
You can install it by running this command:
Install-Package protobuf-net
Let's start with the easy way - by adding attributes to your model.
Configuration without attributes comes next, as you pointed out that the original model can't/shouldn't be modified. This is only for illustration.
Decorated with the appropriate attributes, your model would look like this:
Part 1: Configuration with attributes.
I repeat, this part is only for illustration purposes - read on to "Configuration without attributes"
[ProtoContract]
public struct Fred
{
[ProtoMember(1)]
public string Name;
}
[ProtoContract]
public struct Middle
{
[ProtoMember(1)]
public Fred[] Freds;
}
[ProtoContract]
public struct Top
{
[ProtoMember(1)]
public Middle Middle;
[ProtoMember(2)]
public Fred[] Freds;
}
The only thing to note here is the usage of numbered members, called keys. It's essentially the same thing as giving them property names in the case of JSON or XML serialization, except this is the protobuf way to do it. You simply assign a unique integer value to each member within the same class, and most of the times you're done there.
For convenience let's add a simple Builder from which we can instantiate a Top which is similar to the one in your example:
public class TopTestBuilder
{
public Top BuildDefaultTestTop()
{
var top = new Top
{
Middle = new Middle
{
Freds = new[]
{
new Fred {Name = "Fred20"},
new Fred {Name = "Fred21"}
}
},
Freds = new[]
{
new Fred {Name = "Fred10"},
new Fred {Name = "Fred11"}
}
};
return top;
}
}
We can serialize it like this:
Top topIn = new TopTestBuilder().BuildDefaultTestTop();
string serialized;
using (var stream = new MemoryStream())
{
Protobuf.Serializer.Serialize(stream, topIn);
stream.Position = 0;
var reader = new StreamReader(stream);
serialized = reader.ReadToEnd();
}
// Output: "\nDC4\n\b\nACKFred20\n\b\nACKFred21DC2\b\nACKFred10DC2\b\nACKFred11"
And deserialize it like this:
Top topOut;
using (var stream = new MemoryStream())
{
var writer = new StreamWriter(stream);
writer.Write(serialized);
writer.Flush();
stream.Position = 0;
topOut = Protobuf.Serializer.Deserialize<Top>(stream);
}
As you can see there is a little bit of plumbing for the MemoryStreams, but other than that it should look familiar to how other types of serialization work. Similarly, everything can just as well be accomplished by configuring a custom TypeModel, allowing serialization to be fully decoupled from the model.
Part 2: Configuration without attributes
By default, Protobuf uses the attributes to define the TypeModel and then stores it in ProtoBuf.Meta.RuntimeTypeModel.Default. This property is used when you call the static Protobuf.Serializer directly.
We can also define our own. It took some fiddling around (note to self: RTFM) to get it working but it turned out to be almost as simple:
var model = TypeModel.Create();
// The first parameter (maps to ProtoContractAttribute) is the Type to be included.
// The second parameter determines whether to apply default behavior,
// based on the attributes. Since we're not using those, this has no effect.
model.Add(typeof(Fred), false);
model.Add(typeof(Middle), false);
model.Add(typeof(Top), false);
// The newly added MetaTypes can be accessed through their respective Type indices.
// The first parameter is the unique member number, similar to ProtoMemberAttribute.
// The second parameter is the name of the member as it is declared in the class.
// When the member is a list:
// The third parameter is the Type for the items.
// The fourth parameter is the Type for the list itself.
model[typeof(Fred)].Add(1, "Name");
model[typeof(Middle)].Add(1, "Freds", typeof(Fred), typeof(Fred[]));
model[typeof(Top)].Add(1, "Middle");
model[typeof(Top)].Add(2, "Freds", typeof(Fred), typeof(Fred[]));
Now all we have to do is change one line of code for both functions:
Serialize:
Top topIn = new TopTestBuilder().BuildDefaultTestTop();
string serialized;
using (var stream = new MemoryStream())
{
model.Serialize(stream, top);
stream.Position = 0;
var reader = new StreamReader(stream);
serialized = reader.ReadToEnd();
}
Deserialize:
Top topOut;
using (var stream = new MemoryStream())
{
var writer = new StreamWriter(stream);
writer.Write(serialized);
writer.Flush();
stream.Position = 0;
topOut = (Top) _model.Deserialize(stream, null, typeof (Top));
}
And it works just the same. Perhaps add a class to keep things organized - give it two public methods Serialize and Deserialize, and a private method BuildTypeModel (to call from the constructor and store in a field on the serializer?)
Your calling code would end up looking something like this:
var serializer = new CustomProtoBufSerializer();
var serialized = serializer.Serialize(someClassInput);
SomeClass someClassOutput = serializer.Deserialize(serialized);
One thing quickly became clear though - Protobuf hasn't been as thoroughly documented and tested as most JSON and XML serializers out there. This, along with the serialization results being non-readable to humans, could be a drawback in some situations. Other than that it seems like it's fast, lightweight and compatible with many different environments.
The absence of automatic type resolution bothered me a bit, so I went looking and found something that seems pretty interesting: Protobuf T4 TypeModel Generator. I haven't been able to try it yet. If people are interested, I might do that later and update the answer with a more generic solution.
Let me know if you have any trouble getting it to work.

unit testing xsd

I have been tasked with finding out why a program passes an XML unit test when the attributes in the xsd portion do not match the values in the unit test. The problem is I know next to nothing about XML or xsd. I have read up on it in various places but nohing has helped me understand why the test is failing.
It seems that before I can figure out why it's not working, I need to understand what is happening in the method doing the unit testing. I think I grasp most of it, but there is a portion that I'm not sure I understand. This portion also deals with the xsd.
Here is my xsd:
public static class Constants
{
public const string Xsd = #"
<xs:schema elementFormDefault=""qualified"" xmlns:xs=""http://www.w3.org/2001/XMLSchema"">
<xs:element name=""Datas"">
<xs:complexType>
<xs:sequence>
<xs:element name=""Data"">
<xs:attributeGroup ref=""allocData""/>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:attributeGroup id=""allocData"">
<xs:attribute name=""Site"" type=""string""/>
<xs:attribute name=""MonthNum"" type=""string""/>
<xs:attribute name=""Department"" type=""string""/>
<xs:attribute name=""Numerator"" type=""float""/>
<xs:attribute name=""Data_Indicator"" type=""string""/>
<xs:attribute name=""Industry_Benchmark"" type=""int""/>
<xs:attribute name=""OSHA_Benchmark"" type=""int""/>
<!--xs:attribute name=""Comments"" type=""string""/-->
<xs:attribute name=""Executive_Comments"" type=""string""/>
<xs:attribute name=""Fleet_Executive_Comments"" type=""string""/>
</xs:attributeGroup>
</xs:schema>";
and here is the method for my unit test:
public void XMLTest()
{
List<DataResults> ListOfResults = new List<DataResults>();
DataResults r1 = new DataResults();
DataResults r2 = new DataResults();
HarvesterHandle testData = new HarvesterHandle();
r1.Site = "Springfield Site";
r1.Department = "Freak";
r1.MonthNum = "8";
r1.Numerator = 65807;
r1.Data_Indicator = "A12";
// r1.Comments = "Hello, World";
r1.Industry_Benchmark = 23;
r1.OSHA_Benchmark = 2;
r1.ExecutiveComments = "It costs HOW MUCH?";
r1.FleetExecComments = "No you can't have a new car";
r2.Site = "Joplin Site";
r2.Department = "Rock";
r2.MonthNum = "12";
r2.Numerator = 65625;
r2.Data_Indicator = "Ou812";
r2.Industry_Benchmark = 523;
r2.OSHA_Benchmark = 2456;
// r2.Comments = "GoodBye, World!";
r2.ExecutiveComments = "Cut it!";
r2.FleetExecComments = "It only has 250,000 miles";
ListOfResults.Add(r1);
ListOfResults.Add(r2);
var test = testData.XMLTest(ListOfResults);
var xmlDoc = XDocument.Parse(test.ToString());
XmlSchemaSet set = new XmlSchemaSet();
set.Add(XmlSchema.Read(XElement.Parse(Constants.Xsd).CreateReader(), (o, e) => { })); bool valid = true;
xmlDoc.Validate(set, (o, e) =>
{
valid = false;
});
Assert.IsTrue(valid);
var element = test.Descendants("Data").FirstOrDefault(x => x.Attribute("Site").Value.Equals(r1.Site));
Assert.IsNotNull(element);
Assert.AreEqual(r1.Site, element.Attribute("Site").Value);
Assert.AreEqual(r1.Department, element.Attribute("Department").Value);
Assert.AreEqual(r1.MonthNum.ToString(), element.Attribute("MonthNum").Value);
Assert.AreEqual(r1.Numerator.ToString(), element.Attribute("Numerator").Value);
Assert.AreEqual(r1.Data_Indicator, element.Attribute("Data_Indicator").Value);
Assert.AreEqual(r1.Industry_Benchmark.ToString(), element.Attribute("Industry_Benchmark").Value);
Assert.AreEqual(r1.OSHA_Benchmark.ToString(), element.Attribute("OSHA_Benchmark").Value);
// Assert.AreEqual(r1.Comments.ToString(), element.Attribute(
// "Comments").Value);
Assert.AreEqual(r1.ExecutiveComments.ToString(), element.Attribute("Executive_Comments").Value);
Assert.AreEqual(r1.FleetExecComments.ToString(), element.Attribute("Fleet_Executive_Comments").Value);
element = test.Descendants("Data").FirstOrDefault(x => x.Attribute("Site").Value.Equals(r2.Site));
Assert.IsNotNull(element);
Assert.AreEqual(r2.Site, element.Attribute("Site").Value);
Assert.AreEqual(r2.Department, element.Attribute("Department").Value);
Assert.AreEqual(r2.MonthNum.ToString(), element.Attribute("MonthNum").Value);
Assert.AreEqual(r2.Numerator.ToString(), element.Attribute("Numerator").Value);
Assert.AreEqual(r2.Data_Indicator, element.Attribute("Data_Indicator").Value);
Assert.AreEqual(r2.Industry_Benchmark.ToString(), element.Attribute("Industry_Benchmark").Value);
Assert.AreEqual(r2.OSHA_Benchmark.ToString(), element.Attribute("OSHA_Benchmark").Value);
// Assert.AreEqual(r2.Comments.ToString(), element.Attribute(
// "Comments").Value);
Assert.AreEqual(r2.ExecutiveComments.ToString(), element.Attribute(Executive_Comments").Value);
Assert.AreEqual(r2.FleetExecComments.ToString(), element.Attribute("Fleet_Executive_Comments").Value);
}
the part that appear to be dealing with the xsd is this:
XmlSchemaSet set = new XmlSchemaSet();
set.Add(XmlSchema.Read(XElement.Parse(Constants.Xsd).CreateReader(), (o, e) => { })); bool valid = true;
xmlDoc.Validate(set, (o, e) =>
{
valid = false;
});
Assert.IsTrue(valid);
The problem is if the attribute data types in the xsd do not match the actual data, the test still passes. For instance, if any of the string attributes (MonthNum, Site, etc) are changed to float, the test still passes. Also, if the .xsd element names Datas and Data are changed to something like Fred and Ethyl, the test still passes.
I am assuming that whatever the xsd portion of the code in the xml unit test method is doing, it is not checking for correct datatypes or element names
per request, here is my xml:
public const string XSLT = #"
<xsl:stylesheet version=""1.0"" xmlns:xsl=""http://www.w3.org/1999/XSL/Transform"">
<xsl:output method=""text"" encoding=""us-ascii"" />
<xsl:template match=""/"">Site, MonthNum, Department, Numerator, Denominator, Data_Indicator, Industry_Benchmark, OSHA_Benchmark, <!--Comments--> Executive_Comments, Fleet_Executive_Comments,
<xsl:for-each select=""Datas/Data"">
<xsl:text>'</xsl:text>
<xsl:value-of select=""#site""/>
<xsl:text>','</xsl:text>
<xsl:value-of select=""#monthNum""/>
<xsl:text>','</xsl:text>
<xsl:value-of select=""#Department""/>
<xsl:text>','</xsl:text>
<xsl:value-of select=""#numerator""/>
<xsl:text>','</xsl:text>,
<xsl:value-of select=""Industry_Benchmark""/>
<xsl:text>'</xsl:text>,
<xsl:value-of select=""OSHA_Benchmark""/>
<xsl:text>'</xsl:text>,
<!--xsl:value-of select=""Comments""/-->
<!--xsl:text>'</xsl:text-->,
<xsl:value-of select=""Executive_Comments""/>
<xsl:text>'</xsl:text>,
<xsl:value-of select=""Fleet_Executive_Comments""/>
<xsl:text>'</xsl:text>,
</xsl:for-each>
</xsl:template>
</xsl:stylesheet>";
}
}
edit: This is the XML generated when running the test:
<Data>
<Data Site="Springfield Site" MonthNum="8" Department="Freak" Numerator="65807" Data_Indicator="A12" Industry_Benchmark="23" OSHA_Benchmark="2" Executive_Comments="It costs HOW MUCH?" Fleet_Executive_Comments="No you can't have a new car" />
<Data Site="Joplin Site" MonthNum="12" Department="Rock" Numerator="65625" Data_Indicator="Ou812" Industry_Benchmark="523" OSHA_Benchmark="2456" Executive_Comments="Cut it!" Fleet_Executive_Comments="It only has 250,000 miles" />
</Data>

It appears that the problem was that the xsd was not correct. It was written as:
<xs:attribute name=""Site"" type=""string""/>
when it should have been written as
<xs:attribute name=""Site"" type=""xs:string""/>