We Keep Coding

Different field serialization technique in ntext SQL field while keeping backward compatibility

TL, DR: we've been serializing some data in some tables of our SQL DB. Unfortunately that serialization technique wastes a lot of space for markup characters. We've found a new, more efficient way: to keep backwards compatibility, is it safe to adopt the following logic? -> When the serialization occurs, it always occurs using the new, more efficient way. When deserialization occurs, we check whether the string uses the new serialization format or the old one --> we then deserialize with the appropriate method. Is this robust? Can this be used in production? Aren't there any subtle problems with this approach?
Greetings. I'm working on an application which interacts with a SQL database. To achieve a specific business requirement, we've been serializing some data in a special column of our DB tables, of type ntext. Basically, in each cell of this column, we serialize an array of "Attributo" object, so typeof(T) is Attributo[]:
The "Attributo" definition is like the following:
public class Attributo
{
public virtual String Nome { get; set; }
public virtual String Valore { get; set; }
public virtual String Tipologia { get; set; }
}
- Deserialization to read the actual values:
XMLUtilities.Deserialize<Attributo[]>(value));
Serialization to store the values in the column (for each row..):
XMLUtilities.Serialize(attributes.ToArray());
And this is the helper class, which makes use of the XmlSerializer object:
public static class XMLUtilities
{
public static T Deserialize<T>(String xmlString)
{
XmlSerializer serializer = new XmlSerializer(typeof(T));
using (TextReader reader = new StringReader(xmlString))
{
return (T) serializer.Deserialize(reader);
}
}
public static String Serialize<T>(T xmlObject)
{
MemoryStream stream = new MemoryStream();
XmlSerializer serializer = new XmlSerializer(typeof(T));
XmlSerializerNamespaces xmlnsEmpty = new XmlSerializerNamespaces();
xmlnsEmpty.Add("", "");
serializer.Serialize(stream, xmlObject, xmlnsEmpty);
stream.Seek(0, SeekOrigin.Begin);
using (StreamReader reader = new StreamReader(stream))
{
return reader.ReadToEnd();
}
}
}
Now, the problem with this technique is that it wastes a lot of space for markup characters. This is an example string which is stored on the db:
<?xml version="1.0"?>
<ArrayOfAttributo>
<Attributo>
<Nome>Leakage_test_Time_prg1_p1</Nome>
<Valore>4</Valore>
<Tipologia>Single</Tipologia>
</Attributo>
<Attributo>
<Nome>Leakage_air_Volume_p1</Nome>
<Valore>14</Valore>
<Tipologia>Single</Tipologia>
</Attributo>
</ArrayOfAttributo>
So, we've found a more concise way of serializing these Attributo[], which produces this kind of output:
<ArrayOfA xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<A>
<N>Leakage_test_Time_prg1_p1</N>
<V>4</V>
<T>Single</T>
</A>
<A>
<N>Leakage_air_Volume_p1</N>
<V>14</V>
<T>Single</T>
</A>
</ArrayOfA>
then, to preserve backwards compatibility, which is the core issue we have implemented the following logic:
During serialization:
we always serialize in the new, more concise fashion
During deserialization:
we check whether the string starts with:
<?xml version="1.0"?>
or not. If that is the case, this is an old entry so we deserialize it in the old way. Otherwise, we deserialize using the new format.
We achieved that by decorating "Attributo" this way:
[DataContract(Name = "A", Namespace= "")]
public class Attributo
{
[DataMember(Name = "N")]
public virtual String Nome { get; set; }
[DataMember(Name = "V")]
public virtual String Valore { get; set; }
[DataMember(Name = "T")]
public virtual String Tipologia { get; set; }
}
and by performing the following changes to our serialization/deserialization methods, which now, for the new serialization technique, rely on DataContractSerializer object:
public static T Deserialize<T>(String xmlString)
{
//let's see if it's an old-style entry...
if (xmlString.StartsWith("<?xml version=\"1.0\"?>\r\n<ArrayOfAttributo>"))
{
try
{
XmlSerializer serializer = new XmlSerializer(typeof(T));
using (TextReader reader = new StringReader(xmlString))
{
return (T)serializer.Deserialize(reader);
}
}
catch { }
}
//..then it must be a new-style one
DataContractSerializer ser = new DataContractSerializer(typeof(T));
using (Stream s = _streamFromString(xmlString))
{
return (T) ser.ReadObject(s);
}
}
public static String Serialize<T>(T xmlObject)
{
MemoryStream stream1 = new MemoryStream();
DataContractSerializer ser = new DataContractSerializer(typeof(T));
ser.WriteObject(stream1, xmlObject);
stream1.Position = 0;
StreamReader sr = new StreamReader(stream1);
string xmlString = sr.ReadToEnd();
return xmlString;
}
private static Stream _streamFromString(string s)
{
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
writer.Write(s);
writer.Flush();
stream.Position = 0;
return stream;
}
Everything seems to be working with this approach but we want to assess every possible risk before proceeding any further. Is this safe to use in production?

One more thing to keep in mind, while deserializing the older entry :
deserialize old-entry in old-style
serialize the old-entry in new-style
save the new-style-serialized-old-entry, and delete the old-style-serialized-old-entry.
You're good to go.

How to serialize IEnumerable<char> as string & Confusing Deserialization error

Lets say I have the following simple class:
[XmlRoot]
[XmlType("string")]
public partial class eString : IEnumerable<char>
{
string AsString {get;set;}
public IEnumerator<char> GetEnumerator()
{
return this.AsString.ToCharArray().ToList().GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
return this.AsString.ToCharArray().GetEnumerator();
}
public void Add(char character)
{
this.AsString += character;
}
}
Also a namedValue class:
[XmlRoot]
public partial class eNamedValue: KeyValuePair<eString,object>
{...}
And finally,
[XmlRoot]
public partial class eList<T>: List<eNamedValue>
{...}
Now, I serialize eList or any class that inherits from eList using the following XML serializer:
public static XDocument Serialize<T>(this T obj) where T: new()
{
Type tt = obj.GetType();
XmlSerializer xsSubmit = new XmlSerializer(tt);
StringWriter sww = new StringWriter();
XmlWriter writer = XmlWriter.Create(sww);
xsSubmit.Serialize(writer, obj);
return XDocument.Parse(sww.ToString());
}
Serialization works - but my eString is being serialized as a character array, so instead of getting "string", I get individual characters values:
<eNamedList>
<Key>99</Key>
<Key>111</Key>
<Key>100</Key>
<Key>101</Key>...
Also, on Deserialization (SOLVED, see Update#1 below):
public static T Deserialize<T>(this XDocument xml) where T: new()
{
var serializer = new XmlSerializer(typeof(T));
T result;
using (TextReader reader = new StringReader(xml.ToString()))
{
result = (T)serializer.Deserialize(reader);
}
return result;
}
I receive the following error:
System.Runtime.Serialization.SerializationException: Error in line 1 position 111. Expecting element 'eNamedList' from namespace 'http://schemas.datacontract.org/2004/07/Epic'.. Encountered 'Element' with name 'eNamedList', namespace ''.
So, my questions become:
How do I control serialization of my eString, or any IEnumerable<char> and always serialize as a string?
Why, when the element names match, do I get a failure on deserialization? (Am I just missing a namespace declaration?)
Thanks!
Update #1:
So, I removed the IEnumerable<char> interface from my eString, and just left the IEnumerable<char>.GetEnumerator() method, which allows my string to be used AS an IEnumerable<char> while in a foreach loop, but serializes as a string. #WIN
Also, thanks to dbc, updated the original post with the XML Deserializer (rather than DataContract Serializer) and deserialization works.

To answer your questions:
You probably shouldn't reinvent the wheel with custom string solutions. Regardless, if you want an (insanely) high-level of control over your (de-)serialization, you could implement IXmlSerializable and do the exact same thing yourself.
[XmlRoot("string-wrapper")]
public class CustomString : IXmlSerializable
{
public string Value { get; set; }
public XmlSchema GetSchema()
{
return null; // GetSchema should not be used.
}
public void ReadXml(XmlReader reader)
{
reader.MoveToContent();
bool isEmpty = reader.IsEmptyElement;
reader.ReadStartElement();
if (!isEmpty)
{
Value = reader.ReadString();
reader.ReadEndElement();
}
}
public void WriteXml(XmlWriter writer)
{
writer.WriteString(Value);
}
}
Serialization of a CustomString now yields <string-wrapper>Testing</string-wrapper>. I'll post some test code at the end of this post.
Your deserialization is likely broken because the XmlSerializer doesn't know that the IEnumerable you marked serializable should actually be treated like a string.
And now... Forget what I just told you. Unless you have very specific formatting requirements you should not implement your own version of a string. The built-in formatter knows a lot more formatting tricks (http://www.w3.org/TR/xmlschema-2/#built-in-datatypes), so you should probably let it do it's job.
Serializing classes is where the attributes come in, though I recommend you switch to the WFC data contracts, which may sound scary at first but actually provides a whole lot more for a lot less code. Again, I'm not sure what you're trying to accomplish, but trust me you don't want to get into the habit of hand writing XML.
If you're up for it you might like dynamic objects and the ExpandoObject (http://www.codeproject.com/Tips/227139/Converting-XML-to-an-dynamic-object-using-ExpandoO). These eliminate types all together and allow you to create dictionaries, arrays, named properties, whatever, all on the fly!
Finally, easy on the generics! Deserializing generic classes is not a trivial task. Besides, you probably don't need to. If you want your own collections, try the System.Collections.ObjectModel namespace. You don't have to worry about maintaining lists and implementing interfaces, just what you're actually storing:
class DictionaryOfStringsAndObjects : KeyedCollection<string, object {...}
class CollectionOfStrings : Collection<string> {...}
Also, try to avoid partial classes unless an ORM or a large legacy class forces it on you. You shouldn't actually use them unless you're made to.
All the best, and to a future devoid of XML!
public class CustomSerializer
{
public static void Test()
{
var obj = new CustomString {Value = "Random string!"};
var serializer = new CustomSerializer();
var xml = serializer.Serialize(obj);
Console.WriteLine(xml);
var obj2 = serializer.Deserialize<CustomString>(xml);
}
public string Serialize(object obj)
{
var serializer = new XmlSerializer(obj.GetType());
using (var io = new StringWriter())
{
serializer.Serialize(io, obj);
return io.ToString();
}
}
public T Deserialize<T>(string xml)
{
var serializer = new XmlSerializer(typeof (T));
using (var io = new StringReader(xml))
{
return (T)serializer.Deserialize(io);
}
}
}

Serialize a derived type without the type and namespace attributes

I have an xml API that I have to mimic character for character. I'm trying to use the built-in xml serialization functionality in .NET, but it's adding some extra attributes. In a NORMAL web service or xml API, these attributes wouldn't hurt anything and might even serve a purpose. But they are unexpected characters and, unfortunately, I cannot allow them. So here's what I'd like to do (with hypothetical objects of course):
I have a base type
public abstract class Instrument { }
...and I have a derived type
public class Guitar : Instrument { }
...and I would like to serialize the derived type to something like this:
<Guitar />
Instead, I get this:
<Instrument d1p1:type="Guitar" xmlns:d1p1="http://www.w3.org/2001/XMLSchema-instance" />
Here's a test I'm working with:
[TestClass]
public class when_serializing_a_guitar
{
private XmlSerializer _serializer;
private string _expectedXml;
private StringWriter _stringWriter;
private string _actualXml;
private XmlSerializerNamespaces _ns;
private XmlWriter _xmlWriter;
private void WithThisContext()
{
_ns = new XmlSerializerNamespaces();
_ns.Add("", "");
_stringWriter = new StringWriter();
_xmlWriter = XmlWriter.Create(_stringWriter, new XmlWriterSettings
{
OmitXmlDeclaration = true,
CloseOutput = false
});
_serializer = new XmlSerializer(typeof(Instrument), new[] { typeof(Guitar) });
_expectedXml = #"<Guitar />";
}
private void BecauseOfThisAction()
{
_serializer.Serialize(_xmlWriter, new Guitar(), _ns);
_actualXml = _stringWriter.ToString();
}
[TestMethod]
public void it_should_return_the_expected_properly_formatted_xml()
{
WithThisContext();
BecauseOfThisAction();
Assert.AreEqual(_expectedXml, _actualXml);
}
}
Know how I can do this?

I'm assuming that you need to keep the domain model hierarchy intact. Otherwise you could just do this: var serializer = new XmlSerializer(typeof (Guitar));
If you do need to keep it intact, I suggest you write your own ToXml methods on each of your domain objects.
public interface IXmlWritable
{
string ToXml();
}
public class Instrument : IXmlWritable
{
public string classification { get; set; }
public string ToXml()
{
return "<Instrument classification='" + classification + "' />";
}
}
Or something like that depending on how you want to iterate through the nodes.

You can use XElement from System.Xml.Linq (you need to add the reference to it with 'add reference'). This is the code to simply create a clean xml doc:
XElement el = new XElement("data", new XElement("guitar"));
el.Save(#"D:\test.xml", SaveOptions.None);

Capture serialised XML when deserialising

.net, C#
Is it easily possible (by use of attributes etc) to automatically save the entire XML string (as a string field) that was created when an object was serialised when that object is deserialised?
I ask because I'm receiving an XML stub from a web service, and that stub contains a digital signature that I can use to verify the XML. I can deserialise the XML into a useful object that can be passed down into my application layer for verification, but I need the XML too. Ideally my new object would have an OriginalXML property or something. I could verify the XML at a higher level but it's not so convenient for me.
Cheers,
Chris.

You can load the XML file into a string, no problem. But the OriginalXML property would have to be marked with the [NonSerialized] attribute, because you don't want to store that string when you serialize. You'll have to deserialize, reload as an XmlDocument, and store the resulting string to that property.
XmlDocument xmlDoc = new XmlDocument();
try {
xmlDoc.Load(serializedFile);
}
catch (XmlException exc) {
// Handle the error
}
StringWriter stringWriter = new StringWriter();
XmlTextWriter xmlWriter= new XmlTextWriter(stringWriter);
xmlDoc.WriteTo(xmlWriter);
myObject.OriginalXML = xmlWriter.ToString();
HTH,
James

How about
[DataContract]
class FooBar
{
//this property doesn't have the DataMember attribure
//and thus won't be serialized
public string OriginalXml { get; set; }
[DataMember] private int _foo;
[DataMember] private string _bar;
static public FooBar Deserialize(XmlReader reader)
{
var fooBar =
(FooBar)new DataContractSerializer(typeof(FooBar)).ReadObject(reader);
fooBar.OriginalXml = reader.ToString();
return fooBar;
}
}

Wrapping Serialized Properties in Additional Elements with .NET XML Serialization

I am using C# + VSTS2008 + .Net 3.0 to do XML serialization. The code works fine. Here below is my code and current serialized XML results.
Now I want to add two additional layers to the output XML file. Here is my expected XML results. Any easy way to do this? I am not sure whether NestingLevel could help to do this. I want to find an easy way which does not change the structure of MyClass and MyObject.
Expected XML serialization result,
<?xml version="1.0"?>
<MyClass xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<MyObjectProperty>
<AdditionalLayer1>
<AdditionalLayer2>
<ObjectName>Foo</ObjectName>
</AdditionalLayer1>
</AdditionalLayer2>
</MyObjectProperty>
</MyClass>
Current XML serialization result,
<?xml version="1.0"?>
<MyClass xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<MyObjectProperty>
<ObjectName>Foo</ObjectName>
</MyObjectProperty>
</MyClass>
My current code,
public class MyClass
{
public MyObject MyObjectProperty;
}
public class MyObject
{
public string ObjectName;
}
public class Program
{
static void Main(string[] args)
{
XmlSerializer s = new XmlSerializer(typeof(MyClass));
FileStream fs = new FileStream("foo.xml", FileMode.Create);
MyClass instance = new MyClass();
instance.MyObjectProperty = new MyObject();
instance.MyObjectProperty.ObjectName = "Foo";
s.Serialize(fs, instance);
return;
}
}

I want to find an easy way which does not change the structure of MyClass and MyObject.
Frankly, that isn't going to happen. XmlSerializer (when used simply) follows the structure of the classes / members. So you can't add extra layers without changing the structure.
When used in a bespoke way (IXmlSerializable), the one thing you can say for sure is that it isn't simple... this is not a fun interface to implement.
My advice: introduce a DTO that mimics your desired format, and shim to that before you serialize.

Why do you need these additional two layers?
Is its a business requirement, you should add these two objects to your object model:
public class MyClass
{
public AdditionalLayer1 AdditionalLayer1;
}
public class AdditionalLayer1
{
public AdditionalLayer2 AdditionalLayer2;
}
public class AdditionalLayer2
{
public MyObject MyObjectProperty;
}
public class MyObject
{
public string ObjectName;
}
if its purely because of some compatibility requirements you can either do the thing above, or implement IXmlSerializable on MyClass:
public class MyClass : IXmlSerializable
{
public MyObject MyObjectProperty;
public void WriteXml (XmlWriter writer)
{
//open the two layers
//serialize MyObject
//close the two layers
}
public void ReadXml (XmlReader reader)
{
//read all the layers back, etc
}
public XmlSchema GetSchema()
{
return(null);
}
}

You could do a transformation with XSL after serializing and add the "missing" tags.

I would suggest serializing out to a MemoryStream instead of a FileStream and then loading the XML into an XmlDocument, then using the methods of that class to insert the extra nodes you want. After that, you can save it out.

I did exactly this only recently and didn't find overriding the IXmlSerializable interface at all difficult or complicated. Serialize/deserialize as normal but for the class that contains the sub-class which you need to wrap in extra tags derive from IXmlSerializable:
class ContainingClass : IXmlSerializable
{
...
#region IXmlSerializable Members
public XmlSchema GetSchema()
{
return null;
}
public void ReadXml(XmlReader reader)
{
**reader.ReadStartElement("Equipment");**
XmlSerializer ser = new XmlSerializer(typeof(Host));
while (reader.NodeType != XmlNodeType.EndElement)
{
Host newHost = new Host();
newHost.Name = (string) ser.Deserialize(reader);
_hosts.Add(newHost);
reader.Read();
}
// Read the node to its end.
// Next call of the reader methods will crash if not called.
**reader.ReadEndElement(); // "Equipment"**
}
public void WriteXml(XmlWriter writer)
{
XmlSerializer ser = new XmlSerializer(typeof(Host));
**writer.WriteStartElement("Equipment");**
foreach (Host host in this._hosts)
{
ser.Serialize(writer, host);
}
**writer.WriteEndElement();**
}
#endregion
All I do here is to wrap the de/serialization of the lower level objects. Here I'm wrapping an extra tag "Equipment" around all the Hosts in a collection in this class. You could add as many tags as you like here as long as you close each one you start.
NOTE: Bold is showing as ** text ** - just make sure you get rid of the double asterisks :)