We Keep Coding

Change XML opener(reader) to JSON

I have followed a good tutorial that shows how to make an Automation Framework in C# Selenium.
The config file is in XML at the moment, but I wanted some more practice and change it to a .json file.
At the moment we are using the namespace System.Xml.XPath; and my question is, are there a similar for JSON? Lets say "System.Json;" that works the same as my XML reader. So I don't need to refactor to much code, or is it unavoidably?
This is how it works at the moment.
//Initialize
ConfigReader.SetFrameworkSettings();
public class ConfigReader
{
public static void SetFrameworkSettings()
{
XPathItem aut;
string strFilename = Environment.CurrentDirectory.ToString() + "\\Config\\GlobalConfig.xml";
FileStream stream = new FileStream(strFilename, FileMode.Open);
XPathDocument document = new XPathDocument(stream);
XPathNavigator navigator = document.CreateNavigator();
//Get XML Details and pass it in XPathItem type variables
aut = navigator.SelectSingleNode("AutoFramework/RunSettings/AUT");
Settings.AUT = aut.Value.ToString();
}
}
public class Settings
{
public static string AUT { get; set; }
}
Would be awesome if you could just change this two lines
XPathDocument document = new XPathDocument(stream);
XPathNavigator navigator = document.CreateNavigator()
And the XpathItem
Cheers

I would recommend using Newtonsoft.Json which is available from Nuget.
To "reuse" your current code you would have to make some basic SettingsConverter first:
public static class SettingsConverter
{
public static string FromXmlToJson(string xml)
{
Settings settings = null;
// read your xml file and assign values to the settings object
// now you can "serialize" settings object into Json
return JsonSerialization.Serialize(settings);
}
public static string FromJsonToXml(string json)
{
Settings settings = JsonSerialization.Deserialize<MeSettings>(json);
// save settings using your "xml" serialization
return xmlString; // return xml string
}
}
To use these methods I'm using this JsonSerialization helper class :
public static class JsonSerialiation
{
public static string Serialize<T>(T obj)
{
using (MemoryStream stream = new MemoryStream())
{
using (JsonTextWriter writer = new JsonTextWriter(new StreamWriter(stream))
{
JsonSerializer serializer = new JsonSerializer();
serializer.Serializer(writer, obj);
writer.Flush();
stream.Position = 0;
using (StreamReader reader = new StreamREader(stream))
{
return reader.ReadToEnd();
}
}
}
}
public static T Deserialize<T>(string jsonString)
{
using (JsonTextReader reader = new JsonTextReader(new StringReader(str)))
{
JsonSerializer serializer = new JsonSerializer();
return serializer.Deserialize<T>(reader)
}
}
}
Above example requires from you to change your Settings class from static :
public class Settings
{
public static string AUT { get; set; }
}
To instance :
public class Settings
{
public string AUT { get; set; }
}
If you prefer to keep it as static. You should use JObject from Newtonsoft.Json library :
JObject obj = JObject.Parse(jsonString);
Settings.AUT = obj.SelectToken("AUT").Value<string>();
You can always use JsonConvert.Serialize<T>() and JsonConvert.Deserialize<T>() methods instead of the JsonSerialization helper class that I've made but in my opinion the less control you have over your code the bigger the problems will be.

Xml being deserialized into base class instead of derived classes

I know this is a popular topic and I have researched extensively without finding an answer to my problem.
I have a base class IntroductionAction and 2 derived classes IntroductionActionComplex and IntroductionActionSimple. I have a list of IntroductionAction objects to which I have added objects of both of the derived types. My classes are as follows:
[XmlInclude(typeof(IntroductionActionComplex))]
[XmlInclude(typeof(IntroductionActionSimple))]
public class IntroductionAction
{
public IntroductionAction() { }
}
public class IntroductionActionComplex : IntroductionAction
{
[XmlIgnore]
public string name { get; set; }
[XmlElement(ElementName = "QuestionString")]
public string question { get; set; }
[XmlElement(ElementName = "AnswerString")]
public List<string> answerStrings { get; set; }
public IntroductionActionComplex()
{
name = string.Empty;
question = null;
answerStrings = new List<string>();
}
}
public class IntroductionActionSimple : IntroductionAction
{
[XmlIgnore]
public string name { get; set; }
[XmlText]
public string Value { get; set; }
public IntroductionActionSimple()
{
Value = string.Empty;
}
}
I then create the List as follows
[XmlElement("IntroductionAction")]
public List<IntroductionAction> introductionActions { get; set; }
I am using XmlSerializer and everything serializes correctly. This is the resulting XML of the list containing one of each of the derived classes which is correct.
<IntroductionAction>
<QuestionString>
test
</QuestionString>
<AnswerString>
test
</AnswerString>
<AnswerString>
test
</AnswerString>
</IntroductionAction>
<IntroductionAction>
test
</IntroductionAction>
This XML file is going onto a device which doesn't read it as XML but just searches for the tags and does whatever work it needs to do and because of that the file can't contain any XSI or XSD tags, indentation, etc that is usually associated with proper XML.
My deserialization code is straight forward:
public static T Deserialize_xml_Config<T>(string file1, T obj)
{
XmlSerializer deserializer = new XmlSerializer(obj.GetType());
using (TextReader reader = new StreamReader(file1))
{
return (T)deserializer.Deserialize(reader);
}
}
Finally to my problem. When I deserialize, it is being deserialized to the base class IntroductionAction and not to the derived classes.
These IntroductionAction classes are just part of a much larger object that I am serializing/deserializing. I have tried making the base class abstract since it contains no functionality but I get an error on deserialization saying
The specified type is abstract: name='IntroductionAction'
Despite my XmlIncludes it seems unable to find the derived classes.
I have tried adding the types to the serializer but that didn't work.
Any help is much appreciated.
Edit:
This is what I mean by adding the types to the serializer
XmlSerializer deserializer = new XmlSerializer(obj.GetType(), new Type [] { typeof(IntroductionActionComplex), typeof(IntroductionActionSimple) });
using (TextReader reader = new StreamReader(file1))
{
return (T)deserializer.Deserialize(reader);
}
Also my attempt at using XmlAttributeOverrides:
XmlAttributeOverrides attrOverrides = new XmlAttributeOverrides();
var attrs = new XmlAttributes();
XmlElementAttribute attr = new XmlElementAttribute();
attr.ElementName = "IntroductionAction";
attr.Type = typeof(IntroductionActionComplex);
attrs.XmlElements.Add(attr);
attr.ElementName = "IntroductionAction";
attr.Type = typeof(IntroductionActionSimple);
attrs.XmlElements.Add(attr);
attrOverrides.Add(typeof(IntroductionAction), "IntroductionAction", attrs);
XmlSerializer deserializer = new XmlSerializer(obj.GetType(), attrOverrides);
using (TextReader reader = new StreamReader(file1))
{
return (T)deserializer.Deserialize(reader);
}

I think you are pretty close. Below is the full example of saving and loading the XML file based on derived class types. This will save the nodes as the derived type itself, so loading back in will keep the desired type, rather than convert back to the base type. You'll probably need to add exception handling, this was just a quick solution. I did not change your base IntroductionAction or the derived IntroductionActionComplex / IntroductionActionSimple classes.
public class RootNode
{
[XmlElement("IntroductionAction")]
public List<IntroductionAction> introductionActions { get; set; }
public RootNode()
{
introductionActions = new List<IntroductionAction>();
}
private static XmlAttributeOverrides GetXmlAttributeOverrides()
{
XmlAttributeOverrides xml_attr_overrides = new XmlAttributeOverrides();
XmlAttributes xml_attrs = new XmlAttributes();
xml_attrs.XmlElements.Add(new XmlElementAttribute(typeof(IntroductionActionComplex)));
xml_attrs.XmlElements.Add(new XmlElementAttribute(typeof(IntroductionActionSimple)));
xml_attr_overrides.Add(typeof(RootNode), "introductionActions", xml_attrs);
return xml_attr_overrides;
}
// Add exception handling
public static void SaveToFile(RootNode rootNode, string fileName)
{
using (MemoryStream mem_stream = new MemoryStream())
{
XmlSerializer serializer = new XmlSerializer(rootNode.GetType(), RootNode.GetXmlAttributeOverrides());
serializer.Serialize(mem_stream, rootNode);
using (BinaryWriter output = new BinaryWriter(new FileStream(fileName, FileMode.Create)))
{
output.Write(mem_stream.ToArray());
}
}
}
// Add exception handling
public static RootNode LoadFromFile(string fileName)
{
if (File.Exists(fileName))
{
using (FileStream file = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
using (TextReader reader = new StreamReader(file))
{
XmlSerializer serializer = new XmlSerializer(typeof(RootNode), RootNode.GetXmlAttributeOverrides());
return (RootNode)serializer.Deserialize(reader);
}
}
}
return null;
}
}
Test program:
internal class Program
{
private static void Main(string[] args)
{
RootNode obj = new RootNode();
obj.introductionActions.Add(new IntroductionActionComplex() { question = "qTest", answerStrings = { "aTest1", "aTest2" }, name = "aName1" });
obj.introductionActions.Add(new IntroductionActionSimple() { name = "aName2", Value = "aValue" });
RootNode.SaveToFile(obj, "Test.xml");
RootNode obj2 = RootNode.LoadFromFile("Test.xml");
}
}

Deserialisation doesn't rebuild dictionaries that are members of classes

I've got some serialisation code set up as follows:
static void SerialiseObject(Object o, String path)
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream(path, FileMode.Create);
formatter.Serialize(stream, o);
stream.Close();
}
static Object DeserialiseObject(String path)
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream(path, FileMode.Open, FileAccess.Read);
Object o = (Object)formatter.Deserialize(stream);
stream.Close();
return o;
}
And a class with the following member defined:
[Serializable]
public class CircuitModel
{
public Dictionary<String, Bus> Buses { protected set; get; }
...
}
I populate the Dictionary, and then the following code successfully serialises and deserialises the dictionary, with all Bus objects intact:
SerialiseObject(CircuitModel.Buses, "temp.bin");
Object o = DeserialiseObject("temp.bin");
But when I try to do the same for CircuitModel:
SerialiseObject(CircuitModel, "temp.bin");
Object o = DeserialiseObject("temp.bin");
CircuitModel.Buses has been initialised, but is empty.
I've also tried implementing serialisation with ISerializable (for the Bus and CircuitModel classes) and had exactly the same problem
Any idea as to why this would be happening?

I think you have something more sinister going on with your child collection because binary serialization of Dictionaries within classes does work just fine.
[TestFixture]
public class SerializeTest
{
[Test]
public void TestSer()
{
var parent = new Parent
{
Name = "Test"
};
parent.Children.Add("Child1", new Child {Name = "Child1"});
parent.Children.Add( "Child2", new Child { Name = "Child2" } );
SerialiseObject(parent, "test.bin");
var copy = DeserialiseObject("test.bin") as Parent;
Assert.IsNotNull(copy);
Assert.AreEqual(2, copy.Children.Count);
Assert.IsTrue(copy.Children.ContainsKey("Child1"));
Assert.AreEqual("Child1", copy.Children["Child1"].Name);
}
static void SerialiseObject( Object o, String path )
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream( path, FileMode.Create );
formatter.Serialize( stream, o );
stream.Close();
}
static Object DeserialiseObject( String path )
{
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream( path, FileMode.Open, FileAccess.Read );
Object o = (Object) formatter.Deserialize( stream );
stream.Close();
return o;
}
[Serializable]
private class Parent
{
public string Name { get; set; }
public Dictionary<string, Child> Children { get; protected set; }
public Parent()
{
Children = new Dictionary<string, Child>();
}
}
[Serializable]
private class Child
{
public string Name { get; set; }
}
}
The children deserialize with the parent and contain the details they were initialized with. I would check any code that is setting your Buses collection. My example just did it in the constructor of the parent class, but it may be possible that you have rogue code setting it after it's been deserialized?

Dictionaries are not serializable. Remove the dictionary if you need to serialize that data, and replace it by a list of a custom class that contains the data in the dictionary:
[Serializable]
public class BusItem
{
public string Name {get;set;}
public Bus Bus {get;set;}
}
Edit: I just found out you can actually serialize Dictionaries using the DataContractSerializer instead.
http://theburningmonk.com/2010/05/net-tips-xml-serialize-or-deserialize-dictionary-in-csharp/

If you are talking about XML serialization, it might be because Dictionary is not serializable to XML. Look at Why isn't there an XML-serializable dictionary in .NET.

Querying serialized object file

Is there anyway to achieve that without loading the whole file into memory? If so, what do you suggest me to do?
Class implementation:
[Serializable()]
public class Car
{
public string Brand { get; set; }
public string Model { get; set; }
}
[Serializable()]
public class CarCollection : List<Car>
{
}
Serialization to file:
CarCollection cars = new CarCollection
{
new Cars{ Brand = "BMW", Model = "7.20" },
new Cars{ Brand = "Mercedes", Model = "CLK" }
};
using (Stream stream = File.Open("data", FileMode.Create))
{
BinaryFormatter bin = new BinaryFormatter();
bin.Serialize(stream, cars);
}

If you serialize to XML you can use a SAX parser (XmlReader class), which will read from a stream seqentially.

To deserialize the collection one object at a time, you also need to serialize it one at a time.
Simplest way is to define your own generic class:
public static class StreamSerializer
{
public static void Serialize<T>(IList<T> list, string filename)
{
using (Stream stream = File.Open(filename, FileMode.Create))
{
BinaryFormatter bin = new BinaryFormatter();
// seralize each object separately
foreach (var item in list)
bin.Serialize(stream, item);
}
}
public static IEnumerable<T> Deserialize<T>(string filename)
{
using (Stream stream = File.Open(filename, FileMode.Open))
{
BinaryFormatter bin = new BinaryFormatter();
// deserialize each object separately, and
// return them one at a time
while (stream.Position < stream.Length)
yield return (T)bin.Deserialize(stream);
}
}
}
Then you can simply write:
CarsCollection cars = new CarsCollection
{
new Cars{ Brand = "BMW", Model = "7.20" },
new Cars{ Brand = "Mercedes", Model = "CLK" }
};
// note that you cannot serialize the entire list if
// you want to query without loading - it must be symmetrical
StreamSerializer.Serialize(cars, "data.bin");
// the following expression iterates through objects, processing one
// at a time. "First" method is a good example because it
// breaks early.
var bmw = StreamSerializer
.Deserialize<Cars>("data.bin")
.First(c => c.Brand == "BMW");
A slightly more complex case might be if your CarsCollection belongs to a different class. In that case, you will need to implement ISerializable, but the principle is similar.
On a side note, usual convention is not to name entities in plural (i.e. Cars should be named Car).

Generally you can use some sort of reader (StreamReader, BinaryReader, ...) together with BufferedStream.

How can I send multiple types of objects across Protobuf?

I'm implementing a client-server application, and am looking into various ways to serialize and transmit data. I began working with Xml Serializers, which worked rather well, but generate data slowly, and make large objects, especially when they need to be sent over the net. So I started looking into Protobuf, and protobuf-net.
My problem lies in the fact that protobuf doesn't sent type information with it. With Xml Serializers, I was able to build a wrapper which would send and receive any various (serializable) object over the same stream, since object serialized into Xml contain the type name of the object.
ObjectSocket socket = new ObjectSocket();
socket.AddTypeHandler(typeof(string)); // Tells the socket the types
socket.AddTypeHandler(typeof(int)); // of objects we will want
socket.AddTypeHandler(typeof(bool)); // to send and receive.
socket.AddTypeHandler(typeof(Person)); // When it gets data, it looks for
socket.AddTypeHandler(typeof(Address)); // these types in the Xml, then uses
// the appropriate serializer.
socket.Connect(_host, _port);
socket.Send(new Person() { ... });
socket.Send(new Address() { ... });
...
Object o = socket.Read();
Type oType = o.GetType();
if (oType == typeof(Person))
HandlePerson(o as Person);
else if (oType == typeof(Address))
HandleAddress(o as Address);
...
I've considered a few solutions to this, including creating a master "state" type class, which is the only type of object sent over my socket. This moves away from the functionality I've worked out with Xml Serializers, though, so I'd like to avoid that direction.
The second option would be to wrap protobuf objects in some type of wrapper, which defines the type of object. (This wrapper would also include information such as packet ID, and destination.) It seems silly to use protobuf-net to serialize an object, then stick that stream between Xml tags, but I've considered it. Is there an easy way to get this functionality out of protobuf or protobuf-net?
I've come up with a third solution, and posted it below, but if you have a better one, please post it too!
Information on field bounds bug (using System.String):
Hashing:
protected static int ComputeTypeField(Type type) // System.String
{
byte[] data = ASCIIEncoding.ASCII.GetBytes(type.FullName);
MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
return Math.Abs(BitConverter.ToInt32(md5.ComputeHash(data), 0));
}
Serialization:
using (MemoryStream stream = new MemoryStream())
{
Serializer.NonGeneric.SerializeWithLengthPrefix
(stream, o, PrefixStyle.Base128, field); // field = 600542181
byte[] data = stream.ToArray();
_pipe.Write(data, 0, data.Length);
}
Deserializaion:
using (MemoryStream stream = new MemoryStream(_buffer.Peek()))
{
lock (_mapLock)
{
success = Serializer.NonGeneric.TryDeserializeWithLengthPrefix
(stream, PrefixStyle.Base128, field => _mappings[field], out o);
}
if (success)
_buffer.Clear((int)stream.Position);
else
{
int len;
if (Serializer.TryReadLengthPrefix(stream, PrefixStyle.Base128, out len))
_buffer.Clear(len);
}
}
field => _mappings[field] throws a KeyNotFoundException while looking for 63671269.
If I replace ToInt32 with ToInt16 in the hash function, the field value is set to 29723 and it works. It also works if I explicitly define System.String's field to 1. Explicitly defining the field to 600542181 has the same effect as using the hash function to define it. The value of the string being serialized does not change the outcome.

This functionality is actually built in, albeit not obviously.
In this scenario, it is anticipated that you would designate a unique number per message type. The overload you are using passes them all in as "field 1", but there is an overload that lets you include this extra header information (it is still the job of the calling code to decide how to map numbers to types, though). You can then specify different types as different fields is the stream (note: this only works with the base-128 prefix style).
I'll need to double check, but the intention is that something like the following should work:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using ProtoBuf;
static class Program
{
static void Main()
{
using (MemoryStream ms = new MemoryStream())
{
WriteNext(ms, 123);
WriteNext(ms, new Person { Name = "Fred" });
WriteNext(ms, "abc");
ms.Position = 0;
while (ReadNext(ms)) { }
}
}
// *** you need some mechanism to map types to fields
static readonly IDictionary<int, Type> typeLookup = new Dictionary<int, Type>
{
{1, typeof(int)}, {2, typeof(Person)}, {3, typeof(string)}
};
static void WriteNext(Stream stream, object obj) {
Type type = obj.GetType();
int field = typeLookup.Single(pair => pair.Value == type).Key;
Serializer.NonGeneric.SerializeWithLengthPrefix(stream, obj, PrefixStyle.Base128, field);
}
static bool ReadNext(Stream stream)
{
object obj;
if (Serializer.NonGeneric.TryDeserializeWithLengthPrefix(stream, PrefixStyle.Base128, field => typeLookup[field], out obj))
{
Console.WriteLine(obj);
return true;
}
return false;
}
}
[ProtoContract] class Person {
[ProtoMember(1)]public string Name { get; set; }
public override string ToString() { return "Person: " + Name; }
}
Note that this doesn't currently work in the v2 build (since the "WithLengthPrefix" code is incomplete), but I'll go and test it on v1. If it works, I'll all the above scenario to the test suite to ensure it does work in v2.
Edit:
yes, it does work fine on "v1", with output:
123
Person: Fred
abc

I've come up with another solution, but I decided to put it as an answer, instead of in the question, because that makes more sense to me. It's pretty ugly, in my opinion, and I've been warned against using reflection, so please comment on it or provide better answers if you have them. Thanks!
class Program
{
static void Main(string[] args)
{
Person person = new Person
{
Id = 12345,
Name = "Fred",
Address = new Address
{
Line1 = "Flat 1",
Line2 = "The Meadows"
}
};
object value;
using (Stream stream = new MemoryStream())
{
Send<Person>(stream, person);
stream.Position = 0;
value = Read(stream);
person = value as Person;
}
}
static void Send<T>(Stream stream, T value)
{
Header header = new Header()
{
Guid = Guid.NewGuid(),
Type = typeof(T)
};
Serializer.SerializeWithLengthPrefix<Header>(stream, header, PrefixStyle.Base128);
Serializer.SerializeWithLengthPrefix<T>(stream, value, PrefixStyle.Base128);
}
static object Read(Stream stream)
{
Header header;
header = Serializer.DeserializeWithLengthPrefix<Header>
(stream, PrefixStyle.Base128);
MethodInfo m = typeof(Serializer).GetMethod("DeserializeWithLengthPrefix",
new Type[] {typeof(Stream), typeof(PrefixStyle)}).MakeGenericMethod(header.Type);
Object value = m.Invoke(null, new object[] {stream, PrefixStyle.Base128} );
return value;
}
}
[ProtoContract]
class Header
{
public Header() { }
[ProtoMember(1, IsRequired = true)]
public Guid Guid { get; set; }
[ProtoIgnore]
public Type Type { get; set; }
[ProtoMember(2, IsRequired = true)]
public string TypeName
{
get { return this.Type.FullName; }
set { this.Type = Type.GetType(value); }
}
}
[ProtoContract]
class Person {
[ProtoMember(1)]
public int Id { get; set; }
[ProtoMember(2)]
public string Name { get; set; }
[ProtoMember(3)]
public Address Address { get; set; }
}
[ProtoContract]
class Address {
[ProtoMember(1)]
public string Line1 { get; set; }
[ProtoMember(2)]
public string Line2 { get; set; }
}