We Keep Coding

Loosing an XML file when using XmlSerializer

Sometimes, when I save to XML, I end up with a completely empty XML file.
I can't reproduce the issue on demand yet. It is just occasional. Are there steps that one can take to assist the user in this regard?
At the moment I do this:
public bool SavePublisherData()
{
bool bSaved = false;
try
{
XmlSerializer x = new XmlSerializer(_PublisherData.GetType());
using (StreamWriter writer = new StreamWriter(_strPathXML))
{
_PublisherData.BuildPublisherListFromDictionary();
x.Serialize(writer, _PublisherData);
bSaved = true;
}
}
catch
{
}
return bSaved;
}
The reason I have not put anything in the catch block is because this code is part of a C# DLL and I am calling it from an MFC project. I have read that you can't (or shouldn't) pass exceptions through from one environment to another. Thus, when an exception happens in my DLL I don't really know how I can sensibly feed that information to the user so they can see it. That is a side issue.
But this is how I save it. So, what steps can one take to try and prevent complete data loss?
Thank you.
Update
I have looked at the KB article that the link in the comments refers to and it states:
Use the following XmlSerializer class constructors. These class constructors cache the assemblies.
This is also re-stated in the article itself indicated in the comments:
What is the solution?
The default constructors XmlSerializer(type) and XmlSerializer(type, defaultNameSpace) caches the dynamic assembly so if you use those constructors only one copy of the dynamic assembly needs to be created.
Seems pretty smart… why not do this in all constructors? Hmm… interesting idea, wonder why they didn’t think of that one:) Ok, the other constructors are used for special cases, and the assumption would be that you wouldn’t create a ton of the same XmlSerializers using those special cases, which would mean that we would cache a lot of items we later didn’t need and use up a lot of extra space. Sometimes you have to do what is good for the majority of the people.
So what do you do if you need to use one of the other constructors? My suggestion would be to cache the XmlSerializer if you need to use it often. Then it would only be created once.
My code uses one of these default constructors as you can see:
XmlSerializer(_PublisherData.GetType());
So I don't think I need to worry about this XmlSerializerFactory in this instance.

Is there a serialization system for .net that supports reference tracking across streams?

I would like to track object references across multiple chunks of serialized objects. The serializers I know can track references, but only to serialize one graph as a whole at once. I want to partially serialize a graph and then later serialize another part of it or update some objects from the graph.
I think the code below explains my problem quite well - I'm searching for a Serializer that would print a "Success" if it was used here.
[Serializable]
class TestClass
{
public string Identifier;
public TestClass Reference;
}
class Program
{
static void Main(string[] args)
{
//some test data
TestClass t1 = new TestClass();
t1.Identifier = "t1";
TestClass t2 = new TestClass();
t2.Reference = t1;
t2.Identifier = "t2";
BinaryFormatter formatter = new BinaryFormatter(); //replace this with something that works ^^
MemoryStream stream = new MemoryStream(1024);
formatter.Serialize(stream, t1);
//possibly do lots of things inbetween these two, including,
//but not limited to, switching streams, writing other things
//on the stream etc.
formatter.Serialize(stream, t2);
stream.Position = 0;
object copy1 = formatter.Deserialize(stream);
object copy2 = formatter.Deserialize(stream);
if (((TestClass)copy2).Reference == copy1)
{
Console.WriteLine("Success");
}
else
{
Console.WriteLine("Failure");
}
Console.ReadLine();
}
}
My use case is essentially that I want to send objects over the network and keep the references intact. I have looked into protobuf-net and have written some workarounds around this issue, but they are getting out of hands in terms of complexity. My question is: is there a (good) serialization library that allows me to track references across one call of the deserialization method? Or am I missing some simple technique to archieve this with existing serializers? Or do I have to write such a thing by myself?
EDIT: Clarification: This is networking for a game, so speed and compactness are an issue. That means XML/JSON serializers are a rather bad alternative.

DataContracts can provide this functionality. Basically you would decorate a class with [DataContract(IsReference=true)]. Also, the good news about DataContracts is that they're baked into WCF by default, so you'll already have a communication layer to go along with your project.
See this question for more.
Circular references preventing serialization of object graph
And here's some more information on MSDN's site:
https://msdn.microsoft.com/en-us/library/hh241056(v=vs.100).aspx

enum serialization with protobuf-net

I updated an older version of protobuf to the current one in a huge project (the used version is around 1-2 years old. I don't know the rev).
Sadly the newer version throws an exception
CreateWireTypeException in ProtoReader.cs line 292
in the following test case:
enum Test
{
test1 = 0,
test2
};
static public void Test1()
{
Test original = Test.test2;
using (MemoryStream ms = new MemoryStream())
{
Serializer.SerializeWithLengthPrefix<Test>(ms, original, PrefixStyle.Fixed32, 1);
ms.Position = 0;
Test obj;
obj = Serializer.DeserializeWithLengthPrefix<Test>(ms, PrefixStyle.Fixed32);
}
}
I found out enums are not supposed to serialized directly outside of a class but our system is too huge to simply wrap all the enums in classes. Are there any other solutions to this problem? It works fine with Serialize and Deserialize only the DeserializeWithLengthPrefix throws exceptions.
The testcase works fine in older revisions e.g. r262 of protobuf-net.

Simply, a bug; this is fixed in r640 (now deployed to both NuGet and google-code), along with an additional test based on your code above so that it can't creep back in.
Re performance (comments); the first hint I would look at would be: "prefer groups". Basically, the protobuf specification includes 2 different ways of including sub-objects - "groups" and "length-prefix". Groups was the original implementation, but google have now move towards "length-prefix", and try to advise people not to use "groups". However! Because of how protobuf-net works, "groups" are actually noticeably cheaper to write; this is because unlike the google implementation, protobuf-net does not know the length of things in advance. This means that to write a length-prefix, it needs to do one of:
calculate the length (almost as much work as actually serializing the data, bud adds an entire duplicate of the code) as needed; write the length, then actually serialize the data
serialize to a buffer, write the length, write the buffer
leave a place-holder, serialize, then loop back and write the actual length into the place-holder, adjusting the padding if needed
I've implemented all 3 approaches at different times, but v2 uses the 3rd option. I keep toying with adding a 4th implementation:
leave a place-holder, serialize, then loop back and write the actual length using an overlong form (so no padding adjustments ever needed)
but... consensus seems to be that the "overlong form" is a bit risky; still, it would work nicely for protobuf-net to protobuf-net.
But as you can see: length-prefix always has some overhead. Now imagine fairly deeply nested objects, and you can see a few blips. Groups work very differently; the encoding format for a group is:
write a start marker; serialize; write an end marker
that's it; no length needed; really, really, really cheap to write. On the wire, the main difference between them is:
groups: cheap to write, but you can't skip them if you encounter them as unexpected data; you have to parse the headers of the payload
length-prefix: more expensive to write, but cheap to skip if you encounter them as unexpected data - you just read the length and copy/move that many bytes
But! too much detail!
What does that mean for you? Well, imagine you have:
[ProtoContract]
public class SomeWrapper
{
[ProtoMember(1)]
public List<Person> People { get { return people; } }
private readonly List<Person> people = new List<Person>();
}
You can make the super complex change:
[ProtoContract]
public class SomeWrapper
{
[ProtoMember(1, DataFormat=DataFormat.Group)]
public List<Person> People { get { return people; } }
private readonly List<Person> people = new List<Person>();
}
and it'll use the cheaper encoding scheme. All your existing data will be fine as long as you are using protobuf-net.

.net Serialization: How to use raw binary writer while maintaining which thing is which

I'm making a roguelike game in XNA with procedurally generated levels.
It takes about a second to generate a whole new level but takes about 4 seconds to serialize it and about 8 seconds to deserialize one with my current methods. Also the files are massive (about 10 megs depending on how big the level is)
I serialize like this.
private void SerializeLevel()
{
string name = Globals.CurrentLevel.LvLSaveString;
using (Stream stream = new FileStream("SAVES\\"+name+".lvl", FileMode.Create, FileAccess.Write, FileShare.None))
{
formatter.Serialize(stream, Globals.CurrentLevel);
stream.Close();
}
}
My game engine architecture is basically a load of nested Lists which might go..
Level\Room\Interior\Interiorthing\sprite
This hierarchy is important to maintain for the game/performance. For instance usually only things in the current room are considered for updates and draws.
I want to try something like the Raw Binary formatter shown in this post to improve serialization/deserialization performance
I can just save the ints and floats and bools which correspond to all the positions of/configurations of things and reinstantiate everything when I load a level (which only takes a second)
My question is how do I use this Raw Binary serializer while also maintaining which object is which, what type it is and which nested list it is in.
In the example cited OP is just serializing a huge list of ints and every 3rd one is taken as the start of a new coordinate.
I could have a new stream for each different type of thing in each room but that would result in loads of different files (I think) Is there a way to segregate the raw binary stream with some kind of hierarchy? Ie. split it up into different sections pertaining to different rooms and different lists of things?
UPDATE
Ok, one thind that was throwing me off was that in question I reference OP is referring to "manual serialization" as "raw binary serialization" which I couldnt find any info on.

If you want to serialize each member of Globals independently, and upon deserialization to manually update the member value, you need to know which member you are currently processing upon deserialization. I can suggest you these:
Process items in the same order. The code in your example will put binary data in the stream that it is nearly impossible to extract, unless you deserialize members in the order they have been serialized. This is going to be maintenance hell if new items are added and is not a good solution regarding both code clarity, maintainability and backwards compatibility.
Use dictionary. As per comments, Globals appears to be a static class, therefore it itself is not serializable. When serializing, put all members of the Globals class in a dictionary, and serialize it. Upon deserialization, you will know that you have a dictionary (not a random mess of objects). Then from the deserialized dictionary restore the Globals object
Use custom class. Create a class with all settings (a better approach). Use a single static instance of the class to access settings. You can serialize and deserialize that class
Settings. The second approach gets closer to an already built-in concept in .NET - Settings. Take a look at it, it seems that the Globals class is in fact a custom variant of a settings configuration

How does BinaryFormatter.Deserialize create new objects?

When BinaryFormatter deserializes a stream into objects, it appears to create new objects without calling constructors.
How is it doing this? And why? Is there anything else in .NET that does this?
Here's a demo:
[Serializable]
public class Car
{
public static int constructionCount = 0;
public Car()
{
constructionCount++;
}
}
public class Test
{
public static void Main(string[] args)
{
// Construct a car
Car car1 = new Car();
// Serialize and then deserialize to create a second, identical car
MemoryStream stream = new MemoryStream();
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(stream, car1);
stream.Seek(0, SeekOrigin.Begin);
Car car2 = (Car)formatter.Deserialize(stream);
// Wait, what happened?
Console.WriteLine("Cars constructed: " + Car.constructionCount);
if (car2 != null && car2 != car1)
{
Console.WriteLine("But there are actually two.");
}
}
}
Output:
Cars constructed: 1
But there are actually two.

There are two things calling a constructor does (or at least should do).
One is to set aside a certain amount of memory for the object and does all the housekeeping necessary for it to be an object to the rest of the .NET world (note certain amount of handwaving in this explanation).
The other is to put the object into a valid initial state, perhaps based on parameters - this is what the actual code in the constructor will do.
Deserialisation does much the same thing as the first step by calling FormatterServices.GetUninitializedObject, and then does much the same thing as the second step by setting the values for fields to be equivalent to those that were recorded during serialisation (which may require deserialising other objects to be said values).
Now, the state that deserialisation is putting the object into may not correspond to that possible by any constructor. At best it will be wasteful (all values set by the constructor will be overwritten) and at worse it could be dangerous (constructor has some side-effect). It could also just be impossible (only constructor is one that takes parameters - serialisation has no way of knowing what arguments to use).
You could look at it as a special sort of constructor only used by deserialisation (OO purists will - and should - shudder at the idea of a constructor that doesn't construct, I mean this as an analogy only, if you know C++ think of the way overriding new works as far as memory goes and you've an even better analogy, though still just an analogy).
Now, this can be a problem in some cases - maybe we have readonly fields that can only be set by a constructor, or maybe we have side-effects that we want to happen.
A solution to both is to override serialisation behaviour with ISerializable. This will serialise based on a call to ISerializable.GetObjectData and then call a particular constructor with SerializationInfo and StreamingContext fields to deserialise (said constructor can even be private - meaning most other code won't even see it). Hence if we can deserialise readonly fields and have any side-effects we want (we can also do all manner of things to control just what is serialised and how).
If we just care about ensuring some side-effect happens on deserialisation that would happen on construction, we can implement IDeserializationCallback and we will have IDeserializationCallback.OnDeserialization called when deserialisation is complete.
As for other things that do the same thing as this, there are other forms of serialisation in .NET but that's all I know of. It is possible to call FormatterServices.GetUninitializedObject yourself but barring a case where you have a strong guarantee that subsequent code will put the object produced into a valid state (i.e. precisely the sort of situation you are in when deserialising an object from data produced by serialising the same sort of object) doing such is fraught and a good way to produce a really hard to diagnose bug.

The thing is, BinaryFormatter isn't really making your particular object. It's putting an object graph back into memory. The object graph is basically the representation of your object in memory; this was created when the object is serialized. Then, the deserialize call basically just sticks that graph back in memory as an object at an open pointer, and then it gets casted to what it actually is by the code. If it's casted wrong, then an exception is thrown.
As to your particular example, you are only really constructing one car; you are just making an exact duplicate of that car. When you serialize it off into the stream, you store an exact binary copy of it. When you deserialize it, you don't have to construct anything. It just sticks the graph in memory at some pointer value as an object and lets you do whatever you want with it.
Your comparison of car1 != car2 is true because of that different pointer location, since Car is a reference type.
Why? Frankly, it's easy to just go pull the binary representation, rather than having to go and pull each property and all that.
I'm not sure whether anything else in .NET uses this same procedure; the most likely candidates would be anything else that uses an object's binary in some format during serialization.

Not sure why the constructor does not get called but I use IDeserializationCallback as a work around.
also take a look at
OnSerializingAttribute
OnSerializedAttribute
OnDeserializingAttribute
OnDeserializedAttribute