We Keep Coding

C# generics code-bloat - should I be worried?

I'd like to ask something about generics.
I am trying to keep the code simple, and thus I will be making a single class to handle load/save for a game's savegame files. As each portion of the game has different requirements I'd like to keep this as easily accessible as possible:
public void Load<T>(string path, out T obj)
{
BinaryFormatter bf = new BinaryFormatter();
using (FileStream file = File.Open(Application.persistentDataPath + path, FileMode.Open))
{
obj = (T)bf.Deserialize(file);
}
}
Now I can call this with a simple
TurnData x; s.Load("test.txt", out x);
The alternative would be to make the Load function return the object and then convert it to a TurnData type.
TurnData x = (TurnData)s.Load("test.txt");
I do not know much about C#. I assume that the code inside using(...) { ... } does not get executed if there is an error opening the file for example? If someone can confirm this that would be nice. The example code I have seen did not have any error handling, which seemed weird to me, so I added using?
So in this secondary version where the function returns the object instead of using an out parameter would need more complicated code for error checking and possible return null? It doesn't seem great.
So the real question is ... can I use the next version I have here or are there concerns that I should have due to the use of generics?

There is no generic code bloat for reference types - code is reused. With value types, though, CLR will generate a separate method for each type. See
.NET Generics and Code Bloat.

The using statement has nothing to do with error handling. Using File.Open method you can expect to get the exceptions you will find here. You could avoid the abruptly stop of your program from any such an exception by wrapping your using statement in a try/cath construct like below:
public T Load<T>(string path)
{
T obj = default(T);
var bf = new BinaryFormatter();
try
{
using (var file = File.Open(Application.persistentDataPath + path, FileMode.Open))
{
obj = (T)bf.Deserialize(file);
}
}
catch(Exception exception)
{
// Log the exception
}
return obj;
}
Essentially you attempt to Open the file specified in the path. If that fails you just log the failure and your return null from the function.
Regarding the using statement, it provides
a convenient syntax that ensures the correct use of IDisposable
objects.
as you can read more thoroughly here
As a side note regarding the signature of your method I would make a few comments. Consider the following method body and spot the differences with that we have above.
public T Load<T>(string path, IFormatter formatter)
{
if(path ==null) throw new ArgumentNullException(nameof(path));
if(formatter == null) throw new ArgumentNullException(nameof(formatter));
T obj = default(T);
try
{
using (var file = File.Open(path, FileMode.Open))
{
obj = (T)formatter.Deserialize(file);
}
}
catch(Exception exception)
{
// Log the exception
}
return obj;
}
and
var path = Path.Combine(Application.persistentDataPath, "test.txt");
var binaryFormatter = new BinaryFormatter();
var x = s.Load(path, binaryFormatter);
Making the above changes you make your method more easily to be tested through a unit test and more reliable since you make some precondition checking before the meat and potatoes of your method. What would had happened if you had passed a null path? What would had happened if you had passed a null formatter ? etc...

Deserialization problem in .NET

I created a binary serialized file of my object, so that I could use it as a database file for many other projects. It works quite fine with the same project, but when I try to deserialize that file from other projects, it won't. The error that appears says, "xxxx.xxxx assembly not found". So, how should I serialize the object in order to make it assembly independent???
Here's my code :
// Binary formatter
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream("BinarySerialization.bin",
FileMode.Create,
FileAccess.Write, FileShare.None);
formatter.Serialize(stream, words);
stream.Close();
What amendments do I need to do??? Please provide me with a working code example/sample.
Thanks.

BinaryFormatter is a type-based serialized; it deeply embeds type metadata into the output. I would wager that you have copied the class definition between the projects - that isn't enough for BinaryFormatter, since that is no longer the same Type (types are bound to their assembly).
In your scenario, it sounds to me that the correct thing to do here would be to use a contract-based serializer; for example:
xml (XmlSerializer, DataContractSerializer, etc)
json (JavascriptSerializer, JSON.net, etc)
binary (protobuf-net, etc)
There would would entirely in your scenario, and would also have much better version tolerance (BinaryFormatter is very brittle with versions)
You mention "XMLs aren't safe here, as I don't want the users to know the contents of the main database file." - in that case protobuf-net has the "advantage" of being not human-readable, but note: none of these, nor BinaryFormatter are encrypted; if I wanted to, I could obtain the contents if I really, really wanted to. If you need strong security, use proper encryption. In which case your code becomes (in addition to maybe a few marker attributes):
using(var stream = new FileStream("BinarySerialization.bin", FileMode.Create,
FileAccess.Write, FileShare.None))
{
Serializer.Serialize(stream, words);
}
Edit to show (per comments) how to serialize Dictionary<string, List<Word>> where Word is a class with 2 string members (most of the code here is just plumbing to show a complete example):
using System;
using System.Collections.Generic;
using System.IO;
using ProtoBuf;
[ProtoContract]
public class Word {
[ProtoMember(1)]
public string Foo { get; set; }
[ProtoMember(2)]
public string Bar { get; set; }
}
static class Program {
public static void Main() {
var data = new Dictionary<string, List<Word>>{
{"abc", new List<Word> {
new Word { Foo = "def", Bar = "ghi"},
new Word { Foo = "jkl", Bar = "mno"}
}},
{"pqr", new List<Word> {
new Word {Foo = "stu", Bar = "vwx"}
}}
};
using(var file = File.Create("my.bin")) {
Serializer.Serialize(file, data);
}
Dictionary<string, List<Word>> clone;
using(var file = File.OpenRead("my.bin")) {
clone = Serializer.Deserialize<
Dictionary<string, List<Word>>>(file);
}
foreach(var pair in clone) {
Console.WriteLine(pair.Key);
foreach(var word in pair.Value){
Console.WriteLine("\t{0} | {1}", word.Foo, word.Bar);
}
}
}
}

I would put all my models that have to be serialized into a seperate assembly.
Then you reference this assembly everywhere you need to deserialize the Models.
If not you need some kind of generator, that recreates the models based on some schema (same as WCF does with its utilities) or use plain formats like XML to persist your data.

The serialization mechanism in .NET creates a helper dll out of the type to serialize and deserialize your data in runtime. First it will spit out a code file that gets compiled and then the helper dll is loaded to do the serialization and deserialization.
If for some reason something happens while the helper .dll gets created - let's say a compilation error- then the runtime will not found this dll.
If the dll name in your case is some random character then I would say you are facing the problem described above. You can troubleshoot this by switching on an undocument switch. See the following article:
HOW TO: Debug into a .NET XmlSerializer Generated Assembly

How can I access an existing byte[] in C#?

I have a small issue accessing a byte[]:
I have a binary object (byte[] saved to mssql db) which I get from the db and I want to read it. Whenever I access it, for its length or for its Read() method, I get a Cannot access a closed Stream exception.
What's the best way to treat binaries if they have to be updated in the code and than saved again to the db?
Thanks.
Edit - code
In this application we convert a test object to a generic data object we've created to simplify, so this is the data object:
public class DataObject
{
public Stream Content { get; set; }
public Descriptor Descriptor { get; set; }
}
The descriptor contains metadata only (currently only name and description strings) and is not relevant, I think.
The test is more complicated, I'll start by adding the mapping into data object. The serializer mentioned is NetDataContractSerializer.
public DataObject Map(Test test)
{
using(var stream = new MemoryStream())
{
Serialize(test, stream);
return new DataObject { Content = stream, Descriptor = test.Descriptor };
}
}
private void Serialize(Test test, MemoryStream stream)
{
serializer.WriteObject(stream, test);
stream.Flush();
stream.Position = 0;
}
and vice versa:
public Test Build(DataObject data)
{
using (var stream = data.Content)
{
var test = Deserialize(stream);
test.Descriptor = data.Descriptor;
return test ;
}
}
private Test Deserialize(Stream stream)
{
return serializer.ReadObject(stream) as IPythonTest;
}
Edit II - trying to change the test's content:
This is my first attempt handling streams, I'm not sure I'm doing it right, so I'll explain first what I want to do: The information in data field should be saved into the test's data object.
private static void UpdateTestObject(DataObject data, Test test)
{
var testData = new byte[data.Content.Length];
data.Content.Read(testData, 0, (int) data.Content.Length);
test.TestObject = testData;
}
The exception is thrown in UpdateTestObject when accessing data.Content. I get it after creating some test, mapping it, and trying to save it.

data.Content.Read(testData, 0, (int) data.Content.Length);
Here we go. The data object has a Content stream that it closed.
Result: Error.
Reasno? TOTALLY (!) unrelated to your question. Basically find out why / what is the problem there in your data handling.
Could be a design fubar in which the stream is not available after a certain point and youru sage of the object is past this point.

So the problem is caused by the Map() method - as far as I could understand, since it used:
using (var stream = new MemoryStream())
{ ... }
The stream was disposed of at the end of the block. Changing it to declaring a MemoryStream and then using it afterwards worked.
Thanks to everyone who gave it a thought (not mentioning reading all this code)! :)

TextWriter not writing all files

I've got written a service that has a separate thread running that reads roughly 400 records from a database and serializes them into xml files. It runs fine, there are no errors and it reports all files have been exported correctly, yet only a handful of xml files appear afterwards, and its always a different number each time. I've checked to see if it's a certain record causing problems, but they all read out fine, and seem to write fin, but don't...
After playing around and putting a delay in of 250ms between each write they are all exported properly, so I assume it must have something to do with writing so many files in such a quick succession, but I have no idea why, I would have thought it would report some kind of error if they didn't write properly, yet there's nothing.
Here is the code for anyone who wants to try it:
static void Main(string[] args)
{
ExportTestData();
}
public static void ExportTestData()
{
List<TestObject> testObjs = GetData();
foreach (TestObject obj in testObjs)
{
ExportObj(obj);
//Thread.Sleep(10);
}
}
public static List<TestObject> GetData()
{
List<TestObject> result = new List<TestObject>();
for (int i = 0; i < 500; i++)
{
result.Add(new TestObject()
{
Date = DateTime.Now.AddDays(-1),
AnotherDate = DateTime.Now.AddDays(-2),
AnotherAnotherDate = DateTime.Now,
DoubleOne = 1.0,
DoubleTwo = 2.0,
DoubleThree = 3.0,
Number = 345,
SomeCode = "blah",
SomeId = "wobble wobble"
});
}
return result;
}
public static void ExportObj(TestObject obj)
{
try
{
string path = Path.Combine(#"C:\temp\exports", String.Format("{0}-{1}{2}", DateTime.Now.ToString("yyyyMMdd"), String.Format("{0:HHmmssfff}", DateTime.Now), ".xml"));
SerializeTo(obj, path);
}
catch (Exception ex)
{
}
}
public static bool SerializeTo<T>(T obj, string path)
{
XmlSerializer xs = new XmlSerializer(obj.GetType());
using (TextWriter writer = new StreamWriter(path, false))
{
xs.Serialize(writer, obj);
}
return true;
}
Try commenting\uncommenting the Thread.Sleep(10) to see the problem
Does anybody have any idea why it does this? And can suggest how I can avoid this problem?
Thanks
EDIT: Solved. The time based filename wasn't unique enough and was overwriting previously written files. Should've spotted it earlier, thanks for your help

Perhaps try putting the writer in a using block for immediate disposal? Something like
XmlSerializer xs = new XmlSerializer(obj.GetType());
using(TextWriter writer = new StreamWriter(path, false))
{
xs.Serialize(writer, obj);
}

Ok I've found the problem, I was using a time based filename that I thought would be unique enough for each file, turns out in a loop that tight they're coming out with the same filenames and are over-writing each other.
If I change it to use actually unique filenames it works! Thanks for your help

Dispose the writer
public static bool SerializeTo<T>(T obj, string path)
{
XmlSerializer xs = new XmlSerializer(obj.GetType());
using(TextWriter writer = new StreamWriter(path, false)) {
xs.Serialize(writer, obj);
writer.Close();
}
return true;
}

If you're not getting any exceptions, then the using statements proposed by other answers won't help - although you should change to use them anyway. At that point, you don't need the close call any more:
XmlSerializer xs = new XmlSerializer(obj.GetType());
using(TextWriter writer = new StreamWriter(path, false))
{
xs.Serialize(writer, obj);
}
I don't think the problem lies in this code, however. I suspect it's something like the "capturing a loop variable in a lambda expression" problem which crops up so often. If you can come up with a short but complete program which demonstrates the problem, it will be a lot easier to diagnose.
I suggest you create a simple console application which tries to create (say) 5000 files serializing some simple object. See if you can get that to fail in the same way.

Multi-threading may cause that problem. The 250ms delay is an evidence of that.
Do you have multiple threads doing that?

How to save a human readable file

Currently i have an application that reads and writes several properties from one or two basic classes to a .txt file using the Binary Serializer.
I've opened up the .txt file in NotePad and as it's formatted for the application it's not very readable to the human eye, not for me anyway =D
I've heard of using XML but pretty much most of my searches seem to overcomplicate things.
The kind of data im trying to save is simply a collection of "Person.cs" classes,nothing more than a name and address, all private strings but with properties and marked as Serializable.
What would be the best way to actually save my data in a way that can be easily read by a person? It would also make it easier to make small changes to the application's data directly in the file instead of having to load it, change it and save it.
Edit:
I have added the current way i am saving and loading my data, my _userCollection is as it suggests and the nUser/nMember are an integer.
#region I/O Operations
public bool SaveData()
{
try
{
//Open the stream using the Data.txt file
using (Stream stream = File.Open("Data.txt", FileMode.Create))
{
//Create a new formatter
BinaryFormatter bin = new BinaryFormatter();
//Copy data in collection to the file specified earlier
bin.Serialize(stream, _userCollection);
bin.Serialize(stream, nMember);
bin.Serialize(stream, nUser);
//Close stream to release any resources used
stream.Close();
}
return true;
}
catch (IOException ex)
{
throw new ArgumentException(ex.ToString());
}
}
public bool LoadData()
{
//Check if file exsists, otherwise skip
if (File.Exists("Data.txt"))
{
try
{
using (Stream stream = File.Open("Data.txt", FileMode.Open))
{
BinaryFormatter bin = new BinaryFormatter();
//Copy data back into collection fields
_userCollection = (List<User>)bin.Deserialize(stream);
nMember = (int)bin.Deserialize(stream);
nUser = (int)bin.Deserialize(stream);
stream.Close();
//Sort data to ensure it is ordered correctly after being loaded
_userCollection.Sort();
return true;
}
}
catch (IOException ex)
{
throw new ArgumentException(ex.ToString());
}
}
else
{
//Console.WriteLine present for testing purposes
Console.WriteLine("\nLoad failed, Data.txt not found");
return false;
}
}

Replace your BinaryFormatter with XMLSerializer and run the same exact code.
The only change you need to make is the BinaryFormatter takes an empty constructor, while for the XMLSerializer you need to declare the type in the constructor:
XmlSerializer serializer = new XmlSerializer(typeof(Person));

Using XmlSerializer is not really complicated. Have a look at this MSDN page for an example: http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx

You could implement your own PersonsWriter, that takes a StreamWriter as constructor argument and has a Write method that takes an IList<Person> as input to parse out a nice text representation.
For example:
public class PersonsWriter : IDisposable
{
private StreamWriter _wr;
public PersonsWriter(IList<Person> persons, StreamWriter writer)
{
this._wr = writer;
}
public void Write(IList<Persons> people) {
foreach(Person dude in people)
{
_wr.Write(#"{0} {1}\n{2}\n{3} {4}\n\n",
dude.FirstName,
dude.LastName,
dude.StreetAddress,
dude.ZipCode,
dude.City);
}
}
public void Dispose()
{
_wr.Flush();
_wr.Dispose();
}
}

YAML is another option for human readable markup that is also easy to parse. there are libraries available for c# as well as almost all other popular languages. Here's a sample of what yaml looks like:
invoice: 34843
date : 2001-01-23
bill-to: &id001
given : Chris
family : Dumars
address:
lines: |
458 Walkman Dr.
Suite #292
city : Royal Oak
state : MI
postal : 48046

Frankly, as a human, I don't find XML to be all that readable. In fact, it's not really designed to be read by humans.
If you want a human readable format, then you have to build it.
Say, you have a Person class that has a First Name, a last Name and a SSN as properties. Create your file, and have it write out 3 lines, with a description of the field in the first fifty (random number from my head) and then with character 51 have the value start being written.
This will produce a file that looks like:
First Name-------Stephen
Last Name -------Wrighton
SSN -------------XXX-XX-XXXX
Then, reading it back in, your program would know where the data begins on each line, and what each line is for (the program would know that Line 3 is the SSN value).
But remember, to truly gain human readability, you sacrifice data portability.

Try the DataContractSerializer
It serializes objects to XML and is very easy to use

Write a CSV reader writer if you want a good compromise between human and machine readable in a Windows environment
Loads into Excel too.
There's a discussion about it here:
http://knab.ws/blog/index.php?/archives/3-CSV-file-parser-and-writer-in-C-Part-1.html
EDIT
That is a C# article... it just confusingly has "C" in the URL.

I really think you should go with XML (look into DataContractSerializer). Its not that complicated. You could probably even just replace BinarySerializer with XMLSerializer and go.
If you still don't want to do that, though, you can write a delimited text file. Then you'll have to write your own reader method (although, it could almost just use the split method).
//Inside the Person class:
public override string ToString()
{
List<String> propValues = new List<String>();
// Get the type.
Type t = this.GetType();
// Cycle through the properties.
foreach (PropertyInfo p in t.GetProperties())
{
propValues.add("{0}:={1}", p.Name, p.GetValue(o, null));
}
return String.Join(",". propValues.ToArray())
}
using (System.IO.TextWriter tw = new System.IO.StreamWriter("output.txt"))
{
tw.WriteLine(person.ToString());
}