We Keep Coding

C# How to compile a serialized .NET object to a .NET Assembly

I have a binary file, with serialized .NET object (stream) in it. I need to compile it back to a .NET Assembly (Maybe using CodeDomProvider Class or anything else).
Any pointer will be highly appreciated.
Thanks in Advance.

There is no guarantee that it is possible to deserialize a BinaryFormatter serialized object (BinaryFormatter is the .NET-included binary serializer... and it is considered to be quite "evil") to the source code that generated it. Simple example:
[Serializable]
public class MyClass
{
public DateTime Foo { get; private set; }
public MyClass()
{
Foo = DateTime.Now;
}
}
There is no way in C# to write a MyClass object with a specific Foo value unless you are using reflection. You can't write:
var bar = new MyClass { Foo = new DateTime(2018, 1, 1 }
because there is no setter for Foo.
Specific cases (where there a no private fields and if there are setters they are all setters that only set the value of the backing field without doing extra elaboration) can be converted to C# assignments.
What it is possible to do (but in general it is a bad idea with BinaryFormatter, because it doesn't handle very well changes to the underlying types) is include the binary file as an embedded resource (see this) and then read the file:
var assembly = Assembly.GetExecutingAssembly();
var resourceName = "ConsoleApp2.Folder1.File1.bin";
MyClass mc;
using (Stream stream = assembly.GetManifestResourceStream(resourceName))
{
var bf = new BinaryFormatter();
mc = (MyClass)bf.Deserialize(stream);
}
Note that this is a very very bad idea, because if anything changes in the underlying types (even some private fields), everything will break badly.

I've done it using a tool called ClrGuard. https://github.com/endgameinc/ClrGuard.
It will capture the .NET assembly as it tries to execute and dump it in disk. Then we can load with ilspy or any other .NET de-compiler.
ClrGuard will hook into all .NET processes on the system. From there,
it performs an in-line hook of the native LoadImage() function. This
is what Assembly.Load() calls under the CLR hood.
Reference : https://www.endgame.com/blog/technical-blog/hunting-memory-net-attacks

Are Hashtables Serializable?

I see a pervasive belief (2009 article) throughout the internet that the Hashtable class is not serializable; however, I cannot find any modern documentation that supports this notion.
The belief stems from another ill-documented belief that the IDictionary interface prevents serialization; however, I cannot find anything in MSDN that supports this claim, today.
Further, Hashtable implements ISerializable and contains extension methods that accept serialization information.
So, what's the deal? Is Hashtable serializable? Where is the documentation that supports this notion surrounding IDictionary?
Further Clarification (please read):
The statement that IDictionary is not serializable is supported by plenty of documentation; however, this focuses on the use of XML-based serialization interactions with a class. ISerializable as mentioned both in the comments, below, and through MSDN indicates that a class is serializable. It also means the class the must be responsible for its own serialization.
I think this negates the statement that a Hashtable is not serializable. That is perhaps the genesis of my question.

The pervasive belief is so pervasive because it's true:
var t = new Hashtable();
t.Add("Hi!", "I'm here");
t.Add("Hm", "Yup");
var serializer = new XmlSerializer(typeof(Hashtable));
using (var sw = new StringWriter())
{
serializer.Serialize(sw, t);
Console.WriteLine(sw.ToString());
}
throws
NotSupportedException: The type System.Collections.Hashtable is not supported because it implements IDictionary.
That doesn't mean that it's literally impossible to serialize a hash table. Of course I can just iterate over all the keys and values, write them to a string and then reconstruct the hashtable from that. It's just that I can't use the serialization infrastructure fully.
What's the reasoning here? It's actually quite simple - XmlSerializer is designed to produce good XML, in the spirit of the interchange format XML was designed to be. And XML doesn't have any kind of dictionary or "key-value" mechanism that would fit. So to support hashtable serialization, they'd have to make their own "sub-format" with its own rules. And back when .NET was being designed, this was a huge no-no - XML was an interchange format. Any extension (hah) to the format means you're no longer compatible, no matter how good of an idea you have.
Of course, nowadays, everyone and their grandmother are producing XML data that isn't used for interchange purposes. And it's not entirely a bad thing (after all, .NET config files are also a XML format). But it's also kind of wrong.
In contrast, take something like BinaryFormatter. That's a class where the .NET team designed the whole format, and isn't limited by a standard. And lo and behold - BinaryFormatter can serialize and deserialize a Hashtable just fine.
So the slightly more correct belief would be "Hashtable cannot be serialized to valid standard XML. The XmlSerializer class in particular will throw an error when you attempt to serialize a Hashtable."

Does Hashtable implement ISerializable? Absolutely:
public class Hashtable : IDictionary, ICollection, IEnumerable, ISerializable, IDeserializationCallback, ICloneable
Can we serializae a Hashtable to XML? Let's try it:
var hash = new System.Collections.Hashtable();
hash[7] = "7";
hash[8] = "8";
var serializer = new System.Xml.Serialization.XmlSerializer(typeof(System.Collections.Hashtable));
TextWriter writer = new System.IO.StreamWriter(#"C:\SomeFile.xml");
serializer.Serialize(writer, hash);
Result... Error as you expected
An exception of type 'System.NotSupportedException' occurred in System.Xml.dll but was not handled in user code
Additional information: The type System.Collections.Hashtable is not supported because it implements IDictionary.
So, it would appear that indeed, it's still the case in .Net 4.5+
But lets try one more time with a binary serialization...
var hash = new System.Collections.Hashtable();
hash[7] = "7";
hash[8] = "8";
var formatter = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
Stream stream = new FileStream(#"C:\SomeFolder\SomeFile.bin", FileMode.Create, FileAccess.Write, FileShare.None);
formatter.Serialize(stream, hash);
stream.Close();
Result... No Errors thrown... So the issue appears to be related to IDictionary and XmlSerialization, but not all Serialization
If you really need to do this to XML, ManoDestra had a nice link https://blogs.msdn.microsoft.com/adam/2010/09/10/how-to-serialize-a-dictionary-or-hashtable-in-c/
Also, interestingly, XML Serialization mentions that you can't serialize unsigned longs or collections there-of.
Microsoft XML Serialization (MSDN)

Microsoft seems to say that it is certainly possible to do this.
https://msdn.microsoft.com/en-us/library/b85344hz(v=vs.110).aspx
using System;
using System.IO;
using System.Collections;
using System.Runtime.Serialization.Formatters.Binary;
using System.Runtime.Serialization;
public class App
{
[STAThread]
static void Main()
{
Serialize();
Deserialize();
}
static void Serialize()
{
// Create a hashtable of values that will eventually be serialized.
Hashtable addresses = new Hashtable();
addresses.Add("Jeff", "123 Main Street, Redmond, WA 98052");
addresses.Add("Fred", "987 Pine Road, Phila., PA 19116");
addresses.Add("Mary", "PO Box 112233, Palo Alto, CA 94301");
// To serialize the hashtable and its key/value pairs,
// you must first open a stream for writing.
// In this case, use a file stream.
FileStream fs = new FileStream("DataFile.dat", FileMode.Create);
// Construct a BinaryFormatter and use it to serialize the data to the stream.
BinaryFormatter formatter = new BinaryFormatter();
try
{
formatter.Serialize(fs, addresses);
}
catch (SerializationException e)
{
Console.WriteLine("Failed to serialize. Reason: " + e.Message);
throw;
}
finally
{
fs.Close();
}
}
static void Deserialize()
{
// Declare the hashtable reference.
Hashtable addresses = null;
// Open the file containing the data that you want to deserialize.
FileStream fs = new FileStream("DataFile.dat", FileMode.Open);
try
{
BinaryFormatter formatter = new BinaryFormatter();
// Deserialize the hashtable from the file and
// assign the reference to the local variable.
addresses = (Hashtable) formatter.Deserialize(fs);
}
catch (SerializationException e)
{
Console.WriteLine("Failed to deserialize. Reason: " + e.Message);
throw;
}
finally
{
fs.Close();
}
// To prove that the table deserialized correctly,
// display the key/value pairs.
foreach (DictionaryEntry de in addresses)
{
Console.WriteLine("{0} lives at {1}.", de.Key, de.Value);
}
}
}

Serializing and deserializing a List<List<object>> with BinaryFormatter

Let's say I have
List<object> mainList = new List<object>();
And it contains
List<string> stringList = new List<string();
List<CustomClass> custList = new List<CustomClass>();
mainList.Add(stringList);
mainList.Add(custList);
To serialize
Stream stream;
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(stream, mainList);
To deserialize
Stream stream = (Stream)o;
BinaryFormatter formatter = new BinaryFormatter();
List<object> retrievedList = (List<object>)formatter.Deserialize(stream);
At this point, I receive an error that the stream read (deserialization) reached the end of the stream without retrieving a value.
Do I need to specify something besides...
[Serializable]
public class CustomClass { .... }
in the custom class to make this work? Can I not deserialize a List> that contains different type of object every time?
I tried
IList list = (IList)Activator.CreateInstance(typeof(custClassList[0]))
and tried to send and receive this, but got the same issue.
I can however serialize and deserialize a specified type or List, but I really need it to be dynamic.

Basically, BinaryFormatter is a joke. It works in some cases, but will fail in almost identical scenarios for unknown reasons.
The best and superior alternative to BinaryFormatter is the third-party library protobuf-net (https://github.com/mgravell/protobuf-net), developed by Marc Gravel.
This beauty solved all the problems I was having in one pass. It's much easier to set up and reacts more perfectly to complex, custom classes.
I should also mention that it's faster, in the terms of de/serialization.

In order to fix the issue that causes the error "stream read (deserialization) reached the end of the stream ", the stream position needs to reset to 0 as follows...
stream.Position = 0;
Do I need to specify something besides...
[Serializable] public class CustomClass { .... }
Nope...That should be good for what you are doing.
in the custom class to make this work? Can I not deserialize a List>
that contains different type of object every time?
You should be able to serialize any object.

How to save a list of classes in combination with other variables in c# wpf

Hi I just started working with C# WPF and I read about Serialization to store or load your data. My Question is how can I store a class that contains a list of another class and some additional parameters?
My first class (MeterValues) contains a number of parameters (type,speed,etc..)
public class MeterValues{}
I now made a second class to store a list containing multiple instances of the first class type. (So if I have 3 different meters, this list size = 3)
public class MeterValuesList : IList<MeterValues>{}
Now I wish to add an additional parameter to the second class, something independent of the first class so it should only be saved once. (not for every instance of class1)
To make my problem clear, I could add the extra parameter to the first class, but then If I have 100 different meters, the parameter is stored 100 times, and I only need to store it once.
Any idea on how to do this?
PS: If you need any additional information please just ask, I'm very eager to learn and to assist you in helping me solve this problem. Thanks in advance.
UPDATE:
I'm able to save the class MeterValuesList to a .xml file but only the List gets stored in the file, the extra parameter does not show up (It is in the class right before I write it to the file, checked it with debugger but does not show up in the file)
MeterValuesList meterValuesList = DataContext as MeterValuesList;
meterValuesList.CommSettings = "Com5:19200,8,n,1";
FileStream stream = null;
try
{
stream = new FileStream(filename, FileMode.Create, FileAccess.Write);
XmlSerializer serializer = new XmlSerializer(typeof(MeterValuesList));
serializer.Serialize(stream, meterValuesList);
stream.Close();
}
This is the result after saving the class to an xml file. The extra parameter is missing.
<?xml version="1.0"?>
<ArrayOfMeterValues xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<MeterValues></MeterValues>
</ArrayOfMeterValues>

Unless you plan on overriding or expanding the functionality that an IList has in place, there is no reason to inherit from it in your MeterValuesList class. This feels like a case of using a "has a" relationship instead of an "is a" relationship.
Try this instead:
public class MeterValuesGroup
{
List<MeterValues> MeterList { get; set; }
int ExtraParameter { get; set; }
// whatever additional parameters you need here.
}
If you do need to inherit from IList or IEnumerable, you can do something similar. However, in order to serialize this class correctly, you'll have to implement IXmlSerializable in MeterValues and MeterValuesList.
(Here is an excellent example of how this will look: Proper way to implement IXmlSerializable?)
public class MeterValuesList : IList<MeterValues>, IXmlSerializable
{
MeterValues[] _MeterList { get; set; }
string CommSettings = "Com5:19200,8,n,1";
public void WriteXml(System.Xml.XmlWriter writer)
{
writer.WriteAttributeString("CommSettings ", CommSettings );
foreach (var mv in _MeterList)
{
// kind of a bad example, but hopefully you get the idea
if (mv== null)
return;
writer.WriteStartElement("MeterValues");
mv.WriteXml(writer);
writer.WriteEndElement();
}
}

You can add this property in the second class MeterValueList and serilize this class and when you deserialize it , that additional property will be assigned.
MeterValueList m = new MeterValueList();
m.AdditionalParameter = 100;
MemoryStream memorystream = new MemoryStream();
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(memorystream, m);
byte[] yourBytesToDb = memorystream.ToArray();
//here you write yourBytesToDb to database
//----------read from database---------------------
//here you read from database binary data into yourBytesFromDb
MemoryStream memorystreamd = new MemoryStream(yourBytesFromDb);
BinaryFormatter bfd = new BinaryFormatter();
MeterValueList md = bfd.Deserialize(memorystreamd) as MeterValueList ;
var i = md.AdditinalParameter; // must print 100

How to save a human readable file

Currently i have an application that reads and writes several properties from one or two basic classes to a .txt file using the Binary Serializer.
I've opened up the .txt file in NotePad and as it's formatted for the application it's not very readable to the human eye, not for me anyway =D
I've heard of using XML but pretty much most of my searches seem to overcomplicate things.
The kind of data im trying to save is simply a collection of "Person.cs" classes,nothing more than a name and address, all private strings but with properties and marked as Serializable.
What would be the best way to actually save my data in a way that can be easily read by a person? It would also make it easier to make small changes to the application's data directly in the file instead of having to load it, change it and save it.
Edit:
I have added the current way i am saving and loading my data, my _userCollection is as it suggests and the nUser/nMember are an integer.
#region I/O Operations
public bool SaveData()
{
try
{
//Open the stream using the Data.txt file
using (Stream stream = File.Open("Data.txt", FileMode.Create))
{
//Create a new formatter
BinaryFormatter bin = new BinaryFormatter();
//Copy data in collection to the file specified earlier
bin.Serialize(stream, _userCollection);
bin.Serialize(stream, nMember);
bin.Serialize(stream, nUser);
//Close stream to release any resources used
stream.Close();
}
return true;
}
catch (IOException ex)
{
throw new ArgumentException(ex.ToString());
}
}
public bool LoadData()
{
//Check if file exsists, otherwise skip
if (File.Exists("Data.txt"))
{
try
{
using (Stream stream = File.Open("Data.txt", FileMode.Open))
{
BinaryFormatter bin = new BinaryFormatter();
//Copy data back into collection fields
_userCollection = (List<User>)bin.Deserialize(stream);
nMember = (int)bin.Deserialize(stream);
nUser = (int)bin.Deserialize(stream);
stream.Close();
//Sort data to ensure it is ordered correctly after being loaded
_userCollection.Sort();
return true;
}
}
catch (IOException ex)
{
throw new ArgumentException(ex.ToString());
}
}
else
{
//Console.WriteLine present for testing purposes
Console.WriteLine("\nLoad failed, Data.txt not found");
return false;
}
}

Replace your BinaryFormatter with XMLSerializer and run the same exact code.
The only change you need to make is the BinaryFormatter takes an empty constructor, while for the XMLSerializer you need to declare the type in the constructor:
XmlSerializer serializer = new XmlSerializer(typeof(Person));

Using XmlSerializer is not really complicated. Have a look at this MSDN page for an example: http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx

You could implement your own PersonsWriter, that takes a StreamWriter as constructor argument and has a Write method that takes an IList<Person> as input to parse out a nice text representation.
For example:
public class PersonsWriter : IDisposable
{
private StreamWriter _wr;
public PersonsWriter(IList<Person> persons, StreamWriter writer)
{
this._wr = writer;
}
public void Write(IList<Persons> people) {
foreach(Person dude in people)
{
_wr.Write(#"{0} {1}\n{2}\n{3} {4}\n\n",
dude.FirstName,
dude.LastName,
dude.StreetAddress,
dude.ZipCode,
dude.City);
}
}
public void Dispose()
{
_wr.Flush();
_wr.Dispose();
}
}

YAML is another option for human readable markup that is also easy to parse. there are libraries available for c# as well as almost all other popular languages. Here's a sample of what yaml looks like:
invoice: 34843
date : 2001-01-23
bill-to: &id001
given : Chris
family : Dumars
address:
lines: |
458 Walkman Dr.
Suite #292
city : Royal Oak
state : MI
postal : 48046

Frankly, as a human, I don't find XML to be all that readable. In fact, it's not really designed to be read by humans.
If you want a human readable format, then you have to build it.
Say, you have a Person class that has a First Name, a last Name and a SSN as properties. Create your file, and have it write out 3 lines, with a description of the field in the first fifty (random number from my head) and then with character 51 have the value start being written.
This will produce a file that looks like:
First Name-------Stephen
Last Name -------Wrighton
SSN -------------XXX-XX-XXXX
Then, reading it back in, your program would know where the data begins on each line, and what each line is for (the program would know that Line 3 is the SSN value).
But remember, to truly gain human readability, you sacrifice data portability.

Try the DataContractSerializer
It serializes objects to XML and is very easy to use

Write a CSV reader writer if you want a good compromise between human and machine readable in a Windows environment
Loads into Excel too.
There's a discussion about it here:
http://knab.ws/blog/index.php?/archives/3-CSV-file-parser-and-writer-in-C-Part-1.html
EDIT
That is a C# article... it just confusingly has "C" in the URL.

I really think you should go with XML (look into DataContractSerializer). Its not that complicated. You could probably even just replace BinarySerializer with XMLSerializer and go.
If you still don't want to do that, though, you can write a delimited text file. Then you'll have to write your own reader method (although, it could almost just use the split method).
//Inside the Person class:
public override string ToString()
{
List<String> propValues = new List<String>();
// Get the type.
Type t = this.GetType();
// Cycle through the properties.
foreach (PropertyInfo p in t.GetProperties())
{
propValues.add("{0}:={1}", p.Name, p.GetValue(o, null));
}
return String.Join(",". propValues.ToArray())
}
using (System.IO.TextWriter tw = new System.IO.StreamWriter("output.txt"))
{
tw.WriteLine(person.ToString());
}