I have a binary file, with serialized .NET object (stream) in it. I need to compile it back to a .NET Assembly (Maybe using CodeDomProvider Class or anything else).
Any pointer will be highly appreciated.
Thanks in Advance.
There is no guarantee that it is possible to deserialize a BinaryFormatter serialized object (BinaryFormatter is the .NET-included binary serializer... and it is considered to be quite "evil") to the source code that generated it. Simple example:
[Serializable]
public class MyClass
{
public DateTime Foo { get; private set; }
public MyClass()
{
Foo = DateTime.Now;
}
}
There is no way in C# to write a MyClass object with a specific Foo value unless you are using reflection. You can't write:
var bar = new MyClass { Foo = new DateTime(2018, 1, 1 }
because there is no setter for Foo.
Specific cases (where there a no private fields and if there are setters they are all setters that only set the value of the backing field without doing extra elaboration) can be converted to C# assignments.
What it is possible to do (but in general it is a bad idea with BinaryFormatter, because it doesn't handle very well changes to the underlying types) is include the binary file as an embedded resource (see this) and then read the file:
var assembly = Assembly.GetExecutingAssembly();
var resourceName = "ConsoleApp2.Folder1.File1.bin";
MyClass mc;
using (Stream stream = assembly.GetManifestResourceStream(resourceName))
{
var bf = new BinaryFormatter();
mc = (MyClass)bf.Deserialize(stream);
}
Note that this is a very very bad idea, because if anything changes in the underlying types (even some private fields), everything will break badly.
I've done it using a tool called ClrGuard. https://github.com/endgameinc/ClrGuard.
It will capture the .NET assembly as it tries to execute and dump it in disk. Then we can load with ilspy or any other .NET de-compiler.
ClrGuard will hook into all .NET processes on the system. From there,
it performs an in-line hook of the native LoadImage() function. This
is what Assembly.Load() calls under the CLR hood.
Reference : https://www.endgame.com/blog/technical-blog/hunting-memory-net-attacks
Related
I'm working with some highly custom objects from a nuget package that I need to store in the local SQLite db using Xamarin. I tried using this code
public static byte[] Serialize(ComplexType c)
{
byte[] arrayData;
using(MemoryStream stream = new MemoryStream())
{
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize( stream, c );
arrayData = stream.ToArray();
stream.Close();
}
return arrayData;
}
public static ComplexType Deserialize(byte[] arrayData)
{
ComplexType c;
using(MemoryStream stream = new MemoryStream())
{
stream.Write( arrayData, 0, arrayData.Length );
stream.Seek( 0, SeekOrigin.Begin );
BinaryFormatter formatter = new BinaryFormatter();
c = (ComplexType) formatter.Deserialize( stream );
}
return c;
}
to serialize and store the object then retrieve it and deserialize it. It seems like it worked fine to do the serialization but now it turns out that the object is not marked serializable. I can't add the SerializableAttribute property to it because it's from a nuget. I tried editing the object definition in the nuget but of course it's locked.
Is there any easy way to pull off serializing an object not marked serializable or perhaps some other simple method of storing a non-serializable custom object in a SQLite database? Or should I just byte the bullet and write really complicated setters and getters to store all of the custom object's individual custom properties broken down into their individual lists of standard properties and basically deconstruct and reconstruct the object every time it goes to and from the database?
I'm seeing mixed reports on ISerializable and derived classes. Will implementing ISerializable on a derived class from the custom object allow the derived class to be serialized? What about IXMLSerializable? Will either of these work? I just want to get some clarification if possible before I go way down the wrong rabbit hole.
I'd like to ask something about generics.
I am trying to keep the code simple, and thus I will be making a single class to handle load/save for a game's savegame files. As each portion of the game has different requirements I'd like to keep this as easily accessible as possible:
public void Load<T>(string path, out T obj)
{
BinaryFormatter bf = new BinaryFormatter();
using (FileStream file = File.Open(Application.persistentDataPath + path, FileMode.Open))
{
obj = (T)bf.Deserialize(file);
}
}
Now I can call this with a simple
TurnData x; s.Load("test.txt", out x);
The alternative would be to make the Load function return the object and then convert it to a TurnData type.
TurnData x = (TurnData)s.Load("test.txt");
I do not know much about C#. I assume that the code inside using(...) { ... } does not get executed if there is an error opening the file for example? If someone can confirm this that would be nice. The example code I have seen did not have any error handling, which seemed weird to me, so I added using?
So in this secondary version where the function returns the object instead of using an out parameter would need more complicated code for error checking and possible return null? It doesn't seem great.
So the real question is ... can I use the next version I have here or are there concerns that I should have due to the use of generics?
There is no generic code bloat for reference types - code is reused. With value types, though, CLR will generate a separate method for each type. See
.NET Generics and Code Bloat.
The using statement has nothing to do with error handling. Using File.Open method you can expect to get the exceptions you will find here. You could avoid the abruptly stop of your program from any such an exception by wrapping your using statement in a try/cath construct like below:
public T Load<T>(string path)
{
T obj = default(T);
var bf = new BinaryFormatter();
try
{
using (var file = File.Open(Application.persistentDataPath + path, FileMode.Open))
{
obj = (T)bf.Deserialize(file);
}
}
catch(Exception exception)
{
// Log the exception
}
return obj;
}
Essentially you attempt to Open the file specified in the path. If that fails you just log the failure and your return null from the function.
Regarding the using statement, it provides
a convenient syntax that ensures the correct use of IDisposable
objects.
as you can read more thoroughly here
As a side note regarding the signature of your method I would make a few comments. Consider the following method body and spot the differences with that we have above.
public T Load<T>(string path, IFormatter formatter)
{
if(path ==null) throw new ArgumentNullException(nameof(path));
if(formatter == null) throw new ArgumentNullException(nameof(formatter));
T obj = default(T);
try
{
using (var file = File.Open(path, FileMode.Open))
{
obj = (T)formatter.Deserialize(file);
}
}
catch(Exception exception)
{
// Log the exception
}
return obj;
}
and
var path = Path.Combine(Application.persistentDataPath, "test.txt");
var binaryFormatter = new BinaryFormatter();
var x = s.Load(path, binaryFormatter);
Making the above changes you make your method more easily to be tested through a unit test and more reliable since you make some precondition checking before the meat and potatoes of your method. What would had happened if you had passed a null path? What would had happened if you had passed a null formatter ? etc...
Let's say I have
List<object> mainList = new List<object>();
And it contains
List<string> stringList = new List<string();
List<CustomClass> custList = new List<CustomClass>();
mainList.Add(stringList);
mainList.Add(custList);
To serialize
Stream stream;
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(stream, mainList);
To deserialize
Stream stream = (Stream)o;
BinaryFormatter formatter = new BinaryFormatter();
List<object> retrievedList = (List<object>)formatter.Deserialize(stream);
At this point, I receive an error that the stream read (deserialization) reached the end of the stream without retrieving a value.
Do I need to specify something besides...
[Serializable]
public class CustomClass { .... }
in the custom class to make this work? Can I not deserialize a List> that contains different type of object every time?
I tried
IList list = (IList)Activator.CreateInstance(typeof(custClassList[0]))
and tried to send and receive this, but got the same issue.
I can however serialize and deserialize a specified type or List, but I really need it to be dynamic.
Basically, BinaryFormatter is a joke. It works in some cases, but will fail in almost identical scenarios for unknown reasons.
The best and superior alternative to BinaryFormatter is the third-party library protobuf-net (https://github.com/mgravell/protobuf-net), developed by Marc Gravel.
This beauty solved all the problems I was having in one pass. It's much easier to set up and reacts more perfectly to complex, custom classes.
I should also mention that it's faster, in the terms of de/serialization.
In order to fix the issue that causes the error "stream read (deserialization) reached the end of the stream ", the stream position needs to reset to 0 as follows...
stream.Position = 0;
Do I need to specify something besides...
[Serializable] public class CustomClass { .... }
Nope...That should be good for what you are doing.
in the custom class to make this work? Can I not deserialize a List>
that contains different type of object every time?
You should be able to serialize any object.
I created a binary serialized file of my object, so that I could use it as a database file for many other projects. It works quite fine with the same project, but when I try to deserialize that file from other projects, it won't. The error that appears says, "xxxx.xxxx assembly not found". So, how should I serialize the object in order to make it assembly independent???
Here's my code :
// Binary formatter
IFormatter formatter = new BinaryFormatter();
Stream stream = new FileStream("BinarySerialization.bin",
FileMode.Create,
FileAccess.Write, FileShare.None);
formatter.Serialize(stream, words);
stream.Close();
What amendments do I need to do??? Please provide me with a working code example/sample.
Thanks.
BinaryFormatter is a type-based serialized; it deeply embeds type metadata into the output. I would wager that you have copied the class definition between the projects - that isn't enough for BinaryFormatter, since that is no longer the same Type (types are bound to their assembly).
In your scenario, it sounds to me that the correct thing to do here would be to use a contract-based serializer; for example:
xml (XmlSerializer, DataContractSerializer, etc)
json (JavascriptSerializer, JSON.net, etc)
binary (protobuf-net, etc)
There would would entirely in your scenario, and would also have much better version tolerance (BinaryFormatter is very brittle with versions)
You mention "XMLs aren't safe here, as I don't want the users to know the contents of the main database file." - in that case protobuf-net has the "advantage" of being not human-readable, but note: none of these, nor BinaryFormatter are encrypted; if I wanted to, I could obtain the contents if I really, really wanted to. If you need strong security, use proper encryption. In which case your code becomes (in addition to maybe a few marker attributes):
using(var stream = new FileStream("BinarySerialization.bin", FileMode.Create,
FileAccess.Write, FileShare.None))
{
Serializer.Serialize(stream, words);
}
Edit to show (per comments) how to serialize Dictionary<string, List<Word>> where Word is a class with 2 string members (most of the code here is just plumbing to show a complete example):
using System;
using System.Collections.Generic;
using System.IO;
using ProtoBuf;
[ProtoContract]
public class Word {
[ProtoMember(1)]
public string Foo { get; set; }
[ProtoMember(2)]
public string Bar { get; set; }
}
static class Program {
public static void Main() {
var data = new Dictionary<string, List<Word>>{
{"abc", new List<Word> {
new Word { Foo = "def", Bar = "ghi"},
new Word { Foo = "jkl", Bar = "mno"}
}},
{"pqr", new List<Word> {
new Word {Foo = "stu", Bar = "vwx"}
}}
};
using(var file = File.Create("my.bin")) {
Serializer.Serialize(file, data);
}
Dictionary<string, List<Word>> clone;
using(var file = File.OpenRead("my.bin")) {
clone = Serializer.Deserialize<
Dictionary<string, List<Word>>>(file);
}
foreach(var pair in clone) {
Console.WriteLine(pair.Key);
foreach(var word in pair.Value){
Console.WriteLine("\t{0} | {1}", word.Foo, word.Bar);
}
}
}
}
I would put all my models that have to be serialized into a seperate assembly.
Then you reference this assembly everywhere you need to deserialize the Models.
If not you need some kind of generator, that recreates the models based on some schema (same as WCF does with its utilities) or use plain formats like XML to persist your data.
The serialization mechanism in .NET creates a helper dll out of the type to serialize and deserialize your data in runtime. First it will spit out a code file that gets compiled and then the helper dll is loaded to do the serialization and deserialization.
If for some reason something happens while the helper .dll gets created - let's say a compilation error- then the runtime will not found this dll.
If the dll name in your case is some random character then I would say you are facing the problem described above. You can troubleshoot this by switching on an undocument switch. See the following article:
HOW TO: Debug into a .NET XmlSerializer Generated Assembly
Currently i have an application that reads and writes several properties from one or two basic classes to a .txt file using the Binary Serializer.
I've opened up the .txt file in NotePad and as it's formatted for the application it's not very readable to the human eye, not for me anyway =D
I've heard of using XML but pretty much most of my searches seem to overcomplicate things.
The kind of data im trying to save is simply a collection of "Person.cs" classes,nothing more than a name and address, all private strings but with properties and marked as Serializable.
What would be the best way to actually save my data in a way that can be easily read by a person? It would also make it easier to make small changes to the application's data directly in the file instead of having to load it, change it and save it.
Edit:
I have added the current way i am saving and loading my data, my _userCollection is as it suggests and the nUser/nMember are an integer.
#region I/O Operations
public bool SaveData()
{
try
{
//Open the stream using the Data.txt file
using (Stream stream = File.Open("Data.txt", FileMode.Create))
{
//Create a new formatter
BinaryFormatter bin = new BinaryFormatter();
//Copy data in collection to the file specified earlier
bin.Serialize(stream, _userCollection);
bin.Serialize(stream, nMember);
bin.Serialize(stream, nUser);
//Close stream to release any resources used
stream.Close();
}
return true;
}
catch (IOException ex)
{
throw new ArgumentException(ex.ToString());
}
}
public bool LoadData()
{
//Check if file exsists, otherwise skip
if (File.Exists("Data.txt"))
{
try
{
using (Stream stream = File.Open("Data.txt", FileMode.Open))
{
BinaryFormatter bin = new BinaryFormatter();
//Copy data back into collection fields
_userCollection = (List<User>)bin.Deserialize(stream);
nMember = (int)bin.Deserialize(stream);
nUser = (int)bin.Deserialize(stream);
stream.Close();
//Sort data to ensure it is ordered correctly after being loaded
_userCollection.Sort();
return true;
}
}
catch (IOException ex)
{
throw new ArgumentException(ex.ToString());
}
}
else
{
//Console.WriteLine present for testing purposes
Console.WriteLine("\nLoad failed, Data.txt not found");
return false;
}
}
Replace your BinaryFormatter with XMLSerializer and run the same exact code.
The only change you need to make is the BinaryFormatter takes an empty constructor, while for the XMLSerializer you need to declare the type in the constructor:
XmlSerializer serializer = new XmlSerializer(typeof(Person));
Using XmlSerializer is not really complicated. Have a look at this MSDN page for an example: http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx
You could implement your own PersonsWriter, that takes a StreamWriter as constructor argument and has a Write method that takes an IList<Person> as input to parse out a nice text representation.
For example:
public class PersonsWriter : IDisposable
{
private StreamWriter _wr;
public PersonsWriter(IList<Person> persons, StreamWriter writer)
{
this._wr = writer;
}
public void Write(IList<Persons> people) {
foreach(Person dude in people)
{
_wr.Write(#"{0} {1}\n{2}\n{3} {4}\n\n",
dude.FirstName,
dude.LastName,
dude.StreetAddress,
dude.ZipCode,
dude.City);
}
}
public void Dispose()
{
_wr.Flush();
_wr.Dispose();
}
}
YAML is another option for human readable markup that is also easy to parse. there are libraries available for c# as well as almost all other popular languages. Here's a sample of what yaml looks like:
invoice: 34843
date : 2001-01-23
bill-to: &id001
given : Chris
family : Dumars
address:
lines: |
458 Walkman Dr.
Suite #292
city : Royal Oak
state : MI
postal : 48046
Frankly, as a human, I don't find XML to be all that readable. In fact, it's not really designed to be read by humans.
If you want a human readable format, then you have to build it.
Say, you have a Person class that has a First Name, a last Name and a SSN as properties. Create your file, and have it write out 3 lines, with a description of the field in the first fifty (random number from my head) and then with character 51 have the value start being written.
This will produce a file that looks like:
First Name-------Stephen
Last Name -------Wrighton
SSN -------------XXX-XX-XXXX
Then, reading it back in, your program would know where the data begins on each line, and what each line is for (the program would know that Line 3 is the SSN value).
But remember, to truly gain human readability, you sacrifice data portability.
Try the DataContractSerializer
It serializes objects to XML and is very easy to use
Write a CSV reader writer if you want a good compromise between human and machine readable in a Windows environment
Loads into Excel too.
There's a discussion about it here:
http://knab.ws/blog/index.php?/archives/3-CSV-file-parser-and-writer-in-C-Part-1.html
EDIT
That is a C# article... it just confusingly has "C" in the URL.
I really think you should go with XML (look into DataContractSerializer). Its not that complicated. You could probably even just replace BinarySerializer with XMLSerializer and go.
If you still don't want to do that, though, you can write a delimited text file. Then you'll have to write your own reader method (although, it could almost just use the split method).
//Inside the Person class:
public override string ToString()
{
List<String> propValues = new List<String>();
// Get the type.
Type t = this.GetType();
// Cycle through the properties.
foreach (PropertyInfo p in t.GetProperties())
{
propValues.add("{0}:={1}", p.Name, p.GetValue(o, null));
}
return String.Join(",". propValues.ToArray())
}
using (System.IO.TextWriter tw = new System.IO.StreamWriter("output.txt"))
{
tw.WriteLine(person.ToString());
}