We Keep Coding

Deep copying a Func within an object in C#

I am building a list of unit tests, which are organised as a list of objects, each of which contain the test method to be executed as a Func. Each object has a variable which is within scope of the Func and is used by it. The variable is not passed in as a parameter.
Iterating over the list and running all the tests runs fine, but is is possible to copy a Func from one object, -breaking the reference to that object-, and assign it to a new object? I assume this is possible somehow by creating a Deep Copy, but my attempt using BinaryFormatter has not worked, any tips would be appreciated!
I have a simplified forms application as follows to illustrate my problem:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing; using
using System.IO;
using System.Linq;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;
using System.Threading.Tasks; using System.Windows.Forms;
namespace WindowsFormsApplication4 {
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
public static object DeepClone(object obj)
{
object objResult = null;
using (MemoryStream ms = new MemoryStream())
{
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(ms, obj);
ms.Position = 0;
objResult = bf.Deserialize(ms);
}
return objResult;
}
[Serializable]
public class POCOwithFunc {
public POCOwithFunc(Func<string> myfunc)
{
mqi = myfunc;
}
public POCOwithFunc() { }
public Func<string> mqi;
public object parm;
}
private void button1_Click(object sender, EventArgs e)
{
List<POCOwithFunc> testList = new List<POCOwithFunc>();
for (int x = 0; x < 5; x++)
{
var pc = new POCOwithFunc();
pc.parm = x;
pc.mqi = delegate()
{
var rrq = pc.parm;
return "result: " + pc.parm;
};
testList.Add(pc);
}
String output = "";
foreach (var test in testList)
{
output += test.mqi() + "\r\n";
}
//output:
//result: 0
//result: 1
//result: 2
//result: 3
//result: 4
var pocoToBeCopied = testList[2];
var newpoco = new POCOwithFunc();
newpoco.parm = 10;
newpoco.mqi = pocoToBeCopied.mqi;
var res = newpoco.mqi(); //returns 2
newpoco = (POCOwithFunc)DeepClone(pocoToBeCopied); //fails
}
} }

This is the first time I'm hearing about deep copy a delegate (which would not work, as the delegate (Func is type of delegate) contain reference to its closure (its environment, which contain any variable that that delegate is using).
I would suggest to change the parameter itself, or, to send it as a parameter (there is a delegate type for it too: Func<object, string >).
(and, I think you should think about redesign the whole thing :-/)

7 years later...
So I had encountered pretty much the same problem, also when writing tests (basically, I had a delegate that captured a lot of info from different parts of the program, and at the end of the program displayed it in a elaborate, error-prone way. I wanted to check that in all cases where captured variable turns to null/other weird values, the delegate handles it gracefully). I had a general idea that "no you can't" isn't completely true, so here goes.
TL;DR:
Closures: compiler generates a class that contains references to all
"captured" variables. This class can be accessed and edited via
reflection, thus "redirecting" the closure to point to another
variable.
Delegate target can also be edited via reflection.
To "deep copy" a delegate within an object you need to change
delegates' Target and (if needed) redirect any fields that point to
original object.
Long version:
Turned out, there is two questions here: first (and the one that was more important for me) - redirecting closures. When compiler detects a delegate that forms a closure that is moved beyond the current scope, a class is generated. This class contains references to the objects that were "captured", so that they don't get garbage collected despite being out of scope. (Also, turned out, the fields of this class are helpfully called by the name of variable that got captured.) You can see the generated class with Reflection:
<!-- language: csharp -->
public static void PrintClass(System.Delegate f)
{
var compilerGeneratedType = f.Target.GetType();
var fields = compilerGeneratedType.GetFields(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic);
Console.WriteLine(compilerGeneratedType.Name + " has fields:");
foreach (var field in fields)
{
Console.WriteLine(field.FieldType + "\t" + field.Name);
}
}
and the output when used on a classic closure example:
Action a = () => { };
for (int index = 0; index != 3; index++)
{
a += () => { Console.Write(index + " "); };
}
a(); //output: 3 3 3
PrintClass(a);
output of PrintClass function:
<>c__DisplayClass0_1 has fields: System.Int32 index
Modifying that closure can be done with reflection, like that:
static System.Delegate RedirectClosure(System.Delegate f, object newTarget, string originalVariableName)
{
System.Delegate result = (System.Delegate)f.Clone();
var compilerGeneratedType = f.Target.GetType();
var fields = compilerGeneratedType.GetFields();
foreach (var field in fields)
{
if (field.FieldType == newTarget.GetType() && field.Name == originalVariableName)
field.SetValue(result.Target, newTarget);
}
return result;
}
Using that earlier example:
Action a = () => { };
for (int index = 0; index != 3; index++)
{
a += () => { Console.Write(index + " "); };
}
a(); //output: 3 3 3
int j = 42;
RedirectClosure(a, (object)j, "index");
a(); //output: 42 42 42
Now for this specific example (delegate is inside a class, and it captured a field of this same class), second problem: the delegate's Target also needs to be changed. For this I modified the copying function to also redirect delegate's target. (Using .Clone() on delegate or straight-up copying it has the side effect: if I copy delegate from source to destination and modify only its "captured object" field to point to second destination - source delegate also starts pointing there. I guess they still share a reference to same Target.) Modified function (copies the delegate and redirects to new object):
static System.Delegate CopyDelegateAndRedirectClosure<T1>(System.Delegate f, T1 originalTarget, T1 newTarget)
{
System.Delegate result = (System.Delegate)f.Clone();
// I bet there is a better way to get a copy then this =(
var serialized = JsonSerializer.Serialize(result.Target);
var deserialized = JsonSerializer.Deserialize(serialized, result.Target.GetType());
var targetField = result.GetType().GetField("_target", BindingFlags.Instance | BindingFlags.NonPublic);
targetField.SetValue(result,deserialized);
var compilerGeneratedType = f.Target.GetType();
var fields = compilerGeneratedType.GetFields();
foreach (var field in fields)
{
if (field.FieldType == originalTarget.GetType() && field.GetValue(f.Target) == (object)originalTarget)
field.SetValue(result.Target, newTarget);
}
return result;
}
And here is an example of using it on the same situation the original poster had.
Test class:
class A
{
public int param;
public Func<int> del;
}
"Deep copying" the delegate from one instance to another:
var destination = new A { param = 10 };
var source = new A { param = 2 };
source.del = () => { return source.param + 100; };
Console.WriteLine($"source: param = {source.param}, del() result = {source.del()}");
// output:
// source: param = 2, del() result = 102
destination.del = (System.Func<int>)CopyDelegateAndRedirectClosure(source.del, source, destination);
Console.WriteLine($"destination: param = {destination.param}, del() result = {destination.del()}");
Console.WriteLine($"source: param = {source.param}, del() result = {source.del()}");
// output:
// destination: destination: param = 10, del() result = 110
// source: param = 2, del() result = 102
So there it is: copied the delegate from one instance to other, and it now operates on the new instance. The old one is unaffected.
Now again to "why would one do that" - yep, better architecture would have prevented me from ever finding this question. However, this did allow me to write a test replacing all captured variables of a specific type with "broken" value, and "bad architecture + tests" is better then "bad architecture + no tests".

Defining a method with for loops and conditional statements

I have to define a method with Reflection.Emit that is rather complex, because I have to do a for loop on a field and have a condition with break and return. My method that I want to recreate with reflection looks like this in regular code:
override int GetKeyImpl(Type obj0)
{
int answer = -1;
for(int i = 0; i < knownTypes.length; i++){
if(knowntypes[i] == obj0){
answer = i;
break;
}
}
return answer;
}
My idea to solve this problem was to generate a method with reflection that redirects the call to my original method and returns the int.
I need to know how to do a for loop and breaks with OpCodes to recreate the method while doing conditional checks on an array that is inside a class. I've searched for tutorials but didn't find any that go further than addition of two ints.
Edit: Forgot to mention it, I'm using IKVM.Reflection and knownTypes is an array of Type[]. The method that im writing is one that will override an abstract one.

This should reproduce the method you specified:
TypeBuilder type = /* ... */;
FieldInfo knownFields = /* ... */;
// Finding dependencies via reflection
var baseMethod = type.BaseType.GetMethod(
"GetKeyImpl",
BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance);
var typeEqualsOperator = typeof(Type).GetMethod(
"op_Equality",
BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic,
null,
new[] { typeof(Type), typeof(Type) },
null);
// Declaring the method
var getKeyImpl = type.DefineMethod(
baseMethod.Name,
baseMethod.Attributes & ~(MethodAttributes.Abstract |
MethodAttributes.NewSlot));
// Setting return type
getKeyImpl.SetReturnType(typeof(int));
// Adding parameters
getKeyImpl.SetParameters(typeof(Type));
getKeyImpl.DefineParameter(1, ParameterAttributes.None, "obj0");
// Override the base method
type.DefineMethodOverride(getKeyImpl, baseMethod);
var il = getKeyImpl.GetILGenerator();
// Preparing locals
var answer = il.DeclareLocal(typeof(int));
var i = il.DeclareLocal(typeof(int));
// Preparing labels
var loopCondition = il.DefineLabel();
var loopIterator = il.DefineLabel();
var returnLabel = il.DefineLabel();
var loopBody = il.DefineLabel();
// Writing body
// answer = -1
il.Emit(OpCodes.Ldc_I4_M1);
il.Emit(OpCodes.Stloc, answer);
// i = 0
il.Emit(OpCodes.Ldc_I4_0);
il.Emit(OpCodes.Stloc, i);
// jump to loop condition
il.Emit(OpCodes.Br_S, loopCondition);
// begin loop body
il.MarkLabel(loopBody);
// if (obj0 != knownTypes[i]) continue
il.Emit(OpCodes.Ldarg_0); // omit if 'knownTypes' is static
il.Emit(OpCodes.Ldfld, knownTypes); // use 'Ldsfld' if 'knownTypes' is static
il.Emit(OpCodes.Ldloc, i);
il.Emit(OpCodes.Ldelem_Ref);
il.Emit(OpCodes.Ldarg_1); // use 'Ldarg_0' if 'knownTypes' is static
il.Emit(OpCodes.Call, typeEqualsOperator);
il.Emit(OpCodes.Brfalse_S, loopIterator);
// answer = i; jump to return
il.Emit(OpCodes.Ldloc, i);
il.Emit(OpCodes.Stloc, answer);
il.Emit(OpCodes.Br_S, returnLabel);
// begin loop iterator
il.MarkLabel(loopIterator);
// i = i + 1
il.Emit(OpCodes.Ldloc, i);
il.Emit(OpCodes.Ldc_I4_1);
il.Emit(OpCodes.Add);
il.Emit(OpCodes.Stloc, i);
// begin loop condition
il.MarkLabel(loopCondition);
// if (i < knownTypes.Length) jump to loop body
il.Emit(OpCodes.Ldloc, i);
il.Emit(OpCodes.Ldarg_0); // omit if 'knownTypes' is static
il.Emit(OpCodes.Ldfld, knownTypes); // use 'Ldsfld' if 'knownTypes' is static
il.Emit(OpCodes.Ldlen);
il.Emit(OpCodes.Conv_I4);
il.Emit(OpCodes.Blt_S, loopBody);
// return answer
il.MarkLabel(returnLabel);
il.Emit(OpCodes.Ldloc, answer);
il.Emit(OpCodes.Ret);
// Finished!
The decompiled results are as expected:
override int GetKeyImpl(Type obj0)
{
for (int i = 0; i < this.knownTypes.Length; i++)
{
if (this.knownTypes[i] == obj0)
return i;
}
return -1;
}
If you have access to .NET Reflector, there is a Reflection.Emit Language Add-In that may interest you. Alternatively, write a prototype in C# code, and then run it through a disassembler to see the raw IL.
If it had been okay to make the method static (and accept knownTypes as a parameter or make it a static field), then you could have composed the method body using LINQ expression trees. Unfortunately, you cannot compose instance method bodies using this technique; they have to be static. Example:
var method = typeBuilder.DefineMethod(
"GetKeyImpl",
MethodAttributes.Private |
MethodAttributes.Static |
MethodAttributes.HideBySig);
var type = E.Parameter(typeof(Type), "type");
var knownTypes = E.Parameter(typeof(Type[]), "knownTypes");
var answer = E.Variable(typeof(int), "answer");
var i = E.Variable(typeof(int), "i");
var breakTarget = E.Label("breakTarget");
var continueTarget = E.Label("continueTarget");
var returnTarget = E.Label(typeof(int), "returnTarget");
var forLoop = E.Block(
new[] { i },
E.Assign(i, E.Constant(0)),
E.Loop(
E.Block(
E.IfThen(
E.GreaterThanOrEqual(i, E.ArrayLength(knownTypes)),
E.Break(breakTarget)),
E.IfThen(
E.Equal(E.ArrayIndex(knownTypes, i), type),
E.Return(returnTarget, i)),
E.Label(continueTarget),
E.PreIncrementAssign(i))),
E.Label(breakTarget));
var body = E.Lambda<Func<Type, Type[], int>>(
E.Block(
new[] { answer },
E.Assign(answer, E.Constant(-1)),
forLoop,
E.Label(returnTarget, answer)),
type,
knownTypes);
body.CompileToMethod(method);
return method;
The example above accepts knownTypes as the second parameter. Refactoring to read from a static field instead would be straightforward. The decompiled results, again, are as expected:
private static int GetKeyImpl(Type type, Type[] knownTypes)
{
for (int i = 0; i < knownTypes.Length; i++)
{
if (knownTypes[i] == type)
return i;
}
return -1;
}

The easist way to work out how to generate the IL for a method is to create a simple console application that has your method in. Then build it and run ILDasm against it to look at the IL instructions necessary to make up the method.
Once you can see the instructions it shouldn't be too difficult to write code to emit the OpCodes necessary.

Fast serialization and deserialization using dynamically emitted POCOs

I am currently serializing SQL table rows into a binary format for efficient storage. I serialize/deserialize the binary data into a List<object> per row. I'm trying to upgrade this to use POCOs, that will be dynamically generated (emitted) with one Field per column.
I've been searching online for hours and have stumbled upon ORMs/frameworks like EF, T4, ExpandoObject, but all of these either use a dynamic object (properties can be added/removed on the fly) or simply generate a POCO before compiling. I cannot use templating because the schema of the tables is unknown at compile time, and using dynamic objects would be overkill (and slow) since I know the exact set of properties and their types. I need to generate one POCO per table, with Fields corresponding to columns, and with the data types set accordingly (INT -> int, TEXT -> string).
After generating the POCO, I'll proceed to get/set properties using emitted CIL, much like what PetaPoco does for statically compiled POCOs. I'm hoping all of this rigmarole will be faster than using untyped Lists, and give me high-fidelity POCOs that are strongly-typed and can be accelerated by the CLR. Am I correct to assume this? and can you start me off on generating POCOs at runtime? And will using POCOs be much faster or much more memory-efficient than using a List<object>? Basically, will it be worth the trouble? I already know how to accelerate getting/setting Fields using emitted CIL.

From comments and chat, it seems that a key part of this is still creating a dynamic type; ok, here's a full example that shows a fully serializable (by any common serializer) type. You could of course add more to the type - maybe indexers to get properties by number or by name, INotifyPropertyChanged, etc.
Also - critical point: you must cache and re-use the generated Type instances. Do not keep regenerating this stuff... you will hemorrhage memory.
using Newtonsoft.Json;
using ProtoBuf;
using System;
using System.IO;
using System.Reflection;
using System.Reflection.Emit;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;
using System.Xml.Serialization;
public interface IBasicRecord
{
object this[int field] { get; set; }
}
class Program
{
static void Main()
{
object o = 1;
int foo = (int)o;
string[] names = { "Id", "Name", "Size", "When" };
Type[] types = { typeof(int), typeof(string), typeof(float), typeof(DateTime?) };
var asm = AppDomain.CurrentDomain.DefineDynamicAssembly(
new AssemblyName("DynamicStuff"),
AssemblyBuilderAccess.Run);
var module = asm.DefineDynamicModule("DynamicStuff");
var tb = module.DefineType("MyType", TypeAttributes.Public | TypeAttributes.Serializable);
tb.SetCustomAttribute(new CustomAttributeBuilder(
typeof(DataContractAttribute).GetConstructor(Type.EmptyTypes), new object[0]));
tb.AddInterfaceImplementation(typeof(IBasicRecord));
FieldBuilder[] fields = new FieldBuilder[names.Length];
var dataMemberCtor = typeof(DataMemberAttribute).GetConstructor(Type.EmptyTypes);
var dataMemberProps = new[] { typeof(DataMemberAttribute).GetProperty("Order") };
for (int i = 0; i < fields.Length; i++)
{
var field = fields[i] = tb.DefineField("_" + names[i],
types[i], FieldAttributes.Private);
var prop = tb.DefineProperty(names[i], PropertyAttributes.None,
types[i], Type.EmptyTypes);
var getter = tb.DefineMethod("get_" + names[i],
MethodAttributes.Public | MethodAttributes.HideBySig, types[i], Type.EmptyTypes);
prop.SetGetMethod(getter);
var il = getter.GetILGenerator();
il.Emit(OpCodes.Ldarg_0); // this
il.Emit(OpCodes.Ldfld, field); // .Foo
il.Emit(OpCodes.Ret); // return
var setter = tb.DefineMethod("set_" + names[i],
MethodAttributes.Public | MethodAttributes.HideBySig, typeof(void), new Type[] { types[i] });
prop.SetSetMethod(setter);
il = setter.GetILGenerator();
il.Emit(OpCodes.Ldarg_0); // this
il.Emit(OpCodes.Ldarg_1); // value
il.Emit(OpCodes.Stfld, field); // .Foo =
il.Emit(OpCodes.Ret);
prop.SetCustomAttribute(new CustomAttributeBuilder(
dataMemberCtor, new object[0],
dataMemberProps, new object[1] { i + 1 }));
}
foreach (var prop in typeof(IBasicRecord).GetProperties())
{
var accessor = prop.GetGetMethod();
if (accessor != null)
{
var args = accessor.GetParameters();
var argTypes = Array.ConvertAll(args, a => a.ParameterType);
var method = tb.DefineMethod(accessor.Name,
accessor.Attributes & ~MethodAttributes.Abstract,
accessor.CallingConvention, accessor.ReturnType, argTypes);
tb.DefineMethodOverride(method, accessor);
var il = method.GetILGenerator();
if (args.Length == 1 && argTypes[0] == typeof(int))
{
var branches = new Label[fields.Length];
for (int i = 0; i < fields.Length; i++)
{
branches[i] = il.DefineLabel();
}
il.Emit(OpCodes.Ldarg_1); // key
il.Emit(OpCodes.Switch, branches); // switch
// default:
il.ThrowException(typeof(ArgumentOutOfRangeException));
for (int i = 0; i < fields.Length; i++)
{
il.MarkLabel(branches[i]);
il.Emit(OpCodes.Ldarg_0); // this
il.Emit(OpCodes.Ldfld, fields[i]); // .Foo
if (types[i].IsValueType)
{
il.Emit(OpCodes.Box, types[i]); // (object)
}
il.Emit(OpCodes.Ret); // return
}
}
else
{
il.ThrowException(typeof(NotImplementedException));
}
}
accessor = prop.GetSetMethod();
if (accessor != null)
{
var args = accessor.GetParameters();
var argTypes = Array.ConvertAll(args, a => a.ParameterType);
var method = tb.DefineMethod(accessor.Name,
accessor.Attributes & ~MethodAttributes.Abstract,
accessor.CallingConvention, accessor.ReturnType, argTypes);
tb.DefineMethodOverride(method, accessor);
var il = method.GetILGenerator();
if (args.Length == 2 && argTypes[0] == typeof(int) && argTypes[1] == typeof(object))
{
var branches = new Label[fields.Length];
for (int i = 0; i < fields.Length; i++)
{
branches[i] = il.DefineLabel();
}
il.Emit(OpCodes.Ldarg_1); // key
il.Emit(OpCodes.Switch, branches); // switch
// default:
il.ThrowException(typeof(ArgumentOutOfRangeException));
for (int i = 0; i < fields.Length; i++)
{
il.MarkLabel(branches[i]);
il.Emit(OpCodes.Ldarg_0); // this
il.Emit(OpCodes.Ldarg_2); // value
il.Emit(types[i].IsValueType ? OpCodes.Unbox_Any : OpCodes.Castclass, types[i]); // (SomeType)
il.Emit(OpCodes.Stfld, fields[i]); // .Foo =
il.Emit(OpCodes.Ret); // return
}
}
else
{
il.ThrowException(typeof(NotImplementedException));
}
}
}
var type = tb.CreateType();
var obj = Activator.CreateInstance(type);
// we'll use the index (via a known interface) to set the values
IBasicRecord rec = (IBasicRecord)obj;
rec[0] = 123;
rec[1] = "abc";
rec[2] = 12F;
rec[3] = DateTime.Now;
for (int i = 0; i < 4; i++)
{
Console.WriteLine("{0} = {1}", i, rec[i]);
}
using (var ms = new MemoryStream())
{
var ser = new XmlSerializer(type);
ser.Serialize(ms, obj);
Console.WriteLine("XmlSerializer: {0} bytes", ms.Length);
}
using (var ms = new MemoryStream())
{
using (var writer = new StreamWriter(ms, Encoding.UTF8, 1024, true))
{
var ser = new JsonSerializer();
ser.Serialize(writer, obj);
}
Console.WriteLine("Json.NET: {0} bytes", ms.Length);
}
using (var ms = new MemoryStream())
{
var ser = new DataContractSerializer(type);
ser.WriteObject(ms, obj);
Console.WriteLine("DataContractSerializer: {0} bytes", ms.Length);
}
using (var ms = new MemoryStream())
{
Serializer.NonGeneric.Serialize(ms, obj);
Console.WriteLine("protobuf-net: {0} bytes", ms.Length);
}
using (var ms = new MemoryStream())
{
// note: NEVER do this unless you have a custom Binder; your
// assembly WILL NOT deserialize in the next AppDomain (i.e.
// the next time you load your app, you won't be able to load)
// - shown only for illustration
var bf = new BinaryFormatter();
bf.Serialize(ms, obj);
Console.WriteLine("BinaryFormatter: {0} bytes", ms.Length);
}
}
}
Output:
XmlSerializer: 246 bytes
Json.NET: 81 bytes
DataContractSerializer: 207 bytes
protobuf-net: 25 bytes
BinaryFormatter: 182 bytes

This is actually quite a complex question. Unfortunately, to answer it fully you would have to basically write it and test it, however - I strongly suggest not looking at any on-the-fly POCO generation until you have your answer! Basically, you should ignore that step for now.
The other essential question in performance is: how fast does it need to be? The absolute first thing I would do is the absolutely simplest thing that works, and measure that. And the simplest thing that works is: load it into a DataTable and serialize that datatable (using RemotingFormat = RemotingFormat.Binary;). In 10 lines of code that will give you a
line in the sand:
var dt = new DataTable();
dt.Load(yourDataReader);
//... any access tests
dt.RemotingFormat = SerializationFormat.Binary;
using (var file = File.Create(path))
{
var bf = new BinaryFormatter();
bf.Serialize(file, dt);
}
// ... also check deserialize, if that is perf-critical
Normally I wouldn't recommend either DataTable or BinaryFormatter, but... it doesn't seem far-fetched in this case.
Personally, I suspect you'll find that DataTable in binary-remoting-mode isn't actually terrible.
The next step is to see what else works without any huge effort. For example:
loading a data-source into objects is a solved problem, with tools like dapper
serializing a set of objects in a very efficient way is a solved problem, with tools like protobuf-net
So I would be tempted to create an illustrative class (purely to see if it is any better) along the lines of:
[DataContract]
public class Foo {
[DataMember(Order=1)] public int Id {get;set;}
[DataMember(Order=2)] public string Name {get;set;}
// ... more props
// IMPORTANT: make this representative - basically, the same data
// that you had in the data-table
// note also include any supporting info - any indexers and interface
// support that your core code needs
}
[DataContract]
public class FooWrapper { // just to help in the test
[DataMember(Order=1)] public List<Foo> Items {get;set;}
}
and do the same test (your main code would only use the indexer access, but let dapper use the .Query<Foo>(...) API for now):
var data = conn.Query<Foo>(...).ToList(); // dapper
//... any access tests, just using the indexer API
using (var file = File.Create(path))
{
var wrapper = new FooWrapper { Items = data };
Serializer.Serialize(file, wrapper); // protobuf-net
}
// note that you deserialize via Serializer.Deserialize<FooWrapper>(file)
The point of this is that this will give you some bounds on what is reasonable to expect in terms of what can be achieved. Feel free to use your own materializer/serializer in place of dapper/protobuf-net, but I humbly submit that these two have been heavily optimized for scenarios largely like this.
When you have a lower and upper bound, you have sensible data to answer the "is it worth it" question. Generating objects at run-time isn't massively hard, but it is more work than most people would need to do. You also want to be really careful to re-use the generated types as far as possible. Note that if you go that route, protobuf-net has a fully non-generic API, via Serializer.NonGeneric or RuntimeTypeModel.Default (all three options end up at the same core). Dapper doesn't, but I would be more than happy to add one (accepting a Type instance). In the interim, you could also use MakeGenericMethod / Invoke for that one step.
I realize I haven't directly answered "is it worth it", but that is deliberate: that cannot be answered without direct application to your scenario. Hopefully, I have instead provided some hints at how you can answer it for your scenario. I would be very interested in hearing your findings.
Only when you know that it is worth it (and with the above I would expect that to take about an hour's effort) would I go to the trouble of generating types. If you do, I recommend the use of Sigil - it will make your IL generation far less frustrating.

Dynamically calling a dll and method with arguments

Basically I'm trying to call a dll by name, instantiate an object, then call a method by name in that dll. I'm getting an "Exception has been thrown by the target of an invocation." during the Method.Invoke. I'm fairly sure my problem is with the typecasting of the arguments of the method. I was wondering if anyone had any input on this exception. Additionally, any suggestions on how to revise my approach are welcome.
public void calldll(string dllName, string typeName, string methodName, string arguments) {
string[] argumentArray = arguments.Split(new char[] { '|' }, StringSplitOptions.None);
Assembly assembly = Assembly.LoadFrom(dllName);
System.Type type = assembly.GetType(typeName);
Object o = Activator.CreateInstance(type);
MethodInfo method = type.GetMethod(methodName);
ParameterInfo[] parameters = method.GetParameters();
object[] methodParameters = new object[parameters.GetLength(0)];
for (int i = 0; i < parameters.Length - 1; i++)
{
var converter = TypeDescriptor.GetConverter(parameters[i].GetType());
methodParameters[i] = converter.ConvertFrom(argumentArray[i]);
}
method.Invoke(o, methodParameters); }

I found two issues with your code:
You are not looping over all parameters. You should remove -1 from the for loop.
When you are creating your converter, you call the GetType() method. This returns the Type of the ParameterInfo object, not the Type of the parameter. Use the property ParameterType instead.
All in all, change the first lines in the for loop to this:
for (int i = 0; i < parameters.Length; i++)
{
var converter = TypeDescriptor.GetConverter(parameters[i].ParameterType);
Once you have done these corrections, I believe your code should run as intended. At least it did for me when I tested a simple void Hello(int x, string y) method.

How do I create and access a new instance of an Anonymous Class passed as a parameter in C#?

I have created a function that takes a SQL command and produces output that can then be used to fill a List of class instances. The code works great. I've included a slightly simplified version without exception handling here just for reference - skip this code if you want to jump right the problem. If you have suggestions here, though, I'm all ears.
public List<T> ReturnList<T>() where T : new()
{
List<T> fdList = new List<T>();
myCommand.CommandText = QueryString;
SqlDataReader nwReader = myCommand.ExecuteReader();
Type objectType = typeof (T);
FieldInfo[] typeFields = objectType.GetFields();
while (nwReader.Read())
{
T obj = new T();
foreach (FieldInfo info in typeFields)
{
for (int i = 0; i < nwReader.FieldCount; i++)
{
if (info.Name == nwReader.GetName(i))
{
info.SetValue(obj, nwReader[i]);
break;
}
}
}
fdList.Add(obj);
}
nwReader.Close();
return fdList;
}
As I say, this works just fine. However, I'd like to be able to call a similar function with an anonymous class for obvious reasons.
Question #1: it appears that I must construct an anonymous class instance in my call to my anonymous version of this function - is this right? An example call is:
.ReturnList(new { ClientID = 1, FirstName = "", LastName = "", Birthdate = DateTime.Today });
Question #2: the anonymous version of my ReturnList function is below. Can anyone tell me why the call to info.SetValue simply does nothing? It doesn't return an error or anything but neither does it change the value of the target field.
public List<T> ReturnList<T>(T sample)
{
List<T> fdList = new List<T>();
myCommand.CommandText = QueryString;
SqlDataReader nwReader = myCommand.ExecuteReader();
// Cannot use FieldInfo[] on the type - it finds no fields.
var properties = TypeDescriptor.GetProperties(sample);
while (nwReader.Read())
{
// No way to create a constructor so this call creates the object without calling a ctor. Could this be a source of the problem?
T obj = (T)FormatterServices.GetUninitializedObject(typeof(T));
foreach (PropertyDescriptor info in properties)
{
for (int i = 0; i < nwReader.FieldCount; i++)
{
if (info.Name == nwReader.GetName(i))
{
// This loop runs fine but there is no change to obj!!
info.SetValue(obj, nwReader[i]);
break;
}
}
}
fdList.Add(obj);
}
nwReader.Close();
return fdList;
}
Any ideas?
Note: when I tried to use the FieldInfo array as I did in the function above, the typeFields array had zero elements (even though the objectType shows the field names - strange). Thus, I use TypeDescriptor.GetProperties instead.
Any other tips and guidance on the use of reflection or anonymous classes are appropriate here - I'm relatively new to this specific nook of the C# language.
UPDATE: I have to thank Jason for the key to solving this. Below is the revised code that will create a list of anonymous class instances, filling the fields of each instance from a query.
public List<T> ReturnList<T>(T sample)
{
List<T> fdList = new List<T>();
myCommand.CommandText = QueryString;
SqlDataReader nwReader = myCommand.ExecuteReader();
var properties = TypeDescriptor.GetProperties(sample);
while (nwReader.Read())
{
int objIdx = 0;
object[] objArray = new object[properties.Count];
foreach (PropertyDescriptor info in properties)
objArray[objIdx++] = nwReader[info.Name];
fdList.Add((T)Activator.CreateInstance(sample.GetType(), objArray));
}
nwReader.Close();
return fdList;
}
Note that the query has been constructed and the parameters initialized in previous calls to this object's methods. The original code had an inner/outer loop combination so that the user could have fields in their anonymous class that didn't match a field. However, in order to simplify the design, I've decided not to permit this and have instead adopted the db field access recommended by Jason. Also, thanks to Dave Markle as well for helping me understand more about the tradeoffs in using Activator.CreateObject() versus GenUninitializedObject.

Anonymous types encapsulate a set of read-only properties. This explains
Why Type.GetFields returns an empty array when called on your anonymous type: anonymous types do not have public fields.
The public properties on an anonymous type are read-only and can not have their value set by a call to PropertyInfo.SetValue. If you call PropertyInfo.GetSetMethod on a property in an anonymous type, you will receive back null.
In fact, if you change
var properties = TypeDescriptor.GetProperties(sample);
while (nwReader.Read()) {
// No way to create a constructor so this call creates the object without calling a ctor. Could this be a source of the problem?
T obj = (T)FormatterServices.GetUninitializedObject(typeof(T));
foreach (PropertyDescriptor info in properties) {
for (int i = 0; i < nwReader.FieldCount; i++) {
if (info.Name == nwReader.GetName(i)) {
// This loop runs fine but there is no change to obj!!
info.SetValue(obj, nwReader[i]);
break;
}
}
}
fdList.Add(obj);
}
to
PropertyInfo[] properties = sample.GetType().GetProperties();
while (nwReader.Read()) {
// No way to create a constructor so this call creates the object without calling a ctor. Could this be a source of the problem?
T obj = (T)FormatterServices.GetUninitializedObject(typeof(T));
foreach (PropertyInfo info in properties) {
for (int i = 0; i < nwReader.FieldCount; i++) {
if (info.Name == nwReader.GetName(i)) {
// This loop will throw an exception as PropertyInfo.GetSetMethod fails
info.SetValue(obj, nwReader[i], null);
break;
}
}
}
fdList.Add(obj);
}
you will receive an exception informing you that the property set method can not be found.
Now, to solve your problem, what you can do is use Activator.CreateInstance. I'm sorry that I'm too lazy to type out the code for you, but the following will demonstrate how to use it.
var car = new { Make = "Honda", Model = "Civic", Year = 2008 };
var anothercar = Activator.CreateInstance(car.GetType(), new object[] { "Ford", "Focus", 2005 });
So just run through a loop, as you've done, to fill up the object array that you need to pass to Activator.CreateInstance and then call Activator.CreateInstance when the loop is done. Property order is important here as two anonymous types are the same if and only if they have the same number of properties with the same type and same name in the same order.
For more, see the MSDN page on anonymous types.
Lastly, and this is really an aside and not germane to your question, but the following code
foreach (PropertyDescriptor info in properties) {
for (int i = 0; i < nwReader.FieldCount; i++) {
if (info.Name == nwReader.GetName(i)) {
// This loop runs fine but there is no change to obj!!
info.SetValue(obj, nwReader[i]);
break;
}
}
}
could be simplified by
foreach (PropertyDescriptor info in properties) {
info.SetValue(obj, nwReader[info.Name]);
}

I had the same problem, I resolved it by creating a new Linq.Expression that's going to do the real job and compiling it into a lambda: here's my code for example:
I want to transform that call:
var customers = query.ToList(r => new
{
Id = r.Get<int>("Id"),
Name = r.Get<string>("Name"),
Age = r.Get<int>("Age"),
BirthDate = r.Get<DateTime?>("BirthDate"),
Bio = r.Get<string>("Bio"),
AccountBalance = r.Get<decimal?>("AccountBalance"),
});
to that call:
var customers = query.ToList(() => new
{
Id = default(int),
Name = default(string),
Age = default(int),
BirthDate = default(DateTime?),
Bio = default(string),
AccountBalance = default(decimal?)
});
and do the DataReader.Get things from the new method, the first method is:
public List<T> ToList<T>(FluentSelectQuery query, Func<IDataReader, T> mapper)
{
return ToList<T>(mapper, query.ToString(), query.Parameters);
}
I had to build an expression in the new method:
public List<T> ToList<T>(Expression<Func<T>> type, string sql, params object[] parameters)
{
var expression = (NewExpression)type.Body;
var constructor = expression.Constructor;
var members = expression.Members.ToList();
var dataReaderParam = Expression.Parameter(typeof(IDataReader));
var arguments = members.Select(member =>
{
var memberName = Expression.Constant(member.Name);
return Expression.Call(typeof(Utilities),
"Get",
new Type[] { ((PropertyInfo)member).PropertyType },
dataReaderParam, memberName);
}
).ToArray();
var body = Expression.New(constructor, arguments);
var mapper = Expression.Lambda<Func<IDataReader, T>>(body, dataReaderParam);
return ToList<T>(mapper.Compile(), sql, parameters);
}
Doing this that way, i can completely avoid the Activator.CreateInstance or the FormatterServices.GetUninitializedObject stuff, I bet it's a lot faster ;)

Question #2:
I don't really know, but I would tend to use Activator.CreateObject() instead of FormatterServices.GetUninitializedObject(), because your object might not be created properly. GetUninitializedObject() won't run a default constructor like CreateObject() will, and you don't necessarily know what's in the black box of T...

This method stores one line of a sql query in a variable of anonymous type. You have to pass a prototype to the method. If any property of the anonymous type can not be found within the sql query, it is filled with the prototype-value. C# creates constructors for its anonymous classes, the parameters have the same names as the (read-only) properties.
public static T GetValuesAs<T>(this SqlDataReader Reader, T prototype)
{
System.Reflection.ConstructorInfo constructor = prototype.GetType().GetConstructors()[0];
object[] paramValues = constructor.GetParameters().Select(
p => { try { return Reader[p.Name]; }
catch (Exception) { return prototype.GetType().GetProperty(p.Name).GetValue(prototype); } }
).ToArray();
return (T)prototype.GetType().GetConstructors()[0].Invoke(paramValues);
}