We Keep Coding

How do I create mutually callable MethodInfos from MethodBuilders created from LambdaExpressions?

I'm currently working on an compiler in C#, where the behaviour is defined by LambdaExpressions, and then using CompileToMethod, transformed into MethodBuilders and saved to DLL. All functions are public and static.
However, I could not find a way to extract usable MethodInfo (or another method of reference) from the MethodBuilder until the behaviour is defined and declaring type is created/sealed. That means that at that until that point, it is impossible to use Expression.Call to call these functions. That makes self-recursion or mutual referencing between two functions impossible.
I ended up using Reflection to invoke the functions at runtime, but it's very suboptimal, and I'm still curious if there's a better way.
How do i ensure functions created with LambdaExpression.CompileToMethod(MethodBuilder) can self-call?
Alternatively, is there any other way to use LambdaExpressions which would allow this and support saving as a static method to a dll?

I hope this helps.
This is complete code example which produces runtime defined type with single static recursive method.
For the simplicity of the example the recursive method is infinite - at the end of the Main method the recursive method is called
static void Main(string[] args)
{
var moduleBuilder = CreateDynamicModuleBuilder();
var typeBuilder = moduleBuilder.DefineType("Person", TypeAttributes.Public | TypeAttributes.Class);
var methodBuilder = typeBuilder.DefineMethod("SayHello", MethodAttributes.Static | MethodAttributes.Public);
var methodExpression = CreateRecursiveExpression();
var lambda = Expression.Lambda(methodExpression);
lambda.CompileToMethod(methodBuilder);
var typeInfo = typeBuilder.CreateType();
var methodInfo = typeInfo.GetMethod("SayHello", BindingFlags.Public | BindingFlags.Static);
methodInfo.Invoke(null, null);
}
private static Expression CreateRecursiveExpression()
{
var methodInfo = typeof(Console).GetMethod("WriteLine", new[] { typeof(String) });
var arg = Expression.Constant("Hello");
var consoleCall = Expression.Call(methodInfo, arg);
var sayHelloActionVariable = Expression.Variable(typeof(Action), "sayHelloAction");
var block = Expression.Block(
new[] { sayHelloActionVariable },
Expression.Assign(
sayHelloActionVariable,
Expression.Lambda(
Expression.Block(
consoleCall,
Expression.Invoke(sayHelloActionVariable)
)
)
),
Expression.Invoke(sayHelloActionVariable)
);
return block;
}
private static ModuleBuilder CreateDynamicModuleBuilder()
{
var name = new AssemblyName("Example.DynamicRecursion");
var am = AssemblyBuilder.DefineDynamicAssembly(name, AssemblyBuilderAccess.RunAndSave);
var mb = am.DefineDynamicModule(name.Name, $"{name.Name}.dll");
return mb;
}
This code will create type with the following signature
public class Person
{
public static void SayHello()
{
Action sayHelloAction;
sayHelloAction = () =>
{
Console.WriteLine("Hello");
sayHelloAction();
}
sayHelloAction();
}
}

When implementing an interface that has a method with 'in' parameter by TypeBuilder.CreateType, TypeLoadException is thrown

Before beginning, this is my first question on SO. So there might be faults or lack of information about the problem. Please let me know if there's something that I need to correct. Thanks.
Using TypeBuilder, I'm building a class that implements an interface that contains a method. After implementing that method with ILGenerator, then I call TypeBuilder.CreateType() and everything goes well in the normal case.
But if the method contains any parameter with the in modifier, also known as readonly reference for value types, TypeBuilder.CreateType() throws TypeLoadException("Method 'SomeMethod' ... does not have an implementation.").
Unlike the usual case of TypeLoadException that implemented method with the same signature as the one declared in the interface(s) doesn't exist, this problem is raised only when the method contains in parameter(s) even signatures are the same. When I remove or change the in modifier to ref or out, TypeBuilder.CreateType() successfully recognizes the generated method as an implementation of one declared in the interface, and the type is built normally.
Here's a fully compilable example:
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Reflection;
using System.Reflection.Emit;
using System.Threading;
namespace EmitMethodWithInParamTest
{
public struct StructParam
{
public String Data;
}
public interface ISomeInterface
{
Int32 SomeMethod(in StructParam param);
}
static class EmitExtension
{
public static void ReplicateCustomAttributes(this ParameterBuilder paramBuilder, ParameterInfo paramInfo)
{
foreach (var attrData in paramInfo.GetCustomAttributesData())
{
var ctorArgs = attrData.ConstructorArguments.Select(arg => arg.Value).ToArray();
// Handling variable arguments
var ctorParamInfos = attrData.Constructor.GetParameters();
if (ctorParamInfos.Length > 0 &&
ctorParamInfos.Last().IsDefined(typeof(ParamArrayAttribute)) &&
ctorArgs.Last() is IReadOnlyCollection<CustomAttributeTypedArgument> variableArgs)
{
ctorArgs[ctorArgs.Length - 1] = variableArgs.Select(arg => arg.Value).ToArray();
}
var namedPropArgs = attrData.NamedArguments.Where(arg => !arg.IsField);
var namedPropInfos = namedPropArgs.Select(arg => (PropertyInfo)arg.MemberInfo).ToArray();
var namedPropValues = namedPropArgs.Select(arg => arg.TypedValue.Value).ToArray();
var namedFieldArgs = attrData.NamedArguments.Where(arg => arg.IsField);
var namedFieldInfos = namedFieldArgs.Select(arg => (FieldInfo)arg.MemberInfo).ToArray();
var namedFieldValues = namedFieldArgs.Select(arg => arg.TypedValue.Value).ToArray();
var attrBuilder = new CustomAttributeBuilder(attrData.Constructor,
ctorArgs, namedPropInfos, namedPropValues, namedFieldInfos, namedFieldValues);
paramBuilder.SetCustomAttribute(attrBuilder);
}
}
}
class Program
{
static Program()
{
Thread.CurrentThread.CurrentUICulture = CultureInfo.GetCultureInfo("en-us");
}
static void Main(String[] args)
{
var assemblyBuilder = AssemblyBuilder.DefineDynamicAssembly(new AssemblyName("DynamicAssembly"), AssemblyBuilderAccess.Run);
var moduleBuilder = assemblyBuilder.DefineDynamicModule("MainModule");
var typeBuilder = moduleBuilder.DefineType("SomeClass",
TypeAttributes.Public | TypeAttributes.Class | TypeAttributes.AutoClass | TypeAttributes.AnsiClass | TypeAttributes.BeforeFieldInit | TypeAttributes.AutoLayout,
null /*base class*/,
new[] { typeof(ISomeInterface) });
var methodInfoToImpl = typeof(ISomeInterface).GetMethod(nameof(ISomeInterface.SomeMethod));
var paramInfos = methodInfoToImpl.GetParameters();
var methodBuilder = typeBuilder.DefineMethod(methodInfoToImpl.Name,
MethodAttributes.Public | MethodAttributes.Virtual | MethodAttributes.Final,
CallingConventions.HasThis,
methodInfoToImpl.ReturnType,
paramInfos.Select(pi => pi.ParameterType).ToArray());
foreach (var paramInfo in paramInfos)
{
// paramInfo.Position is zero-based but DefineParameter requires 1-based index.
var paramBuilder = methodBuilder.DefineParameter(paramInfo.Position + 1, paramInfo.Attributes, paramInfo.Name);
if (paramInfo.Attributes.HasFlag(ParameterAttributes.HasDefault))
{
paramBuilder.SetConstant(paramInfo.DefaultValue);
}
paramBuilder.ReplicateCustomAttributes(paramInfo);
}
// Dummy implementation for example. Always throws NotImplementedException.
var ilGen = methodBuilder.GetILGenerator();
ilGen.Emit(OpCodes.Newobj, typeof(NotImplementedException).GetConstructor(Type.EmptyTypes));
ilGen.Emit(OpCodes.Throw);
var builtType = typeBuilder.CreateType(); // <- TypeLoadException("Method 'SomeMethod' in type 'SomeClass' from assembly 'DynamicAssembly, ...' does not have an implementation.") is thrown.
var generatedObj = (ISomeInterface)Activator.CreateInstance(builtType);
var someParam = new StructParam() { Data = "SomeData" };
var result = generatedObj.SomeMethod(in someParam); // <- NotImplementedException expected by dummy implementation if executed.
Console.WriteLine($"Result: {result}");
}
}
}
This code is also uploaded to Pastebin.
While digging down this problem, I found that the in parameter has two custom attributes, InteropServices.InAttribute and CompilerServices.IsReadOnlyAttribute. But when I generate a method without implementing the interface (this succeeds normally because no signature matching required), in parameter of generated method has only one custom attribute, InAttribute. So I replicated all custom attributes of parameters from the interface, but still TypeLoadException is being raised.
I've tested this on .NET Framework 4.6.1 and .NET Core 2.2 with C# 7.2 and 7.3. And all environments gave me the same exception. I'm using Visual Studio 2017 on Windows.
Is there anything that I have missed or are there any workarounds?
Thank you for any help in advance.

After writing the question above, I've been investigated built binary of sample code in IL and source code of CoreCLR for a few days, and now I found the problem and solution.
In short, required and optional custom modifiers of return type and each parameter type take a part of method signature like each types do, and it had to be replicated manually. I thought that it will be done by passing ParameterAttributes.In to MethodBuilder.DefineParameter and replicating the custom attribute InAttribute, but it was wrong.
And, among in, ref and out modifiers, only in emits a required custom modifier to specified parameter. In contrast, ref and out are represented only with their type itself. This is the reason why only in didn't work as expected.
To replicate custom modifiers, call to TypeBuilder.DefineMethod need be modified like this:
var methodBuilder = typeBuilder.DefineMethod(methodInfoToImpl.Name,
MethodAttributes.Public | MethodAttributes.Virtual | MethodAttributes.Final,
CallingConventions.HasThis,
methodInfoToImpl.ReturnType,
methodInfoToImpl.ReturnParameter.GetRequiredCustomModifiers(), // *
methodInfoToImpl.ReturnParameter.GetOptionalCustomModifiers(), // *
paramInfos.Select(pi => pi.ParameterType).ToArray(),
paramInfos.Select(pi => pi.GetRequiredCustomModifiers()).ToArray(), // *
paramInfos.Select(pi => pi.GetOptionalCustomModifiers()).ToArray() // *
);
Marked lines with // * are newly added to replicate custom modifiers of return/parameter types.
Or, we can do this by calling MethodBuilder.SetSignature method after calling DefineMethod without any type and custom modifiers arguments. If we decided to call SetSignature separately, we need to call it before any DefineParameter, SetCustomAttribute, Equals(Object), SetImplementationFlags, getter of property Signature and many other methods that call the internal method MethodBuilder.GetMethodSignature() that cache bytes representing method signature.
Thank you for reading and giving me advice. :)

Can't get MethodInfo instance in reflection-only context

The following code shows method's name correctly:
static void Main(string[] args)
{
try
{
Assembly assembly = Assembly.Load("ClassLibrary1");
foreach (var referencedAssembly in assembly.GetReferencedAssemblies())
{
Assembly.Load(referencedAssembly.Name);
}
Type bl1Type = assembly.GetType("ClassLibrary1.Bl1");
var types = new[] {typeof(MyEnum) };
var method = bl1Type.GetMethod("Method1", BindingFlags.Instance | BindingFlags.Public, Type.DefaultBinder, types, null);
Console.WriteLine(method == null ? "Method was null" : $"Found {method.Name}");
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
But if I try to resolve Method1 in a reflection-only context to improve performance and change the call to Assembly.Load("ClassLibrary1"); with Assembly.ReflectionOnlyLoad("ClassLibrary1"); then method is always null and don't get resolved. Any ideas about how to resolve a method in a reflection-only context?

If you try the following you will see that your method is found in the list
var methods = bl1Type.GetMethods(BindingFlags.Instance | BindingFlags.Public);
Also,if you search for the method that accepts the string parameter it will also work
var types = new[] {typeof(string)};
I think it's confused by the MyEnum.
If you load the MyEnum from the second dll it will work
Assembly assembly2 = Assembly.ReflectionOnlyLoad("ClassLibrary2");
Type bl2Type = assembly2.GetType("ClassLibrary2.MyEnum");
var types = new[] { bl2Type };
The full code
using System;
using System.Linq;
using System.Reflection;
namespace ReflectionTest
{
class Program
{
static void Main(string[] args)
{
Assembly assembly = Assembly.ReflectionOnlyLoad("ClassLibrary1");
Type bl1Type = assembly.GetType("ClassLibrary1.Bl1");
var types = new[] { Assembly.ReflectionOnlyLoad(assembly.GetReferencedAssemblies().Single(a => a.Name == "ClassLibrary2").Name).GetType("ClassLibrary2.MyEnum") };
//var types = new[] {typeof(MyEnum)}; //doesn't work
var method = bl1Type.GetMethod("Method1", BindingFlags.Instance | BindingFlags.Public, Type.DefaultBinder, types, null);
Console.WriteLine(method == null ? "Method was null" : $"Found {method.Name}");
Console.ReadLine();
}
}
}
So,your initial code doesn't work because it tries to find a method that uses a different "MyEnum" (and of course it doesn't exist)

ReflectionOnly context is separate loader context from the regular loader context. It loads own copy of the assemblies and types, the system types from mscorlib are shared with the regular loader context though.
Also, there is typically no performance advantage in loading assemblies into ReflectionOnly context. ReflectionOnly context uses the exact same loading mechanism as regular assemblies, except that attempt to execute any code from them is blocked and a few sanity checks are suppressed (https://blogs.msdn.microsoft.com/junfeng/2004/08/24/reflection-only-assembly-loading/ describes the details).

How best to create and execute a method in a .NET (C#) class dynamically through configuration

I'm thinking of something along the lines of the "Inline Task" in MsBuild. For reference: http://msdn.microsoft.com/en-us/library/dd722601.aspx
I'd like to find or create a framework which allows me to override a method via configuration. For example if I have a well known base class which has a method Execute(args), how can I supply an overridden method implementation at deployment time, without requiring new code, build, release cycle? I would like to actually plug in the method body into a config file or preferably a database table.
I assume this would be done either with code dom, dynamic language integration, or perhaps something like powershell(?). I'm looking for recommendations or perhaps a library someone has already written.
The application is written in C#. Preferably the extension would also be in C#, but I'm open to other ideas as well.
Update: Technically I don't even have to actually override a method. It would be sufficient to just be able to dynamically execute some external source code, passing in an arg and returning a result.
Update. I ended up writing code to instantiate a PowerShell object and execute a script dynamically to return a value. Here is a snippet of code I used.
public static Collection<PSObject> ExecuteScript(string code, string variableName, object variableValue)
{
PowerShell ps = PowerShell.Create();
ps.AddScript(code);
if (!string.IsNullOrWhiteSpace(variableName))
{
ps.Runspace.SessionStateProxy.SetVariable(variableName, variableValue);
}
var result = ps.Invoke();
return result;
}
Then in the calling code, I simply check the first PSObject in the return value, and pull the resulting value from it. It works great. Thanks for all the responses.

Here are two examples of dynamic execution. I have used neither though so I can't comment further.
http://www.codeproject.com/KB/dotnet/evaluator.aspx
http://www.csharpfriends.com/articles/getarticle.aspx?articleid=118
Regarding namespaces, from the second article you can add assemblies through the CompilerParameter class.
// Create the C# compiler
CSharpCodeProvider csCompiler = new CSharpCodeProvider();
ICodeCompiler iCodeCompiler = csCompiler.CreateCompiler();
// input params for the compiler
CompilerParameters compilerParams = new CompilerParameters();
compilerParams.OutputAssembly = "CSharpFriends.dll";
compilerParams.ReferencedAssemblies.Add("system.dll");

One option would be to use Iron Python (or another DLR language). Your Execute method would then lookup the script in your configuration file, compile it and execute it all at runtime.
Including the necessary Iron Python assemblies with your project isn't a significant overhead.
You might need to do some plumbing to expose other parts of your application to the python runtime environment but this is quite easy to do.

You can use interfaces and then resolve the concrete classes at runtime e.g. using configuration files.
Check the various Dependency Injection Containers at http://www.hanselman.com/blog/ListOfNETDependencyInjectionContainersIOC.aspx

Managed Extensibility Framework (MEF) might be suitable as well. It was included as part of .NET 4.
http://msdn.microsoft.com/en-us/library/dd460648.aspx
http://mef.codeplex.com/
If the extensibility is just for one method then MEF would be overkill. If what you are extending will grow over time then I think MEF would provide the most robust and long-term manageable framework.

It looks like you might want to have a look at the Factory Pattern; returning delegates. Unfortunately you will need a type to 'house' the method, so you would typically generate code like:
namespace Dynamic {
public static int Foo(int bar) {
// .. Configured body here.
}
}
It's important that your factory does not generate methods it has seen before. Here is an example:
static class Delegates
{
private static Func<Func<int, string>> _test;
public static Func<int, string> Test
{
get
{
return _test();
}
}
static Delegates()
{
// Use your config variables instead of the "return arg.ToString();"
CreateFactory<Func<int, string>>(x => _test = x, "return arg.ToString();");
}
private static void CreateFactory<TDelegate>(Action<Func<TDelegate>> locationSetter, string identifier)
{
locationSetter(() =>
{
var result = Generate<TDelegate>(identifier);
locationSetter(() => result);
return result;
});
}
private static string GenerateSignature<TDelegate>()
{
// Create the signature of the delegate.
var t = typeof(TDelegate);
if (!typeof(Delegate).IsAssignableFrom(t))
throw new Exception("TDelegate must be delegate type.");
var invoke = t.GetMethod("Invoke");
var sig = new StringBuilder();
// Append the return type.
if (invoke.ReturnType == typeof(void))
sig.Append("void");
else
sig.Append(invoke.ReturnType.FullName);
sig.Append(" ");
sig.Append("Invoke(");
// Append the parameters.
var param = invoke.GetParameters();
for (var i = 0; i < param.Length; i++)
{
if (i != 0)
sig.Append(", ");
sig.Append(param[i].ParameterType.FullName);
sig.Append(" ");
sig.Append(param[i].Name);
}
sig.Append(")");
return sig.ToString();
}
private static TDelegate Generate<TDelegate>(string code)
{
// Generate the containing class and method.
var codeBuilder = new StringBuilder(50);
codeBuilder.AppendLine("using System;");
codeBuilder.Append("namespace Dynamic { class DynamicClass { public static ");
codeBuilder.Append(GenerateSignature<TDelegate>());
codeBuilder.AppendLine("{");
codeBuilder.AppendLine(code);
codeBuilder.AppendLine("} } }");
var compilerVersion = new Version(1, 0, 0, 0);
// Create the compiler parameters.
var parameters = new CompilerParameters();
parameters.GenerateInMemory = true;
parameters.GenerateExecutable = false;
parameters.ReferencedAssemblies.Clear();
foreach (var referenceAssembly in AppDomain.CurrentDomain.GetAssemblies())
{
parameters.ReferencedAssemblies.Add(referenceAssembly.Location);
// Figure out which version we are compiling against.
var an = new AssemblyName(referenceAssembly.FullName);
if (an.Name == "mscorlib" && compilerVersion < an.Version)
{
compilerVersion = an.Version;
}
}
var cp = new CSharpCodeProvider(
new Dictionary<string, string>() { { "CompilerVersion", string.Format("v{0}.{1}", compilerVersion.Major, compilerVersion.Minor) } }
);
var results = cp.CompileAssemblyFromSource(parameters, codeBuilder.ToString());
if (results.Errors.HasErrors)
throw new Exception("Method failed to compile.");
var assembly = results.CompiledAssembly;
if (assembly == null)
throw new Exception("Method failed to compile.");
var t = assembly.GetType("Dynamic.DynamicClass");
if (t == null)
throw new Exception("Method failed to compile.");
var m = t.GetMethod("Invoke");
if (m == null)
throw new Exception("Method failed to compile.");
return (TDelegate)(object)Delegate.CreateDelegate(typeof(TDelegate), m);
}
}

Mono Cecil - Initializing a local variable

I have the following method:
public void DoSomething()
{
Console.WriteLine("");
}
I want to modify this code with Mono Cecil. I want to create an instance of a custom class within the method:
public void DoSomething()
{
MyClass instance = new MyClass();
Console.WriteLine("");
}
Currently I use the following code:
var constructorInfo = typeof(MyClass).GetConstructor(new Type[] { });
MethodReference myClassConstructor = targetAssembly.MainModule.Import(constructorInfo);
var processor = targetMethod.Body.GetILProcessor();
var firstInstruction = targetMethod.Body.Instructions[1];
var instructions = new List<Instruction>() {
processor.Create(OpCodes.Newobj, myClassConstructor),
processor.Create(OpCodes.Stloc_0)
};
foreach (var instruction in instructions)
{
processor.InsertBefore(firstInstruction, instruction);
}
After applying those changes, the program is invalid and cannot be executed.
If i use 'IL DASM' to look at the generated code the following statement is missing:
.locals init ([0] class [MyAssembly]MyNamespace.MyClass instance)
The rest of the IL is the same, as if I directly compile the full code.
Any ideas what I am doing wrong?

I have not tried it but by looking at the Cecil Source Code you should first create the local variable which is part of your MethodBody.
MethodBody has a Variables collection which can be filled via
body.Variables.Add( new VariableDefinition(typedef) )
Then you need to call processor.Create(xxx,indexto local variable);
That should do the trick. The only thing I did not see yet how you can get a TypeDefinition out from a Type. I think you need to load the assembly into Mono Cecil first before you can use this approach.