We Keep Coding

How do I create mutually callable MethodInfos from MethodBuilders created from LambdaExpressions?

I'm currently working on an compiler in C#, where the behaviour is defined by LambdaExpressions, and then using CompileToMethod, transformed into MethodBuilders and saved to DLL. All functions are public and static.
However, I could not find a way to extract usable MethodInfo (or another method of reference) from the MethodBuilder until the behaviour is defined and declaring type is created/sealed. That means that at that until that point, it is impossible to use Expression.Call to call these functions. That makes self-recursion or mutual referencing between two functions impossible.
I ended up using Reflection to invoke the functions at runtime, but it's very suboptimal, and I'm still curious if there's a better way.
How do i ensure functions created with LambdaExpression.CompileToMethod(MethodBuilder) can self-call?
Alternatively, is there any other way to use LambdaExpressions which would allow this and support saving as a static method to a dll?

I hope this helps.
This is complete code example which produces runtime defined type with single static recursive method.
For the simplicity of the example the recursive method is infinite - at the end of the Main method the recursive method is called
static void Main(string[] args)
{
var moduleBuilder = CreateDynamicModuleBuilder();
var typeBuilder = moduleBuilder.DefineType("Person", TypeAttributes.Public | TypeAttributes.Class);
var methodBuilder = typeBuilder.DefineMethod("SayHello", MethodAttributes.Static | MethodAttributes.Public);
var methodExpression = CreateRecursiveExpression();
var lambda = Expression.Lambda(methodExpression);
lambda.CompileToMethod(methodBuilder);
var typeInfo = typeBuilder.CreateType();
var methodInfo = typeInfo.GetMethod("SayHello", BindingFlags.Public | BindingFlags.Static);
methodInfo.Invoke(null, null);
}
private static Expression CreateRecursiveExpression()
{
var methodInfo = typeof(Console).GetMethod("WriteLine", new[] { typeof(String) });
var arg = Expression.Constant("Hello");
var consoleCall = Expression.Call(methodInfo, arg);
var sayHelloActionVariable = Expression.Variable(typeof(Action), "sayHelloAction");
var block = Expression.Block(
new[] { sayHelloActionVariable },
Expression.Assign(
sayHelloActionVariable,
Expression.Lambda(
Expression.Block(
consoleCall,
Expression.Invoke(sayHelloActionVariable)
)
)
),
Expression.Invoke(sayHelloActionVariable)
);
return block;
}
private static ModuleBuilder CreateDynamicModuleBuilder()
{
var name = new AssemblyName("Example.DynamicRecursion");
var am = AssemblyBuilder.DefineDynamicAssembly(name, AssemblyBuilderAccess.RunAndSave);
var mb = am.DefineDynamicModule(name.Name, $"{name.Name}.dll");
return mb;
}
This code will create type with the following signature
public class Person
{
public static void SayHello()
{
Action sayHelloAction;
sayHelloAction = () =>
{
Console.WriteLine("Hello");
sayHelloAction();
}
sayHelloAction();
}
}

When implementing an interface that has a method with 'in' parameter by TypeBuilder.CreateType, TypeLoadException is thrown

Before beginning, this is my first question on SO. So there might be faults or lack of information about the problem. Please let me know if there's something that I need to correct. Thanks.
Using TypeBuilder, I'm building a class that implements an interface that contains a method. After implementing that method with ILGenerator, then I call TypeBuilder.CreateType() and everything goes well in the normal case.
But if the method contains any parameter with the in modifier, also known as readonly reference for value types, TypeBuilder.CreateType() throws TypeLoadException("Method 'SomeMethod' ... does not have an implementation.").
Unlike the usual case of TypeLoadException that implemented method with the same signature as the one declared in the interface(s) doesn't exist, this problem is raised only when the method contains in parameter(s) even signatures are the same. When I remove or change the in modifier to ref or out, TypeBuilder.CreateType() successfully recognizes the generated method as an implementation of one declared in the interface, and the type is built normally.
Here's a fully compilable example:
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Reflection;
using System.Reflection.Emit;
using System.Threading;
namespace EmitMethodWithInParamTest
{
public struct StructParam
{
public String Data;
}
public interface ISomeInterface
{
Int32 SomeMethod(in StructParam param);
}
static class EmitExtension
{
public static void ReplicateCustomAttributes(this ParameterBuilder paramBuilder, ParameterInfo paramInfo)
{
foreach (var attrData in paramInfo.GetCustomAttributesData())
{
var ctorArgs = attrData.ConstructorArguments.Select(arg => arg.Value).ToArray();
// Handling variable arguments
var ctorParamInfos = attrData.Constructor.GetParameters();
if (ctorParamInfos.Length > 0 &&
ctorParamInfos.Last().IsDefined(typeof(ParamArrayAttribute)) &&
ctorArgs.Last() is IReadOnlyCollection<CustomAttributeTypedArgument> variableArgs)
{
ctorArgs[ctorArgs.Length - 1] = variableArgs.Select(arg => arg.Value).ToArray();
}
var namedPropArgs = attrData.NamedArguments.Where(arg => !arg.IsField);
var namedPropInfos = namedPropArgs.Select(arg => (PropertyInfo)arg.MemberInfo).ToArray();
var namedPropValues = namedPropArgs.Select(arg => arg.TypedValue.Value).ToArray();
var namedFieldArgs = attrData.NamedArguments.Where(arg => arg.IsField);
var namedFieldInfos = namedFieldArgs.Select(arg => (FieldInfo)arg.MemberInfo).ToArray();
var namedFieldValues = namedFieldArgs.Select(arg => arg.TypedValue.Value).ToArray();
var attrBuilder = new CustomAttributeBuilder(attrData.Constructor,
ctorArgs, namedPropInfos, namedPropValues, namedFieldInfos, namedFieldValues);
paramBuilder.SetCustomAttribute(attrBuilder);
}
}
}
class Program
{
static Program()
{
Thread.CurrentThread.CurrentUICulture = CultureInfo.GetCultureInfo("en-us");
}
static void Main(String[] args)
{
var assemblyBuilder = AssemblyBuilder.DefineDynamicAssembly(new AssemblyName("DynamicAssembly"), AssemblyBuilderAccess.Run);
var moduleBuilder = assemblyBuilder.DefineDynamicModule("MainModule");
var typeBuilder = moduleBuilder.DefineType("SomeClass",
TypeAttributes.Public | TypeAttributes.Class | TypeAttributes.AutoClass | TypeAttributes.AnsiClass | TypeAttributes.BeforeFieldInit | TypeAttributes.AutoLayout,
null /*base class*/,
new[] { typeof(ISomeInterface) });
var methodInfoToImpl = typeof(ISomeInterface).GetMethod(nameof(ISomeInterface.SomeMethod));
var paramInfos = methodInfoToImpl.GetParameters();
var methodBuilder = typeBuilder.DefineMethod(methodInfoToImpl.Name,
MethodAttributes.Public | MethodAttributes.Virtual | MethodAttributes.Final,
CallingConventions.HasThis,
methodInfoToImpl.ReturnType,
paramInfos.Select(pi => pi.ParameterType).ToArray());
foreach (var paramInfo in paramInfos)
{
// paramInfo.Position is zero-based but DefineParameter requires 1-based index.
var paramBuilder = methodBuilder.DefineParameter(paramInfo.Position + 1, paramInfo.Attributes, paramInfo.Name);
if (paramInfo.Attributes.HasFlag(ParameterAttributes.HasDefault))
{
paramBuilder.SetConstant(paramInfo.DefaultValue);
}
paramBuilder.ReplicateCustomAttributes(paramInfo);
}
// Dummy implementation for example. Always throws NotImplementedException.
var ilGen = methodBuilder.GetILGenerator();
ilGen.Emit(OpCodes.Newobj, typeof(NotImplementedException).GetConstructor(Type.EmptyTypes));
ilGen.Emit(OpCodes.Throw);
var builtType = typeBuilder.CreateType(); // <- TypeLoadException("Method 'SomeMethod' in type 'SomeClass' from assembly 'DynamicAssembly, ...' does not have an implementation.") is thrown.
var generatedObj = (ISomeInterface)Activator.CreateInstance(builtType);
var someParam = new StructParam() { Data = "SomeData" };
var result = generatedObj.SomeMethod(in someParam); // <- NotImplementedException expected by dummy implementation if executed.
Console.WriteLine($"Result: {result}");
}
}
}
This code is also uploaded to Pastebin.
While digging down this problem, I found that the in parameter has two custom attributes, InteropServices.InAttribute and CompilerServices.IsReadOnlyAttribute. But when I generate a method without implementing the interface (this succeeds normally because no signature matching required), in parameter of generated method has only one custom attribute, InAttribute. So I replicated all custom attributes of parameters from the interface, but still TypeLoadException is being raised.
I've tested this on .NET Framework 4.6.1 and .NET Core 2.2 with C# 7.2 and 7.3. And all environments gave me the same exception. I'm using Visual Studio 2017 on Windows.
Is there anything that I have missed or are there any workarounds?
Thank you for any help in advance.

After writing the question above, I've been investigated built binary of sample code in IL and source code of CoreCLR for a few days, and now I found the problem and solution.
In short, required and optional custom modifiers of return type and each parameter type take a part of method signature like each types do, and it had to be replicated manually. I thought that it will be done by passing ParameterAttributes.In to MethodBuilder.DefineParameter and replicating the custom attribute InAttribute, but it was wrong.
And, among in, ref and out modifiers, only in emits a required custom modifier to specified parameter. In contrast, ref and out are represented only with their type itself. This is the reason why only in didn't work as expected.
To replicate custom modifiers, call to TypeBuilder.DefineMethod need be modified like this:
var methodBuilder = typeBuilder.DefineMethod(methodInfoToImpl.Name,
MethodAttributes.Public | MethodAttributes.Virtual | MethodAttributes.Final,
CallingConventions.HasThis,
methodInfoToImpl.ReturnType,
methodInfoToImpl.ReturnParameter.GetRequiredCustomModifiers(), // *
methodInfoToImpl.ReturnParameter.GetOptionalCustomModifiers(), // *
paramInfos.Select(pi => pi.ParameterType).ToArray(),
paramInfos.Select(pi => pi.GetRequiredCustomModifiers()).ToArray(), // *
paramInfos.Select(pi => pi.GetOptionalCustomModifiers()).ToArray() // *
);
Marked lines with // * are newly added to replicate custom modifiers of return/parameter types.
Or, we can do this by calling MethodBuilder.SetSignature method after calling DefineMethod without any type and custom modifiers arguments. If we decided to call SetSignature separately, we need to call it before any DefineParameter, SetCustomAttribute, Equals(Object), SetImplementationFlags, getter of property Signature and many other methods that call the internal method MethodBuilder.GetMethodSignature() that cache bytes representing method signature.
Thank you for reading and giving me advice. :)

Convert Reflection.Emit to Roslyn

I need to convert an existing code that uses Reflection.Emit to Roslyn.
The code I have currently is basically this:
var assemblyName = new AssemblyName("AssemblyName");
var assemblyBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Save);
var builder = assemblyBuilder.DefineDynamicModule("test", "test.dll");
var type = builder.DefineType("Entry", TypeAttributes.Public, typeof(object), null);
var method = type.DefineMethod("###Start_v1.4.3.0", MethodAttributes.Public | MethodAttributes.HideBySig);
method.SetReturnType(typeof(void));
var generator = method.GetILGenerator();
generator.Emit(OpCodes.Nop);
generator.Emit(OpCodes.Ret);
type.CreateType();
assemblyBuilder.Save(#"test.dll");
As you can see, there is a class named Entry with a method called ###Start_v1.4.3.0.
We're using this for more than 7 years now but evereytime we need to change anything, it's a pain because we need to use those Emits and it's not trivial.
It would be great if we could just have Roslyn to compile the code:
public class Entry
{
public void ###Start_v1.4.3.0()
{
}
}
But it doesn't work due to the method name being invalid.
The compiled dll is used by a third party component and it looks for this class and method name to execute. We tried to reach the developers to have a new version but no luck.
I think Roslyn won't compile this at all, but I believe there might be a way to rename the method name later from let's say just Start() to ###Start_v1.4.3.0()... I just don't know how to do this.
Any help will be very welcome.

If the only problem is the illegal method name, you can easily resolve that issue.
Compile the dll with a legal name, and then you have several ways to change the method name.
With mono.cecil its pretty simple.
public void ChangeMethodName()
{
//Before changing the method name
var assem = Assembly.LoadFile(#"C:\temp\ClassLibrary1.dll");
Console.WriteLine(
assem.GetType("ClassLibrary1.Class1").
GetMethod("Start", BindingFlags.Static | BindingFlags.Public).
Invoke(null, null));
// Change the name
var module = ModuleDefinition.ReadModule(#"C:\temp\ClassLibrary1.dll");
TypeDefinition myType =
module.Types.First(type => type.Name == "Class1");
var method = myType.Methods.First(m => m.Name == "Start");
method.Name = "###Start_v1.4.3.0";
module.Write(#"C:\temp\ClassLibrary1_new.dll");
//After changing the method name
assem = Assembly.LoadFile(#"C:\temp\ClassLibrary1_new.dll");
Console.WriteLine(
assem.GetType("ClassLibrary1.Class1").
GetMethod("###Start_v1.4.3.0",
BindingFlags.Static|BindingFlags.Public).
Invoke(null, null));
}
public class Class1
{
public static string Start()
{
return $"my name is {MethodBase.GetCurrentMethod().Name}";
}
}

Generating IL for Recursive Methods

I tried to generate IL for recursive method using following strategy,
Firstly I defined type using following code snippet
private void InitializeAssembly(string outputFileName)
{
AppDomain appDomain = AppDomain.CurrentDomain;
AssemblyName assemblyName = new AssemblyName(outputFileName);
assemblyBuilder = appDomain.DefineDynamicAssembly(assemblyName,
AssemblyBuilderAccess.Save);
moduleBuilder = assemblyBuilder.DefineDynamicModule(outputFileName, outputFileName + ".exe");
typeBuilder = moduleBuilder.DefineType(typeName, TypeAttributes.Public);
methodBuilder = typeBuilder.DefineMethod("Main",
MethodAttributes.Static | MethodAttributes.Public,
typeof(void),
System.Type.EmptyTypes);
ilGen = methodBuilder.GetILGenerator();
}
Next I started to generate IL for recursive method as given below.
MethodBuilder method = typeBuilder.DefineMethod(
“MethodName”,
MethodAttributes.Static | MethodAttributes.Public,
NodeTypeToDotNetType(func.RetType),
parameters);
ILGenerator ilOfMethod = method.GetILGenerator();
method.DefineParameter();
For calling method itself inside the method body I used following construct,
ilOfMethod.Emit(OpCodes.Call, typeBuilder.GetMethod("MethodName", new System.Type[] {typeof(arg1),typeof(arg2),etc}));
Finally save generated assembly using following method.
private void SaveAssembly(string outputFileName)
{
ilGen.Emit(OpCodes.Ret);
typeBuilder.CreateType();
moduleBuilder.CreateGlobalFunctions();
assemblyBuilder.SetEntryPoint(methodBuilder);
assemblyBuilder.Save(outputFileName + ".exe");
}
Unfortunately this is not working since recursive method calling construct, inside the method returns null. Issue here is that recursive call inside the method ( i.e. ilOfMethod.Emit(OpCodes.Call, typeBuilder.GetMethod("MethodName", new System.Type[] {typeof(arg1),typeof(arg2),etc}));
) returns null. Since we actually create the type inside the SaveAssembly() method, this is acceptable. So my question is that: is it possible to generate IL for recursive methods using above construct? If it is not possible, Please let me know that alternative constructs for generating IL for recursive methods.

I haven't tested it, but if I remember correctly you should be able to simply use the result of DefineMethod to emit the Call instruction:
MethodBuilder method = typeBuilder.DefineMethod("MethodName", ...);
...
ILGenerator ilOfMethod = method.GetILGenerator();
...
ilOfMethod.Emit(OpCodes.Call, method);

How do I get an IL bytearray from a DynamicMethod?

As a bit of a novelty, I'm trying to see how different the IL from light weight code generated at runtime looks vs code generated by the VS compiler, as I noticed that VS code tends to run with a different performance profile for things like casts.
So I wrote the following code::
Func<object,string> vs = x=>(string)x;
Expression<Func<object,string>> exp = x=>(string)x;
var compiled = exp.Compile();
Array.ForEach(vs.Method.GetMethodBody().GetILAsByteArray(),Console.WriteLine);
Array.ForEach(compiled.Method.GetMethodBody().GetILAsByteArray(),Console.WriteLine);
Unfortunately, this throws an exception as GetMethodBody is apparently an illegal operation on code generated by expression trees. How can I in a library manner (i.e. not with an external tool unless the tool has an API) look at the code generated by code using lightweight codegen?
Edit: the error occurs on line 5, compiled.Method.GetMethodBody() throws the exception.
Edit2:
Does anyone know how to recover the local variables declared in the method? Or is there no way to GetVariables?

Yeah, doesn't work, the method is generated by Reflection.Emit. The IL is stored in the MethodBuilder's ILGenerator. You can dig it out but you have to be pretty desperate. Reflection is needed to get to the internal and private members. This worked on .NET 3.5SP1:
using System.Linq.Expressions;
using System.Reflection;
using System.Reflection.Emit;
...
var mtype = compiled.Method.GetType();
var fiOwner = mtype.GetField("m_owner", BindingFlags.Instance | BindingFlags.NonPublic);
var dynMethod = fiOwner.GetValue(compiled.Method) as DynamicMethod;
var ilgen = dynMethod.GetILGenerator();
var fiBytes = ilgen.GetType().GetField("m_ILStream", BindingFlags.Instance | BindingFlags.NonPublic);
var fiLength = ilgen.GetType().GetField("m_length", BindingFlags.Instance | BindingFlags.NonPublic);
byte[] il = fiBytes.GetValue(ilgen) as byte[];
int cnt = (int)fiLength.GetValue(ilgen);
// Dump <cnt> bytes from <il>
//...
On .NET 4.0 you'll have to use ilgen.GetType().BaseType.GetField(...) because the IL generator was changed, DynamicILGenerator, derived from ILGenerator.

The ILReader here should work.
ILVisualizer 2010 Solution

Based off Hans Passant's work I was able to dig a little deeper there appears to be a method that you should call, called BakeByteArray so the following works::
var dynMethod = fiOwner.GetValue(compiled.Method) as DynamicMethod;
var ilgen =dynamicMethod.GetILGenerator();
byte[] il = ilgen.GetType().GetMethod("BakeByteArray", BindingFlags.NonPublic | BindingFlags.Instance).Invoke(ilgen, null) as byte[];
This certainly helps, but I still have no way to resolve VariableInfo's just yet which is something that would help in my work.

The current solutions here aren't addressing the current situation in .NET 4 very well. You can use either DynamicILInfo or ILGenerator to create the dynamic method, but the solutions listed here do not work with DynamicILInfo dynamic methods at all.
Whether you use the DynamicILInfo method of generating IL or the ILGenerator method, the IL bytecode ends up in DynamicMethod.m_resolver.m_code. You don't have to check both methods and it's a less complex solution.
This is the version you should be using:
public static byte[] GetILBytes(DynamicMethod dynamicMethod)
{
var resolver = typeof(DynamicMethod).GetField("m_resolver", BindingFlags.Instance | BindingFlags.NonPublic).GetValue(dynamicMethod);
if (resolver == null) throw new ArgumentException("The dynamic method's IL has not been finalized.");
return (byte[])resolver.GetType().GetField("m_code", BindingFlags.Instance | BindingFlags.NonPublic).GetValue(resolver);
}
See this answer for more helper methods and a solution for the DynamicMethod token resolution issue.

I just merged #Hans Passant and #jnm2 solutions into a extension method and added useful comments:
public static byte[] GetIlAsByteArray(this DynamicMethod dynMethod)
{
// First we try to retrieve the value of "m_resolver" field,
// which will always be null unless the dynamic method is completed
// by either calling 'dynMethod.CreateDelegate()' or 'dynMethod.Invoke()' function.
// Source: https://learn.microsoft.com/en-us/dotnet/api/system.reflection.emit.dynamicmethod.getilgenerator
// (Remarks section)
// Note that the dynamic method object does not know when it is ready for use
// since there is not API which indicates that IL generation has completed.
// Source: https://referencesource.microsoft.com/#mscorlib/system/reflection/emit/dynamicmethod.cs,7fc135a2ceea0854,references
// (Comment lines)
// So, if the dynamic method is not completed, we will retrieve the "m_ILStream" field instead.
// The only difference I notice between "m_resolver" and "m_ILStream" fields is that the IL bytes in "m_ILStream"
// will have trailing zeros / null bytes depending on the amount of unused bytes in this buffer.
// ( The buffer size for "m_ILStream" is allocated by a call to 'dynMethod.GetILGenerator(streamSize)' function. )
BindingFlags bindingFlags = bindingFlags.Instance | bindingFlags.NonPublic;
object fiResolver = typeof(DynamicMethod).GetField("m_resolver", bindingFlags).GetValue(dynMethod);
if (fiResolver == null)
{
ILGenerator ilGen = dynMethod.GetILGenerator();
FieldInfo fiIlStream = null;
// Conditional for .NET 4.x because DynamicILGenerator class derived from ILGenerator.
// Source: https://stackoverflow.com/a/4147132/1248295
if (Environment.Version.Major >= 4)
{
fiIlStream = ilGen.GetType().BaseType.GetField("m_ILStream", bindingFlags);
}
else // This worked on .NET 3.5
{
fiIlStream = ilGen.GetType().GetField("m_ILStream", bindingFlags);
}
return fiIlStream.GetValue(ilGen) as byte[];
}
else
{
return (byte[])(fiResolver.GetType().GetField("m_code", bindingFlags).GetValue(fiResolver));
}
}