Question
How does a delegate store a reference to a function? The source code appears to refer to it as an Object, and the manner in which it invokes the method seems redacted from the source code. Can anyone explain how C# is handling this?
Original Post
It seems I'm constantly fighting the abstractions C# imposes on its programmers. One that's been irking me is the obfuscation of Functions/Methods. As I understand it, all methods are in fact anonymous methods assigned to properties of a class. This is the reason why no function is prefixed by a datatype. For example...
void foo() { ... }
... would be written in Javascript as...
Function foo = function():void { ... };
In my experience, Anonymous functions are typically bad form, but here it's replete throughout the language standard. Because you cannot define a function with its datatype (and apparently the implication/handling is assumed by the compiler), how does one store a reference to a method if the type is never declared?
I'm trying very hard to avoid Delegates and its variants (Action & Func), both because...
it is another abstraction from what's actually happening
the unnecessary overhead required to instantiate these classes (which in turn carry their own pointers to the methods being called).
Looking at the source code for the Delegate.cs, it appears to refer to the reference of a function as simply Object (see lines 23-25).
If these really are objects, how are we calling them? According to the delegate.cs trail, it dead-ends on the following path:
Delegate.cs:DynamicInvoke() > DynamicInvokeImpl() > methodinfo.cs:UnsafeInvoke() > UnsafeInvokeInternal() > RuntimeMethodHandle.InvokeMethod() > runtimehandles.cs:InvokeMethod()
internal extern static object InvokeMethod(object target, object[] arguments, Signature sig, bool constructor);
This really doesn't explain how its invoked if indeed the method is an object. It feels as though this is not code at all, and the actual code called has been redacted from source repository.
Your help is appreciated.
Response to Previous Comments
#Amy: I gave an example immediately after that statement to explain what I meant. If a function were prefixed by a datatype, you could write a true anonymous function, and store it as a property to an Object such as:
private Dictionary<string, Function> ops = new Dictionary<string, Function> {
{"foo", int (int a, int b) { return a + b } }
};
As it stands, C# doesn't allow you to write true anonymous functions, and walls that functionality off behind Delegates and Lambda expressions.
#500 Internal server error: I already explained what I was trying to do. I even bolded it. You assume there's any ulterior motive here; I'm simply trying to understand how C# stores a reference to a method. I even provided links to the source code so that others could read the code for themselves and help answer the question.
#Dialecticus: Obviously if I already found the typical answer on Google, the only other place to find the answer I'm looking for would be here. I realize this is outside the knowledge of most C# developers, and that's why I've provided the source code links. You don't have to reply if you don't know the answer.
While I'm not fully understanding your insights about "true anonymous functions", "not prefixed by a data type" etc, I can explain you how applications written in C# call methods.
First of all, there is no such a thing "function" in C#. Each and every executable entity in C# is in fact a method, that means, it belongs to a class. Even if you define lambdas or anonymous functions like this:
collection.Where(item => item > 0);
the C# compiler creates a compiler-generated class behind the scenes and puts the lambda body return item > 0 into a compiler-generated method.
So assuming you have this code:
class Example
{
public static void StaticMethod() { }
public void InstanceMethod() { }
public Action Property { get; } = () => { };
}
static class Program
{
static void Main()
{
Example.StaticMethod();
var ex = new Example();
ex.InstanceMethod();
ex.Property();
}
}
The C# compiler will create an IL code out of that. The IL code is not executable right away, it needs to be run in a virtual machine.
The IL code will contain a class Example with two methods (actually, four - a default constructor and the property getter method will be automatically generated) and a compiler-generated class containing a method whose body is the body of the lambda expression.
The IL code of Main will look like this (simplified):
call void Example::StaticMethod()
newobj instance void Example::.ctor()
callvirt instance void Example::InstanceMethod()
callvirt instance class [mscorlib]System.Action Example::get_Prop()
callvirt instance void [mscorlib]System.Action::Invoke()
Notice those call and callvirt instructions: these are method calls.
To actually execute the called methods, their IL code needs to be compiled into machine code (CPU instructions). This occurs in the virtual machine called .NET Runtime. There are several of them like .NET Framework, .NET Core, Mono etc.
A .NET Runtime contains a JIT (just-in-time) compiler. It converts the IL code to the actually executable code during the execution of your program.
When the .NET Runtime first encounters the IL code "call method StaticMethod from class Example", it first looks in the internal cache of already compiled methods. When there are no matches (which means this is the first call of that method), the Runtime asks the JIT compiler to create such a compiled-and-ready-to-run method using the IL code. The IL code is converted into a sequence of CPU operations and stored in the process' memory. A pointer to that compiled code is stored in the cache for future reuse.
This all will happen behind the call or callvirt IL instructions (again, simplified).
Once this happened, the Runtime is ready to execute the method. The CPU gets the compiled code's first operation address as the next operation to execute and goes on until the code returns. Then, the Runtime takes over again and proceeds with next IL instructions.
The DynamicInvoke method of the delegates does the same thing: it instructs the Runtime to call a method (after some additional arguments checks etc). The "dead end" you mention RuntimeMethodHandle.InvokeMethod is an intrinsic call to the Runtime directly. The parameters of this method are:
object target - the object on which the delegate invokes the instance method (this parameter).
object[] arguments - the arguments to pass to the method.
Signature sig - the actual method to call, Signature is an internal class that provides the connection between the managed IL code and native executable code.
bool constructor - true if this is a constructor call.
So in summary, methods are not represented as objects in C# (while you of course can have a delegate instance that is an object, but it doesn't represent the executable method, it rather provides an invokable reference to it).
Methods are called by the Runtime, the JIT compiler makes the methods executable.
You cannot define a global "function" outside of classes in C#. You could get a direct native pointer to the compiled (jitted) method code and probably even call it manually by directly manipulating own process' memory. But why?
You clearly misunderstand main differences between script languages, C/C++ and C#.
I guess the main difficulty is that there is no such thing as a function in C#. At all.
C#7 introduced the new feature "a local function", but that is not what a function in JS is.
All pieces of code are methods.
That name is intentionally different from function or a procedure to emphasize the fact that all executable code in C# belongs to a class.
Anonymous methods and lambdas are just a syntax sugar.
A compiler will generate a real method in the same (or a nested) class, where the method with anonymous method declaration belongs to.
This simple article explains it. You can take the examples, compile them and check the generated IL code yourself.
So all the methods (anonymous or not) do belong to a class. It's impossible to answer your updated question, besides saying It does not store a reference to a function, as there is no such thing in C#.
How does one store a reference to a method?
Depending on what you mean by reference, it can be either
An instance of MethodInfo class, used to reference reflection information for a method,
RuntimeMethodHandle (obtainable via RuntimeMethodInfo.MethodHandle) stores a real memory pointer to a JITed method code
A delegate, that is very different from just a memory pointer, but logically could be used to "pass a method reference to another method" .
I believe you are looking for the MethodInfo option, it has a MethodInfo.Invoke method which is very much alike Function..apply function in JS. You have already seen in the Delegate source code how that class is used.
If by "reference" you mean the C-style function pointer, it is in RuntimeMethodHandle struct. You should never use it without solid understanding how a particular .Net platform implementation and a C# compiler work.
Hopefully it clarifies things a bit.
A delegate is simply a pointer(memory location to jump to) to a method with the specified parameters and return type. Any Method that matches the signature(Parameters and return type) is eligible to fulfill the role, irrespective of the defined object. Anonymous simply means the delegate is not named.
Most times the type is implied(if it is not you will get a compiler error):
C# is a strongly typed language. That means every expression (including delegates) MUST have a return type(including void) as well as strongly typed parameters(if any). Generics were created to permit explicit types to be used within general contexts, such as Lists.
To put it another way, delegates are the type-safe managed version of C++ callbacks.
Delegates are helpful in eliminating switch statements by allowing the code to jump to the proper handler without testing any conditions.
A delegate is similar to a Closure in Javascript terminology.
In your response to Amy, you are attempting to equate a loosely typed language like JS, and a strongly typed language C#. In C# it is not possible to pass an arbitrary(loosely-typed) function anywhere. Lambdas and delegates are the only way to guarantee type safety.
I would recommend trying F#, if you are looking to pass functions around.
EDIT:
If you are trying to mimic the behavior of Javascipt, I would try looking at using inheritance through Interfaces. I can mimic multiple inheritance, and be type safe at the same time. But, be aware that it cannot fully supplant Javascript's dependency injection model.
As you probably found out C# doesn't have the concept of a function as in your JavaScript example.
C# is a statically typed language and the only way you can use function pointers is by using the built in types (Func,Action) or custom delegates.(I'm talking about safe,strongly typed pointers)
Javascript is a dynamic language that's why you can do what you describe
If you are willing to lose type safety, you can use the "dynamic" features of C# or refection to achieve what you want like in the following examples (Don't do this,use Func/Action)
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
namespace ConsoleApp1
{
class Program
{
private static Dictionary<string, Func<int, int, int>> FuncOps = new Dictionary<string, Func<int, int, int>>
{
{"add", (a, b) => a + b},
{"subtract", (a, b) => a - b}
};
//There are no anonymous delegates
//private static Dictionary<string, delegate> DelecateOps = new Dictionary<string, delegate>
//{
// {"add", delegate {} }
//};
private static Dictionary<string, dynamic> DynamicOps = new Dictionary<string, dynamic>
{
{"add", new Func<int, int, int>((a, b) => a + b)},
{"subtract", new Func<int, int, int>((a, b) => a - b)},
{"inverse", new Func<int, int>((a) => -a )} //Can't do this with Func
};
private static Dictionary<string, MethodInfo> ReflectionOps = new Dictionary<string, MethodInfo>
{
{"abs", typeof(Math).GetMethods().Single(m => m.Name == "Abs" && m.ReturnParameter.ParameterType == typeof(int))}
};
static void Main(string[] args)
{
Console.WriteLine(FuncOps["add"](3, 2));//5
Console.WriteLine(FuncOps["subtract"](3, 2));//1
Console.WriteLine(DynamicOps["add"](3, 2));//5
Console.WriteLine(DynamicOps["subtract"](3, 2));//1
Console.WriteLine(DynamicOps["inverse"](3));//-3
Console.WriteLine(ReflectionOps["abs"].Invoke(null, new object[] { -1 }));//1
Console.ReadLine();
}
}
}
one more example that you shouldn't use
delegate object CustomFunc(params object[] paramaters);
private static Dictionary<string, CustomFunc> CustomParamsOps = new Dictionary<string, CustomFunc>
{
{"add", parameters => (int) parameters[0] + (int) parameters[1]},
{"subtract", parameters => (int) parameters[0] - (int) parameters[1]},
{"inverse", parameters => -((int) parameters[0])}
};
Console.WriteLine(CustomParamsOps["add"](3, 2)); //5
Console.WriteLine(CustomParamsOps["subtract"](3, 2)); //1
Console.WriteLine(CustomParamsOps["inverse"](3)); //-3
I will provide a really short and simplified answer compared to the others. Everything in C# (classes, variables, properties, structs, etc) has a backed with tons of things your programs can hook into. This network of backend stuff slightly lowers the speed of C# when compared to "deeper" languages like C++, but also gives programmers a lot more tools to work with and makes the language easier to use. In this backend is included things like "garbage collection," which is a feature that automatically deletes objects from memory when there are no variables left that reference them. Speaking of reference, the whole system of passing objects by reference, which is default in C#, is also managed in the backend. In C#, Delegates are possible because of features in this backend that allow for something called "reflection."
From Wikipedia:
Reflection is the ability of a computer program to examine,
introspect, and modify its own structure and behavior at runtime.
So when C# compiles and it finds a Delegate, it is just going to make a function, and then store a reflective reference to that function in the variable, allowing you to pass it around and do all sorts of cool stuff with it. You aren't actually storing the function itself in the variable though, you are storing a reference, which is kinda like an address that points you to where the function is stored in RAM.
I have C++ wrapped in C dll. The dll is called in my C# project.
In my wrapper functions I call a lot of Singletons, they are setup as follows:
ComponentManager &ComponentManager::_cmpManager()
{
static ComponentManager ONLY_ONE;
return ONLY_ONE;
}
The above function is a static function inside my ComponentManager class.
Here is the specific problem:
bool createNewEntity(char *c)
{
if (ComponentManager::_cmpManager().nameAvailable(c))
{
Entity e(c);
Transform t;
ComponentManager::_cmpManager().addComponent(c, t);
SceneNode sc(CMP_MANAGER2.getComponent<Transform>(c));
SCENE_MANAGER.addSceneNode(sc, e.entityName);
return true;
}
return false;
}
Essentially what this does is the singleton has a Hash Map with a key type string, this function checks to see if this key already exists. The behaviour is always returning true. When I use a global object of type componentManager instead of the singleton it behaves correctly, so something is telling me the singleton keeps leaving scope and deleting itself. Also if I use the singleton in an application exe rather than a dll it behaves correctly. So I have 2 questions,
Is there a way to keep my singleton from traveling out of scope? If
not.
Is there another way of setting up singletons to not be deleted
after leaving scope?
C++ static keyword is a bit different from C# static.
See https://msdn.microsoft.com/en-us/library/y5f6w579.aspx for description.
In item 2 there it says: 2. When you declare a variable in a function, the static keyword specifies that the variable retains its state between calls to that function.
Try to declare your static not inside the method but in class scope (as per item 3 in above reference).
3. When you declare a data member in a class declaration, the static keyword specifies that one copy of the member is shared by all instances of the class. A static data member must be defined at file scope. An integral data member that you declare as const static can have an initializer.
You will also need to declare that static member at file scope.
Why it works when it is not in a dll -- it is not exactly clear. Probably a peculiar behaviour of linker. If class is declared in a dll it probably tries to instantiate class every time and your static inside the function is a new one every time. But if class is inside the exe file it is somehow the same class every time and when you call your method _cmpManager() it always accesses the same instance of the class.
Just my two cents :-).
I got curious as to where Type.GetType() is implemented, so I took a peek at the assembly and noticed Type.GetType() calls base.GetType() and since Type inherits from MemberInfo I took a look and it is defined as _MemberInfo.GetType() which returns this.GetType(). Since I cannot find the actual code that shows how C# can get type information I would like to know:
How does the CLR get Type and MemberInfo from objects at Runtime?
The ACTUAL source for .NET Framework 2.0 is available on the internet (for educational purposes) here: http://www.microsoft.com/en-us/download/details.aspx?id=4917
This is the C# Language implementation. You can use 7zip to unpack it. You will find the reflection namespace here (relatively):
.\sscli20\clr\src\bcl\system\reflection
I am digging for the specific implementation you are asking about, but this is a good start.
UPDATE: Sorry, but I think its a dead end. Type.GetType() calls to the base implementation which comes from System.Object. If you inspect that codefile (.\sscli20\clr\src\bcl\system\object.cs) you will find the method is extern (see code below). Further inspect could uncover the implementation, but its not in the BCL. I suspect it will be in C++ code somewhere.
// Returns a Type object which represent this object instance.
//
[MethodImplAttribute(MethodImplOptions.InternalCall)]
public extern Type GetType();
UPDATE (AGAIN): I dug deeper and found the answer in the implementation of the CLR virtual machine itself. (Its in C++).
The first piece of puzzle is here:
\sscli20\clr\src\vm\ecall.cpp
Here we see the code that maps the external call to an C++ function.
FCFuncStart(gObjectFuncs)
FCIntrinsic("GetType", ObjectNative::GetClass, CORINFO_INTRINSIC_Object_GetType)
FCFuncElement("InternalGetHashCode", ObjectNative::GetHashCode)
FCFuncElement("InternalEquals", ObjectNative::Equals)
FCFuncElement("MemberwiseClone", ObjectNative::Clone)
FCFuncEnd()
Now, we need to go find ObjectNative::GetClass ... which is here:
\sscli20\clr\src\vm\comobject.cpp
and here is the implementation of GetType:
FCIMPL1(Object*, ObjectNative::GetClass, Object* pThis)
{
CONTRACTL
{
THROWS;
SO_TOLERANT;
DISABLED(GC_TRIGGERS); // FCallCheck calls ForbidenGC now
INJECT_FAULT(FCThrow(kOutOfMemoryException););
SO_TOLERANT;
MODE_COOPERATIVE;
}
CONTRACTL_END;
OBJECTREF objRef = ObjectToOBJECTREF(pThis);
OBJECTREF refType = NULL;
TypeHandle typeHandle = TypeHandle();
if (objRef == NULL)
FCThrow(kNullReferenceException);
typeHandle = objRef->GetTypeHandle();
if (typeHandle.IsUnsharedMT())
refType = typeHandle.AsMethodTable()->GetManagedClassObjectIfExists();
else
refType = typeHandle.GetManagedClassObjectIfExists();
if (refType != NULL)
return OBJECTREFToObject(refType);
HELPER_METHOD_FRAME_BEGIN_RET_ATTRIB_2(Frame::FRAME_ATTR_RETURNOBJ, objRef, refType);
if (!objRef->IsThunking())
refType = typeHandle.GetManagedClassObject();
else
refType = CRemotingServices::GetClass(objRef);
HELPER_METHOD_FRAME_END();
return OBJECTREFToObject(refType);
}
FCIMPLEND
One last thing, the implementation of GetTypeHandle along with some other supporting functions can be found in here:
\sscli20\clr\src\vm\object.cpp
The most significant parts of reflection are implemented as part of the CLI itself. As such, you could look at either the MS CLI reference source (aka "Rotor"), or the mono source. But: it will mostly be C/C++. The public API implementation details (MethodInfo, Type etc) may be C#.
It might not answer you question directly. However, here is a little outline of how managed code knows everything about types.
Whenever you compile code the compiler analyzes/parses the source files and collects information it encounters. For example take a look at class below.
class A
{
public int Prop1 {get; private set;}
protected bool Met2(float input) {return true;}
}
The compiler can see that this is an internal class with two members. Member one is a property of type int with private setter. Member 2 is a protected method with name Met2 and type boolean that takes float input (input name is 'input'). So, it has all this information.
It stores this information in the assembly. There are a couple of tables. For example classes (types) all leave in one table, methods live in another table. Think in turms of SQL tables, though they are definitely are not.
When a user (developer) wants to know information about a type it calls GetType method. This method relies on objects hidden field - type object pointer. This object is basically a pointer to a class table. Each class table will have a pointer to the first method in methods table. Each method record will have a pointer to the first parameter in the parameters table.
PS: this mechamism is key to making .NET assemblies more secure. You cannot replace pointers to methods. It will break the signature of the assebmly.
JIT compilation relies heavily on this tables as well
As #GlennFerrieLive points out, the call to GetType is an InternalCall which means the implementation is within the CLR itself and not in any of the BCL.
My understanding is that the internal CLR method takes the runtime type information from the this pointer, which basically amounts to the name of the type. It then look up the complete type information from the metadata present in all loaded assemblies (presumably, in the current appdomain), which is what makes reflection rather expensive. The metadata area is basically a database of all the types and members present in the assembly and it constructs an instance of Type or Method|Property|FieldInfo from this data.
We have a helper assembly that assists developers with logging information. The specific method used for logging has two signatures:
LogToULS(string message)
LogToULS(string message, Microsoft.Sharepoint.Administration.SPDiagnosticsCategory category)
In my application I have created a static class that contains an instance of the SPDiagnosticsCategory that I would like to be used any time the application logs something. If the first signature is used, a generic category is assigned and it is harder to find logged information specific to this application.
My question is if it's possible to force people to use the second signature any time LogToULS is called from this application or does this need to be accomplished through programmer education?
If you can't remove the method from the codebase, you could mark it as deprecated, so other programmers get a compiler warning whenever they call it (and IntelliSense will warn against its usage):
[Obsolete("Use LogToULS(string, SPDiagnosticsCategory) instead."]
public void LogToULS(string message)
{
// ...
}
As per the ObsoleteAttribute documentation, you can pass true as a second parameter to its constructor to cause a compiler error (not just a warning), but this may break existing code.
Define the first overload as:
public void LogToULS(string message)
{
LogToULS(message, genericErrorCodeHere);
}
Then, make Resharper inline this method so all existing call sites get updated. Next, delete the old, obsolete overload.
Is there a way via System.Reflection, System.Diagnostics or other to get a reference to the actual instance that is calling a static method without passing it in to the method itself?
For example, something along these lines
class A
{
public void DoSomething()
{
StaticClass.ExecuteMethod();
}
}
class B
{
public void DoSomething()
{
SomeOtherClass.ExecuteMethod();
}
}
public class SomeOtherClass
{
public static void ExecuteMethod()
{
// Returns an instance of A if called from class A
// or an instance of B if called from class B.
object caller = getCallingInstance();
}
}
I can get the type using System.Diagnostics.StackTrace.GetFrames, but is there a way to get a reference to the actual instance?
I am aware of the issues with reflection and performance, as well as static to static calls, and that this is generally, perhaps even almost univerally, not the right way to approach this. Part of the reason of this question is I was curious if it was doable; we are currently passing the instance in.
ExecuteMethod(instance)
And I just wondered if this was possible and still being able to access the instance.
ExecuteMethod()
#Steve Cooper:
I hadn't considered extension methods. Some variation of that might work.
Consider making the method an extension method. Define it as:
public static StaticExecute(this object instance)
{
// Reference to 'instance'
}
It is called like:
this.StaticExecute();
I can't think of a way to do what you want to do directly, but I can only suggest that if you find something, you watch out for static methods, which won't have one, and anonymous methods, which will have instances of auto-generated classes, which will be a little odd.
I do wonder whether you should just pass the invoking object in as a proper parameter. After all, a static is a hint that this method doesn't depend on anything other than its input parameters. Also note that this method may be a bitch to test, as any test code you write will not have the same invoking object as the running system.
I do not believe you can. Even the StackTrace and StackFrame classes just give you naming information, not access to instances.
I'm not sure exactly why you'd want to do this, but know that even if you could do it it would likely be very slow.
A better solution would be to push the instance to a thread local context before calling ExecuteMethod that you can retrieve within it or just pass the instance.
In the case of a static method calling your static method, there is no calling instance.
Find a different way to accomplish whatever you are trying to do.
Just have ExecuteMethod take an object. Then you have the instance no matter what.
I feel like I'm missing something, here. The static method can be called from literally anywhere. There's no guarantee that a class A or class B instance will appear anywhere in the call stack.
There's got to be a better way to accomplish whatever you're trying to do.