How does C# Compiler and Virtual Machine (.NET Common-Language-Runtime) handle lambda functions in C#? Are they generally translated into normal functions but marked as "anonymous" in a sense in the compiler and then treated as normal functions or is there a different approach?
It probably is too broad. That said, some basic information can be easily provided:
First, "anonymous" simply means that there is no name by which you can refer to the method in your code. Any anonymous method, whether declared using the older delegate syntax or the newer lambda syntax, still winds up being compiled as a method, with an actual name, into some class (either your own or a new one the compiler creates for the purpose…see below). You simply don't have access to the name in your own code. Other than that, it is fundamentally just like any other method.
The main exception to the above is how it will deal with variable capturing. If the anonymous method captures a local variable, then a new class (also hidden from your code) is declared for the purpose and that variable winds up being stored in the class instead of as an actual local variable, and the anonymous method of course winds up being in that class too.
Finally note that the lambda syntax is use not just for anonymous methods, but also to declare instances of the Expression class. In that scenario, some of the handing is similar, but of course you don't get a type compiled into your code the way you would for an anonymous method.
As others have suggested, if you need more detail than that, then StackOverflow probably is not the best place for the answer (or, if it is the best place for the answer, surely it's already been asked and answered and you just need to search for the answer). You can use ildasm.exe, dotPeek, etc. to inspect the actual generated code, you can read the C# specification, or you can search the web for other details.
Related
When it comes to extension methods class names seem to do nothing, but provide a grouping which is what name-spaces do. As soon as I include the namespace I get all the extension methods in the namespace. So my question comes down to this: Is there some value I can get from the extension methods being in the static class?
I realize it is a compiler requirement for them to be put into a static class, but it seems like from an organizational perspective it would be reasonable for it to be legal to allow extension methods to be defined in name-spaces without classes surrounding them. Rephrasing the above question another way: Is there any practical benefit or help in some scenario I get as a developer from having extension methods attached to the class vs. attached to the namespace?
I'm basically just looking to gain some intuition, confirmation, or insight - I suspect it's may be that it was easiest to implement extension methods that way and wasn't worth the time to allow extension methods to exist on their own in name-spaces.
Perhaps you will find a satisfactory answer in Eric Lippert's blog post Why Doesn't C# Implement "Top Level" Methods? (in turn prompted by SO question Why C# is not allowing non-member functions like C++), whence (my emphasis):
I am asked "why doesn't C# implement feature X?" all the time. The
answer is always the same: because no one ever designed, specified,
implemented, tested, documented and shipped that feature. All six of
those things are necessary to make a feature happen. All of them cost
huge amounts of time, effort and money. Features are not cheap, and we
try very hard to make sure that we are only shipping those features
which give the best possible benefits to our users given our
constrained time, effort and money budgets.
I understand that such a general answer probably does not address the
specific question.
In this particular case, the clear user benefit was in the past not
large enough to justify the complications to the language which would
ensue. By restricting how different language entities nest inside each
other we (1) restrict legal programs to be in a common, easily
understood style, and (2) make it possible to define "identifier
lookup" rules which are comprehensible, specifiable, implementable,
testable and documentable.
By restricting method bodies to always be inside a struct or class, we make it easier to reason about the meaning of an unqualified
identifier used in an invocation context; such a thing is always an
invocable member of the current type (or a base type).
To me putting them in the class is all about grouping related functions inside a class. You may have a number of extension methods in the same namespace. If I wanted to write some extension methods for the DirectoryInfo and FileInfo classes I would create two classes in an IO namespace called DirectoryInfoExtensions and FileInfoExtensions.
You can still call the extension methods like you would any other static method. I dont know how the compiler works but perhaps the output assembly if compiled for .net 2 can still be used by legacy .net frameworks. It also means the existing reflection library can work and be used to run extension methods without any changes. Again I am no compiler expert but I think the "this" keyword in the context of an extension method is to allow for syntactical sugar that allows us to use the methods as though they belong to the object.
The .NET Framework requires that every method exist in a class which is within an assembly. A language could allow methods or fields to be declared without an explicitly-specified enclosing class, place all such methods in assembly Fnord into a class called Fnord_TopLevelDefault, and then search the Fnord_TopLevelDefault class of all assemblies when performing method lookup; the CLS specification would have to be extended for this feature to work smoothly for mixed-language projects, however. As with extension methods, such behavior could be CLS compliant if the CLS didn't acknowledge it, since code in a language which didn't use such a feature could use a "free-floating" method Foo in assembly Fnord by spelling it Fnord_TopLevelDefault.Foo, but that would be a bit ugly.
A more interesting question is the extent to which allowing an extension method Foo to be invoked from an arbitrary class without requiring a clearly visible reference to that class is less evil than would be allowing a non-extension static methods to be likewise invoked. I don't think Math.Sqrt(x) is really more readable than Sqrt; even if one didn't want to import Math everywhere, being able to do so at least locally could in some cases improve code legibility considerably.
They can reference other static class members internally.
You should not only consider the consumer side aspect, but also the code maintenance aspect.
Even though intellisense doesn't distinguish with respect to the owner class, the information is still there through tool tips and whatever productivity tools you have added to your IDE. This can easily be used to provide some context for the method in what otherwise would be a flat (and sometimes very long) list.
Consumer wise, bottom line, I do not think it matters much.
I've been playing around with c++ lately and wondered why there were so many global functions. Then I started thinking about programming in c# and how member functions are stored, so I guess my question is if I have a:
public class Foo {
public void Bar() { ... }
}
and then I do something silly like adding 1,000,000 Foo's to a list; does this mean I have 1,000,000 Foo objects sitting in memory each with there own Bar() function? Or does something much more clever happen?
Thanks.
Nope, there is only one instance. All instances of a class point to an object that contains all the instance methods that take an implicit first parameter affectionately called this. When you invoke an instance method on an instance the this pointer for that instance is passed as the first parameter to that method. That is how the method knows all the instance fields and properties for that instance.
For details see CLR via C#.
This is, of course, complicated by virtual methods. CLR via C# will spell out the distinction for you and is highly recommended if you are interested in this subject. Either way, there is still only one instance of each instance method. The issue is just how these methods are resolved.
An instance method is simply a static method with a hidden this parameter.
(virtual methods are a little more complicated)
In C++ a member function doesn't normally require any per-object storage (an exception - virtual functions - is discussed in the next paragraph). Normally, at each point where the function is used the compiler generates CPU-specific machine code to directly call that function, and for inline functions the call may be avoided and the function's affect may be optimally integrated into the caller's code (which can be ~10x faster for small functions such as "getters and setters" that simply read or write one member variable).
For those classes that have one or more virtual functions, each object will have one extra pointer to a per-class table of function pointers and other information. Thus, each object grows by the size of a pointer - typically 4 or 8 bytes.
Addressing your original observation: C++ has more non-member functions (usually in the std namespace), but a namespace serves this purpose better than a class anyway. Indeed, namespaces are effectively logical interfaces for "static" functions and data that can span many "physical" header files. Why should the logical API of a program be compromised by considerations related to physical files and their implications to build times, file-modification-timestamp triggered make tools etc? In trivial cases where the namespace is in one header, C++ could use a class or struct to scope the same declarations, but that's less convenient as it prevents the use of namespace aliases, using namespaces, and Koenig lookup for implicitly searching namespaces matching a function arguments' namespaces - forcing very explicit prefixing at every point of use. It also gives the false impression that the user is intended to instantiate an object from the content.
I know in Ruby can add and modify method of class dynamically in run time. what about other language? what is C# ,can in this language modify or add some method and ... in run time and dynamically?
I think you are looking for prototype inheritance. A list of languages is mentioned in the same wikipedia page.
There is a similar question on SO which you can look up.
Yes, in C# you can add methods at runtime through reflection and the Emitter object.
In C# 4.0 you can even do it in plain C# code with the Expando object. This is arguably closer to the Ruby way (it's practically a carbon copy if I remember correctly) and a lot easier to use.
Edit: All of this applies to all .NET languages, including VB.Net and F#.
The whole point of static type systems like C#'s is that all functionality defined for a particular type is known (and checked) at compile-time.
If you write
foo.jump(42);
the compiler verifies that whatever type foo has, it supports an operation called jump taking an integer parameter.
Recently, C# got the possibility of having dynamically checked objects through the dynamic type, which basically allows what you described in a very limited context, but nevertheless, the overall language is statically typed.
So what's left are dynamic languages like Ruby, where method existence is just checked at run-time (or call-time).
I think JavaScript can change so called prototypes to add methods to objects and basically achive the same thing as Ruby.
Python excels at this operation - here are bunch of examples: Python: changing methods and attributes at runtime
Lisp's object system is also quite dynamic:
http://en.wikipedia.org/wiki/Common_Lisp_Object_System
"CLOS is dynamic, meaning that not only the contents, but also the structure of its objects can be modified at runtime. CLOS supports changing class definitions on-the-fly (even when instances of the class in question already exist) as well as changing the class membership of a given instance through the change-class operator. CLOS also allows one to add, redefine and remove methods at runtime."
in C# 4 you have dynamic object which you can add/modify at run time.
I've spent my professional life as a C# developer. As a student I occasionally used C but did not deeply study it's compilation model. Recently I jumped on the bandwagon and have begun studying Objective-C. My first steps have only made me aware of holes in my pre-existing knowledge.
From my research, C/C++/ObjC compilation requires all encountered symbols to be pre-declared. I also understand that building is a two-step process. First you compile each individual source file into individual object files. These object files might have undefined "symbols" (which generally correspond to the identifiers declared in the header files). Second you link the object files together to form your final output. This is a pretty high-level explanation but it satisfies my curiosity enough. But I'd also like to have a similar high-level understanding of the C# build process.
Q: How does the C# build process get around the need for header files? I'd imagine perhaps the compilation step does two-passes?
(Edit: Follow up question here How do C/C++/Objective-C compare with C# when it comes to using libraries?)
UPDATE: This question was the subject of my blog for February 4th 2010. Thanks for the great question!
Let me lay it out for you. In the most basic sense the compiler is a "two pass compiler" because the phases that the compiler goes through are:
Generation of metadata.
Generation of IL.
Metadata is all the "top level" stuff that describes the structure of the code. Namespaces, classes, structs, enums, interfaces, delegates, methods, type parameters, formal parameters, constructors, events, attributes, and so on. Basically, everything except method bodies.
IL is all the stuff that goes in a method body -- the actual imperative code, rather than metadata about how the code is structured.
The first phase is actually implemented via a great many passes over the sources. It's way more than two.
The first thing we do is take the text of the sources and break it up into a stream of tokens. That is, we do lexical analysis to determine that
class c : b { }
is class, identifier, colon, identifier, left curly, right curly.
We then do a "top level parse" where we verify that the token streams define a grammaticaly-correct C# program. However, we skip parsing method bodies. When we hit a method body, we just blaze through the tokens until we get to the matching close curly. We'll come back to it later; we only care about getting enough information to generate metadata at this point.
We then do a "declaration" pass where we make notes about the location of every namespace and type declaration in the program.
We then do a pass where we verify that all the types declared have no cycles in their base types. We need to do this first because in every subsequent pass we need to be able to walk up type hierarchies without having to deal with cycles.
We then do a pass where we verify that all generic parameter constraints on generic types are also acyclic.
We then do a pass where we check whether every member of every type -- methods of classes, fields of structs, enum values, and so on -- is consistent. No cycles in enums, every overriding method overrides something that is actually virtual, and so on. At this point we can compute the "vtable" layouts of all interfaces, classes with virtual methods, and so on.
We then do a pass where we work out the values of all "const" fields.
At this point we have enough information to emit almost all the metadata for this assembly. We still do not have information about the metadata for iterator/anonymous function closures or anonymous types; we do those late.
We can now start generating IL. For each method body (and properties, indexers, constructors, and so on), we rewind the lexer to the point where the method body began and parse the method body.
Once the method body is parsed, we do an initial "binding" pass, where we attempt to determine the types of every expression in every statement. We then do a whole pile of passes over each method body.
We first run a pass to transform loops into gotos and labels.
(The next few passes look for bad stuff.)
Then we run a pass to look for use of deprecated types, for warnings.
Then we run a pass that searches for uses of anonymous types that we haven't emitted metadata for yet, and emit those.
Then we run a pass that searches for bad uses of expression trees. For example, using a ++ operator in an expression tree.
Then we run a pass that looks for all local variables in the body that are defined, but not used, to report warnings.
Then we run a pass that looks for illegal patterns inside iterator blocks.
Then we run the reachability checker, to give warnings about unreachable code, and tell you when you've done something like forgotten the return at the end of a non-void method.
Then we run a pass that verifies that every goto targets a sensible label, and that every label is targetted by a reachable goto.
Then we run a pass that checks that all locals are definitely assigned before use, notes which local variables are closed-over outer variables of an anonymous function or iterator, and which anonymous functions are in reachable code. (This pass does too much. I have been meaning to refactor it for some time now.)
At this point we're done looking for bad stuff, but we still have way more passes to go before we sleep.
Next we run a pass that detects missing ref arguments to calls on COM objects and fixes them. (This is a new feature in C# 4.)
Then we run a pass that looks for stuff of the form "new MyDelegate(Foo)" and rewrites it into a call to CreateDelegate.
Then we run a pass that transforms expression trees into the sequence of factory method calls necessary to create the expression trees at runtime.
Then we run a pass that rewrites all nullable arithmetic into code that tests for HasValue, and so on.
Then we run a pass that finds all references of the form base.Blah() and rewrites them into code which does the non-virtual call to the base class method.
Then we run a pass which looks for object and collection initializers and turns them into the appropriate property sets, and so on.
Then we run a pass which looks for dynamic calls (in C# 4) and rewrites them into dynamic call sites that use the DLR.
Then we run a pass that looks for calls to removed methods. (That is, partial methods with no actual implementation, or conditional methods that don't have their conditional compilation symbol defined.) Those are turned into no-ops.
Then we look for unreachable code and remove it from the tree. No point in codegenning IL for it.
Then we run an optimization pass that rewrites trivial "is" and "as" operators.
Then we run an optimization pass that looks for switch(constant) and rewrites it as a branch directly to the correct case.
Then we run a pass which turns string concatenations into calls to the correct overload of String.Concat.
(Ah, memories. These last two passes were the first things I worked on when I joined the compiler team.)
Then we run a pass which rewrites uses of named and optional parameters into calls where the side effects all happen in the correct order.
Then we run a pass which optimizes arithmetic; for example, if we know that M() returns an int, and we have 1 * M(), then we just turn it into M().
Then we do generation of the code for anonymous types first used by this method.
Then we transform anonymous functions in this body into methods of closure classes.
Finally, we transform iterator blocks into switch-based state machines.
Then we emit the IL for the transformed tree that we've just computed.
Easy as pie!
I see that there are multiple interpretations of the question. I answered the intra-solution interpretation, but let me fill it out with all the information I know.
The "header file metadata" is present in the compiled assemblies, so any assembly you add a reference to will allow the compiler to pull in the metadata from those.
As for things not yet compiled, part of the current solution, it will do a two-pass compilation, first reading namespaces, type names, member names, ie. everything but the code. Then when this checks out, it will read the code and compile that.
This allows the compiler to know what exists and what doesn't exist (in its universe).
To see the two-pass compiler in effect, test the following code that has 3 problems, two declaration-related problems, and one code problem:
using System;
namespace ConsoleApplication11
{
class Program
{
public static Stringg ReturnsTheWrongType()
{
return null;
}
static void Main(string[] args)
{
CallSomeMethodThatDoesntExist();
}
public static Stringg AlsoReturnsTheWrongType()
{
return null;
}
}
}
Note that the compiler will only complain about the two Stringg types that it cannot find. If you fix those, then it complains about the method-name called in the Main method, that it cannot find.
It uses the metadata from the reference assemblies. That contains a full type declaration, same thing as you'd find in a header file.
It being a two-pass compiler accomplishes something else: you can use a type in one source file before it is declared in another source code file.
It's a 2-pass compiler. http://en.wikipedia.org/wiki/Multi-pass_compiler
All the necessary information can be obtained from the referenced assemblies.
So there are no header files but the compiler does need access to the DLL's being used.
And yes, it is a 2-pass compiler but that doesn't explain how it gets information about library types.
Before I ask my question, please take a look at this example function:
DateTime.TryParse("01/01/2000", out oDate)
Why do I need to specify the out keyword? Shouldn't the compiler know this from the function's definition?
I'm asking this out of pure curiosity in the hope that I will learn something new about the compiler.
I should also clarify that I'm asking about the C# .NET 3.5 compiler in particular.
The out keyword could be implied by the compiler but my understanding is that the C# team decided to make the out keyword explicitly required by the caller of the function to increase visibility as to the nature of the parameter.
The compiler does know, you may not. It's a way of letting you know that the parameter you are passing can change in this function you are passing it to.
It's not about what the compiler knows, it's all about making sure the developer realizes this call can and will change the value of variable X.
A lot of this has it's roots in C++ where a reference value needs no call site monitor. It's impossible to look at a C++ call and know exactly what it will do. Parameters passed by reference and value in C++ have huge differences in semantics.
Yeah the compiler could figure it out, but this way you know that it is going to modify the variable you are passing in.
the C# language has a lot of what I would call safety nets that explicitely tell the programmer what is going on. A couple of examples are:
No fall through in switch statements.
You can't assign a value in an if statement: if(x = 5) throws an error instead of evaluating to true.
http://msdn.microsoft.com/en-us/library/t3c3bfhx(VS.80).aspx
"The out keyword causes arguments to be passed by reference. This is similar to the ref keyword, except that ref requires that the variable be initialized before being passed. To use an out parameter, both the method definition and the calling method must explicitly use the out keyword."
Since the DateTime.TryParse does not require oDate to be initialized, you must pass the out keyword.
OK, I'm not a C# expert, so if I mess up will somebody please correct me?
There's two ways to pass a parameter to a C# function: by value and by reference. The big difference here is whether modifying the parameter inside the function modifies the variable used to call it. This is not something I'd trust the compiler to decide for itself.
Since you want oDate to be a variable passed in from the caller, and changed, you want it passed by reference.
The other question is whether it should be initialized or not. C# likes to catch when variables are used while uninitialized, since that's almost always an error. In this case, you might well just declare what you're passing in, and use TryParse() to give it its first value. This is a perfectly legitimate technique, so the compiler should allow it. This is another thing I wouldn't trust the compiler to get right. (I assume the compiler also checks to make sure an out parameter is initialized before use in TryParse().)
So, "out" serves two purposes. It establishes that the parameter is passed in by reference, and that it is expected to be initialized inside the function. Neither of these can be determined by the compiler.