We Keep Coding

Writing and changing code at runtime

I have a situation where I'd like to build MVC style views at runtime using their EditorFor/DisplayFor templates (or something similar).
Ideally our application would let the user choose which fields they want in their UI (so they can add /remove any as they see fit), to this end I thinking it'd be handy to be create viewmodel classess at runtime and add the various dataannotation attributes to them according to what user selects (ie. stringlength, required etc).
One thing I need to be able to support is changing of the generated classes at runtime without affecting other users or having to do a full iisreset.
To go about this I've been doing a bit of research and it seems like there might be 3 different approaches, CodeDom, RunSharp / Relfection.Emit,Roslyn.
From what I can tell reflection.Emit/Runsharp would allow me to create the classes and add attibutes and properties to them at runtime and probably also modify them when I need to without adverse effects.
I'm not sure if Roslyn would allow this, I haven't been able to track down any simple examples of creating a class with properties or attributes in it, and I've seen a few mentions that Roslyn's output is immutable so I'm not sure how that goes for allowing me to modify it at a later date without adverse effects.
In general from what I've seen most people don't recommend CodeDom so I'm not entirely sure if I should bother going down that route.
Can anyone give me an idea of which of these directions might be viable for me?

So, none of these solutions are going to work, and honestly, generating type at runtime really isn't what you want here.
When it comes to the CLR, once you have a type with fields and methods, you can't really add new members or change members at runtime. The closest we come to doing that is the edit-and-continue features in Visual Studio, we're highly restricted to what changes we can make. We often 'cheat' by not adding methods or attributes where you think they added, but we hide them somewhere else and emit IL that references this secret location when you make an edit. Crazy things like removing members is entirely unsupported. Even if it was supported, lots of code likes to presume that doing someObject.GetType().GetMembers() returns the same thing over and over again.
As far as Roslyn is concerned, when we say the results are "immutable" we don't mean that puts any requirement on any IL that you might generate with it. Rather, when you ask Roslyn to parse something or analyze source code, the objects (syntax trees, type information, etc) is immutable. Still, it doesn't matter since you can't modify types in the CLR once they exist.
I'm with svick in his comment -- this isn't what you want to do. Use some appropriate data structures to represent your information at runtime, rather than trying to think of this as a concrete class that can be mutated somehow.

How to identify what state variables are read/written in a given method in C#

What is the simplest way to identify if a given method is reading or writing a member variable or property?
I am writing a tool to assist in an RPC system, in which access to remote objects is expensive. Being able to detect if a given object is not used in a method could allow us to avoid serializing its state. Doing it on source code is perfectly reasonable (but being able to do it on compiled code would be amazing)
I think I can either write my own simple parser, I can try to use one of the existing C# parsers and work with the AST. I am not sure if it is possible to do this with Assemblies using Reflection. Are there any other ways? What would be the simplest?
EDIT: Thanks for all the quick replies. Let me give some more information to make the question clearer. I definitely prefer correct, but it definitely shouldn't be extremely complex. What I mean is that we can't go too far checking for extremes or impossibles (as the passed-in delegates that were mentioned, which is a great point). It would be enough to detect those cases and assume everything could be used and not optimize there. I would assume that those cases would be relatively uncommon.
The idea is for this tool to be handed to developers outside of our team, that should not be concerned about this optimization. The tool takes their code and generates proxies for our own RPC protocol. (we are using protobuf-net for serialization only, but no wcf nor .net remoting). For this reason, anything we use has to be free or we wouldn't be able to deploy the tool for licensing issues.

You can have simple or you can have correct - which do you prefer?
The simplest way would be to parse the class and the method body. Then identify the set of tokens which are properties and field names of the class. The subset of those tokens which appears in the method body are the properties and field names you care about.
This trivial analysis of course is not correct. If you had
class C
{
int Length;
void M() { int x = "".Length; }
}
Then you would incorrectly conclude that M references C.Length. That's a false positive.
The correct way to do it is to write a full C# compiler, and use the output of its semantic analyzer to answer your question. That's how the IDE implements features like "go to definition".

Before attempting to write this kind of logic yourself, I would check to see if you can leverage NDepend to meet your needs.
NDepend is a code dependency analysis tool ... and much more. It implements a sophisticated analyzer for examining relationships between code constructs and should be able to answer that question. It also operates on both source and IL, if I'm not mistaken.
NDepend exposes CQL - Code Query Language - which allows you to write SQL-like queries against the relationships between structures in your code. NDepend has some support for scripting and is capable of being integrated with your build process.

To complete the LBushkin answer on NDepend (Disclaimer: I am one of the developer of this tool), NDepend can indeed help you on that. The Code LINQ Query (CQLinq) below, actually match methods that...
shouldn't provoque any RPC calls but
that are reading/writing any fields of any RPC types,
or that are reading/writing any properties of any RPC types,
Notice how first we define the 4 sets: typesRPC, fieldsRPC, propertiesRPC, methodsThatShouldntUseRPC - and then we match methods that violate the rule. Of course this CQLinq rule needs to be adapted to match your own typesRPC and methodsThatShouldntUseRPC:
warnif count > 0
// First define what are types whose call are RDC
let typesRPC = Types.WithNameIn("MyRpcClass1", "MyRpcClass2")
// Define instance fields of RPC types
let fieldsRPC = typesRPC.ChildFields()
.Where(f => !f.IsStatic).ToHashSet()
// Define instance properties getters and setters of RPC types
let propertiesRPC = typesRPC.ChildMethods()
.Where(m => !m.IsStatic && (m.IsPropertyGetter || m.IsPropertySetter))
.ToHashSet()
// Define methods that shouldn't provoke RPC calls
let methodsThatShouldntUseRPC =
Application.Methods.Where(m => m.NameLike("XYZ"))
// Filter method that should do any RPC call
// but that is using any RPC fields (reading or writing) or properties
from m in methodsThatShouldntUseRPC.UsingAny(fieldsRPC).Union(
methodsThatShouldntUseRPC.UsingAny(propertiesRPC))
let fieldsRPCUsed = m.FieldsUsed.Intersect(fieldsRPC )
let propertiesRPCUsed = m.MethodsCalled.Intersect(propertiesRPC)
select new { m, fieldsRPCUsed, propertiesRPCUsed }

My intuition is that detecting which member variables will be accessed is the wrong approach. My first guess at a way to do this would be to just request serialized objects on an as-needed basis (preferably at the beginning of whatever function needs them, not piecemeal). Note that TCP/IP (i.e. Nagle's algorithm) should stuff these requests together if they are made in rapid succession and are small

Eric has it right: to do this well, you need what amounts to a compiler front end. What he didn't emphasize enough is the need for strong flow analysis capabilities (or a willingness to accept very conservative answers possibly alleviated by user annotations). Maybe he meant that in the phrase "semantic analysis" although his example of "goto definition" just needs a symbol table, not flow analysis.
A plain C# parser could only be used to get very conservative answers (e.g., if method A in class C contains identifier X, assume it reads class member X; if A contains no calls then you know it can't read member X).
The first step beyond this is having a compiler's symbol table and type information (if method A refers to class member X directly, then assume it reads member X; if A contains **no* calls and mentions identifier X only in the context of accesses to objects which are not of this class type then you know it can't read member X). You have to worry about qualified references, too; Q.X may read member X if Q is compatible with C.
The sticky point are calls, which can hide arbitrary actions. An analysis based on just parsing and symbol tables could determine that if there are calls, the arguments refer only to constants or to objects which are not of the class which A might represent (possibly inherited).
If you find an argument that has an C-compatible class type, now you have to determine whether that argument can be bound to this, requiring control and data flow analysis:
method A( ) { Object q=this;
...
...q=that;...
...
foo(q);
}
foo might hide an access to X. So you need two things: flow analysis to determine whether the initial assignment to q can reach the call foo (it might not; q=that may dominate all calls to foo), and call graph analysis to determine what methods foo might actually invoke, so that you can analyze those for accesses to member X.
You can decide how far you want to go with this simply making the conservative assumption "A reads X" anytime you don't have enough information to prove otherwise. This will you give you a "safe" answer (if not "correct" or what I'd prefer to call "precise").
Of frameworks that might be helpful, you might consider Mono, which surely parses and builds symbol tables. I don't know what support it provides for flow analysis or call graph extraction; I would not expect the Mono-to-IL front-end compiler to do a lot of that, as people usually hide that machinery in the JIT part of JIT-based systems. A downside is that Mono may be behind the "modern C#" curve; last time I heard, it handled only C# 2.0 but my information may be stale.
An alternative is our DMS Software Reengineering Toolkit and its C# Front End.
(Not an open source product).
DMS provides general source code parsing, tree building/inspection/analysis, general symbol table support and built-in machinery for implementing control-flow analysis, data flow analysis, points-to analysis (needed for "What does object O point to?"), and call graph construction. This machinery has all been tested by fire with DMS's Java and C front ends, and the symbol table support has been used to implement full C++ name and type resolution, so its pretty effective. (You don't want to underestimate the work it takes to build all that machinery; we've been working on DMS since 1995).
The C# Front End provides for full C# 4.0 parsing and full tree building. It presently does not build symbol tables for C# (we're working on this) and that's a shortcoming compared to Mono. With such a symbol table, however, you would have access to all that flow analysis machinery (which has been tested with DMS's Java and C front ends) and that might be a big step up from Mono if it doesn't provide that.
If you want to do this well, you have a considerable amount of work in front of you. If you want to stick with "simple", you'll have to do with just parsing the tree and being OK with being very conservative.
You didn't say much about knowing if a method wrote to a member. If you are going to minimize traffic the way you describe, you want to distinguish "read", "write" and "update" cases and optimize messages in both directions. The analysis is obviously pretty similar for the various cases.
Finally, you might consider processing MSIL directly to get the information you need; you'll still have the flow analysis and conservative analysis issues. You might find the following technical paper interesting; it describes a fully-distributed Java object system that has to do the same basic analysis you want to do,
and does so, IIRC, by analyzing class files and doing massive byte code rewriting.
Java Orchestra System

By RPC do you mean .NET Remoting? Or DCOM? Or WCF?
All of these offer the opportunity to monitor cross process communication and serialization via sinks and other constructs, but they are all platform specific, so you'll need to specify the platform...

You could listen for the event that a property is being read/written to with an interface similar to INotifyPropertyChanged (although you obviously won't know which method effected the read/write.)

I think the best you can do is explicitly maintain a dirty flag.

How does const correctness help write better programs?

This question is from a C# guy asking the C++ people. (I know a fair bit of C but only have some general knowledge of C++).
Allot of C++ developers that come to C# say they miss const correctness, which seems rather strange to me. In .Net in order to disallow changing of things you need to create immutable objects or objects with read only properties/fields, or to pass copies of objects (by default structs are copied).
Is C++ const correctness, a mechanism to create immutable/readonly objects, or is there more to it? If the purpose is to create immutable/readonly objects, the same thing can be done in an environment like .Net.

A whole section is devoted to Const Correctness in the FAQ. Enjoy!

const correctness in C++ is at heart simply a way of saying exactly what you mean:
void foo( const sometype & t ) {
...
}
says "I will not change the object referred to by t, and if I try to, please Mr. Compiler warn me about it".
It is not a security feature or a way of (reliably) creating read-only objects, because it is trivial to subvert using C-style casts.

I bet this is a matter of mindsets. I'm working myself with long-time C++ people and have noticed they seem to think in C++ (of course they do!). It will take time for them to start seeing multiple answers to the ones they've grown familiar to have the 'stock' answers like const correctness. Give them time, and try to educate them gently.
Having said that, I do like the C++ 'const' way of guarantees. It's mainly promising the user of an interface that the object, data behind the pointer or whatever won't be messed with. I would see this as the #1 value of const correctness. #2 value is what it can allow the compiler in terms of optimization.
The problem is, of course, if the whole sw stack hasn't been built with const in mind from the ground up.
I'm sure C# has its own set of paradigms to do the same. This shows why it's so important to "learn one new (computing or natural) language a year". Simply to exercise one's mind of the other ways to see the world, and solve its problems.

With const-correctness, you can expose a mutable object as read-only without making a copy of it and wasting precious memory (especially relevant for collections). I know, there is this CPU-and-memory-are-cheap philosophy among desktop developers, but it still matters in embedded world.
On top of that, if you add complexities of memory ownership in C++, it is almost always better not to copy non-trivial objects that contain or reference other objects, hence there is a method of exposing existing objects as read-only.

Excellent Scott Meyer's book Effective C++ has dedicated an item to this issue:
ITEM 3: Use const whenever possible.
It really enlightens you :-)

The real answer, even if it is really hard to grasp if you have not used it, is that you loose expressiveness. The language has concepts that you cannot possibly express in C#, and they have meaning, they are part of the design, but lost in the translation to code.
This is not an answer, but rather an example:
Consider that you have a container that is sorted on a field of the elements that are stored. Those objects are really big. You need to offer access to the data for readers (consider showing the information in the UI).
Now, in C#/Java, you can go in one of two ways: either you make a copy for the caller to use (guarantees that the data will not change, but inefficient) or you return a reference to your internally held object (just hoping the caller will not change your data through setters).
If the user gets the reference and changes through it the field that serves as index, then your container invariants are broken.
In C++ you can return a constant reference/pointer, and the compiler will disable calling setter methods (mutating methods) on the instance you return. You get both the security that the user will not change (*) and efficiency in the call.
The third not mentioned before option is making the object inmutable, but that disables changes everywhere. Perfectly controlled changes to the object will be disallowed, and the only possibility of change is creating a new element with the changes performed. That amounts to time and space.

C++ provides plenty of ways you can mess up your code. I see const correctness as a first, efficient way of putting constraints to the code, in order to control the "side" effects (unwanted changes).
Marking a parameter as const (usually reference to const object or pointer to const object), will ensure that the passed object can't be changed, so you have no "side" effect of that function/method.
Marking a method as const will guarantee that that method will not change the state of the object it works with. If it does, the compiler will generate an error.
Marking a const data member will ensure that the data member can only be initialized in the initialization list of the constructor and can't be changed.
Also, smart compilers can use the constness as hints for various performance optimizations.
Of course, you can override constness. Constness is compile time checking. You can't break the rules by default, but you can use casting (e.g. const_cast) to make a const object non-const so it can be changed.

Just enlisting the compiler's help when writing code would be enough for me to advocate const-correctness. But today there is an additional advantage: multi-threading code is generally easier to write when you know where can our objects change and where they cannot.

I think it's a pity C# doesn't support const correctness as C++ does. 95% of all parameters I pass to a function are constant values, the const feature guarantees that the data you pass by reference won't be modified after the call. The "const" keyword provides compiler-checked documentation, which is a great help in large programs.

Why can I not edit a method that contains an anonymous method in the debugger?

So, every time I have written a lambda expression or anonymous method inside a method that I did not get quite right, I am forced to recompile and restart the entire application or unit test framework in order to fix it. This is seriously annoying, and I end up wasting more time than I saved by using these constructs in the first place. It is so bad that I try to stay away from them if I can, even though Linq and lambdas are among my favourite C# features.
I suppose there is a good technical reason for why it is this way, and perhaps someone knows? Furthermore, does anyone know if it will be fixed in VS2010?
Thanks.

Yes there is a very good reason for why you cannot do this. The simple reason is cost. The cost of enabling this feature in C# (or VB) is extremely high.
Editing a lambda function is a specific case of a class of ENC issues that are very difficult to solve with the current ENC (Edit'n'Continue) architecture. Namely, it's very difficult to ENC any method which where the ENC does one of the following:-
Generates Metadata in the form of a class
Edits or generates a generic method
The first issue is more of a logic constraint but it also bumps into a couple of limitations in the ENC architecture. Namely the problem is generating the first class isn't terribly difficult. What's bothersome is generating the class after the second edit. The ENC engine must start tracking the symbol table for not only the live code, but the generated classes as well. Normally this is not so bad, but this becomes increasingly difficult when the shape of a generated class is based on the context in which it is used (as is the case with lambdas because of closures). More importantly, how do you resolve the differences against instances of the classes that are already alive in the process?
The second issue is a strict limitation in the CLR ENC architecture. There is nothing that C# (or VB) can do to work around this.
Lambdas unfortunately hit both of these issues dead on. The short version is that ENC'ing a lambda involves lots of mutations on existing classes (which may or may not have been generated from other ENC's). The big problem comes in resolving the differences between the new code and the existing closure instances alive in the current process space. Also, lambdas tend to use generics a lot more than other code and hit issue #2.
The details are pretty hairy and a bit too involved for a normal SO answer. I have considered writing a lengthy blog post on the subject. If I get around to it I'll link it back into this particular answer.

According to a list of Supported Code Changes, you cannot add fields to existing types. Anonymous methods are compiled into oddly-named classes (kinda <>_c__DisplayClass1), which are precisely that: types. Even though your modifications to the anonymous method may not include changing the set of enclosed variables (adding those would alter fields of an existing class), I guess that's the reason it's impossible to modify anonymous methods.

It is a bit a shame that this feature is partially supported in VB but not in C#:
http://msdn.microsoft.com/en-us/library/bb385795.aspx
Implementing the same behaviour in C# would reduce the pain level by 80% for functions that contain lambda expressions, where we do not need to modify the lambda expressions nor any expression that depends on them, and probably not for a "monster cost".

Restarting a unit test should take a matter of seconds, if that. I've never liked the "edit and continue" model to be honest - you should always rerun from scratch IMO, just in case the change midway through execution would have affected the code which ran earlier. Given that, you're better off using unit tests which can be run with a very quick turnaround. If your individual unit tests take an unbearable time to start, that's something you should look at addressing.
EDIT: As for why it doesn't work - you may find that it works for some lambdas but not others. Lambda expressions which don't capture any variables (including this) are cached in a private static variable, so that only one instance of the delegate is ever created. Changing the code means reinitialising that variable which could have interesting side-effects I suspect.

I just want to point out that Visual Studio's consideration of "editing" in this context is (or at least can be) a bit stupid. When I was checking out an older commit as part of doing an interactive rebase in git and then attempting to run an unit test, that resulted in 9 error (with ENC0014 and some others).
So with no files modified, every time I attempted to debug the unit test I got those errors. Restarting Visual Studio made the errors go away, so I guess that the underlying problem is missing cache invalidation where Visual Studio does not detect/react to files being changed outside editing via its editor windows.

How do you programmatically identify the number of references to a method with C#

I've recently inherited C# console application that is in need of some pruning and clean up. Long story short, the app consists of a single class containing over 110,000 lines of code. Yup, over 110,000 lines in a single class. And, of course, the app is core to our business, running 'round the clock updating data used on a dynamic website. Although I'm told my predecessor was "a really good programmer", it obvious he was not at all into OOP (or version control).
Anyway... while familiarizing myself with the code I've found plenty of methods that are declared, but never referenced. It looks as if copy/paste was used to version the code, for example say I have a method called getSomethingImportant(), chances are there is another method called getSomethingImortant_July2007() (the pattern is functionName_[datestamp] in most cases). It looks like when the programmer was asked to make a change to getSomethingImportant() he would copy/paste then rename to getSomethingImortant_Date, make changes to getSomethingImortant_Date, then change any method calls in the code to the new method name, leaving the old method in the code but never referenced.
I'd like to write a simple console app that crawls through the one huge class and returns a list of all methods with the number of times each method was referenced. By my estimates there are well over 1000 methods, so doing this by hand would take a while.
Are there classes within the .NET framework that I can use to examine this code? Or any other usefull tools that may help identify methods that are declared but never referenced?
(Side question: Has anyone else ever seen a C# app like this, one reeeealy big class? It's more or less one huge procedural process, I know this is the first I've seen, at least of this size.)

You could try to use NDepend if you just need to extract some stats about your class. Note that this tool relies on Mono.Cecil internally to inspect assemblies.

To complete the Romain Verdier answer, lets dig a bit into what NDepend can bring to you here. (Disclaimer: I am a developer of the NDepend team)
NDepend lets query your .NET code with some LINQ queries. Knowing which methods call and is called by which others, is as simple as writing the following LINQ query:
from m in Application.Methods
select new { m, m.MethodsCalled, m.MethodsCallingMe }
The result of this query is presented in a way that makes easy to browse callers and callees (and its 100% integrated into Visual Studio).
There are many other NDepend capabilities that can help you. For example you can right click a method in Visual Studio > NDepend > Select methods... > that are using me (directly or indirectly) ...
The following code query is generated...
from m in Methods
let depth0 = m.DepthOfIsUsing("NUnit.Framework.Constraints.ConstraintExpression.Property(String)")
where depth0 >= 0 orderby depth0
select new { m, depth0 }
... which matches direct and indirect callers, with the depth of calls (1 means direct caller, 2 means caller of direct callers and so on).
And then by clicking the button Export to Graph, you get a call graph of your pivot method (of course it could be the other way around, i.e method called directly or indirectly by a particular pivot method).

Download the free trial of Resharper. Use the Resharper->Search->Find Usages in File (Ctrl-Shift-F7) to have all usages highlighted. Also, a count will appear in the status bar. If you want to search across multiple files, you can do that too using Ctrl-Alt-F7.
If you don't like that, do text search for the function name in Visual Studio (Ctrl-Shift-F), this should tell you how many occurrences were found in the solution, and where they are.

I don't think you want to write this yourself - just buy NDepend and use its Code Query Language

There is no easy tool to do that in .NET framework itself. However I don't think you really need a list of unused methods at once. As I see it, you'll just go through the code and for each method you'll check if it's unused and then delete it if so. I'd use Visual Studio "Find References" command to do that. Alternatively you can use Resharper with its "Analize" window. Or you can just use Visual Studio code analysis tool to find all unused private methods.

FXCop has a rule that will identify unused private methods. So you could mark all the methods private and have it generate a list.
FXCop also has a language if you wanted to get fancier
http://www.binarycoder.net/fxcop/

If you don't want to shell out for NDepend, since it sounds like there is just a single class in a single assembly - comment out the methods and compile. If it compiles, delete them - you aren't going to have any inheritance issues, virtual methods or anything like that. I know it sounds primitive, but sometimes refactoring is just grunt work like this. This is kind of assuming you have unit tests you run after each build until you've got the code cleaned up (Red/Green/Refactor).

The Analyzer window in Reflector can show you where a method is called (Used By).
Sounds like it would take a very long time to get the information that way though.
You might look at the API that Reflector provides for writing add-ins and see if you can get the grunt work of the analysis that way. I would expect that the source code for the code metrics add-in could tell you a bit about how to get information about methods from the reflector API.
Edit: Also the code model viewer add-in for Reflector could help too. It's a good way to explore the Reflector API.

I don't know of anything that's built to handle this specific case, but you could use Mono.Cecil. Reflect the assemblies and then count references in the IL. Shouldn't be too tough.

Try having the compiler emit assembler files, as in x86 instructions, not .NET assemblies.
Why? Because it's much easier to parse assembler code than it is C# code or .NET assemblies.
For instance, a function/method declaration looks something like this:
.string "w+"
.text
.type create_secure_tmpfile, #function
create_secure_tmpfile:
pushl %ebp
movl %esp, %ebp
subl $24, %esp
movl $-1, -8(%ebp)
subl $4, %esp
and function/method references will look something like this:
subl $12, %esp
pushl 24(%ebp)
call create_secure_tmpfile
addl $16, %esp
movl 20(%ebp), %edx
movl %eax, (%edx)
When you see "create_secure_tmpfile:" you know you have a function/method declaration, and when you see "call create_secure_tmpfile" you know you have a function/method reference. This may be good enough for your purposes, but if not it's just a few more steps before you can generate a very cute call-tree for your entire application.