We Keep Coding

How to create a script engine class in C# [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Is it possible to make a call to a C# application from a C++ application?
I am playing with the idea of writing a proof of concept application that contains a script engine that runs (executes) a .CLI language (e.g. C#, VB.Net etc).
I had originally wanted to create the script engine application in C++, but that appears to be fraught with problems and work arounds. Instead, I want to write the script engine in C# instead now.
I have sketched together a very rough idea of what it is I'm trying to do below:
The code is still pseudo C++, but hopefully, the semantics should be clear:
class GenericDotNetLangInterpreter
{
public:
Results run(const Arguments& args);
protected:
GenericDotNetLangInterpreter(const std::string &script);
};
class MyInterpreter : private GenericDotNetLangInterpreter
{
public:
MyInterpreter(const LanguageType l);
Results run(const Arguments& args);
}
Couple of questions:
Has someone done this kind of thing before, and is there some code I can use as a reference point?
what are some gotchas I need to be aware of going down this path?

Not really an answer, but pointing out that C# and VB.Net aren't interpreted, but compiled to IL (similar to Java's Bytecode). In other words, your API name and signature would probably change. You would need to invoke the associated compiler and then execute the code.
There are .Net languages that can run interpreted on the DLR, but I'm not clear on calling that from a native language.

The limit between managed and unmanaged applications is thinner than it looks. Your C++ app can link with mscorelib (basic .NET support).
From there on, your C++ app can compile your text "script" to .NET byte code and execute it.
Whether it makes sense or is a good idea is left up to you.

Is it possible to call C# lexical/syntactic analyzers without compilation?

Considering this question of SO, where whole C# in-memory compiler is being called. When only lexical and syntactic analyzing is required: parse text as a stream of lexemes, check them and exit.
Is it possible in current version of System.CodeDom.Compiler, if not - will it be?

If you can use Mono, I believe it has a C# parser/lexer you may be able to use.
Here's a link to look into. As for what the MS C# team is planning to do, there is some talk of at some point making the C# compiler into a "service" - but it's unclear what that means or when that will happen.

While it might look like the code is compiled in-memory (CompilerParameters.GenerateInMemory), that's not what actually happens. The same compiler as the one used in Visual Studio is used to compile the code (csc.exe). It gets started by CreateProcess (much like Process.Start) and runs out-of-process to compile the code to an assembly on disk in a temporary folder. The GenerateInMemory option invokes Assembly.LoadFrom() to load the assembly.
You'll get the equivalent of a syntax check simply by setting GenerateInMemory to false and delete the OutputAssembly after it is done.
While this might sound kinda backwards, the huge benefit it has is that this won't put any memory pressure on your process. This will hold you over until C# 5.0 ships.

Where would you use C# Runtime Compilation?

I happened upon a brief discussion recently on another site about C# runtime compilation recently while searching for something else and thought the idea was interesting. Have you ever used this? I'm trying to determine how/when one might use this and what problem it solves. I'd be very interested in hearing how you've used it or in what context it makes sense.
Thanks much.

Typically, I see this used in cases where you are currently using Reflection and need to optimize for performance.
For example, instead of using reflection to call method X, you generate a Dynamic Method at runtime to do this for you.

You can use this to add scripting support to your application. For examples look here or here.
It is quite easily possible to publish parts of your internal object framework to the scripting part, so you could with relative ease add something to your application that has the same effect as for example VBA for Office.

I've seen this (runtime compilation / use of System.Reflection.Emit classes) in generating dynamic proxies ( Code project sample ) or other means of optimizing reflection calls (time-wise).

At least one case you might use it is when generating dynamic code. For example, the framework is using this internally to generate XML serializers on the fly. After looking into a class at runtime, it can generate the code to serialize / deserialize the class. It then compiles that code and users it as needed.
In the same way you can generate code to handle arbitrary DB tables etc. and then compile and load the generated assembly.

Well, all C# code is run-time compiled, since it's a JIT (just-in-time) compiler. I assume you are referring to Reflection.Emit to create classes etc. on the fly. Here's an example I have seen recently in the Xml-Rpc.Net library.
I create a C# interface that has the same signature as an XML-RPC service's method calls, e.g.
IMyProxy : IXmlRpcProxy
{
[XmlRpcMethod]
int Add(int a, int b);
}
Then in my code I call something like
IMyProxy proxy = (IMyProxy)XmlRcpFactory.Create(typeof(IMyProxy));
This uses run-time code generation to create a fully functional proxy for me, so I can use it like this:
int result = proxy.Add(1, 2);
This then handles the XML-RPC call for me. Pretty cool.

I used runtime compiler services from .NET in my diploma thesis. Basically, it was about visually creating some graphical component for a process visualization, which is generated as C# code, compiled into an assembly and can then be used on the target system without being interpreted, to make it faster and more compact. And, as a bonus, the generated images could be packaged into the very same assembly as resources.
The other use of that was in Java. I had an application that had to plot a potentially expensive function using some numerical algorithm (was back at university) the user could enter. I put the entered function into a class, compiled and loaded it and it was then available for relatively fast execution.
So, these are my two experiences where runtime code generation was a good thing.

something I used it for was for allowing C# and VB code to bu run by the user ad-hoc. They could type in a line of code (or a couple lines) and it would be compiled, loaded into an app domain, and executed, and then unloaded. This probably isnt the best example of its usage, but an example of it none-the-less

Assembler library for .NET, assembling runtime-variable strings into machine code for injection

Is there such a thing as an x86 assembler that I can call through C#? I want to be able to pass x86 instructions as a string and get a byte array back. If one doesn't exist, how can I make my own?
To be clear - I don't want to call assembly code from C# - I just want to be able to assemble code from instructions and get the machine code in a byte array.
I'll be injecting this code (which will be generated on the fly) to inject into another process altogether.

As part of some early prototyping I did on a personal project, I wrote quite a bit of code to do something like this. It doesn't take strings -- x86 opcodes are methods on an X86Writer class. Its not documented at all, and has nowhere near complete coverage, but if it would be of interest, I would be willing to open-source it under the New BSD license.
UPDATE:
Ok, I've created that project -- Managed.X86

See this project:
https://github.com/ZenLulz/MemorySharp
This project wraps the FASM assembler, which is written in assembly and as a compiled as Microsoft coff object, wrapped by a C++ project, and then again wrapped in C#. This can do exactly what you want: given a string of x86/x64 assembly, this will produce the bytes needed.
If you require the opposite, there is a port of the Udis86 disassembler, fully ported to C#, here:
https://github.com/spazzarama/SharpDisasm
This will convert an array of bytes into the instruction strings for x86/x64

Take a look at Phoenix from Microsoft Research.

Cosmos also has some interesting support for generating x86 code:
http://www.gocosmos.org/blog/20080428.en.aspx

Not directly from C# you can't. However, you could potentially write your own wrapper class that uses an external assembler to compile code. So, you would potentially write the assembly out to a file, use the .NET Framework to spin up a new process that executes the assembler program, and then use System.IO to open up the generated file by the assembler to pull out the byte stream.
However, even if you do all that, I would be highly surprised if you don't then run into security issues. Injecting executable code into a completely different process is becoming less and less possible with each new OS. With Vista, I believe you would definitely get denied. And even in XP, I think you would get an access denied exception when trying to write into memory of another process.
Of course, that raises the question of why you are needing to do this. Surely there's got to be a better way :).

Take a look at this: CodeProject: Using unmanaged code and assembly in C#.

I think you would be best off writing a native Win32 dll. You can then write a function in assembler that is exported from the dll. You can then use C# to dynamically link to the dll.
This is not quite the same as passing in a string and returning a byte array. To do this you would need an x86 assembler component, or a wrapper around masm.exe.

i don't know if this is how it works but you could just shellexecute an external compiler then loading the object generated in your byte array.

Translate C++/CLI to C#

I have a small to medium project that is in C++/CLI. I really hate the syntax extensions of C++/CLI and I would prefer to work in C#. Is there a tool that does a decent job of translating one to the other?
EDIT: When I said Managed c++ before I apparently meant c++/CLI

You can only translate Managed C++ code (and C++/CLI code) to C# if the C++ code is pure managed. If it is not -- i.e. if there is native code included in the sources -- tools like .NET Reflector won't be able to translate the code for you.
If you do have native C++ code mixed in, then I'd recommend trying to move the native code into a separate DLL, replace your calls to DLL functions by easily identifiable stub functions, compile your project as a pure .NET library, then use .NET reflector to de-compile into C# code. Then you can replace the calls to the stub functions by p-invoke calls to your native DLL.
Good luck! I feel for you!

.NET Managed C++ is like a train wreck. But have you looked into C++ CLI? I think Microsoft did a great job in this field to make C++ a first class .NET citizen.
http://msdn.microsoft.com/en-us/magazine/cc163852.aspx

I'm not sure if this will work, but try using .Net Reflector along with ReflectionEmitLanguage plug-in. The RelelectionEmitLanguage plug-in claims to convert your assembly to c# code.

It has to be done manually unfortunately, but if the code is mostly C++/CLI (not native C++) then it can actually be done pretty quickly. I managed to port around 250,000 lines of C++/CLI code into C# in less than a couple of months, and I don't even know C++ very well at all.
If preserving Git history is important, you might want to git mv your cpp file into a cs file, commit, then start porting. The reason for this is that Git will think your file is new if you modify it too much after renaming it.
This was my approach when porting large amounts of code (so that it wouldn't take forever):
Create another worktree / clone of the branch and keep it open at all times
This is extremely important as you will want to compare your C# to the old C++/CLI code
Rename cpp to cs, delete header file, commit
I chose to rename the cpp file since its git history is probably more important than the header file
Create namespace + class in cs file, add any base classes/interfaces (if abstract sealed, make static in C#)
Copy fields first, then constructors, then properties, and finally functions
Start replacing with Ctrl+H:
^ to empty
:: to .
-> to .
nullptr to null
for each to foreach
gcnew to new
L" to "
Turn on case sensitivity to avoid accidental renames (for example L"cool" should become "cool", not "coo"
Prefixes like ClassName:: to empty, so that MyClass::MyMethod becomes MyMethod
Go through the red code and port manually code that cannot be just replaced (e.g. some special C++ casts), unless you have some cool regex to do it fast
Once code compiles, go through it again, compare to C++/CLI line by line, check for errors, clean it up, move on.
If you encounter a dependency that needs to be ported, you could pause, port that, then come back. I did that, but it might not be so easy.
Properties were the most annoying to port, because I had to remove everything before and after the getters and setters. I could have maybe written a regex for it but didn't bother doing so.
Once the porting is done, it's very important that you go through the changes line by line, read the code, and compare with C++/CLI code and fix possible errors.
One problem with this approach is that you can introduce bugs in variable declarations, because in C++/CLI you can declare variables in 2 ways:
MyType^ variable; <- null
MyType variable; <- calls default constructor
In the latter case, you want to actually do MyType variable = new MyType(); but since you already removed all the ^ you have to just manually check and test which one is correct. You could of course just replace all ^'s manually, but for me it would have taken too long (plus laziness) so I just did it this way.
Other recommendations:
Have a dummy C++/CLI project and a tool like LinqPad or another C# project to test differences between C++/CLI and C# if you're unsure of a piece of ported code
Install Match Margin to help highlight similar code (helped me when porting WinForms code)
ReSharper! It helped with finding bugs and cleaning up the code a LOT. Truly worth the money.
Some gotchas that I encountered while porting:
Base classes can be called in C++/CLI like so: BaseClass->DoStuff, but in C# you would have to do base.DoStuff instead.
C++/CLI allows such statements: if (foo), but in C# this has to be explicit. In the case of integers, it would be if (foo != 0) or for objects if (foo != null).
Events in base classes can be invoked in C++/CLI, but in C# it's not possible. The solution is to create a method, like OnSomeEvent, in the base class, and inside that to invoke the event.
C++/CLI automatically generates null checks for event invocations, so in C# make sure to add an explicit null check: MyEvent?.Invoke(this, EventArgs.Empty);. Notice the question mark.
dynamic_cast is equivalent to as cast in C#, the rest can be direct casts ((int) something).
gcnew can be done without parentheses. In C# you must have them with new.
Pay attention to virtual override keywords in the header files, you can easily forget to mark the C# methods with override keyword.
Intefaces can have implementations! In this case, you might have to rethink the architecture a bit. One option is to pull the implementation into an abstract class and derive from it
Careful when replacing casts with Convert calls in C#
Convert.ToInt32 rounds to the narest int, but casting always rounds down, so in this case we should not use the converter.
Always try casting first, and if that doesn't work, use the Convert class.
Variables in C++/CLI can be re-declared in a local scope, but in C# you get naming conflicts. Code like this easily lead to hard to find bugs if not ported carefully.
Example: An event handler can take a parameter e, but also has a try-catch like catch (Exception e) which means there are 2 e variables.
Another example:
// number is 2
int number = 2;
for (int number = 0; number < 5; number++)
{
// number is now 0, and goes up to 4
}
// number is again 2!
The above code is illegal in C#, because there is a naming conflict. Find out exactly how the code works in C++ and port it with the exact same logic, and obviously use different variable names.
In C++/CLI, it's possible to just write throw; which would create a generic C++ exception SEHException. Just replace it with a proper exception.
Be careful when porting code that uses the reference % sign, that usually means that you will have to use ref or out keywords in C#.
Similarly, pay attention to pointers * and & references. You might have to write additional code to write changes back whereas in C++ you can just modify the data pointed to by the pointer.
It's possible to call methods on null object instances in C++/CLI. Yes seriously. So inside the function you could do If (this == null) { return; }.
Port this type of code carefully. You might have to create an extension method that wraps over this type of method in order to avoid breaking the code.
Check and make sure everything in the old project file vcxproj was ported correctly. Did you miss any embedded resources?
Careful when porting directives like #ifdef, the "if not" (#ifndef) looks awfully similar but can have disastrous consequences.
C++/CLI classes automatically implement IDisposable when adding a destructor, so in C# you'll need to either implement that interface or override the Dispose method if it's available in the base class.
Other tips:
If you need to call Win32 functions, just use P/Invoke instead of creating a C++/CLI wrapper
For complex native C++ code, better create a C++/CLI project with managed wrappers
Again, pay attention to pointers. I had forgotten to do Marshal.StructureToPtr in my P/Invoke code which wasn't necessary in the C++ version since we had the actual pointer and not a copy of its data.
I have surely missed some things, but hopefully these tips will be of some help to people who are demoralized by the amount of code that needs to be ported, especially in a short period of time :)
After porting is done, use VS/ReSharper to refactor and clean up the code. Not only is it nice for readability, which is my top priority when writing code, but it also forces you to interact with the code and possibly find bugs that you otherwise would have missed.
Oh and one final FYI that could save you headaches: If you create a C++/CLI wrapper that exposes the native C++ pointer, and need to use that pointer in an external C++/CLI assembly, you MUST make the native type public by using #pragma make_public or else you'll get linker errors:
// put this at the top of the wrapper class, after includes
#pragma make_public(SomeNamespace::NativeCppClass)
If you find a bug in the C++/CLI code, keep it. You want to port the code, not fix the code, so keep things in scope!
For those wondering, we got maybe around 10 regressions after the port. Half were mistakes because I was already on autopilot mode and didn't pay attention to what I was doing.
Happy porting!

Back ~2004 Microsoft did have a tool that would convert managed C++ to C++/CLI ... sort of. We ran it on a couple of projects, but to be honest the amount of work left cleaning up the project was no less than the amount of work it would have been to do the conversion by hand in the first place. I don't think the tool ever made it out into a public release though (maybe for this reason).
I don't know which version of Visual Studio you are using, but we have managed C++ code that will not compile with Visual Studio 2005/2008 using the /clr:oldSyntax switch and we still have a relic VS 2003 around for it.
I don't know of any way of going from C++ to C# in a useful way ... you could try round tripping it through reflector :)

Such projects are often done in c++/cli because C# isn't really an elegant option for the task. e.g. if you have to interface with some native C++ libraries, or do very high performance stuff in low level C. So just make sure whoever chose c++/cli didn't have a good reason to do it before doing the switch.
Having said that, I'm highly skeptical there's something that does what you ask, for the simple reason that not all C++/cli code is translatable to C# (and probably vice versa too).