I've searched good and Stack Overflow but couldn't find an answer to what I was looking for. Is there anyway to hook the call of Python functions from within C++/C#? Capture the function call plus it parameters?
Edit with an example:
def a(pOne):
//do stuff
def b():
a("a")
So on the call to 'a' I want C++ to hook that function (assuming the function name is known) and execute a C++ function/event on the call to 'a' with the parameters passed to the C++ function/event being the value of what was passed in for 'pOne'.
Thanks
There a couple of different options for C++ depending on what you want to do.
The best way to do what you want to do is to 'extend' Python.
My Favorite way to do this is Boost.Python
You can also use the raw C-API though I wouldn't recommend this.
For some examples of embedding and extending with Boost.Python you can look at this project I have been working on.
If you just want to write some C++ code and make it callable from Python, you do that by Extending and Embedding the Python Interpreter. (Actually, just Extending; ignore the other half.)
Basically, if you write a new module foo, anyone can import foo and call foo.a("a"), and it doesn't matter if module foo is implemented as a Python file foo.py, or compiled from foo.cpp into a dynamic library foo.so.
For that, as Kyle C suggests, there are a number of higher-level ways to do this so you don't need to deal with the C API directly. (In addition to his suggestion of Boost.Python, I'd also suggest looking at Cython, which lets you write the code in an almost-Python language that can talk directly to your C++, while also exposing things directly to Python.)
However, that isn't what you asked for. What you want to be able to do is take some function that's defined in Python code, and hook it to do something different. For that, you really do need to read the Extending and Embedding documentation linked above. You're going to have to write an embedded interpreter, reproduce some of the behavior (exactly how much depends on where exactly you want to hook it), and make sure that wherever PyObject_Call or a similar function would have been called, it first checks whether that object is the a function and, if so, calls your hook code.
This is going to be pretty difficult. If you haven't written an embedded Python interpreter yet, go do that before you even think about how to hook it.
It's worth noting that it's probably much, much easier to do the hooking from within Python than from outside the interpreter. (If you've never heard of "monkeypatching", go google that.) And you can always make your Python hook call code from a module that you built in C++, or even call directly into a compiled .so file via ctypes.
Finally, if you want to hook some running interpreter instance at runtime, that's even more difficult. You obviously need to be able to do some kind of debug/trace/etc. attach and code insertion, and the details of that are entirely dependent on your platform. Then, you'll want to do the same thing you would have done in the previous hard version, except that you'll have to do it by intercepting calls to, e.g., PyObject_Call (from libpython.so/Python.dll/Python.framework/whatever) to go through your hooks first. If you don't already know how to hook SO calls on your platform, you need to learn that before you can even think about hooking Python code from outside.
Do you have reasons not to alter the in-python code? If not, why not write an extension module or a python ctypes call to your dll, and wrap a with a decorator that calls your hook?
import my_cpp_module
def hook_a_to_call_my_cpp_module(orig_a):
def replacement_a(pOne):
try:
my_cpp_module.my_cpp_function(pOne)
finally:
return orig_a(pOne)
#
return replacement_a
#hook_a_to_call_my_cpp_module
def a(pOne):
pass
you can even do this without touching the source file which has a, like this:
# in this other file you import the module that contains function a
import the_py_that_has_a
# next line just to avoid copy-pasting the above code from my answer here again
from the_decorator_from_the_previous_example import hook_a_to_call_my_cpp_module
the_py_that_has_a.a = hook_a_to_call_my_cpp_module(the_py_that_has_a.a)
If you have some reasons not to alter your py code at all, could you state them in a comment?
Related
I have two apps -- a C# app and a VBA app. I would like to create a variable in each app that points to the same memory location. The intent is, if I change the value in one app, the other app immediately has access to that value without any additional logic to transfer that data between the apps. In essence, I want to "remotely control" the value of the C# variable from a VBA app.
I'm not sure if this is even possible, so any help is greatly appreciated.
EDIT: I am considering a slightly different approach, now. I may use a non-persistent Memory Mapped File to transfer the data between the two apps. Although some code would be required to write/read the memory-mapped file, it is better than disk i/o. I am transferring a real number between the two apps (e.g. 1.2345). Not sure which class to use -- StreamWriter, StringWriter or TextWriter? Any suggestions??
Thank you.
You will need to pass the variable by reference to your code in C#
While I have not tried this directly in C#, the C++ equivalent of what you're looking for is given below. Do note that this approach uses global variables and is very crude and potentially unsafe, so be very careful with dealing with memory this way. I am sure there might be more elegant ways to express this.
double* monitored_variable_global_pointer;
void __stdcall DoSomething(double &monitored_variable)
{
// since monitored_variable is passed by reference, any change here, will change it on the VBA side, e.g.
monitored_variable_global_pointer = &monitored_variable;
}
// On the VBA side, the passed variable will have to be global too for this to work.
// Also, you need the function calling convention to be stdcall for it to work with VBA, at least when making functions in C++
Once the code is written,
The C++ code is compiled into a DLL
The function is then called once in VBA (preferably triggered by the event that the workbook is opened, after declaration of the global variable)
You would need to do the equivalent of these steps in C# for achieving the desired effect.
after long time of reading on google I still don't know how can I work with this EDB http://msdn.microsoft.com/en-us/library/aa912256 at C#. I read a lot of about wrapers and SWIG solution. But how can I wrap functions which are at windbase.h.
I want to make EDB in my c# program. Can I make some way dll from <.h>?
After some effort I think now, it is not good idea try to use windbase.h in C#. There are great and useful articles about P/invok. Just write "Call Unmanaged DLLs from C#" to google. One of best for me was:
http://msdn.microsoft.com/en-us/magazine/cc301501.aspx
But because our unmanaged function requires a structure as a parameter, the structure needs to be defined in the managed code as well as in the unmanaged code. In other words, it is needed to rewrite all structures and constants and stuffs to c#. And there are also other dependencies on windbase_edb.h, winnt.h, windef.h, winbase.h...
From my point of view it is too difficult way. Or it can be done differently?
Ok, so I was wondering how one would go about creating a program, that creates a second program(Like how most compression programs can create self extracting self excutables, but that's not what I need).
Say I have 2 programs. Each one containing a class. The one program I would use to modify and fill the class with data. The second file would be a program that also had the class, but empty, and it's only purpose is to access this data in a specific way. I don't know, I'm thinking if the specific class were serialized and then "injected" into the second file. But how would one be able to do that? I've found modifying files that were already compiled fascinating, though I've never been able to make changes that didn't cause errors.
That's just a thought. I don't know what the solution would be, that's just something that crossed my mind.
I'd prefer some information in say c or c++ that's cross-platform. The only other language I'd accept is c#.
also
I'm not looking for 3-rd party library's, or things such as Boost. If anything a shove in the right direction could be all I need.
++also
I don't want to be using a compiler.
Jalf actually read what I wrote
That's exactly what I would like to know how to do. I think that's fairly obvious by what I asked above. I said nothing about compiling the files, or scripting.
QUOTE "I've found modifying files that were already compiled fascinating"
Please read and understand the question first before posting.
thanks.
Building an executable from scratch is hard. First, you'd need to generate machine code for what the program would do, and then you need to encapsulate such code in an executable file. That's overkill unless you want to write a compiler for a language.
These utilities that generate a self-extracting executable don't really make the executable from scratch. They have the executable pre-generated, and the data file is just appended to the end of it. Since the Windows executable format allows you to put data at the end of the file, caring only for the "real executable" part (the exe header tells how big it is - the rest is ignored).
For instance, try to generate two self-extracting zip, and do a binary diff on them. You'll see their first X KBytes are exactly the same, what changes is the rest, which is not an executable at all, it's just data. When the file is executed, it looks what is found at the end of the file (the data) and unzips it.
Take a look at the wikipedia entry, go to the external links section to dig deeper:
http://en.wikipedia.org/wiki/Portable_Executable
I only mentioned Windows here but the same principles apply to Linux. But don't expect to have cross-platform results, you'll have to re-implement it to each platform. I couldn't imagine something that's more platform-dependent than the executable file. Even if you use C# you'll have to generate the native stub, which is different if you're running on Windows (under .net) or Linux (under Mono).
Invoke a compiler with data generated by your program (write temp files to disk if necessary) and or stored on disk?
Or is the question about the details of writing the local executable format?
Unfortunately with compiled languages such as C, C++, Java, or C#, you won't be able to just ``run'' new code at runtime, like you can do in interpreted languages like PHP, Perl, and ECMAscript. The code has to be compiled first, and for that you will need a compiler. There's no getting around this.
If you need to duplicate the save/restore functionality between two separate EXEs, then your best bet is to create a static library shared between the two programs, or a DLL shared between the two programs. That way, you write that code once and it's able to be used by as many programs as you want.
On the other hand, if you're really running into a scenario like this, my main question is, What are you trying to accomplish with this? Even in languages that support things like eval(), self modifying code is usually some of the nastiest and bug-riddled stuff you're going to find. It's worse even than a program written completely with GOTOs. There are uses for self modifying code like this, but 99% of the time it's the wrong approach to take.
Hope that helps :)
I had the same problem and I think that this solves all problems.
You can put there whatever code and if correct it will produce at runtime second executable.
--ADD--
So in short you have some code which you can hard-code and store in the code of your 1st exe file or let outside it. Then you run it and you compile the aforementioned code. If eveything is ok you will get a second executable runtime- compiled. All this without any external lib!!
Ok, so I was wondering how one would
go about creating a program, that
creates a second program
You can look at CodeDom. Here is a tutorial
Have you considered embedding a scripting language such as Lua or Python into your app? This will give you the ability to dynamically generate and execute code at runtime.
From wikipedia:
Dynamic programming language is a term used broadly in computer science to describe a class of high-level programming languages that execute at runtime many common behaviors that other languages might perform during compilation, if at all. These behaviors could include extension of the program, by adding new code, by extending objects and definitions, or by modifying the type system, all during program execution. These behaviors can be emulated in nearly any language of sufficient complexity, but dynamic languages provide direct tools to make use of them.
Depending on what you call a program, Self-modifying code may do the trick.
Basically, you write code somewhere in memory as if it were plain data, and you call it.
Usually it's a bad idea, but it's quite fun.
Take the following function as an example:
void Changing(string var1, string var2, string var3)
{
}
I have code written in .txt files, and I want it to be dynamically loaded into that function. Potentially the code could be different every time the function is called. How could I do this?
Code is in text files on the disk
Code should be dynamically loaded without stopping execution of program
Code should be indistinguishable from code loaded from the beginning, should pretend as if it was always there.
Location of the file to execute is included in var1.
You can't 100% pretend it was there from the beginning; you're going to have to work around it.
If the code is in an external text file, you have a number of options:
use CSharpCodeProvider to compile the code at runtime; you'd need to add extra fluff to make it a well-defined class (presumably implementing a common interface); then use reflection to create an instance of the type; yeuch - and beware you can't unload, etc.
wait until .NET 5.0 and hope that the compiler-as-a-service stays
use Mono, where compiler-as-a-service already exists
run the external file as (for example) a python script via IronPython (note that this changes the script language)
I'd look at the last option (IronPython) first... seems made for this job.
You can give scripting languages like Lua or boo a try. I've not implemented these but have seen the similar stuff work in java.
I have a small to medium project that is in C++/CLI. I really hate the syntax extensions of C++/CLI and I would prefer to work in C#. Is there a tool that does a decent job of translating one to the other?
EDIT: When I said Managed c++ before I apparently meant c++/CLI
You can only translate Managed C++ code (and C++/CLI code) to C# if the C++ code is pure managed. If it is not -- i.e. if there is native code included in the sources -- tools like .NET Reflector won't be able to translate the code for you.
If you do have native C++ code mixed in, then I'd recommend trying to move the native code into a separate DLL, replace your calls to DLL functions by easily identifiable stub functions, compile your project as a pure .NET library, then use .NET reflector to de-compile into C# code. Then you can replace the calls to the stub functions by p-invoke calls to your native DLL.
Good luck! I feel for you!
.NET Managed C++ is like a train wreck. But have you looked into C++ CLI? I think Microsoft did a great job in this field to make C++ a first class .NET citizen.
http://msdn.microsoft.com/en-us/magazine/cc163852.aspx
I'm not sure if this will work, but try using .Net Reflector along with ReflectionEmitLanguage plug-in. The RelelectionEmitLanguage plug-in claims to convert your assembly to c# code.
It has to be done manually unfortunately, but if the code is mostly C++/CLI (not native C++) then it can actually be done pretty quickly. I managed to port around 250,000 lines of C++/CLI code into C# in less than a couple of months, and I don't even know C++ very well at all.
If preserving Git history is important, you might want to git mv your cpp file into a cs file, commit, then start porting. The reason for this is that Git will think your file is new if you modify it too much after renaming it.
This was my approach when porting large amounts of code (so that it wouldn't take forever):
Create another worktree / clone of the branch and keep it open at all times
This is extremely important as you will want to compare your C# to the old C++/CLI code
Rename cpp to cs, delete header file, commit
I chose to rename the cpp file since its git history is probably more important than the header file
Create namespace + class in cs file, add any base classes/interfaces (if abstract sealed, make static in C#)
Copy fields first, then constructors, then properties, and finally functions
Start replacing with Ctrl+H:
^ to empty
:: to .
-> to .
nullptr to null
for each to foreach
gcnew to new
L" to "
Turn on case sensitivity to avoid accidental renames (for example L"cool" should become "cool", not "coo"
Prefixes like ClassName:: to empty, so that MyClass::MyMethod becomes MyMethod
Go through the red code and port manually code that cannot be just replaced (e.g. some special C++ casts), unless you have some cool regex to do it fast
Once code compiles, go through it again, compare to C++/CLI line by line, check for errors, clean it up, move on.
If you encounter a dependency that needs to be ported, you could pause, port that, then come back. I did that, but it might not be so easy.
Properties were the most annoying to port, because I had to remove everything before and after the getters and setters. I could have maybe written a regex for it but didn't bother doing so.
Once the porting is done, it's very important that you go through the changes line by line, read the code, and compare with C++/CLI code and fix possible errors.
One problem with this approach is that you can introduce bugs in variable declarations, because in C++/CLI you can declare variables in 2 ways:
MyType^ variable; <- null
MyType variable; <- calls default constructor
In the latter case, you want to actually do MyType variable = new MyType(); but since you already removed all the ^ you have to just manually check and test which one is correct. You could of course just replace all ^'s manually, but for me it would have taken too long (plus laziness) so I just did it this way.
Other recommendations:
Have a dummy C++/CLI project and a tool like LinqPad or another C# project to test differences between C++/CLI and C# if you're unsure of a piece of ported code
Install Match Margin to help highlight similar code (helped me when porting WinForms code)
ReSharper! It helped with finding bugs and cleaning up the code a LOT. Truly worth the money.
Some gotchas that I encountered while porting:
Base classes can be called in C++/CLI like so: BaseClass->DoStuff, but in C# you would have to do base.DoStuff instead.
C++/CLI allows such statements: if (foo), but in C# this has to be explicit. In the case of integers, it would be if (foo != 0) or for objects if (foo != null).
Events in base classes can be invoked in C++/CLI, but in C# it's not possible. The solution is to create a method, like OnSomeEvent, in the base class, and inside that to invoke the event.
C++/CLI automatically generates null checks for event invocations, so in C# make sure to add an explicit null check: MyEvent?.Invoke(this, EventArgs.Empty);. Notice the question mark.
dynamic_cast is equivalent to as cast in C#, the rest can be direct casts ((int) something).
gcnew can be done without parentheses. In C# you must have them with new.
Pay attention to virtual override keywords in the header files, you can easily forget to mark the C# methods with override keyword.
Intefaces can have implementations! In this case, you might have to rethink the architecture a bit. One option is to pull the implementation into an abstract class and derive from it
Careful when replacing casts with Convert calls in C#
Convert.ToInt32 rounds to the narest int, but casting always rounds down, so in this case we should not use the converter.
Always try casting first, and if that doesn't work, use the Convert class.
Variables in C++/CLI can be re-declared in a local scope, but in C# you get naming conflicts. Code like this easily lead to hard to find bugs if not ported carefully.
Example: An event handler can take a parameter e, but also has a try-catch like catch (Exception e) which means there are 2 e variables.
Another example:
// number is 2
int number = 2;
for (int number = 0; number < 5; number++)
{
// number is now 0, and goes up to 4
}
// number is again 2!
The above code is illegal in C#, because there is a naming conflict. Find out exactly how the code works in C++ and port it with the exact same logic, and obviously use different variable names.
In C++/CLI, it's possible to just write throw; which would create a generic C++ exception SEHException. Just replace it with a proper exception.
Be careful when porting code that uses the reference % sign, that usually means that you will have to use ref or out keywords in C#.
Similarly, pay attention to pointers * and & references. You might have to write additional code to write changes back whereas in C++ you can just modify the data pointed to by the pointer.
It's possible to call methods on null object instances in C++/CLI. Yes seriously. So inside the function you could do If (this == null) { return; }.
Port this type of code carefully. You might have to create an extension method that wraps over this type of method in order to avoid breaking the code.
Check and make sure everything in the old project file vcxproj was ported correctly. Did you miss any embedded resources?
Careful when porting directives like #ifdef, the "if not" (#ifndef) looks awfully similar but can have disastrous consequences.
C++/CLI classes automatically implement IDisposable when adding a destructor, so in C# you'll need to either implement that interface or override the Dispose method if it's available in the base class.
Other tips:
If you need to call Win32 functions, just use P/Invoke instead of creating a C++/CLI wrapper
For complex native C++ code, better create a C++/CLI project with managed wrappers
Again, pay attention to pointers. I had forgotten to do Marshal.StructureToPtr in my P/Invoke code which wasn't necessary in the C++ version since we had the actual pointer and not a copy of its data.
I have surely missed some things, but hopefully these tips will be of some help to people who are demoralized by the amount of code that needs to be ported, especially in a short period of time :)
After porting is done, use VS/ReSharper to refactor and clean up the code. Not only is it nice for readability, which is my top priority when writing code, but it also forces you to interact with the code and possibly find bugs that you otherwise would have missed.
Oh and one final FYI that could save you headaches: If you create a C++/CLI wrapper that exposes the native C++ pointer, and need to use that pointer in an external C++/CLI assembly, you MUST make the native type public by using #pragma make_public or else you'll get linker errors:
// put this at the top of the wrapper class, after includes
#pragma make_public(SomeNamespace::NativeCppClass)
If you find a bug in the C++/CLI code, keep it. You want to port the code, not fix the code, so keep things in scope!
For those wondering, we got maybe around 10 regressions after the port. Half were mistakes because I was already on autopilot mode and didn't pay attention to what I was doing.
Happy porting!
Back ~2004 Microsoft did have a tool that would convert managed C++ to C++/CLI ... sort of. We ran it on a couple of projects, but to be honest the amount of work left cleaning up the project was no less than the amount of work it would have been to do the conversion by hand in the first place. I don't think the tool ever made it out into a public release though (maybe for this reason).
I don't know which version of Visual Studio you are using, but we have managed C++ code that will not compile with Visual Studio 2005/2008 using the /clr:oldSyntax switch and we still have a relic VS 2003 around for it.
I don't know of any way of going from C++ to C# in a useful way ... you could try round tripping it through reflector :)
Such projects are often done in c++/cli because C# isn't really an elegant option for the task. e.g. if you have to interface with some native C++ libraries, or do very high performance stuff in low level C. So just make sure whoever chose c++/cli didn't have a good reason to do it before doing the switch.
Having said that, I'm highly skeptical there's something that does what you ask, for the simple reason that not all C++/cli code is translatable to C# (and probably vice versa too).