I am writing an application in C# which uses a C++/CLI Wrapper .dll which again uses another C++/CLI Wrapper for native C code.
Further Explanation:
My C# Application (to which I refer as Reporter) is nothing more than a windows form which calls the first C++/CLI Wrapper (to which I refer to as Control) which contains a UserControl. This UserControl is a GUI in order to call the last .dll (refered to as Generator). I do this because I want to use my Control in other projects and I do not want to hassle with the marshalling of my types like char *.
So here's my problem: For some times I can call my Generator-Function just as planned. But after some calls, I get an AccessViolationException.
My Generator contains loads of C-functions and also global C variables. Everything is marked properly as extern "C". As I determined, the Violation occurs when I try to a global variable.
I was trying to put all the global variables in my wrapper-class in Generator but I failed, because I could not convert all the C-types into a managed-type.
After I call the functions I was free(x)ing the space of my variables. Before commented that out, I wasn't able to call the function more than twice. Now (after commenting out) I am able to call the function in Generator 4 times. Always.
How can I work around this? Is there a way to give my function like "administrative rights" in order to allow them to do what they want with the global variables?
Thanks to all in advance, I am stuck with this for almost a month now and did much research on how to write wrapper-classes.
Leon
EDIT:
This is how I declare the concerning global variable:
"globals.h"
extern "C"{
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h> // fuer va_start, va_end
#include <string.h>
#include <malloc.h>
#include <windows.h>
//#include <omp.h>
// Used to prevent redefinition, _HAUPT_ is only defined in .dll-Header
#ifdef _HAUPT_
/*#define _HAUPT_*/
#define TYPE
int extreme = 0;
#else
#define TYPE extern
TYPE int extreme;
#endif
}
However, while writing my edit here I found out the problem was on my side. I mixed up a self-written LinkedList, error occured when I was trying to declare an unallocated pointer with a value.
I agree with Taus:
AccessViolationExceptions are most often caused by accessing
freed/unallocated memory or similar. Admin rights have nothing to do
with it. A minimal example would go along way towards identifying the
problem
The marshaler will generally try to create a managed (C#) representation of that data and then attempt to deallocate/free the unmanaged data (C++). However, you're probably not allocating memory in C/C++ in the way that the marshaler is expecting, or maybe you're were not expecting the marshaler to try and free the memory for you.
If the default deallocation behaviour doesn't match your use case then you can handle the deallocation (if at all) of the C++ object manually by using an IntPtr in your C# code. For example, if you're returning a string literal from unmanaged code, then the memory should not be deallocated. See this post for an example.
If you post the code snippet showing how you're allocating memory in your unmanaged application, how you're exposing the data, and finally, how you're accessing it in your C# application, we can help you pinpoint the problem.
You cannot get a useful answer for a question like this. Access violations are the standard way in which C code fails and there are numerous ways to invoke undefined behavior in that language. The kind that corrupts memory, such corruption eventually crashes your code. The exact place where the code fails is very rarely close to the statement with the bug. And it can take a while before the corruption has an affect. Spending a week or more to find the bug is not unusual at all.
Basic ways to go about it:
Write unit tests that exercise the C code, helps to narrow down the number of code execution paths that cause the corruption.
Be sure to build the C code with all debug features turned on, enabled by default for the Debug build in the MSVC compiler. You need all the help you can get from /MDd and /RTC.
Use Application Verifier, a tool that can detect heap corruption and show you what statement caused it. Best used on a unit test that fails.
Contemplate if the C language actually is useful to you to get the job done. You can squeeze an extra ~25% perf out of native code over managed code, give or take, but the price is rather a high one if you can't make it work reliably or lose a month of your life. Hardware is a lot cheaper than your hours.
Related
I have just started a C# project and want to import a C++ .lib and it's corresponding header (.h) file.
I've read various posts that all mention .dll, rather than .lib, which is confusing me.
The image below shows the .lib and .h file I'm referring to, all I've done is drag them into the project.
Can anyone point me to a clearer explanation of how to go about doing this? I'm sure it can't be as hard as it seems.
This is, unfortunately, a non-trivial problem.
The reason is primarily due to the fact that C++ is an unmanaged language. C# is a managed language. Managed and unmanaged refers to how a language manages memory.
C++ you must do your own memory management (allocating and freeing),
C# .NET Framework does memory management with a garbage collector.
In your library code
You must make sure all of the places you call new, must call delete, and the same goes for malloc and free if you are using the C conventions.
You will have to create a bunch of wrapper classes around your function calls, and make sure you aren't leaking any memory in your C++ code.
The problem
Your main problem (to my knowledge) is you won't be able to call those functions straight in C# because you can't statically link unmanaged code into managed code.
You will have to write a .dll to wrap all your library functions in C++. Once you do, you can use the C# interop functionality to call those functions from the dll.
[DllImport("your_functions.dll", CharSet = CharSet.Auto)]
public extern void your_function();
What you could do, is creating a C++/CLI wrapper and expose the functionality of the lib you want to use via your wrapper. The created wrapper dll you can easily reference in your C# project. This of course takes a little bit of work to create the managed/unmanaged wrapper, but will pay off in the long run.
To create a managed C++ project select under the C++ project templates CLR and Class Library. Here you can link to your lib, use the header file the way you are used to.
Next create a new class (ref class) and wrap your library in it. An example might look something like this:
LibHeader.h
int foo(...);
You write a wrapper class like this:
Header:
Wrapper.h
public ref class MyWrapper
{
public:
int fooWrapped();
};
Your Implementation:
Wrapper.cpp
#include Libheader.h
int MyWrapper::fooWrapped()
{
return foo();
}
Namespaces and all the good stuff omitted for simplicity.
Now you can use MyWrapper in your C# code just as easy as any other managed class. Of course when the interface of the lib gets more complicated you have to think about it a bit more, but it might help to separate the lib-code from your application. Hope to have shed some light on it.
It is 'as hard as it seems'. C++ and C# are ambivalent. The first has deterministic destruction, the second not. Now, you write C++/cli delaying the destruction to some finalizer called by the garbage collector working in it's own thread, introducing problems all over the place (thread safety, are C++ members (used in c++/cli) valid?, ...). Even worse the GC might suggest C++ objects being tiny (a pointer) and provoke a kind of memory leak (due to late deallocating tiny objects). Essentially you end up in writing a GC on top of the C++/cli GC to delete non C++/cli (!) objects in the main thread or another. All that is insane, ...
I have just started a C# project and want to import a C++ .lib and it's corresponding header (.h) file.
I've read various posts that all mention .dll, rather than .lib, which is confusing me.
The image below shows the .lib and .h file I'm referring to, all I've done is drag them into the project.
Can anyone point me to a clearer explanation of how to go about doing this? I'm sure it can't be as hard as it seems.
This is, unfortunately, a non-trivial problem.
The reason is primarily due to the fact that C++ is an unmanaged language. C# is a managed language. Managed and unmanaged refers to how a language manages memory.
C++ you must do your own memory management (allocating and freeing),
C# .NET Framework does memory management with a garbage collector.
In your library code
You must make sure all of the places you call new, must call delete, and the same goes for malloc and free if you are using the C conventions.
You will have to create a bunch of wrapper classes around your function calls, and make sure you aren't leaking any memory in your C++ code.
The problem
Your main problem (to my knowledge) is you won't be able to call those functions straight in C# because you can't statically link unmanaged code into managed code.
You will have to write a .dll to wrap all your library functions in C++. Once you do, you can use the C# interop functionality to call those functions from the dll.
[DllImport("your_functions.dll", CharSet = CharSet.Auto)]
public extern void your_function();
What you could do, is creating a C++/CLI wrapper and expose the functionality of the lib you want to use via your wrapper. The created wrapper dll you can easily reference in your C# project. This of course takes a little bit of work to create the managed/unmanaged wrapper, but will pay off in the long run.
To create a managed C++ project select under the C++ project templates CLR and Class Library. Here you can link to your lib, use the header file the way you are used to.
Next create a new class (ref class) and wrap your library in it. An example might look something like this:
LibHeader.h
int foo(...);
You write a wrapper class like this:
Header:
Wrapper.h
public ref class MyWrapper
{
public:
int fooWrapped();
};
Your Implementation:
Wrapper.cpp
#include Libheader.h
int MyWrapper::fooWrapped()
{
return foo();
}
Namespaces and all the good stuff omitted for simplicity.
Now you can use MyWrapper in your C# code just as easy as any other managed class. Of course when the interface of the lib gets more complicated you have to think about it a bit more, but it might help to separate the lib-code from your application. Hope to have shed some light on it.
It is 'as hard as it seems'. C++ and C# are ambivalent. The first has deterministic destruction, the second not. Now, you write C++/cli delaying the destruction to some finalizer called by the garbage collector working in it's own thread, introducing problems all over the place (thread safety, are C++ members (used in c++/cli) valid?, ...). Even worse the GC might suggest C++ objects being tiny (a pointer) and provoke a kind of memory leak (due to late deallocating tiny objects). Essentially you end up in writing a GC on top of the C++/cli GC to delete non C++/cli (!) objects in the main thread or another. All that is insane, ...
I am trying to replace the C code below by something more efficient:
void mstr2str(MonoString *mstr)
{
char *str = mono_string_to_utf8(mstr);
printf("mono string: %s\n", str);
g_free(str);
}
My goal is to avoid the memory allocation and data copy that comes with mono_string_to_utf8() because the string returned by C# to C can be very large.
I have read the suggestions about using Windows C++ COM interfaces but being under Linux I try to use a direct way to address the monostring from C.
The reference below indicates that this is "impossible":
http://www.mono-project.com/Interop_with_Native_Libraries#marshal-step-4
"there is no way to control how the runtime allocates the marshaled memory, or how long it lasts. This is crucial. If the runtime marshals
a string (e.g. UTF-16 to Ansi conversion), the marshaled string will
only last as long as the call. The unmanaged code CANNOT keep a
reference to this memory, as it WILL be freed after the call ends.
Failure to heed this restriction can result in "strange behavior",
including memory access violations and process death".
But the subject is later addressed with Marshalling, inPtr, and SafeHandles in the same document (without ready-to-use examples while the reference blogs are from 2004/2005).
Any more up-to-date documentation or code examples available?
Thank you for the solution Lupus. For those who, like me, don't know C#, these calls are macros defined in a file called object.h and therefore they are reported as unresolved symbols if you load the runtime dynamically. Here they are:
#define mono_string_chars(s) ((gunichar2*)(s)->chars)
#define mono_string_length(s) ((s)->length)
It seems that C# uses pointers finally.
That documentation is about marshaling on P/Invoke, which is a different topic, unrelated to the C code you're showing.
As for your question: you can access the string characters and do what you want with them by using mono_string_chars() and mono_string_length(). For example you could use iconv() which is available on Linux and OSX, or you can manually convert from UTF16 to UTF8 for maximum control.
What can be done to speed up calling native methods from managed code?
I'm writing a program which needs to be able to manage arbitrarily-sized lists of objects and retrieve information from them at high speed, which it feeds into scripts. Scripts are bits of compiled C# code. I'm writing a basic interface layer from the C++ (native) DLL/SO/etc to the C# (.Net or Mono) management layer.
Now, I've been doing some testing, and I've found that on average, PInvoking a native method from managed code is something like 100 times slower than doing it all in managed (all native and all managed are identically fast, for reference).
The syntax I was using is:
[DllImport("test.dll")]
extern static public String test_method(String value);
String returnedValue = test_method("hello world");
Is there a way to cache a pointer to the function, some code of fast invoker, that would increase speed after loading the native library? That would solve the problem quite neatly, so I doubt it exists. :P
Edit: I didn't specify, but this needs to work on Windows, Linux (Ubuntu at least) and Mac OS X, all for both x86 and x64. Otherwise I would've gone with a C++/CLI interface and been done with it, but unless that works for all 3 platforms, I can't use it.
Further to my question comment, we've established that it was a debug build with the debugger attached. This has a massive impact on runtime performance of .NET code. Easy mistake to make. :)
I'm guessing with a release build and no debugger attached, the performance difference is now much more reasonable.
If you have a very chatty API, and the native methods being called are cheap, then method call overhead can be a performance issue. Try and design a less chatty API. This is a typical technique used to increase the performance of boundary\systems communications.
If performance is acceptable after sorting the debugger issue, there is a simple technique that I have used to easily get a substantial performance increase in chatty APIs, by just adding a single attribute.
In the classes where you have your imported functions (i.e. the DllImport functions), place the SuppressUnmanagedCodeSecurity attribute on the classes. This will remove some expensive security checking from each P/Invoke call. Please see the documentation on SuppressUnmanagedCodeSecurity to understand the ramifications of this. I tend to keep my imported functions grouped together in internal classes (that only contain imported functions) with this attribute applied.
Perhaps the string marshalling is what is causing a slowdown. For comparison sake, try to profile a function that takes and returns elementary C++ types like int.
You can also try to experiment with C++/CLI. That way you can take explicit control over the marshalling and maybe see an improvement.
In C++/CLI assembly:
System::String ^ test_method(System::String ^ args)
{
pin_ptr<const wchar_t> pp = PtrToStringChars(args);
//This might leak, probably needs a smart pointer to wrap it
wchar_t* ret = native_method(pp);
return gcnew String^(ret);
}
I have a small to medium project that is in C++/CLI. I really hate the syntax extensions of C++/CLI and I would prefer to work in C#. Is there a tool that does a decent job of translating one to the other?
EDIT: When I said Managed c++ before I apparently meant c++/CLI
You can only translate Managed C++ code (and C++/CLI code) to C# if the C++ code is pure managed. If it is not -- i.e. if there is native code included in the sources -- tools like .NET Reflector won't be able to translate the code for you.
If you do have native C++ code mixed in, then I'd recommend trying to move the native code into a separate DLL, replace your calls to DLL functions by easily identifiable stub functions, compile your project as a pure .NET library, then use .NET reflector to de-compile into C# code. Then you can replace the calls to the stub functions by p-invoke calls to your native DLL.
Good luck! I feel for you!
.NET Managed C++ is like a train wreck. But have you looked into C++ CLI? I think Microsoft did a great job in this field to make C++ a first class .NET citizen.
http://msdn.microsoft.com/en-us/magazine/cc163852.aspx
I'm not sure if this will work, but try using .Net Reflector along with ReflectionEmitLanguage plug-in. The RelelectionEmitLanguage plug-in claims to convert your assembly to c# code.
It has to be done manually unfortunately, but if the code is mostly C++/CLI (not native C++) then it can actually be done pretty quickly. I managed to port around 250,000 lines of C++/CLI code into C# in less than a couple of months, and I don't even know C++ very well at all.
If preserving Git history is important, you might want to git mv your cpp file into a cs file, commit, then start porting. The reason for this is that Git will think your file is new if you modify it too much after renaming it.
This was my approach when porting large amounts of code (so that it wouldn't take forever):
Create another worktree / clone of the branch and keep it open at all times
This is extremely important as you will want to compare your C# to the old C++/CLI code
Rename cpp to cs, delete header file, commit
I chose to rename the cpp file since its git history is probably more important than the header file
Create namespace + class in cs file, add any base classes/interfaces (if abstract sealed, make static in C#)
Copy fields first, then constructors, then properties, and finally functions
Start replacing with Ctrl+H:
^ to empty
:: to .
-> to .
nullptr to null
for each to foreach
gcnew to new
L" to "
Turn on case sensitivity to avoid accidental renames (for example L"cool" should become "cool", not "coo"
Prefixes like ClassName:: to empty, so that MyClass::MyMethod becomes MyMethod
Go through the red code and port manually code that cannot be just replaced (e.g. some special C++ casts), unless you have some cool regex to do it fast
Once code compiles, go through it again, compare to C++/CLI line by line, check for errors, clean it up, move on.
If you encounter a dependency that needs to be ported, you could pause, port that, then come back. I did that, but it might not be so easy.
Properties were the most annoying to port, because I had to remove everything before and after the getters and setters. I could have maybe written a regex for it but didn't bother doing so.
Once the porting is done, it's very important that you go through the changes line by line, read the code, and compare with C++/CLI code and fix possible errors.
One problem with this approach is that you can introduce bugs in variable declarations, because in C++/CLI you can declare variables in 2 ways:
MyType^ variable; <- null
MyType variable; <- calls default constructor
In the latter case, you want to actually do MyType variable = new MyType(); but since you already removed all the ^ you have to just manually check and test which one is correct. You could of course just replace all ^'s manually, but for me it would have taken too long (plus laziness) so I just did it this way.
Other recommendations:
Have a dummy C++/CLI project and a tool like LinqPad or another C# project to test differences between C++/CLI and C# if you're unsure of a piece of ported code
Install Match Margin to help highlight similar code (helped me when porting WinForms code)
ReSharper! It helped with finding bugs and cleaning up the code a LOT. Truly worth the money.
Some gotchas that I encountered while porting:
Base classes can be called in C++/CLI like so: BaseClass->DoStuff, but in C# you would have to do base.DoStuff instead.
C++/CLI allows such statements: if (foo), but in C# this has to be explicit. In the case of integers, it would be if (foo != 0) or for objects if (foo != null).
Events in base classes can be invoked in C++/CLI, but in C# it's not possible. The solution is to create a method, like OnSomeEvent, in the base class, and inside that to invoke the event.
C++/CLI automatically generates null checks for event invocations, so in C# make sure to add an explicit null check: MyEvent?.Invoke(this, EventArgs.Empty);. Notice the question mark.
dynamic_cast is equivalent to as cast in C#, the rest can be direct casts ((int) something).
gcnew can be done without parentheses. In C# you must have them with new.
Pay attention to virtual override keywords in the header files, you can easily forget to mark the C# methods with override keyword.
Intefaces can have implementations! In this case, you might have to rethink the architecture a bit. One option is to pull the implementation into an abstract class and derive from it
Careful when replacing casts with Convert calls in C#
Convert.ToInt32 rounds to the narest int, but casting always rounds down, so in this case we should not use the converter.
Always try casting first, and if that doesn't work, use the Convert class.
Variables in C++/CLI can be re-declared in a local scope, but in C# you get naming conflicts. Code like this easily lead to hard to find bugs if not ported carefully.
Example: An event handler can take a parameter e, but also has a try-catch like catch (Exception e) which means there are 2 e variables.
Another example:
// number is 2
int number = 2;
for (int number = 0; number < 5; number++)
{
// number is now 0, and goes up to 4
}
// number is again 2!
The above code is illegal in C#, because there is a naming conflict. Find out exactly how the code works in C++ and port it with the exact same logic, and obviously use different variable names.
In C++/CLI, it's possible to just write throw; which would create a generic C++ exception SEHException. Just replace it with a proper exception.
Be careful when porting code that uses the reference % sign, that usually means that you will have to use ref or out keywords in C#.
Similarly, pay attention to pointers * and & references. You might have to write additional code to write changes back whereas in C++ you can just modify the data pointed to by the pointer.
It's possible to call methods on null object instances in C++/CLI. Yes seriously. So inside the function you could do If (this == null) { return; }.
Port this type of code carefully. You might have to create an extension method that wraps over this type of method in order to avoid breaking the code.
Check and make sure everything in the old project file vcxproj was ported correctly. Did you miss any embedded resources?
Careful when porting directives like #ifdef, the "if not" (#ifndef) looks awfully similar but can have disastrous consequences.
C++/CLI classes automatically implement IDisposable when adding a destructor, so in C# you'll need to either implement that interface or override the Dispose method if it's available in the base class.
Other tips:
If you need to call Win32 functions, just use P/Invoke instead of creating a C++/CLI wrapper
For complex native C++ code, better create a C++/CLI project with managed wrappers
Again, pay attention to pointers. I had forgotten to do Marshal.StructureToPtr in my P/Invoke code which wasn't necessary in the C++ version since we had the actual pointer and not a copy of its data.
I have surely missed some things, but hopefully these tips will be of some help to people who are demoralized by the amount of code that needs to be ported, especially in a short period of time :)
After porting is done, use VS/ReSharper to refactor and clean up the code. Not only is it nice for readability, which is my top priority when writing code, but it also forces you to interact with the code and possibly find bugs that you otherwise would have missed.
Oh and one final FYI that could save you headaches: If you create a C++/CLI wrapper that exposes the native C++ pointer, and need to use that pointer in an external C++/CLI assembly, you MUST make the native type public by using #pragma make_public or else you'll get linker errors:
// put this at the top of the wrapper class, after includes
#pragma make_public(SomeNamespace::NativeCppClass)
If you find a bug in the C++/CLI code, keep it. You want to port the code, not fix the code, so keep things in scope!
For those wondering, we got maybe around 10 regressions after the port. Half were mistakes because I was already on autopilot mode and didn't pay attention to what I was doing.
Happy porting!
Back ~2004 Microsoft did have a tool that would convert managed C++ to C++/CLI ... sort of. We ran it on a couple of projects, but to be honest the amount of work left cleaning up the project was no less than the amount of work it would have been to do the conversion by hand in the first place. I don't think the tool ever made it out into a public release though (maybe for this reason).
I don't know which version of Visual Studio you are using, but we have managed C++ code that will not compile with Visual Studio 2005/2008 using the /clr:oldSyntax switch and we still have a relic VS 2003 around for it.
I don't know of any way of going from C++ to C# in a useful way ... you could try round tripping it through reflector :)
Such projects are often done in c++/cli because C# isn't really an elegant option for the task. e.g. if you have to interface with some native C++ libraries, or do very high performance stuff in low level C. So just make sure whoever chose c++/cli didn't have a good reason to do it before doing the switch.
Having said that, I'm highly skeptical there's something that does what you ask, for the simple reason that not all C++/cli code is translatable to C# (and probably vice versa too).