I'm using an Intel Ivy Bridge CPU and want to use the RDRAND opcode (https://software.intel.com/en-us/articles/intel-digital-random-number-generator-drng-software-implementation-guide) in C#.
How can I call this CPU instruction via C#? I've seen an example of executing assembly code from c# here: x86/x64 CPUID in C#
But I'm not sure how to use it for RDRAND. The code doesn't need to check whether the CPU executing the code supports the instruction or not.
I've seen this C++ example of executing assembly byte code coming from drng_samples of Intel:
int rdrand32_step (uint32_t *rand)
{
unsigned char ok;
/* rdrand edx */
asm volatile(".byte 0x0f,0xc7,0xf0; setc %1"
: "=a" (*rand), "=qm" (ok)
:
: "edx"
);
return ok;
}
How can the example of executing assembly code in C# be combined with the C++ code coming from the Intel drng sample code?
There are answers out on SO that will generate (unmanaged) assembly code at runtime for managed code to call back into. That's all very interesting, but I propose that you simply use C++/CLI for this purpose, because it was designed to simplify interop scenarios. Create a new Visual C++ CLR class library and give it a single rdrandwrapper.cpp:
#include <immintrin.h>
using namespace System;
namespace RdRandWrapper {
#pragma managed(push, off)
bool getRdRand(unsigned int* pv) {
const int max_rdrand_tries = 10;
for (int i = 0; i < max_rdrand_tries; ++i) {
if (_rdrand32_step(pv)) return true;
}
return false;
}
#pragma managed(pop)
public ref class RandomGeneratorError : Exception
{
public:
RandomGeneratorError() : Exception() {}
RandomGeneratorError(String^ message) : Exception(message) {}
};
public ref class RdRandom
{
public:
int Next() {
unsigned int v;
if (!getRdRand(&v)) {
throw gcnew RandomGeneratorError("Failed to get hardware RNG number.");
}
return v & 0x7fffffff;
}
};
}
This is a very bare-bones implementation that just tries to mimic Random.Next in getting a single non-negative random integer. Per the question, it does not attempt to verify that RDRAND is actually available on the CPU, but it does handle the case where the instruction is present but fails to work. (This "cannot happen" on current hardware unless it's broken, as detailed here.)
The resulting assembly is a mixed assembly that can be consumed by managed C# code. Make sure to compile your assembly as either x86 or x64, same as your unmanaged code (by default, projects are set to compile as "Any CPU", which will not work correctly since the unmanaged code has only one particular bitness).
using System;
using RdRandWrapper;
class Program {
static void Main(string[] args) {
var r = new RdRandom();
for (int i = 0; i != 10; ++i) {
Console.WriteLine(r.Next());
}
}
}
I make no claims as to performance, but it's probably not great. If you wanted to get many random values this way, you would probably want a Next(int[] values) overload to get many random values in one call, to reduce the overhead of interop.
I have a C (not C++) library that consistently uses the first parameter of functions as context object (let's call the type t_context), and I'd like to use SWIG to generate C# wrappers keep this style of call (i.e. instead of the functions being more or less isolated, wrap them as methods in some class and access the t_context via a reference from the this object within the methods).
Example (C signature):
void my_lib_function(t_context *ctx, int some_param);
Desired C# API:
class Context
{
// SWIG generated struct reference
private SWIG_t_context_ptr ctx;
public void my_lib_function(int some_param)
{
// call SWIG generated my_lib_function with ctx
}
}
I'd also be happy if someone points out to me a SWIG generated wrapper for an existing C (again: not C++) library that uses this API style; I could not find anything.
Alternatively, are there wrapper generators for the C to C# use case other than SWIG that offer more control over the API (perhaps by exposing the templates used for code generation)?
In order to work through this problem I've created the following mini header-file to demonstrate all the pieces we (probably) care about to do this for real. My goals in doing this are:
C# users shouldn't even realise there's anything non-OO happening here.
The maintainer of your SWIG module shouldn't have to echo everything and write lots of proxy functions by hand if possible.
To kick things off I wrote the following header file, test.h:
#ifndef TEST_H
#define TEST_H
struct context;
typedef struct context context_t;
void init_context(context_t **new);
void fini_context(context_t *new);
void context_func1(context_t *ctx, int arg1);
void context_func2(context_t *ctx, const char *arg1, double arg2);
#endif
And a corresponding test.c with some stub implementations:
#include <stdlib.h>
#include "test.h"
struct context {};
typedef struct context context_t;
void init_context(context_t **new) {
*new = malloc(sizeof **new);
}
void fini_context(context_t *new) {
free(new);
}
void context_func1(context_t *ctx, int arg1) {
(void)ctx;
(void)arg1;
}
void context_func2(context_t *ctx, const char *arg1, double arg2) {
(void)ctx;
(void)arg1;
(void)arg2;
}
There are a few different problems we need to solve to make this into a neat, usable OO C# interface. I'll work through them one at a time and present my preferred solution at the end. (This problem can be solved in a simpler way for Python, but the solution here will be applicable to Python, Java, C# and probably others)
Problem 1: Constructor and destructor.
Typically in an OO style C API you'd have some kind of constructor and destructor functions written that encapsulate whatever setup of your (likely opaque). To present them to the target language in a sensible way we can use %extend to write what looks rather like a C++ constructor/destructor, but is still comes out after the SWIG processing as C.
%module test
%{
#include "test.h"
%}
%rename(Context) context; // Make it more C# like
%nodefaultctor context; // Suppress behaviour that doesn't work for opaque types
%nodefaultdtor context;
struct context {}; // context is opaque, so we need to add this to make SWIG play
%extend context {
context() {
context_t *tmp;
init_context(&tmp);
// we return context_t * from our "constructor", which becomes $self
return tmp;
}
~context() {
// $self is the current object
fini_context($self);
}
}
Problem 2: member functions
The way I've set this up allows us to use a cute trick. When we say:
%extend context {
void func();
}
SWIG then generates a stub that looks like:
SWIGEXPORT void SWIGSTDCALL CSharp_Context_func(void * jarg1) {
struct context *arg1 = (struct context *) 0 ;
arg1 = (struct context *)jarg1;
context_func(arg1);
}
The two things to take away from that are:
The function that implements the extended context::func call is called context_func
There's an implicit 'this' equivalent argument going into this function as argument 1 always
The above pretty much matches what we set out to wrap on the C side to begin with. So to wrap it we can simply do:
%module test
%{
#include "test.h"
%}
%rename(Context) context;
%nodefaultctor context;
%nodefaultdtor context;
struct context {};
%extend context {
context() {
context_t *tmp;
init_context(&tmp);
return tmp;
}
~context() {
fini_context($self);
}
void func1(int arg1);
void func2(const char *arg1, double arg2);
}
This doesn't quite meet point #2 of my goals as well as I'd hoped, you have to write out the function declarations manually (unless you use a trick with %include and keeping themin individual header files). With Python you could pull all the pieces together at import time and keep it much simpler but I can't see a neat way to enumerate all the functions that match a pattern into the right place at the point where SWIG generates the .cs files.
This was sufficient for me to test (using Mono) with the following code:
using System;
public class Run
{
static public void Main()
{
Context ctx = new Context();
ctx.func2("", 0.0);
}
}
There are other variants of C OO style design, using function pointers which are possible to solve and a similar question looking at Java I've addressed in the past.
I have a set of C functions that I need to use on an ARM target, in C++ and in C#. I can successfully wrap up the C into a C++ DLL and then into a C# DLL and use all the C functions I've bound successfully. However, I have a debug function that I want to be able to print to the C# GUI and the delegate it uses is being garbage collected rather than left in place for the duration.
Managed Debugging Assistant 'CallbackOnCollectedDelegate' has detected a
problem in 'C:\utm\pc\utm_win32_app\bin\Debug\utm_win32_app.vshost.exe'.
Additional Information: A callback was made on a garbage collected delegate of
type
'utm_dll_wrapper_cs!MessageCodec.MessageCodec_dll+guiPrintToConsoleCallback::
Invoke'. This may cause application crashes, corruption and data loss. When
passing delegates to unmanaged code, they must be kept alive by the managed
application until it is guaranteed that they will never be called.
Here's the snippet of C code that uses and sets up the callback mp_guiPrintToConsole:
#ifdef WIN32
static void (* mp_guiPrintToConsole) (const char*) = NULL;
void logMsg (const char * pFormat, ...)
{
char buffer[MAX_DEBUG_MESSAGE_LEN];
va_list args;
va_start (args, pFormat);
vsnprintf (buffer, sizeof (buffer), pFormat, args);
va_end (args);
#ifdef WIN32
if (mp_guiPrintToConsole)
{
(*mp_guiPrintToConsole) (buffer);
}
#else
// Must be on ARM
printf (buffer);
#endif
}
void initDll (void (*guiPrintToConsole) (const char *))
{
#ifdef WIN32
mp_guiPrintToConsole = guiPrintToConsole;
// This is the signal to the GUI that we're done with initialisation
logMsg ("ready.\r\n");
#endif
}
Here's the C++ code, built into a DLL along with the C code, that can be called from C# and passes in the function pointer printToConsole:
void msInitDll (void (*printToConsole) (const char *))
{
initDll (printToConsole);
}
Here's the snippet code from the C# DLL that calls msInitDll(), passing in guiPrintToConsole(), and defines the delegate onConsoleTrace, which I guess is the thing that is disappearing:
[UnmanagedFunctionPointer (CallingConvention.Cdecl)]
public delegate void _msInitDll([MarshalAs (UnmanagedType.FunctionPtr)] guiPrintToConsoleCallback callbackPointer);
public _msInitDll msInitDll;
public delegate void ConsoleTrace(string data);
public event ConsoleTrace onConsoleTrace;
public void guiPrintToConsole(StringBuilder data)
{
if (onConsoleTrace != null)
{
onConsoleTrace (data.ToString ());
}
}
public void bindDll(string dllLocation)
{
IntPtr ptrDll = LoadLibrary (dllLocation);
if (ptrDll == IntPtr.Zero) throw new Exception (String.Format ("Cannot find {0}", dllLocation));
//...
// All the other DLL function bindings are here
//...
msInitDll = (_msInitDll)bindItem(ptrDll, "msInitDll", typeof(_msInitDll));
msInitDll(guiPrintToConsole);
}
I've looked at the various answers here and the most promising seemed to be to create a static variable in the C# code:
static GCHandle gch;
...and then use that to reference onConsoleTrace in the C# bindDll() function:
gch = GCHandle.Alloc(onConsoleTrace);
However, that doesn't do me any good. I've tried a few other attempts at declaring things static but nothing seems to get me where I want to be. Can anyone suggest another approach to fixing the problem? I have a bug that I need to fix and the lack of any debug is proving quite annoying.
Rob
The following line uses some syntactic sugar:
msInitDll(guiPrintToConsole);
The full syntax is:
msInitDll(new guiPrintToConsoleCallback(guiPrintToConsole));
Hopefully now you see why the delegate can get garbage-collected.
One simple workaround:
var callback = new guiPrintToConsoleCallback(guiPrintToConsole);
msInitDll(callback);
// ... some other code
GC.KeepAlive(callback);
Now the delegate is guaranteed to be alive up to the GC.KeepAlive call.
But you most probably need the delegate to stay alive for longer. As the error message says, simply keep a reference to it. If you need it for the full C# app lifetime duration, turn the callback local to a static field in your class. Static fields are treated as GC roots as their values are always reachable.
And the answer was, in the C# DLL code, add the static variable:
public static guiPrintToConsoleCallback debugCallback;
...and then, in C# bindDLL(), change:
msInitDll(guiPrintToConsole);
...to
debugCallback = new guiPrintToConsoleCallback(guiPrintToConsole);
msInitDll(debugCallback);
Simple when you know how.
I have written two managed C++ wrappers for native C++ classes and I need a unmanaged object of native Class B as a return param in function of managed Wrapper A that construct native Class A!
Example:
// Wrapper A
WrapperA::WrapperA(ClassB *classB)
{
ClassA *classA = new ClassA(classB);
...
}
// native c++
ClassA::ClassA(ClassB *classB)
{
m_classB = classB; // ClassB *m_classB; in .h
...
}
// Wrapper B
ClassB* WrapperB::GetNativeClassB()
{
return m_classB; // ClassB *m_classB; in .h
}
// in C#
...
WrapperB wrapperB = new WrapperB();
unsafe // need for C++ pointer
{
WrapperA wrapperA = new WrapperA(wrapperB.GetNativeClassB() );
// Error: is inaccessible due to its protection level
// -> is set to public
}
...
Is there a better way without unsafe and why I get an access error ???
Thank you in advance!
greets leon22
Protection level: i'm sure you have public defined, but what about the dll containing the symbol? Are you sure you have the last release?
Unsafe: in order to use/wrap unsafe/native code as C++, the best option it is to use C++/CLI (ex Managed C++), provided starting from the Visual Studio 2005 release. just define a ref class that wraps your native/unmanaged class, that one will be directly accessible from managed code, as C#. Hint to start with Visual Studio: open a new dll CLR project from the Visual C++ section;
C++/CLI is the best solution in my opinion
Solution from: Pass a C++/CLI wrapper of a native type to another C++/CLI assembly
// in WrapperB
property IntPtr _classB
{
IntPtr get() { return IntPtr(classB); }
}
// in WrapperA
ClassB *classB = static_cast<ClassB*>(wrapperB->_classB.ToPointer());
// ... do something ...
I'm writing a DLL wrapper for my C++ library, to be called from C#. This wrapper should also have callback functions called from the library and implemented in C#. These functions have for instance std::vector<unsigned char> as output parameters. I don't know how to make this. How do I pass a buffer of unknown size from C# to C++ via a callback function?
Let's take this example
CallbackFunction FunctionImplementedInCSharp;
void FunctionCalledFromLib(const std::vector<unsigned char>& input, std::vector<unsigned char>& output)
{
// Here FunctionImplementedInCSharp (C# delegate) should somehow be called
}
void RegisterFunction(CallbackFunction f)
{
FunctionImplementedInCSharp = f;
}
How should CallbackFunction be defined and what is the code inside FunctionCalledFromLib?
One of the things that dumb me is: how do I delete a buffer created by C# inside C++ code?
As of Visual Studio 2013 at least, there is a safe way to pass callbacks from C# to C++ and have C++ store them and invoke them asynchronously later from unmanaged code. What you can do is create a managed C++/CX class (e.g., named "CallbackManager") to hold the callback delegate references in a map, keyed off an enum value for each. Then your unmanaged code can retrieve a managed delegate reference from the managed C++/CX CallbackManager class via the delegate's associated enum value. That way you don't have to store raw function pointers and so you don't have to worry about the delegate being moved or garbage-collected: it stays in the managed heap throughout its lifecycle.
On the C++ side in CallbacksManager.h:
#include <unordered_map>
#include <mutex>
using namespace Platform;
namespace CPPCallbacks
{
// define callback IDs; this is what unmanaged C++ code will pass to the managed CallbacksManager class to retrieve a delegate instance
public enum class CXCallbackType
{
cbtLogMessage,
cbtGetValueForSetting
// TODO: add additional enum values as you add more callbacks
}
// defines the delegate signatures for our callbacks; these are visible to the C# side as well
public delegate void LogMessageDelegate(int level, String^ message);
public delegate bool GetValueForSettingDelegate(String^ settingName, String^* settingValueOut);
// TODO: define additional callbacks here as you need them
// Singleton WinRT class to manage C# callbacks; since this class is marked 'public' it is consumable from C# as well
public ref class CXCallbacksManager sealed
{
private:
CXCallbacksManager() { } // this is private to prevent incorrect instantiation
public:
// public methods and properties are all consumable by C# as well
virtual ~CXCallbacksManager() { }
static property CXCallbacksManager^ Instance
{
CXCallbacksManager^ get();
}
bool UnregisterCallback(CXCallbackType cbType);
void UnregisterAllCallbacks();
Delegate^ GetCallback(CXCallbackType cbType);
// define callback registration methods
RegisterLogMessageCallback(LogMessageDelegate^ cb) { RegisterCallback(CXCallbackType::cbtLogMessage, cb); }
RegisterGetValueForSettingCallback(GetValueForSettingDelegate^ cb) { RegisterCallback(CXCallbackType::GetValueForSetting, cb); }
// TODO: define additional callback registration methods as you add more callbacks
private:
void RegisterCallback(CXCallbackType cbType, Delegate^ rCallbackFunc);
typedef unordered_map<CXCallbackType, Delegate^> CALLBACK_MAP;
typedef pair<CXCallbackType, Delegate^> CBType_Delegate_Pair;
// Note: IntelliSense errors shown for static data is a Visual Studio IntellSense bug; the code below builds fine
// See http://social.msdn.microsoft.com/Forums/windowsapps/en-US/b5d43215-459a-41d6-a85e-99e3c30a162e/about-static-member-of-ref-class?forum=winappswithnativecode
static mutex s_singletonMutex;
static CXCallbacksManager^ s_rInstance;
mutex m_callbackMapMutex;
CALLBACK_MAP m_callbacksMap; // key=CallbackType, value = C# delegate (function) pointer
};
}
In CallbacksManager.cpp we implement the managed C++/CX class accessed by both C# and our unmanaged C++ code:
#include <assert.h>
#include "CXCallbacksManager.h"
using namespace Platform;
namespace CPPCallbacks
{
// define static class data
CXCallbacksManager^ CXCallbacksManager::s_rInstance;
mutex CXCallbacksManager::s_singletonMutex;
// Returns our singleton instance; this method is thread-safe
CXCallbacksManager^ CXCallbacksManager::Instance::get()
{
s_singletonMutex.lock();
if (s_rInstance == nullptr)
s_rInstance = ref new CXCallbacksManager(); // this lives until the application terminates
s_singletonMutex.unlock();
return s_rInstance;
}
// Register a C# callback; this method is thread-safe
void CXCallbacksManager::RegisterCallback(const CXCallbackType cbType, Delegate^ rCallbackFunc)
{
_ASSERTE(rCallbackFunc);
m_callbackMapMutex.lock();
m_callbacksMap.insert(CBType_Delegate_Pair(cbType, rCallbackFunc));
m_callbackMapMutex.unlock();
}
// Unregister a C# callback; this method is thread-safe
// Returns: true on success, false if no callback was registered for callbackType
bool CXCallbacksManager::UnregisterCallback(const CXCallbackType cbType)
{
m_callbackMapMutex.lock();
const bool bRemoved = (m_callbacksMap.erase(cbType) > 0);
m_callbackMapMutex.unlock();
return bRemoved;
}
// Unregister all callbacks; this method is thread-safe
void CXCallbacksManager::UnregisterAllCallbacks()
{
// must lock the map before iterating across it
// Also, we can't change the contents of the map as we iterate across it, so we have to build a vector of all callback types in the map first.
vector<CXCallbackType> allCallbacksList;
m_callbackMapMutex.lock();
for (CALLBACK_MAP::const_iterator it = m_callbacksMap.begin(); it != m_callbacksMap.end(); it++)
allCallbacksList.push_back(it->first);
for (unsigned int i = 0; i < allCallbacksList.size(); i++)
{
CALLBACK_MAP::const_iterator it = m_callbacksMap.find(allCallbacksList[i]);
if (it != m_callbacksMap.end()) // sanity check; should always succeed
UnregisterCallback(it->first);
}
m_callbackMapMutex.unlock();
}
// Retrieve a registered C# callback; returns NULL if no callback registered for type
Delegate^ CXCallbacksManager::GetCallback(const CXCallbackType cbType)
{
Delegate^ rCallbackFunc = nullptr;
m_callbackMapMutex.lock();
CALLBACK_MAP::const_iterator it = m_callbacksMap.find(cbType);
if (it != m_callbacksMap.end())
rCallbackFunc = it->second;
else
_ASSERTE(false); // should never happen! This means the caller either forgot to register a callback for this cbType or already unregistered the callback for this cbType.
m_callbackMapMutex.unlock();
return rCallbackFunc;
}
}
The delegate instances remain stored in the managed heap by our CXCallbacksManager class, so now it's easy and safe to store callbacks on the C++ side for unmanaged code to invoke later asynchronously. Here is the C# side registering two callbacks:
using CPPCallbacks;
namespace SomeAppName
{
internal static class Callbacks
{
// invoked during app startup to register callbacks for unmanaged C++ code to invoke asynchronously
internal static void RegisterCallbacks()
{
CPPCallbacks.CXCallbacksManager.Instance.RegisterLogMessageCallback(new LogMessageDelegate(LogMessageDelegateImpl));
CPPCallbacks.CXCallbacksManager.Instance.RegisterGetValueForSettingCallback(new GetValueForSettingDelegate(GetValueForSettingDelegateImpl));
// TODO: register additional callbacks as you add them
}
//-----------------------------------------------------------------
// Callback delegate implementation methods are below; these are invoked by C++
// Although these example implementations are in a static class, you could also pass delegate instances created
// from inside a non-static class, which would maintain their state just like any other instance method (i.e., they have a 'this' object).
//-----------------------------------------------------------------
private static void LogMessageDelegateImpl(int level, string message)
{
// This next line is shown for example purposes, but at this point you can do whatever you want because
// you are running in a normal C# delegate context.
Logger.WriteLine(level, message);
}
private static bool GetValueForSettingDelegateImpl(String settingName, out String settingValueOut)
{
// This next line is shown for example purposes, but at this point you can do whatever you want because
// you are running in a normal C# delegate context.
return Utils.RetrieveEncryptedSetting(settingName, out settingValueOut);
}
};
}
Lastly, here is how to invoke your registered C# callbacks from unmanaged C++ code:
#include <assert.h>
#include <atlstr.h> // for CStringW
#include "CXCallbacksManager.h"
using namespace CPPCallbacks;
// this is an unmanaged C++ function in the same project as our CXCallbacksManager class
void LogMessage(LogLevel level, const wchar_t *pMsg)
{
_ASSERTE(msg);
auto rCallback = static_cast<LogMessageDelegate^>(CXCallbacksManager::Instance->GetCallback(CXCallbackType::cbtLogMessage));
_ASSERTE(rCallback);
rCallback(level, ref new String(pMsg)); // invokes C# method
}
// this is an unmanaged C++ function in the same project as our CXCallbacksManager class
// Sets settingValue to the value retrieved from C# for pSettingName
// Returns: true if the value existed and was set, false otherwise
bool GetValueForSetting(const wchar_t *pSettingName, CStringW &settingValue)
{
bool bRetCode = false;
auto rCallback = static_cast<GetValueForSettingDelegate^>(CXCallbacksManager::Instance->GetCallback(CXCallbackType::cbtGetValueForSetting));
_ASSERTE(rCallback);
if (rCallback) // sanity check; should never be null
{
String^ settingValueOut;
bRetCode = rCallback(ref new String(pSettingName), &settingValueOut);
// store the retrieved setting value to our unmanaged C++ CStringW output parameter
settingValue = settingValueOut->Data();
}
return bRetCode;
}
This all works because although you cannot store a managed delegate reference as a member variable inside an unmanaged class, you can still retrieve and invoke a managed delegate from unmanaged code, which is what the above two native C++ methods do.
There are some things you should be aware of. The first is that if you are calling a .NET delegate from unmanaged code, then unless you follow some pretty narrow constraints, you will be in for pain.
Ideally, you can create a delegate in C# pass it into managed code, marshal it into a function pointer, hold onto it for as long as you like, then call it with no ill effects. The .NET documentation says so.
I can tell you that this is simply not true. Eventually, part of your delegate or its thunk will get garbage collected and when you call the function pointer from unmanaged code you will get sent into oblivion. I don't care what Microsoft says, I've followed their prescription to the letter and watched function pointers get turned into garbage, especially in server-side code behinds.
Given that, the most effective way to use function pointers is thus:
C# code calls unmanaged code, passing in delegate.
Unmanaged code marshals the delegate to a function pointer.
Unmanaged code does some work, possible calling the function pointer.
Unmanaged code drops all references to the function pointer.
Unmanaged code returns to managed code.
Given that, suppose we have the following in C#:
public void PerformTrick(MyManagedDelegate delegate)
{
APIGlue.CallIntoUnamangedCode(delegate);
}
and then in managed C++ (not C++/CLI):
static CallIntoUnmanagedCode(MyManagedDelegate *delegate)
{
MyManagedDelegate __pin *pinnedDelegate = delegate;
SOME_CALLBACK_PTR p = Marshal::GetFunctionPointerForDelegate(pinnedDelegate);
CallDeepIntoUnmanagedCode(p); // this will call p
}
I haven't done this recently in C++/CLI - the syntax is different - I think it ends up looking like this:
// This is declared in a class
static CallIntoUnamangedCode(MyManagedDelegate ^delegate)
{
pin_ptr<MyManagedDelegate ^> pinnedDelegate = &delegate;
SOME_CALLBACK_PTR p = Marshal::GetFunctionPointerForDelegate(pinnedDelegate);
CallDeepIntoUnmanagedCode(p); // This will call p
}
When you exit this routines, the pinning gets released.
When you really, really need to have function pointers hanging around for a while before calling, I have done the following in C++/CLI:
Made a hashtable that is a map from int -> delegate.
Made register/unregister routines that add new delegates into the hashtable, bumping up a counter for the hash int.
Made a single static unmanaged callback routine that is registered into unmanaged code with an int from the register call. When this routine is called, it calls back into managed code saying "find the delegate associated with <int> and call it on these arguments".
What happens is that the delegates don't have thunks that do transitions anymore since they're implied. They're free to hang around in limbo being moved by the GC as needed. When they get called, the delegate will get pinned by the CLR and released as needed. I have also seen this method fail, particularly in the case of code that statically registers callbacks at the beginning of time and expects them to stay around to the end of time. I've seen this fail in ASP.NET code behind as well as server side code for Silverlight working through WCF. It's rather unnerving, but the way to fix it is to refactor your API to allow late(r) binding to function calls.
To give you an example of when this will happen - suppose you have a library that includes a function like this:
typedef void * (*f_AllocPtr) (size_t nBytes);
typedef void *t_AllocCookie;
extern void RegisterAllocFunction(f_AllocPtr allocPtr, t_AllocCookie cookie);
and the expectation is that when you call an API that allocates memory, it will be vectored off into the supplied f_AllocPtr. Believe it or not, you can write this in C#. It's sweet:
public IntPtr ManagedAllocMemory(long nBytes)
{
byte[] data = new byte[nBytes];
GCHandle dataHandle = GCHandle.Alloc(data, GCHandleType.Pinned);
unsafe {
fixed (byte *b = &data[0]) {
dataPtr = new IntPtr(b);
RegisterPointerHandleAndArray(dataPtr, dataHandle, data);
return dataPtr;
}
}
}
RegisterPointerHandleAndArray stuffs the triplet away for safe keeping. That way when the corresponding free gets called, you can do this:
public void ManagedFreeMemory(IntPtr dataPointer)
{
GCHandle dataHandle;
byte[] data;
if (TryUnregister(dataPointer, out dataHandle, out data)) {
dataHandle.Free();
// do anything with data? I dunno...
}
}
And of course this is stupid because allocated memory is now pinned in the GC heap and will fragment it to hell - but the point is that it's doable.
But again, I have personally seen this fail unless the actual pointers are short lived. This typically means wrapping your API, so that when you call into a routine that accomplishes a specific task, it registers callbacks, does the task, and then pulls the callbacks out.
As it turns out, the answer to the original question is rather simple, once you know it, and the whole callback issue was no issue. The input buffer parameter is replaced with parameter pair unsigned char *input, int input_length, and the output buffer parameter is replaced with parameter pair unsigned char **output, int *output_length. The C# delegate should be something like this
public delegate int CallbackDelegate(byte[] input, int input_length,
out byte[] output, out int output_length);
And wrapper in C++ should be something like this
void FunctionCalledFromLib(const std::vector<unsigned char>& input, std::vector<unsigned char>& output)
{
unsigned char *output_aux;
int output_length;
FunctionImplementedInCSharp(
&input[0], input.size(), &ouput_aux, &output_length);
output.assign(output_aux, output_aux + output_length);
CoTaskMemFree(output_aux); // IS THIS NECESSARY?
}
The last line is the last part of the mini-puzzle. Do I have to call CoTaskMemFree, or will the marshaller do it for me automagically?
As for the beautiful essay by plinth, I hope to bypass the whole problem by using a static function.
There is no point to using C++/cli.
And here is a real world example from my project.
public ImageSurface(byte[] pngData)
: base(ConstructImageSurfaceFromPngData(pngData), true)
{
offset = 0;
}
private static int offset;
private static IntPtr ConstructImageSurfaceFromPngData(byte[] pngData)
{
NativeMethods.cairo_read_func_t func = delegate(IntPtr closure, IntPtr out_data, int length)
{
Marshal.Copy(pngData, offset, out_data, length);
offset += length;
return Status.Success;
};
return NativeMethods.cairo_image_surface_create_from_png_stream(func, IntPtr.Zero);
}
That is used to transfer PNG data from C# to the native cairo API.
You can see how the C function pointer cairo_read_func_t is implemented in C# and then used as a callback for cairo_image_surface_create_from_png_stream.
Here is a similar example.