Callbacks to C# from unmanaged C++ are tricky.
I learned most of the required cruft from this
MSDN article
and this
stackoverflow tip,
and the result works fine in the debugger.
But outside of the debugger it fails with "Object reference not set to an instance of an object".
Here's the (simplified) C# code:
class CSharpCode
{
delegate void CallbackDelegate();
void DoCSharp()
{
CallbackDelegate callbackDelegate = TheCallback;
IntPtr callbackDelegatePointer = Marshal.GetFunctionPointerForDelegate(callbackDelegate);
GCHandle gchCallbackDelegate = GCHandle.Alloc(callbackDelegatePointer);
GC.Collect(); // create max space for unmanaged allocations
CppCliCode.DoCppCli(callbackDelegatePointer);
}
public static void TheCallback()
{
MessageBox.Show("It worked");
}
}
And here's the C++ code:
#pragma managed
public ref class CppCliCode
{
static void DoCppCli(IntPtr^ callbackDelegatePointer)
{
callback theCallback = static_cast<callback>(callbackDelegatePointer->ToPointer());
DoCpp(theCallback);
}
}
#pragma unmanaged
typedef void (__stdcall *callback)();
void DoCpp(callback theCallback)
{
theCallback();
}
The error occurs somewhere between invoking theCallback() and arriving at TheCallback(). The error suggests that some invisible managed object has become null.
If I remove the GC.Collect() the problem goes away. But that just means it will reappear someday as an intermittent mystery when a GC happens to occur at the wrong moment.
The GCHandle protects the delegate from being collected but allows it to be relocated. The MSDN article says "If a delegate is re-located by a garbage collection, it will not affect the underlaying managed callback, so Alloc is used to add a reference to the delegate, allowing relocation of the delegate, but preventing disposal. Using GCHandle instead of pin_ptr reduces fragmentation potential of the managed heap."
What's wrong?
You must allocate the delegate itself, not its IntPtr. Also you must free the GCHandle when you are done with CSharpCode instance.
class CSharpCode : IDisposible
{
delegate void CallbackDelegate();
GCHandle gchCallbackDelegate;
void DoCSharp()
{
CallbackDelegate callbackDelegate = TheCallback;
IntPtr callbackDelegatePointer = Marshal.GetFunctionPointerForDelegate(callbackDelegate);
gchCallbackDelegate = GCHandle.Alloc(callbackDelegate); // !!!!
GC.Collect(); // create max space for unmanaged allocations
CppCliCode.DoCppCli(callbackDelegatePointer);
}
public void Dispose()
{
CleanUp();
}
~CSharpCode()
{
CleanUp();
}
CleanUp()
{
if(gchCallbackDelegate.IsAllocated)
gchCallbackDelegate.Free();
}
}
By the way I hope you have more powerful naming system of yours. Names like DoCSharp, TheCallBack, theCallBack etc. gave me a hard time to understand the question.
Related
I have a TestNet wrapper for a native component. The native component exposes a blocking TestNative::Foo() that communicates with managed part through calling managed callbacks and a weak GCHandle that is used to retrieve the reference to the .NET wrapper and provides a context. The GCHandle is weak since the .NET wrapper is meant to hide the fact that is handling unmanaged resources to user and deliberately doesn't implement the IDisposable interface: being non weak it would prevent TestNet instances from being collected at all, creating a memory leak. What's happening is that in Release build only the garbage collector will collect reference to .NET wrapper while executing the managed callback, even before both TestNative::Foo() and surprisingly TestNet::Foo() unblocks. I understood the problem my self and I can fix it by issuing a GC.KeepAlive(this) after the P/Invoke call but since the knowledge of this is not very widespread, it seems a lot of people are doing it wrong. I have few questions:
Is GC.KeepAlive(this) always needed in a managed method if last instruction is a P/Invoke call on unmanaged resources or it's just needed in this special case, namely the switch to managed execution context while marshaling the managed callback from native code? The question could be: should I put GC.KeepAlive(this) everywhere? This old microsoft blog (original link is 404, here is cached) seems to suggest so! But this would be game changer and basically it would mean that most people never did P/Invoke correctly, because this would require reviewing most P/Invoke calls in wrappers. Is there for example a rule that say that garbage collector (EDIT: or better the finalizer) can't run for objects that belong to the current thread while execution context is unamanaged (native)?
Where I can find proper documentation? I could find CodeAnalysis policy CA2115 pointing to generically use GC.KeepAlive(this) any time a unmanaged resource is accessed with P/Invoke. In general GC.KeepAlive(this) seems to be very rarely needed when dealing with finalizers.
Why is this happening only in Release build? It looks like an optimization but not being needed at all in Debug build hides an important behavior of the garbage collector.
NOTE: I have no problem with delegates being collected, that is a different issue which I know how to handle properly. The issue here is with objects holding unmanaged resources being collected when P/Invoke calls are not finished yet.
It follows code that clearly manifest the problem. Creates a C# console application and a C++ Dll1 project and build them in Release mode:
Program.cs:
using System;
using System.Runtime.InteropServices;
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
var test = new TestNet();
try
{
test.Foo();
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
}
class TestNet
{
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate void Callback(IntPtr data);
static Callback _callback;
IntPtr _nativeHandle;
GCHandle _thisHandle;
static TestNet()
{
// NOTE: Keep delegates references so they can be
// stored persistently in unmanaged resources
_callback = callback;
}
public TestNet()
{
_nativeHandle = CreateTestNative();
// Keep a weak handle to self. Weak is necessary
// to not prevent garbage collection of TestNet instances
_thisHandle = GCHandle.Alloc(this, GCHandleType.Weak);
TestNativeSetCallback(_nativeHandle, _callback, GCHandle.ToIntPtr(_thisHandle));
}
~TestNet()
{
Console.WriteLine("this.~TestNet()");
FreeTestNative(_nativeHandle);
_thisHandle.Free();
}
public void Foo()
{
Console.WriteLine("this.Foo() begins");
TestNativeFoo(_nativeHandle);
// This is never printed when the object is collected!
Console.WriteLine("this.Foo() ends");
// Without the following GC.KeepAlive(this) call
// in Release build the program will consistently collect
// the object in callback() and crash on next iteration
//GC.KeepAlive(this);
}
static void callback(IntPtr data)
{
Console.WriteLine("TestNet.callback() begins");
// Retrieve the weak reference to self. As soon as the istance
// of TestNet exists.
var self = (TestNet)GCHandle.FromIntPtr(data).Target;
self.callback();
// Enforce garbage collection. On release build
self = null;
GC.Collect();
GC.WaitForPendingFinalizers();
Console.WriteLine("TestNet.callback() ends");
}
void callback()
{
Console.WriteLine("this.callback()");
}
[DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
static extern IntPtr CreateTestNative();
[DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
static extern void FreeTestNative(IntPtr obj);
[DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
static extern void TestNativeSetCallback(IntPtr obj, Callback callback, IntPtr data);
[DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
static extern void TestNativeFoo(IntPtr obj);
}
}
Dll1.cpp:
#include <iostream>
extern "C" typedef void (*Callback)(void *data);
class TestNative
{
public:
void SetCallback(Callback callback1, void *data);
void Foo();
private:
Callback m_callback;
void *m_data;
};
void TestNative::SetCallback(Callback callback, void * data)
{
m_callback = callback;
m_data = data;
}
void TestNative::Foo()
{
// Foo() will never end
while (true)
{
m_callback(m_data);
}
}
extern "C"
{
__declspec(dllexport) TestNative * CreateTestNative()
{
return new TestNative();
}
__declspec(dllexport) void FreeTestNative(TestNative *obj)
{
delete obj;
}
__declspec(dllexport) void TestNativeSetCallback(TestNative *obj, Callback callback1, void * data)
{
obj->SetCallback(callback1, data);
}
__declspec(dllexport) void TestNativeFoo(TestNative *obj)
{
obj->Foo();
}
}
The output is consistently:
this.Foo() begins
TestNet.callback() begins
this.callback()
this.~TestNet()
TestNet.callback() ends
TestNet.callback() begins
System.NullReferenceException: Object reference not set to an instance of an object.
If one uncomment the GC.KeepAlive(this) call in TestNet.Foo() the program correctly never ends.
Summarizing very useful comments and research done:
1) Is GC.KeepAlive(this) always needed in a managed instance method if last instruction is a P/Invoke call using unmanaged resources hold by the instance?
Yes, if you don't want the user of the API to have last responsibility of holding a non-collectible reference for the instance of the managed object in pathological cases, look the example below. But it's not the only way: HandleRef or SafeHandle techiniques can also be used to prolong the lifetime of a managed object when doing P/Invoke Interop.
The example will subsequently call native methods through managed instances holding native resources:
using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Threading;
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
new Thread(delegate()
{
// Run a separate thread enforcing GC collections every second
while(true)
{
GC.Collect();
Thread.Sleep(1000);
}
}).Start();
while (true)
{
var test = new TestNet();
test.Foo();
TestNet.Dump();
}
}
}
class TestNet
{
static ManualResetEvent _closed;
static long _closeTime;
static long _fooEndTime;
IntPtr _nativeHandle;
public TestNet()
{
_closed = new ManualResetEvent(false);
_closeTime = -1;
_fooEndTime = -1;
_nativeHandle = CreateTestNative();
}
public static void Dump()
{
// Ensure the now the object will now be garbage collected
GC.Collect();
GC.WaitForPendingFinalizers();
// Wait for current object to be garbage collected
_closed.WaitOne();
Trace.Assert(_closeTime != -1);
Trace.Assert(_fooEndTime != -1);
if (_closeTime <= _fooEndTime)
Console.WriteLine("WARN: Finalize() commenced before Foo() return");
else
Console.WriteLine("Finalize() commenced after Foo() return");
}
~TestNet()
{
_closeTime = Stopwatch.GetTimestamp();
FreeTestNative(_nativeHandle);
_closed.Set();
}
public void Foo()
{
// The native implementation just sleeps for 250ms
TestNativeFoo(_nativeHandle);
// Uncomment to have all Finalize() to commence after Foo()
//GC.KeepAlive(this);
_fooEndTime = Stopwatch.GetTimestamp();
}
[DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
static extern IntPtr CreateTestNative();
[DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
static extern void FreeTestNative(IntPtr obj);
[DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
static extern void TestNativeFoo(IntPtr obj);
}
}
For the native call to be always safe we expect finalizer to be called only after Foo() return. Instead we can easily enforce violations by manually invoking garbage collection in a background thread. Output follows:
Finalize() commenced after Foo() return
WARN: Finalize() commenced before Foo() return
Finalize() commenced after Foo() return
Finalize() commenced after Foo() return
Finalize() commenced after Foo() return
WARN: Finalize() commenced before Foo() return
Finalize() commenced after Foo() return
2) Where I can find documentation?
Documentation of GC.KeepAlive() provides an example very similar to the managed callback in the original question. HandleRef has also very interesting considerations about lifecycle of managed objects and Interop:
If you use platform invoke to call a managed object, and the object is
not referenced elsewhere after the platform invoke call, it is
possible for the garbage collector to finalize the managed object.
This action releases the resource and invalidates the handle, causing
the platform invoke call to fail. Wrapping a handle with HandleRef
guarantees that the managed object is not garbage collected until the
platform invoke call completes.
Also link[1] found by #GSerg explains when an object is eligible for collection, pointing that this reference is not in the root set, allowing it to be collected also when instance method has not returned.
3) Why is this happening only in Release build?
It's an optimization and can happen also in Debug build, with optimization enabled, as pointed by #SimonMourier. It's not enabled by default also in Debug because it could prevent debugging of variables in the current method scope, as explained in these other answers.
[1]
https://devblogs.microsoft.com/oldnewthing/20100810-00/?p=13193?
I'm writing a DLL in C# that is called from Delphi using dllexport.
The basics is very simple and works, but I wanted to expose a function that returns fills a string with specific message (not returning a string). The way I'm doing it is that the Delphi sends the function a pointer of type wide string (LPWSTR) and I fill it. This works well. However, when I do it, I need to free the memory and this is where I'm not sure I did the right thing.
The DLL is build in one class that implement IDisposable. This is because I need to free the memory after the function executes.
Here is the class and this one function.
In order to release the memory I allocate, I declare a global static intPtr, which I release as dispose. Here is the code.
Would be happy to get any feedback
namespace simpleDLL
{
public class Test : IDisposable
{
public delegate int prog(int i);
static IntPtr s_copy;
bool disposed = false;
[DllExport("getString", CallingConvention = CallingConvention.Cdecl)]
public static int getString(out IntPtr s, [MarshalAs(UnmanagedType.LPWStr)] string str)
{
s_copy = (IntPtr)Marshal.StringToHGlobalUni(str);
s = s_copy;
return str.Length;
}
public void Dispose()
{
Dispose(true);
}
// Protected implementation of Dispose pattern.
protected virtual void Dispose(bool disposing)
{
if (disposed)
return;
if (disposing)
{
Marshal.FreeHGlobal(s_copy);
// Free any other managed objects here.
//
}
// Free any unmanaged objects here.
//
disposed = true;
}
~Test()
{
Dispose(false);
}
}
The following sample is from Microsoft's documentation:
public delegate bool CallBack(int handle, IntPtr param);
public class LibWrap
{
// passing managed object as LPARAM
// BOOL EnumWindows(WNDENUMPROC lpEnumFunc, LPARAM lParam);
[DllImport("user32.dll")]
public static extern bool EnumWindows(CallBack cb, IntPtr param);
}
public class App
{
public static void Main()
{
Run();
}
[SecurityPermission(SecurityAction.Demand, UnmanagedCode=true)]
public static void Run()
{
TextWriter tw = System.Console.Out;
GCHandle gch = GCHandle.Alloc(tw);
CallBack cewp = new CallBack(CaptureEnumWindowsProc);
// platform invoke will prevent delegate to be garbage collected
// before call ends
LibWrap.EnumWindows(cewp, GCHandle.ToIntPtr(gch));
gch.Free();
}
private static bool CaptureEnumWindowsProc(int handle, IntPtr param)
{
GCHandle gch = GCHandle.FromIntPtr(param);
TextWriter tw = (TextWriter)gch.Target;
tw.WriteLine(handle);
return true;
}
}
There are two things that mystify me.
Firstly, the documentation of GCHandle.Alloc only talks about preventing a an object to be garbage collected. If that was all, you wouldn't need GCHandle.Alloc: Obviously in the sample, tw isn't going to be collected during the call to EnumWindows - there's a reference to it in the function scope.
The concern here is that one needs to make sure that it isn't moved. But the documentation of GCHandle.Alloc doesn't talk about that. So what's going on?
Secondly, what about the delegate? There may not be a problem in this sample, but what if the delegate is bound to an object (a lambda with a closure or a non-static method of a class)? In that case, one needs to take care of the delegate too, right? Is that another GCHandle.Alloc(myDelegate) or are there more things to consider?
Moving is not a concern. GCHandle.ToIntPtr promises to give you an integer value that you can, at a later date, pass to GCHandle.FromIntPtr to retrieve the original handle. That's all. If you need to stop the object moving in memory then you would have to pin it. But you don't actually need to pin the object, you just need to stop it being collected, and be able to retrieve it in your callback.
The delegate's lifetime is not an issue here because the p/invoke framework will make sure it is not collected during the external call to EnumWindows, as noted in the comment. If you pass a delegate to unmanaged code, and that unmanaged code holds a reference to the delegate, then you have work to do. You must ensure that the delegate outlives the unmanaged reference to it.
Following is a stripped-down version of c# code, which helps to capture the PrintScreen key. This work as I expected and tested.
Question: I am aware of deterministic destruction/disposal pattern, which I started to draft as below. However, I need some expert advice to complete my dispose and finalize method. Any advise ?
public class RegisterPrintKey : IDisposable
{
public delegate void HotKeyPass();
public event HotKeyPass HotKey;
private IntPtr m_WindowHandle = IntPtr.Zero;
private MKEY m_ModKey = MKEY.MOD_CONTROL;
private Keys m_Keys = Keys.A;
private HotKeyWndProc m_HotKeyWnd = new HotKeyWndProc();
[DllImport("user32.dll")]
public static extern bool RegisterHotKey(IntPtr wnd, int id, MKEY mode, Keys vk);
[DllImport("user32.dll")]
public static extern bool UnregisterHotKey(IntPtr wnd, int id);
private class HotKeyWndProc : NativeWindow
{
public int m_WParam = 10000;
public HotKeyPass m_HotKeyPass;
protected override void WndProc(ref Message m)
{
if (m.Msg == 0x0312 && m.WParam.ToInt32() == m_WParam)
{
if (m_HotKeyPass != null) m_HotKeyPass.Invoke();
}
base.WndProc(ref m);
}
}
private bool hasDisposed = false;
protected virtual void Dispose(bool dispose)
{
if (hasDisposed) return;
if (dispose)
{
//release objects owned by this instance
HotKey = null;
hasDisposed=true;
}
m_WindowHandle = IntPtr.Zero; // I presume this is not required.
m_Keys = null; //Do i need to dispose this or relay on stack ( clean up when thread unwind its stack)
m_ModKey = null;
m_HotKeyWnd.DestroyHandle();
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
~RegisterPrintKey()
{
Dispose(false);
}
}
public enum MKEY
{
MOD_ALT = 0x0001,
MOD_CONTROL = 0x0002,
MOD_SHIFT = 0x0004,
MOD_WIN = 0x0008,
}
Some suggestions on your code
public class RegisterPrintKey : IDisposable {
...
// This class can allocate the Window Handle resource (HWND)
private class HotKeyWndProc : NativeWindow {
}
// Explicit resource (HWND) allocation
private HotKeyWndProc m_HotKeyWnd = new HotKeyWndProc();
// I'd rather make a property from your "hasDisposed" field:
// - it's make easier to check instance's state (esp. while debugging)
// - IsDisposed is more popular name for this
public Boolean IsDisposed {
get;
protected set; // <- or even "private set"
}
protected virtual void Dispose(Boolean dispose) {
if (IsDisposed)
return;
if (disposed) {
// Release any Objects here
// You've allocated the HWND resource so you have to dispose it:
m_HotKeyWnd.DestroyHandle(); // <- Free HWND you've created
}
// Here, you may work with structures only!
// You don't need these assignments, but you can safely do them:
// mayhaps, they'll be useful for you while debugging
m_WindowHandle = IntPtr.Zero;
m_Keys = null;
m_ModKey = null;
// Finally, we've done with disposing the instance
IsDisposed = true;
}
public void Dispose() {
Dispose(true);
// We've done with Dispose: "GC, please, don't bother the disposed instance"
GC.SuppressFinalize(this);
}
// A treacherous enemy I've commented out:
// if you've made a error in Dispose() it'll be resource leak
// or something like AccessViolation. The only good thing is that
// you can easily reproduce (and fix) the problem.
// If you've uncommented ~RegisterPrintKey() this leak/EAV will become
// floating error: it'll appear and disappear on different workstations
// OSs etc: you can't control GC when to run. Floating error is
// much harder to detect.
// Another bad issue with finalizer is that it prevents the instance
// from being in zero generation, see
// http://stackoverflow.com/questions/12991692/why-doesnt-object-with-finalizer-get-collected-even-if-it-is-unrooted
//~RegisterPrintKey() {
// // This code can be called (if called!) at random time
// Dispose(false); // <- That's an enemy!
// }
}
The basic idea is not to touch anything managed in the Dispose(false) variant of your method. Also, there's usually no need to explicitly set anything to null (especially in the Dispose(false) variant, where those objects are probably already garbage collected).
So, you've got the basic pattern right, but you don't need to do anything in Dispose(false) except for the m_HotKeyWnd.DestroyHandle().
To explain a bit more, when the finalizer code is run by the garbage collector, all of the other managed objects referenced by this object (that aren't referenced by others) are probably garbage collected already. Ignore everything that's managed in the finalizer, it probably doesn't even exist anymore. It's also something you can expect in the native callback (ie. your WndProc in this case) - it's very much possible that the managed objects you interact with don't exist anymore, if the callback comes while the object is on the finalizer queue. This is a fairly common cause of application crashes (or at least of unexpected exceptions) in a managed application interacting with native code.
So, Dispose(true) should handle everything you want to clean up, managed and unmanaged (and you're correctly using GC.SuppressFinalize - you've already disposed of the unmanaged resources, so there's no need for GC to put your object on the finalizer queue), Dispose(false) should only ever handle the unmanaged bits.
EDIT: I didn't realize that your m_HotKeyWnd is actually a managed object - it takes care of its unmanaged resources on its own, and you should in no case call its DestroyHandle from the finalizer. In fact, you don't even need the finalizer, its completely redundant (and thus, slightly harmful). Just implement a simple Dispose (instead of the usual finalizer-dispose pattern) that disposes of the m_HotKeyWnd and sets it to null (NullReferenceException is better in this case than AccessViolationException or undefined behaviour you could get when using the object after its been disposed of - unmanaged stuff gets tricky fast), and does nothing else.
I have a problems with passing string from unmanaged code to managed.
In my unmanaged class (unmanagedClass.cpp) I have a pointer to function from managed code:
TESTCALLBACK_FUNCTION testCbFunc;
TESTCALLBACK_FUNCTION takes one string and returns nothing:
typedef void (*TESTCALLBACK_FUNCTION )(char* msg);
Unmanaged class inherites from ITest interface which has only one method:
STDMETHOD(put_TestCallBack) (THIS_
LPVOID FnAddress
) PURE;
In managedClass.cs I write this code:
public class ManagedClass
{
ITest unmanaged = new unmanagedClass();
public delegate void TestDelegate(string info);
ManagedClass()
{
unmanaged.put_TestCallBack(new TestDelegate(this.Test));
}
void Test(string info)
{
MessageBox.Show(info);
}
}
[ComImport, Guid("<my guid here>")]
public class unmanagedClass
{
}
[ComImport, System.Security.SuppressUnmanagedCodeSecurity,
Guid("<my guid here>"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface ITest
{
[PreserveSig]
int put_TestCallBack([MarshalAs(UnmanagedType.FunctionPtr), In] Capture.TestDelegate func);
}
To call Test func from unmanaged code I use this
(*testCbFunc)("Func Uragan33::Execute has been started!");
But when Test method from managedClass.cs is called I always received null string.
Why does it happen?
Thank in advance!
You have a mismatch on the calling convention. The typedef in your C++ code declares a function pointer with the default calling convention, which is __cdecl. But the default for a delegate in managed code is __stdcall.
You will need an attribute to tell the pinvoke marshaller otherwise. Make that look like this:
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate void TestDelegate(string info);
Drop the [MarshalAs] in the function declaration. Fixing the typedef in your C++ code might be preferable, if you can, clearly making everything consistent is the preferred solution:
typedef void (__stdcall * TESTCALLBACK_FUNCTION )(char* msg);
Unrelated, this a bug you'll need to fix:
unmanaged.put_TestCallBack(new TestDelegate(this.Test));
The delegate object you create is not visible to the garbage collector. If will be collected on the next GC, your code will crash when the native code makes the callback. You have to store the delegate object somewhere so the GC always sees a reference. Either as a field in the class, with the additional requirement that the class object needs to stay alive long enough, or in a static variable.
Note how all of these problems disappear when you declare a callback interface instead of a delegate. The COM way.