I want to put a reference to a C# object into unmanaged memory (C), I guess as a pointer (int), and when the C code calls back into C# later on, I want to get the reference back from the unmanaged memory, so I can resolve it, and access the object. The reason is that the C code controls which object should be used, there's no real alternative. I have limited control over the C code and C++/CLI is not an option.
Question: Is that possible and safe, if so, how?
Well, it is possible. Primary concern is that your scheme is very incompatible with the garbage collector, it moves objects in memory when it compacts the heap. That's something you can stop, you can pin the object so the GC cannot move it. You use GCHandle.Alloc() to allocate a GCHandleType.Pinned handle and pass the return value of GCHandle.AddrOfPinnedObject() to your C code, presumably with a pinvoke call.
You have to fret about how long that object needs to stay pinned. A couple of seconds, tops, is okay, but it gets pretty detrimental to the GC if you keep it pinned for a long time. It is a rock in the road that the GC constantly has to drive around. And the heap segment can never be recycled, that single object can cost you a handful of megabytes.
In which case you should consider allocating unmanaged memory and copying the object into it. Use Marshal.AllocHGlobal() to allocate, Marshal.StructureToPtr() to copy the object into it. Possibly multiple times if you modify the object and the changes need to be visible to the C code as well.
Either way, the object must be blittable or you get a runtime error. An expensive word that just means that the object must have simple field types, the kind that a C program has a shot at reading correctly. Don't use bool. Be careful with the declaration in the C program, pretty easy to corrupt the heap when you get it wrong.
When you control the 'handing out' and the 'use after receiving back' phases you can simply use a List or array and pass around the index.
It's possible to consume C# objects via COM and proxies created by the CLR called COM-Callable Wrappers.
You just need assign a GUID assembly attribute to identify the COM type library, e.g.:
[assembly: Guid ("39ec755f-022e-497a-9ac8-70ba92cfdb7c")]
And then use the Type Library Exporter tool (tlbexp.exe) to genereate the COM type library (.tlb) file which can be consumed in the COM world:
tlbexp.exe YourLibrary.dll
If you mean safe in the C#'s sense of the word, then certainly unsafe, as you'll be using the objects in the unmanaged world, and lifetimes are controlled from the COM side via reference counting as opposed to CLR's GC.
Related
I have a C# class, having a static ImageList object. This image list will be shared with various ListView headers (via SendMessage... HDM_SETIMAGELIST) on several forms in my application.
Though I understand that static objects are not eligible for garbage collection, it is not clear to me if they are also ineligible for relocation (compaction) by the garbage collector. Do I also need to pin this object since it is shared with unmanaged code, say, using GCHandle.Alloc?
Environment is VS 2008, Compact Framework 3.5.
The instance itself is not static. The reference is. If you null the reference the instance becomes eligible for GC. Internally, all static instances are references through a pinned handle to an array of statics. I.e. the instance is implicitly pinned by the runtime.
If you look at the GCroot of an instance declared as a static member, you'll see something like this:
HandleTable:
008113ec (pinned handle)
-> 032434c8 System.Object[]
-> 022427b0 System.Collections.Generic.List`1[[System.String, mscorlib]]
If you null the static reference the corresponding entry in the pinned array is nulled as well.
Now, these are obviously implementation details so they could potentially change.
Yes. You need to pin the object.
While it's true that the reference is static, that is, you may access this location anywhere from your member it's reference is still a GC handle. That is, it's eligible for garbage collection (and/or compaction) but it will of course never happen.
I don't think it's necessarily wrong to think that the static modifier implies that it will eventually have a static location in memory but there bigger issue is that there's no API that allows you to get at the memory address without pinning the object. Whether it's being moved by the GC or not.
Moreover, each static member is unqiue per AppDomain (not process). The same static member could exist in different memory locations in the same process and it can be garbage collected when the AppDomain unloads. This is quite the edge case I'll admit but there's no real advantage of not pinning objects even if it could be done without pinning.
Do I also need to pin this object since it is shared with unmanaged
code, say, using GCHandle.Alloc?
Yes. If the pointer is not pinned, GC is free to move that memory, so you may have dangling C++ pointers, pointing to some non valid, or worse, non their memory at all.
Also, "shared" word should be clarified. If you allocate and pass to unmanaged memory, which copies it somewhere, you may avoid to pin them constantly. Depends on what happens once you pass control to unmanaged environment.
EDIT
Even considering interesting answer from #Brian, I would still opt for pinning the pointer. To make explicit in the code the notion of the fixed pointer, avoid any possible misguide in future code maintenance and keep clarity.
I am in the situation where I have a c# object containing an HGLOBAL and a List<Delegate>. The HGLOBAL holds references to the delegates and the HGLOBAL is registered in unmanaged code as a callback (specifically, a vtbl-style interface).
I'm trying to not get bogged down in the details of the overall design (I'm already expecting some screaming from those that understand what I'm doing in the previous paragraph). My question is currently one of lifetime management using functions I wish were documented differently.
I have converted the class holding the HGLOBAL and the List<Delegate> into a subclass of SafeHandle (because of CriticalFinalizerObject and because HGLOBAL is kinda a handle), and I do have the class that is instantiating this using CER-style atomic transactions to inform C# when the unmanaged code stops expecting it to stick around.
Does calling DangerousAddRef effectively bump up a reference count for the entire SafeHandle object? Will the List<Delegate> be included in this reference count? Another way of asking this is whether DangerousAddRef is effectively constructing a GCHandle to the entire SafeHandle object, or whether it is doing so simply to the SafeHandle.handle member inside?
If it is not applying to the object as a whole then the List<Delegate> is likely to be collected early and cause access violations when unmanaged code tries to call into it.
If it is applying to the object as a whole then I have just created a block of reference-counted objects within a garbage-collected heap and I had better be darned sure that I haven't created any reference loops.
I suspect it's the latter, but there are very few Google searches on this kinda stuff, and when the only symptoms you're dealing with are unmanaged access violations from the finalizer thread on process shutdown when no actual managed code is being ran then it starts to get difficult to see whether you're going in the right direction.
There are lots of questions about managed vs unmanaged resources. I understand the basic definition of the two. However, I have a hard time knowing when a resource or object is managed or unmanaged.
When I think of unmanaged resources I tend to think of native code that isn't directly part of .NET such as pinvoke or marshaling resources. I would normally think of resources meant to interface to something that will use HW such as a file handle or network connection also being unmanaged.
What about .NET objects that wrap native unmanaged resources such as a FileStream.
A FileStream must use unmanaged resources, but when I implement the IDisposable pattern, should I consider this a managed or unmanaged resources?
I've been assuming thus far that if the object implements IDisposable, then it is managed. How would I know that IntPtr should be handled as an unmanaged resoruce?
A FileStream must use unmanaged resources, but when I implement the IDisposable pattern, should I consider this a managed or unmanaged resources?
A FileStream is a managed resource.
Managed resources are classes that contain (and must manage) unmanaged resources. Usually the actual resource is several layers down.
I've been assuming thus far that if the object implements IDisposable, then it is managed.
Correct.
How would I know that IntPtr should be handled as an unmanaged resoruce?
From the documentation of the API that you got its value from. But do note that in practice, most programmers never deal with unmanaged resources directly. And when you do have to, use the SafeHandle class to turn an unmanaged resource into a managed resource.
It is pretty straight-forward, you can never accidentally allocate an unmanaged resource. A pinvoke call is required to allocate it, you'd know about it. The term "object" is overloaded, but there is no such thing as an unmanaged object, all objects in a .NET program are managed. You may interop with code written in another language that supports creating objects, like C++. But you cannot directly use such an object, a C++/CLI wrapper is required. Which makes it a managed class that implements IDisposable.
If you work with a poorly documented library then do pay attention when you get an IntPtr back. That's a pretty strong indication that an unmanaged allocation is involved, either a pointer to unmanaged memory or an operating system handle. That library should then also give you a way to release it, if it doesn't otherwise manage it automatically. Contact the owner of the library if you are not sure how to properly deal with it.
It was Microsoft's job to provide managed wrapper classes around all common operating system resources. Like FileStream, Socket, etcetera. Those classes almost always implement IDisposable. The only thing you have to do in your code when you store such an class object in your own class is to implement IDisposable yourself, just so you call the Dispose() method on those object. Or use the using statement if you use them as a local variable in a method.
It is most helpful to think of a "resource" in this context as meaning "something which an object has asked something else to do on its behalf, until further notice, to the detriment of everyone else". An object constitutes a "managed resource" if abandoning it would result in the garbage collector notifying the object of abandonment, and the object in turn instructing anything that was acting on its behalf to stop doing so. An "unmanaged resource" is a resource which is not encapsulated within a managed resource.
If some object Foo allocates a handle to unmanaged memory, it asks the memory manager to grant it exclusive use of some area of memory, making it unavailable to any other code that might otherwise want to use it, until such time as Foo informs the memory manager that the memory is no longer needed and should thus be made available for other purposes. What makes the handle an unmanaged resource is not the fact that it was received via an API, but rather the fact that even if all deliberate references to it were abandoned the memory manager would forever continue granting exclusive use of the memory to an object which no longer needs it (and likely no longer exists).
While API handles are the most common kind of unmanaged resource, there are countless other kinds as well. Things like monitor locks and events exist entirely within the managed-code world of .net, but can nonetheless represent unmanaged resources since acquiring a lock and abandoning while code is waiting on it may result in that code waiting forever, and since a short-lived object which subscribes to an event from a long-lived object and fails to unsubscribe before it is abandoned may cause that long-lived object to continue carrying around the event reference indefinitely (a small burden if only one subscriber is abandoned, but an unbounded burden if an unbounded number of subscribers are created and abandoned).
Addendum
A fundamental assumption of the garbage collector is that when object X holds a reference to object Y, it is because X is "interested" in Y. In some situations, however, the reference may be held because X wants Y to hold a reference to it even though Y doesn't "care" one way or the other. Such situations occur frequently with notification event handlers. Object Y may want to be notified every time something happens to object X. Although X has to keep a reference to Y so it can perform such notifications, X itself doesn't care about the notifications. It only performs them because of a presumption that some rooted object might care about Y's receiving them.
In some cases, it's possible to use what's called a "weak event pattern". Unfortunately, while there are many weak event patterns in .net, all of them have quirks and limitations due to the lack of a proper WeakDelegate type. Further, while weak events are helpful, they're not a panacea. Suppose, for example, that Y has asked long-lived object X to notify it when something happens, the only existing reference to Y is the one X uses for such notification, the only thing Y does with such notification is to increment a property in some object Z, and that setting that property modifies nothing outside Z. Under that scenario, even though object Z will be the only thing in the universe that "cares" about object Y, Z won't hold any sort of reference to Y whatsoever, and so the garbage collector will have no way of tying Y's lifetime to that of Z. If a X holds a strong reference to Y, the latter will be kept alive even after nobody's interested in it. If X only holds a weak reference, then Y may be garbage-collected even if Z is interested in it. There is no mechanism by which the garbage collector can automatically infer that Z is interested in Y.
I'm converting a C# project to C++ and have a question about deleting objects after use. In C# the GC of course takes care of deleting objects, but in C++ it has to be done explicitly using the delete keyword.
My question is, is it ok to just follow each object's usage throughout a method and then delete it as soon as it goes out of scope (ie method end/re-assignment)?
I know though that the GC waits for a certain size of garbage (~1MB) before deleting; does it do this because there is an overhead when using delete?
As this is a game I am creating there will potentially be lots of objects being created and deleted every second, so would it be better to keep track of pointers that go out of scope, and once that size reachs 1MB to then delete the pointers?
(as a side note: later when the game is optimised, objects will be loaded once at startup so there is not much to delete during gameplay)
Your problem is that you are using pointers in C++.
This is a fundamental problem that you must fix, then all your problems go away. As chance would have it, I got so fed up with this general trend that I created a set of presentation slides on this issue. – (CC BY, so feel free to use them).
Have a look at the slides. While they are certainly not entirely serious, the fundamental message is still true: Don’t use pointers. But more accurately, the message should read: Don’t use delete.
In your particular situation you might find yourself with a lot of long-lived small objects. This is indeed a situation which a modern GC handles quite well, and which reference-counting smart pointers (shared_ptr) handle less efficiently. If (and only if!) this becomes a performance problem, consider switching to a small object allocator library.
You should be using RAII as much as possible in C++ so you do not have to explicitly deleteanything anytime.
Once you use RAII through smart pointers and your own resource managing classes every dynamic allocation you make will exist only till there are any possible references to it, You do not have to manage any resources explicitly.
Memory management in C# and C++ is completely different. You shouldn't try to mimic the behavior of .NET's GC in C++. In .NET allocating memory is super fast (basically moving a pointer) whereas freeing it is the heavy task. In C++ allocating memory isn't that lightweight for several reasons, mainly because a large enough chunk of memory has to be found. When memory chunks of different sizes are allocated and freed many times during the execution of the program the heap can get fragmented, containing many small "holes" of free memory. In .NET this won't happen because the GC will compact the heap. Freeing memory in C++ is quite fast, though.
Best practices in .NET don't necessarily work in C++. For example, pooling and reusing objects in .NET isn't recommended most of the time, because the objects get promoted to higher generations by the GC. The GC works best for short lived objects. On the other hand, pooling objects in C++ can be very useful to avoid heap fragmentation. Also, allocating a larger chunk of memory and using placement new can work great for many smaller objects that need to be allocated and freed frequently, as it can occur in games. Read up on general memory management techniques in C++ such as RAII or placement new.
Also, I'd recommend getting the books "Effective C++" and "More effective C++".
Well, the simplest solution might be to just use garbage collection in
C++. The Boehm collector works well, for example. Still, there are
pros and cons (but porting code originally written in C# would be a
likely candidate for a case where the pros largely outweigh the cons.)
Otherwise, if you convert the code to idiomatic C++, there shouldn't be
that many dynamically allocated objects to worry about. Unlike C#, C++
has value semantics by default, and most of your short lived objects
should be simply local variables, possibly copied if they are returned,
but not allocated dynamically. In C++, dynamic allocation is normally
only used for entity objects, whose lifetime depends on external events;
e.g. a Monster is created at some random time, with a probability
depending on the game state, and is deleted at some later time, in
reaction to events which change the game state. In this case, you
delete the object when the monster ceases to be part of the game. In
C#, you probably have a dispose function, or something similar, for
such objects, since they typically have concrete actions which must be
carried out when they cease to exist—things like deregistering as
an Observer, if that's one of the patterns you're using. In C++, this
sort of thing is typically handled by the destructor, and instead of
calling dispose, you call delete the object.
Substituting a shared_ptr in every instance that you use a reference in C# would get you the closest approximation at probably the lowest effort input when converting the code.
However you specifically mention following an objects use through a method and deleteing at the end - a better approach is not to new up the object at all but simply instantiate it inline/on the stack. In fact if you take this approach even for returned objects with the new copy semantics being introduced this becomes an efficient way to deal with returned objects also - so there is no need to use pointers in almost every scenario.
There are a lot more things to take into considerations when deallocating objects than just calling delete whenever it goes out of scope. You have to make sure that you only call delete once and only call it once all pointers to that object have gone out of scope. The garbage collector in .NET handles all of that for you.
The construct that is mostly corresponding to that in C++ is tr1::shared_ptr<> which keeps a reference counter to the object and deallocates when it drops to zero. A first approach to get things running would be to make all C# references in to C++ tr1::shared_ptr<>. Then you can go into those places where it is a performance bottleneck (only after you've verified with a profile that it is an actual bottleneck) and change to more efficient memory handling.
GC feature of c++ has been discussed a lot in SO.
Try Reading through this!!
Garbage Collection in C++
I have a unmanaged DLL that exposes a function that takes a pointer to a data structure. I have C# code that creates the data structure and calls the dll function without any problem. At the point of the function call to the dll the pointer is correct.
My problem is that the DLL keeps the pointer to the structure and uses the data structure pointer at a later point in time. When the DLL comes to use the pointer it has become invalid (I assume the .net runtime has moved the memory somewhere else).
What are the possible solutions to this problem?
The possible solutions I can think of are:
Fix the memory location of the data structure somehow? I don't know how you would do this in C# or even if you can.
Allocate memory manually so that I have control over it e.g. using Marshal.AllocHGlobal
Change the DLL function contract to copy the structure data (this is what I'm currently doing as a short term change, but I don't want to change the dll at all if I can help it as it's not my code to begin with).
Are there any other better solutions?
You can allocate the structure using AllocHGlobal, which puts it in unmanaged memory, where the GC won't move it around or release it. You could also use a helper class like this to have the GC pin the memory, so it won't be moved or released until un-pinned.
See the fixed C# statement: http://msdn.microsoft.com/en-us/library/f58wzh21(VS.80).aspx
Allocate memory manually so that I have control over it e.g. using Marshal.AllocHGlobal
Pretty close.
In this specific case I'd P/Invoke LocalAlloc to allocate the memory block and use StructureToPtr to initialize it.
UPDATE: Since you can edit the DLL I'd change the DLL to provide AllocXXX and FreeXXX functions.
Without having the DLL on hand to try this, it's hard to say if this would work. I would try making the object "fixed" in your C# class, that way the memory hangs out for the life of you application. Then just pass the static object to the DLL.
The GCHandle class was designed to handle this exact scenario. Essentially, you box a copy of your struct onto the heap, and then call GCHandle.Alloc with GCHandleType.Pinned. When the DLL is done with the structure, call GCHandle.Free. To give the DLL function the address of the object, pass it GCHandle.AddrOfPinnedObject. As long as it is pinned, the GC won't move it.